Coder Social home page Coder Social logo

ibm / dbb Goto Github PK

View Code? Open in Web Editor NEW
55.0 27.0 123.0 5.7 MB

The helpful and handy location for finding and sharing example IBM Dependency Based Build (DBB) scripts and snippets.

License: Apache License 2.0

COBOL 0.09% Groovy 16.60% Shell 10.79% Assembly 0.01% REXX 63.10% Game Maker Language 8.43% JCL 0.80% Java 0.17%

dbb's Introduction

DBB Community Repository

Welcome to the IBM Dependency Based Build (DBB) community repository. The helpful and handy location for finding and sharing example DBB scripts and snippets.

Resources

Versions

Branches and tags are used in this repository to help clarify the appropriate DBB version for the example scripts and snippets.

This repository has different branches, currently:

  • main - The branch where current development occurs, and the DBB repository is for the most recent DBB major version. This branch may not be appropriate for older versions of DBB. Users of DBB 1.x may need to look to the dbb_1_x branch.
  • dbb_1_x - For users of the DBB 1.x version, this is the appropriate matching branch for the repository.

The individual specific versions are tagged, such as v2.0.0 and v1.1.14 tags.

Contributing

For instructions on how to contribute new samples and bug fixes, please read the Contributions Guidelines.

Content

Sample Description
Build/HelloWorld The HelloWorld sample provides simple source file types and the Groovy build scripts to compile them
Build/MultiThreadMVSJob Sample showing how to modilfy a compilation script to run in a multi-thread environment.
IDE/GitISPFClient An ISPF interface that interacts with a Git repository to allow cloning, staging, checking in, pushing and pulling as well as other git commands.
Migration/jcl Sample script to migrate JCL from a Z/OS dataset to a local Groovy script.
Migration/sclm This sample provides scripts to migrate source members to local Git repository and convert the build information in SCLM into build Groovy scripts.
Pipeline/AnalyzeCodeCoverageReport Sample script to extract and print Code Coverage information as collected by IBM Debug.
Pipeline/CreateUCDComponentVersion Post-build script to parse the DBB Build report to generate a UCD component shiplist file and to create a new UCD component version.
Pipeline/DeployUCDComponentVersion Sample script to trigger a UCD deployment from the pipeline, where the pipeline orchestrator does not provide standard plugins for this task.
Pipeline/PackageBuildOutputs Post-build script to create a generic package with the produced build outputs, optionally uploads results to an Artifactory repository. Artifactory deploy/download sample script.
Pipeline/PublishSharedInterfaces Post-build script to publish shared copybooks to a shared git repository managing all shared copybooks.
Pipeline/RunIDZCodeReview Post-build script to integrate IBM IDz Code Review application into a pipeline.
Pipeline/SimplePackageDeploy Post-build script to deploy the tar package contents to the target libraries.
Scanners  Sample dependency scanner implementations using the extension framework of the DBB toolkit.
Templates/Common-Backend-Scripts Asset to encapsulate pipeline steps to simplify the pipeline implementation.
Utilities/DeletePDS Sample script to delete PDSes on z/OS that are no longer needed.
Utilities/Jenkins Utility shell scripts supplied to address issues when running Jenkins remote agents on z/OS UNIX System Services (USS).
Utilities/ReadSMFRecords Groovy scripts to read System Management Facilities (SMF) records using IBM's Dependency Based Build capabilities.
Utilities/Validation Groovy scripts that can aid in the validation of product installs.
Utilities/PermissionCheck Groovy script to check the DBB role for a provided user.

dbb's People

Contributors

anuprakash-moothedath avatar arthiireddy avatar cachapman avatar dangeville-n avatar degilio avatar dennis-behm avatar donnellt avatar drbruce-git avatar drbruce-github avatar gcavalcante avatar geraldmit avatar hannah-justine avatar ibmdbbdev avatar imgbot[bot] avatar jbyibm avatar johnmnemec avatar jsmurray2 avatar kant avatar lburgess07 avatar m-dlb avatar monvora avatar rbsmall avatar reachsenthilnathan avatar rswoffor avatar shabaz213 avatar stevemart avatar wmathis avatar xhouis avatar zach-houston avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

dbb's Issues

org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed

Hi All,

Am trying to perform DBB Team build using Jenkins, am getting below reposes with unreadable format. Attached Complete Jenkins
dbb error log.txt
log. Any help highly appreciated

org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:
/u/ibmuser/jenkins/workspace/FintechUse/MortgageApplication/build/build.groovy: 1: unexpected char: 0x89 @ line 1, column 1.
�����£@���K���K���K�¤���K\������£@���K���K���K�¤���K�����£K\������£@���K���K���K����¢�£��¨K\������£@���K���K���K���������¨K\�

ERROR: Panel definition too large, greater than screen size

We have cloned the GIT repository of the GIT ISPF client and moved that to USS server. We also ran the bgzgit.JCL to setup everything and it ran fine with return code of 0. We then modified the BGZCONF member to point to rocket GIT and other libraries.

When we ran the SBGZEXEC(BGZGIT), we got an error shown below -

ISPF Dialog Error
Command ===>


  • ISPP121 *
  • Panel 'BGZMAIN' error *
  • Panel definition too large, greater than screen size. *
  • Panel line where error was detected: *
  • Pé×ÙÉÔ@~@èÅâ@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ *
  • Enter HELP command for further information regarding this error. *
  • Press ENTER key to terminate the dialog. *

Migration of JCL to Groovy

Hi All,

Here I got the sample which provides a script to migrate JCL to local groovy script.

I am using Windows, so i installed cygwin to run shell script. But I need to specify DBB_HOME environment variable.

I referred this one. https://www.ibm.com/support/knowledgecenter/SS6T76_1.0.6/setup_dbb_toolkit.html

I got IBM Dependency Based Build z/OS Toolkit as zip file. Please help to install this one.

Please help to set environment variable. Can you help me to do all initial setup to get output. And please let me know where we adding JCL code which needed to be converted into Groovy script?. I hardly needed to learn this.

Can you please advice me on this.

Thanks.

Small usability changes when cloning a repo fails

If you clone a repo and the clone fails, for example if you put an invalid directory in, then you are dumped out of the BGZREPOS screen. This leaves the table open, which causes an ISPF error.

In fixing the above it was also noticed that if you clone a repo, but there is an error, however something is cloned, then the repo was not added to the repo list. Add a check to see if the local repo directory has been added, and if it has, add the repo to the list.

FATAL: Invalid id: fatal: ambiguous argument 'refs/remotes/origin/master^{commit}': unknown revision or path not in the working tree.

Started by user Raymond Lam
Running as SYSTEM
Building remotely on zOS-Agent (tvt7111) in workspace /u/rlam/Jenkins-PC/workspace/pocPipeline
The recommended git tool is: NONE
using credential rlamprivatekey

/var/dbb/bin/git-jenkins.sh rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
/var/dbb/bin/git-jenkins.sh config remote.origin.url [email protected]:raylamgit/poc-workspace.git # timeout=10
Fetching upstream changes from [email protected]:raylamgit/poc-workspace.git
/var/dbb/bin/git-jenkins.sh --version # timeout=10
git --version # 'git version 2.14.4_zos_b09'
using GIT_SSH to set credentials Private Key from USS /u/user/.ssh for Jenkins
/var/dbb/bin/git-jenkins.sh fetch --tags --progress -- [email protected]:raylamgit/poc-workspace.git +refs/heads/:refs/remotes/origin/ # timeout=10
/var/dbb/bin/git-jenkins.sh rev-parse refs/remotes/origin/master^{commit} # timeout=10
FATAL: Invalid id: fatal: ambiguous argument 'refs/remotes/origin/master^{commit}': unknown revision or path not in the working tree.
Use '--' to separate paths from revisions, like this:
'git [...] -- [...]'
refs/remotes/origin/master^{commit}
Also: hudson.remoting.Channel$CallSiteStackTrace: Remote call to zOS-Agent
at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1800)
at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357)
at hudson.remoting.Channel.call(Channel.java:1001)
at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:286)
at com.sun.proxy.$Proxy62.revParse(Unknown Source)
at org.jenkinsci.plugins.gitclient.RemoteGitImpl.revParse(RemoteGitImpl.java:577)
at hudson.plugins.git.util.DefaultBuildChooser.getHeadRevision(DefaultBuildChooser.java:129)
at hudson.plugins.git.util.DefaultBuildChooser.getCandidateRevisions(DefaultBuildChooser.java:111)
at hudson.plugins.git.GitSCM.determineRevisionToBuild(GitSCM.java:1145)
at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1298)
at hudson.scm.SCM.checkout(SCM.java:505)
at hudson.model.AbstractProject.checkout(AbstractProject.java:1206)
at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574)
at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
at hudson.model.Run.execute(Run.java:1894)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:428)
org.eclipse.jgit.errors.InvalidObjectIdException: Invalid id: fatal: ambiguous argument 'refs/remotes/origin/master^{commit}': unknown revision or path not in the working tree.
Use '--' to separate paths from revisions, like this:
'git [...] -- [...]'
refs/remotes/origin/master^{commit}
at org.eclipse.jgit.lib.ObjectId.fromString(ObjectId.java:203)
at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.revParse(CliGitAPIImpl.java:1041)
at hudson.plugins.git.GitAPI.revParse(GitAPI.java:331)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:90)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)
at java.lang.reflect.Method.invoke(Method.java:508)
at hudson.remoting.RemoteInvocationHandler$RPCRequest.perform(RemoteInvocationHandler.java:936)
at hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:909)
at hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:860)
at hudson.remoting.UserRequest.perform(UserRequest.java:211)
at hudson.remoting.UserRequest.perform(UserRequest.java:54)
at hudson.remoting.Request$2.run(Request.java:375)
at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:73)
at java.util.concurrent.FutureTask.run(FutureTask.java:277)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1160)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.lang.Thread.run(Thread.java:820)
Finished: FAILURE

UCD packaging scripts - provide consistent property name for dbb build result url

In the ucd-packaging-scripts a property is added to the UCD component version to link it back to the DBB build result. The property name needs to be the same, and not include any reference to the current build result.

See in:

property(name : buildResult.getGroup() + "-" + buildResult.getLabel(), value : buildResult.getUrl())

And in the Mortgage Sample:

property(name : buildResult.getGroup() + "-" + buildResult.getLabel(), value : buildResult.getUrl())

Please refer to the Pipeline/CreateUCDComponentVersion/dbb-ucd-packaging.groovy.

There are no plans to include new features in the MortgageSample.

Documentation error on Pruner.groovy . Wrong parameter specified.

At the docs (README.md) it says:
...
The supported options are:
...

-g,--group
: will prune all the builds in the specified group

-g is fine but -group is NOT. the correct parameter is: groupName

Also I suggest adding an example how to invoke the utilities since is not so obvious..
One example like below:
/var/dbb/v.1.0.6/bin/groovyz Pruner.groovy --groupName DemoHealthCare-teste

@drbruce-git replied on that at:
https://ibm-systems-z.slack.com/archives/C2TBPA725/p1587132738112500?thread_ts=1586352800.067200&cid=C2TBPA725

bgzclone missing

Tried to use Menu->Clone and bgzclone is missing. Looked in the repo and didn't find it anywhere.

LinkEdit Groovy Script failing

Hi, my LinkEdit Groovy script is failing for below line. I am not sure what is the purpose of the line.Please help:

tools.updateBuildResult(file:"$file", rc:rc, maxRC:0, log:logFile)

i am getting java null exception error for this line.

SCLM to Groovy migration - Incorrect allocations when Pass(true) in effect

Groovy script generation is not generating a PASS(true) and subsequently removing the passthrough DD from the allocation. So I have 2 steps in my BMS. The first uses SYSLIN to a temporary and it gets allocated. Then in the 2nd step, it gets used as input. But as it is allocating the temporary in both steps it gets wiped out.

Support new CopyToPDS build report records in CreateUCDComponentVersion Sample

In DBB v1.0.8 the CopyToPDS build report record was extended so that it can contain the same information as the Execute records which would allow a copied JCL file from Git to a PDS to appear in the build report as a build step with output dataset and deploy type. However it is not a DefaultRecordFactory.TYPE_EXECUTE record, but rather a DefaultRecordFactory.COPY_TO_PDS record. In order for the new record to be collected by the CreateUCDComponentVersion sample a small change needs to be made to dbb-ucd-packaging.groovy. When collecting the execute records, it also needs to collect CopyToPDS records:

// parse build report to find the build outputs to be deployed. 
println("** Find deployable outputs in the build report ")
// the following example finds all the build output with deployType set
def executes= buildReport.getRecords().findAll{
	(it.getType()==DefaultRecordFactory.TYPE_EXECUTE || it.getType()==DefaultRecordFactory.TYPE_COPY_TO_PDS)  && 
	!it.getOutputs().findAll{ o -> 
		o.deployType != null
	}.isEmpty()
} 

DBB Jenkins integration issue

I had set up the jenkins integration on my jenkins 2.60.3 many months back and all of this worked great and i am able to successfully build the mortgage application.

https://www.ibm.com/support/knowledgecenter/SS6T76_1.0.2/jenkinsintegration.html

Old set up
Jenkins : 2.60.3
SSH Slave Plugin : 1.26
Git client plugin : 2.7.3
Git Plugin : 3.9.1
Groovy Plugin: 2.0

We recently updated jenkins to a much higher 2.222.1. This is our new set up. Now the build does not work.I can see it runs the git-jenkins.sh rev-parse . Then i see a FATAL:Invalid id and the git command shows junk chars. Below is the plugin versions on my new server.

Jenkins : 2.222.1
SSH Build Agent Plugin : 1.31.2
Git Plugin : 4.2.2
Git Client Plugin: 3.2.1
Groovy Plugin: 2.2

Have you been able to get this mortgage app build to run on a newer version of jenkins ?

I also see warning in the log when creating the new mainframe agent . I use the ssh build agent plugin to connect on my new jenkins server. I see the agent successfully connected message at the end of log ...but before that there is a warning 'linkage error while performing user request' java.lang.Unsatisfiedlinkerror: Native library (com/sun/jna/z/os-390x/libjnidispatch.so) not found in resource path. There is also an error in the log "ERROR" Failed to monitor free swap space'.

The new jenkins no longer puts a slave.jar but instead a remoting.jar on the agent machine.

Please let me know if there are any changes needed when trying to run with a more latest jenkins version

CORS policy error trying to open build report during userbuild

I run DBB user build and then, click on show build report when it completes , it just opens an empty browser with just a header "build report"

It looks like the report it is trying to open is from a local cached location in the workspace, so its ending up with a CORS policy error trying to open the BuildReport.json in the local folder.

Access to XMLHttpRequest at 'file:///C:/Users/xxxx/IBM/rationalsdp/workspace/RemoteSystemsTempFiles/xxxx/u/xxxx/DBB-Sandbox/work/BuildReport.json' from origin 'null' has been blocked by CORS policy: Cross origin requests are only supported for protocol schemes: http, data, chrome-extension, edge, https, chrome-untrusted.

Is there any way to resolve this ?

convert the json file lookup to a search

DBB output dir is dynamically created with a timestamp making it hard to pass the absolute path of the buildreport.json file in a pipeline script

Maybe we can replace
def jsonOutputFile = new File("${properties.workDir}/BuildReport.json")

with
def File jsonOutputFile = new FileNameFinder().getFileNames(properties.workDir, "**/BuildReport.json")

and change the readme to say the workDir is the path of the dbb workSpace

PublishLoadModules Jar files- groovyx.net.http.RestClient error

Hi ,

I'm trying to publish the artifacts to nexus repository once the build is completed, Document mentioned it requires some jar which are mentioned in ArtifactoryHelpers.groovy. we tried without adding any jar while we build we got error like "unable to resolve groovyx.net.http.RestClient" - Is this issue is related to missing of jar ?

*In case to import the jar files to <dbb_install_dir>/lib , is that requires all jar files which are mentioned in ArtifactoryHelpers.groovy in binary format?

Please help me to resolve this.

zAppBuild does not handle the case doLinkEdit=false correct

There is an issue building the MortgageSample using zAppBuild when using cobol_linkEdit=false
to skip link editing a file, like used in the MortgageApplicationV2 sample:

# Skip creating a load module for these programs as they will be statically linked to other programs
cobol_linkEdit = false :: **/cobol/epsnbrvl.cbl, **/cobol/epsmlist.cbl

Instead of producing a member in the OBJ dataset, it uses a temp allocation for SYSLIN in the createCompileCommand method:

if ( (linkEditStream && doLinkEdit && doLinkEdit.toBoolean()) || (props.debug && linkDebugExit!= null && doLinkEdit)) {
compile.dd(new DDStatement().name("SYSLIN").dsn("${props.cobol_objPDS}($member)").options('shr').output(true))
}
else {
compile.dd(new DDStatement().name("SYSLIN").dsn("&&TEMPOBJ").options(props.cobol_tempOptions).pass(true))
}

Groory script to run the cobol pgm

Hello
First I have complied the program then link edit and finally I am trying to execute it.
i am getting return code : 8413184 while executing the below script.
//Way to run the JCL

  println ("Run the cobol program")
  def run = new MVSExec().pgm("hello").parm("")
  run.dd(new DDStatement().name("TASKLIB").dsn("TT6PBE1.BUILD.LOAD(hello)").options("shr"))
  run.dd(new DDStatement().name("SYSOUT").options("cyl space(5,5) unit(vio) blksize(80) lrecl(80) recfm(f,b) new"))
  run.dd(new DDStatement().name("SYSPRINT").options("cyl space(5,5) unit(vio) blksize(80) lrecl(80) recfm(f,b) new"))
  run.copy(new CopyToHFS().ddName("SYSOUT").file(new File("/u/tt6pbe1/TT6PBE1/test1/cobol/out.log")))
  run = run.execute()
  println("run return code: $run")

Issue / RFE for the DBB migration tool

Info about migrating datasets to git: https://www.ibm.com/support/knowledgecenter/en/SS6T76_1.0.7/migration.html

On that page it says:
'After the local Git repository has been created on zFS, you can start using the the migration tool to copy source files from data set libraries to the local repository.
In addition to copying the source from PDS to zFS, the migration tool also creates and updates the .gitattributes file. The .gitattributes file is required by Rocket's Git client to perform automatic codepage conversion between the Git server and the local Git repository during the migration.'

The migration tool might work ok when you are using the codepage IBM-1047 but when you are using something else it doesn't work so well.

What I would like is that the step where the DBB migration tool creates/updates the .gitattributes file is a choice (new parameter) for the user, If I want it created/updated or not.

What happens today?
We are using codepage IBM-278 (Swedish). My mappings file is correct. My .gitattributes file is correct.
After migration the code is correct, but the .gitattributes file is corrupt. Migration tool has added rows to the .gitattributes file with the standard codepage of IBM-1047.

Filing an issue/RFE on request by D. Behm

Thanks!

/N

SCLM Migration having issues - dbb/Migration/sclm

extrmetadata.sh
Executes EXTARCH
Creates data set 'F03923F.SCLMMIG.REXX'
Copies file /u/f03923f/DBB/Migration/sclm/groovy/../rexx/EXTARCH.rexx to F03923F.SCLMMIG.REXX(EXTARCH)
Copies /u/f03923f/DBB/Migration/sclm/groovy/../rexx/SORTDB.rexx macro to F03923F.SCLMMIG.REXX(SORTDB)
Caught: java.lang.UnsatisfiedLinkError: DBB_JNI (Not found in java.library.path)
Copies /u/f03923f/DBB/Migration/sclm/groovy/../rexx/SORTVER.rexx macro to F03923F.SCLMMIG.REXX(SORTVER)
Copies /u/f03923f/DBB/Migration/sclm/groovy/../rexx/VERRETR.rexx macro to F03923F.SCLMMIG.REXX(VERRETR)
java.lang.UnsatisfiedLinkError: DBB_JNI (Not found in java.library.path)
Copies /u/f03923f/DBB/Migration/sclm/groovy/../conf/sclmmig.config to F03923F.SCLMMIG.REXX(MIGCFG)
at com.ibm.dbb.build.internal.jni.Executable.loadNativeLibrary(Executable.java:37)
at com.ibm.dbb.build.internal.jni.Executable.(Executable.java:29)
Creates data set 'F03923F.SCLMMIG.SKELS'
Copies /u/f03923f/DBB/Migration/sclm/groovy/../rexx/VERJOBC.skel macro to F03923F.SCLMMIG.SKELS(VERJOBC)
at com.ibm.dbb.build.internal.ProductRegistrationChecker.getProductStatus(ProductRegistrationChecker.java:197)
at com.ibm.dbb.build.internal.ProductRegistrationChecker.check(ProductRegistrationChecker.java:103)
Copies /u/f03923f/DBB/Migration/sclm/groovy/../rexx/VERRETR.skel macro to F03923F.SCLMMIG.SKELS(VERRETR)
at com.ibm.dbb.build.internal.ProductRegistrationChecker.check(ProductRegistrationChecker.java:67)
at com.ibm.dbb.build.AbstractExec.execute(AbstractExec.java:63)
Executes F03923F.SCLMMIG.REXX(EXTARCH)
/u/f03923f/DBB/Migration/sclm/bin>
at com.ibm.dbb.build.IExecute$execute.call(Unknown Source)
at SclmExtract.run(SclmExtract.groovy:196)
** Build finished
/u/f03923f/DBB/Migration/sclm/bin>
Executes EXTMEMB
Creates data set 'F03923F.SCLMMIG.REXX'
Caught: java.lang.UnsatisfiedLinkError: DBB_JNI (Not found in java.library.path)
Copies file /u/f03923f/DBB/Migration/sclm/groovy/../rexx/EXTMEMB.rexx to F03923F.SCLMMIG.REXX(EXTMEMB)
Copies /u/f03923f/DBB/Migration/sclm/groovy/../rexx/SORTDB.rexx macro to F03923F.SCLMMIG.REXX(SORTDB)
Copies /u/f03923f/DBB/Migration/sclm/groovy/../rexx/SORTVER.rexx macro to F03923F.SCLMMIG.REXX(SORTVER)
Copies /u/f03923f/DBB/Migration/sclm/groovy/../rexx/VERRETR.rexx macro to F03923F.SCLMMIG.REXX(VERRETR)
java.lang.UnsatisfiedLinkError: DBB_JNI (Not found in java.library.path)
at com.ibm.dbb.build.internal.jni.Executable.loadNativeLibrary(Executable.java:37)
Copies /u/f03923f/DBB/Migration/sclm/groovy/../conf/sclmmig.config to F03923F.SCLMMIG.REXX(MIGCFG)
at com.ibm.dbb.build.internal.jni.Executable.(Executable.java:29)
Creates data set 'F03923F.SCLMMIG.SKELS'
at com.ibm.dbb.build.internal.ProductRegistrationChecker.getProductStatus(ProductRegistrationChecker.java:197)
Copies /u/f03923f/DBB/Migration/sclm/groovy/../rexx/VERJOBC.skel macro to F03923F.SCLMMIG.SKELS(VERJOBC)
at com.ibm.dbb.build.internal.ProductRegistrationChecker.check(ProductRegistrationChecker.java:103)
at com.ibm.dbb.build.internal.ProductRegistrationChecker.check(ProductRegistrationChecker.java:67)
at com.ibm.dbb.build.AbstractExec.execute(AbstractExec.java:63)
at com.ibm.dbb.build.IExecute$execute.call(Unknown Source)
Copies /u/f03923f/DBB/Migration/sclm/groovy/../rexx/VERRETR.skel macro to F03923F.SCLMMIG.SKELS(VERRETR)
at SclmExtract.run(SclmExtract.groovy:196)
Executes F03923F.SCLMMIG.REXX(EXTMEMB)
/u/f03923f/DBB/Migration/sclm/bin>
** Build finished

Pipeline: Adapt the UCD packaging script with UCD 7.1.x

With UCD 7.1.x for deploying binaries to Artifactory the -ar option is replaced with -prop options.

The properties must have this format in the property file:

# Common external repository inputs             
EXTREPO.REPOSITORY_TYPE=Artifactory
EXTREPO.USERNAME=repo_user_id                            
EXTREPO.PASSWORD=repo_password                     
EXTREPO.URL=repo uri
EXTREPO.REPOSITORY=repo_path    
EXTREPO.APIKEY=XXXX

UCD packaging failed after code update

Now we have the COPY in the package with deploy type LOAD.

The code here need to be changed:

replace L78 …

	compile.dd(new DDStatement().name("SYSPUNCH").dsn("${props.bms_cpyPDS}($member)").options('shr').output(true).deployType(deployType))

by

	compile.dd(new DDStatement().name("SYSPUNCH").dsn("${props.bms_cpyPDS}($member)").options('shr').output(true))

@dennis-behm FYI

UCD Package Script Duplicate Input Record

I am facing an issue with a duplicate INPUT record , leading BUZTOOL.sh to fail with:

Shiplist pre-processing error: Input 'App-EPSM/link/epsmlist.lnk' duplicated in container JENKINS.PROG.EPSM.S4.LOAD.

The below referenced section needs to make sure that the Main Input is not referenced a second time:

inputUrl = (buildResultProperties != null && properties.git_treeURL_prefix && githash!="") ? "${properties.git_treeURL_prefix}/${githash}/"+ execute.getFile() : ""
inputs(url : "${inputUrl}"){
input(name : execute.getFile(), compileType : "Main", url : inputUrl)
// dependencies
def dependencySets = buildReport.getRecords().findAll{
it.getType()==DefaultRecordFactory.TYPE_DEPENDENCY_SET && it.getFile()==execute.getFile()
};
Set<String> dependencyCache = new HashSet<String>()
dependencySets.unique().each{
it.getAllDependencies().each{
if (it.isResolved() && !dependencyCache.contains(it.getLname())){
def displayName = it.getFile() ? it.getFile() : it.getLname()
def dependencyUrl =""
if (it.getFile() && (it.getCategory()=="COPY"||it.getCategory()=="SQL INCLUDE")) dependencyUrl = (buildResultProperties != null && properties.git_treeURL_prefix && githash!="") ? "${properties.git_treeURL_prefix}/${githash}/"+ it.getFile() : ""
input(name : displayName , compileType : it.getCategory(), url : dependencyUrl)
dependencyCache.add(it.getLname())
}
}
}
}
}
}
}
}
}

<Q>CA certificate during Setting up the CI Pipeline Agent on IBM Z as a Started Task

We were following the documentation for setup a started task
"Setting up the CI Pipeline Agent on IBM Z as a Started Task: " https://www.ibm.com/support/pages/node/6355797

On execution of the CURL command for downloading the latest version of the Jenkins agent from the Jenkins server we receive following error

curl: (60) SSL certificate problem: self signed certificate in certificate chain
More details here: https://curl.haxx.se/docs/sslcerts.html
curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.

As a workaround we have added --insecure parameter for now
Also for cloning a GIT repo we have similar issues. For git clone to HTTPS server we added git parameter http.sslverify=false

As we understand we can setup CA certificate store on USS file system.
Normally we use RACF keyrings

We understand that linking to RACF is not possible at this moment

Are there plans to support linking to RACF keyrings?
If this is not the correct place to ask, who should I contact for this questions

While Binding DBB program using the BIND jcl result in SQLCODE:000000080N

Due to some reason , we have provided bind property as false in Bind properties and tried to bind the DBB compiled program using BIND jcl which uses the PLAN. Here even though the program changes are not made for Db2 it throwing SQL error if it's not bind and after binding also it not making any changes to Db2 but program is running, is there anything specific we need to provide here for binding DBB programs ?
JCL we used for compilation:

//SYSTSIN DD *
DSN SYSTEM(DBXX)
BIND PLAN(CBXX) MEMBER(CBDBBB) -
LIBRARY ('DBB.OLD.DBRM') -
ACTION (REPLACE), VALIDATE(RUN), ISOLATION (CS) , FLAG (I), -
ACQUIRE (USE) , RELEASE (COMMIT) , EXPLAIN (NO), RETAIN, -
OWNER(COREBK) , QUALIFIER(COREBK), ENCODING(EBCDIC)
END

Please help us to bind the program which will reflect the Db2 changes

Input fields for Repository Address and Working Directory are too small

The input fields for the Repository Address and Working Directory are too small.
Mostly because we use on-premise git repo's

Repository Address is limited to only 44 characters
Working Directory is limited to only 19 characters

Due to the fact that for unknown reasons the panels are compiled (why would you do that)
I cant adjust the panels myself, which severely limits the use of the ISPF client.

Please provide alot more characters for Repository Address and Working Directory.

ps.
Request: provide panels just in plane text pls.

Error thrown during User Build thru the GIT ISPF client

Hi,

While performing a DBB user build thru the GIT ISPF client, we are getting the error below. Can you please to point what we are missing on the setup? Are there any customization need apart from updating the configs on BGZCONF file?

JVMJ9VM015W Initialization error for library j9gc29(2): Failed to instantiate compressed references metadata. 200M requested
Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.
** Build finished

Warning instead of Assertion-Exception

When leveraging the PackageBuildOutputs for a specific set of deployTypes through --deployTypes, the packaging process terminates the pipeline, if no matches are found.

assert executes.size() > 0, "There are no outputs found in the build report"

However, it is a valid scenario, when the script is used to package a dedicated deploy type, which is not part of the BuildReport.json

Rather provide a warning instead of throwing an Assertion-Exception

DBB Integrating with Jenkins Not Working - ERROR: Error cloning remote repo 'origin'

Hi All,
For the first time i am integrating DBB with Jenkins and i am getting 'ERROR: Error cloning remote repo 'origin'' error. Not sure where exactly the issue resides, i have followed IBM DBB jenkins configuration mentioned in IBM portal but it is not working for me. Can any one help me to find out the issue?

Cloning the remote Git repository
Cloning repository D:\Udhay\MortgageApplication

/Z23D/usr/lpp/IBM/dbb/bin/git-jenkins.sh init /DevOps/jenkins/workspace/Mortgage Application # timeout=10
ERROR: Error cloning remote repo 'origin'
hudson.plugins.git.GitException: Could not init /DevOps/jenkins/workspace/Mortgage Application
at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$5.execute(CliGitAPIImpl.java:989)
at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$2.execute(CliGitAPIImpl.java:747)
at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$GitCommandMasterToSlaveCallable.call(RemoteGitImpl.java:161)
at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$GitCommandMasterToSlaveCallable.call(RemoteGitImpl.java:154)
at hudson.remoting.UserRequest.perform(UserRequest.java:211)
at hudson.remoting.UserRequest.perform(UserRequest.java:54)
at hudson.remoting.Request$2.run(Request.java:369)
at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
at java.util.concurrent.FutureTask.run(FutureTask.java:277)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1160)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.lang.Thread.run(Thread.java:812)
Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote call to IBM DBB Node
at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1741)
at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:356)
at hudson.remoting.Channel.call(Channel.java:955)
at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.execute(RemoteGitImpl.java:146)
at sun.reflect.GeneratedMethodAccessor1455.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.invoke(RemoteGitImpl.java:132)
at com.sun.proxy.$Proxy88.execute(Unknown Source)
at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1114)
at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1159)
at hudson.scm.SCM.checkout(SCM.java:505)
at hudson.model.AbstractProject.checkout(AbstractProject.java:1205)
at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574)
at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
at hudson.model.Run.execute(Run.java:1853)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:427)
Caused by: hudson.plugins.git.GitException: Command "/Z23D/usr/lpp/IBM/dbb/bin/git-jenkins.sh init /DevOps/jenkins/workspace/Mortgage Application" returned status code 127:
stdout: test: /Z23D/usr/lpp/IBM/dbb/bin/git-jenkins.sh 10: not found
git: /Z23D/usr/lpp/IBM/dbb/bin/git-jenkins.sh 29: not found
iconv: /Z23D/usr/lpp/IBM/dbb/bin/git-jenkins.sh 29: not found

stderr:
at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2429)
at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2359)
at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2355)
at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommand(CliGitAPIImpl.java:1915)
at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$5.execute(CliGitAPIImpl.java:987)
... 11 more
ERROR: Error cloning remote repo 'origin'
Finished: FAILURE

CobolCompile.groovy failing

Starting off with Groovy and DBB.
Using the samples provided, DBB and z Open Development...

Running DBB Build on the PrintApp.cbl file, we get messages saying the DBB User Build is complete for PRINTAPP.CBL. Looking at more details in the log we see...

RSEG1106: File .../work/PRINTAPP.log not found on host....

Together with the resultant java exception

** Build start at 20190912.012102.021
/u/home/.../DBB_Sandbox>
Caught: com.ibm.jzos.ErrnoException: EDC5129I No such file or directory. ERRNO=129 ERRNO2=0x594003d
com.ibm.jzos.ErrnoException: EDC5129I No such file or directory. ERRNO=129 ERRNO2=0x594003d
at com.ibm.jzos.FileAttribute.stat(Native Method)
at com.ibm.jzos.FileAttribute.getStat(FileAttribute.java:595)
at com.ibm.dbb.build.internal.Utils.retrieveHFSEncoding(Utils.java:119)
at com.ibm.dbb.build.BuildProperties.load(BuildProperties.java:120)
at com.ibm.dbb.build.BuildProperties$load$0.call(Unknown Source)
at Tools.loadProperties(Tools.groovy:107)
at Tools$loadProperties.call(Unknown Source)
at build.run(build.groovy:63)

Looking at the z/OS file system, all assets (.cbl, properties files and groovy scripts) are there and look fine. It is true that the .log file hasn't been created yet and thus the exception.

Online documentation states that one should "add the project folder to the file.properties file" (https://www.ibm.com/support/knowledgecenter/en/SSUFAU_1.0.0/com.ibm.idz.dbbz.doc/topics/DBBz_troubleshooting.html).

Is the sample missing this information? I couldn't fine any direction as to what to put into the file.properties file.

DBB Bind Failed

Hi All,

Am trying to Bind the Cobol-CICS-DB2 program, while compiling am getting IKJ56479I , am trying to do user build. Not sure where the problem resides and i believe it may due to ISPF gateway configuration, I have attached the log, please help me to resolve this issue

Dbb bind log.txt

Please add a simple groovy that shows how to delete a DBB collection

It may sound easy for experts, but customers might need to delete "old left" collections at DBB repo.
@drbruce-github showed a nice simple example at
https://ibm-systems-z.slack.com/archives/C2TBPA725/p1587140011114100?thread_ts=1586352800.067200&cid=C2TBPA725
Here for people that do not have slack:
``//Delete a collection XXX
//repositoryClient.deleteCollection("XXX")
@groovy.transform.BaseScript com.ibm.dbb.groovy.ScriptLoader baseScript
import com.ibm.dbb.build.*
import com.ibm.dbb.repository.*
//Set up repository client
BuildProperties properties = BuildProperties.getInstance()
properties.load(new File("${getScriptDir()}/user.properties"))
if(!properties.url||!properties.username){
println("You must update the user.properties file first")
System.exit(0)
}
def client = new RepositoryClient()
client.setUrl(properties.url)
client.setUserId(properties.username)
if(properties.password!=null){
client.setPassword(properties.password)
}else if(properties.passwordFile!=null){
client.setPasswordFile(new File(properties.passwordFile))
}else{
println("Please enter either a password or the path for a password file in user.properties")
System.exit(0)
}
client.setForceSSLTrusted(true)
//Delete a collection XXX
client.deleteCollection("XXX")
println("XXX is deleted")

Error when initializing a repo

I'm getting this error when initializing a new repo.

record not written, error codes 71 571011C
record not written, error codes 71 571011C
record not written, error codes 71 571011C
record not written, error codes 71 571011C
record not written, error codes 71 571011C


TSO BPXMTEXT shows this for the 571011C error

571011C
BPXFSRDW 08/23/18
JRFileNotOpen: The file is not opened

Action: Reissue the request specifying an open file descriptor.


The repo is init'd, the .git dir has files and directories and the .gitattributes is empty.

I get this on both LPARs I have DBB 1.0.7 installed.

Am I doing something wrong or is there a bug?

SCLM to Groovy Migration - zImport does not support same language being used for different types

I have 2 different types, HPP and JCL that both use a language of TEXT. If I use an import type of LANG the wrong language is appended to these files with all the files getting .jcl.

I think the problem exists in both GenerateBuildProperties.groovy and GenerateZImport.groovy

The scriptMappings look OK however:

scriptMapping = TEXT :: hpp/text/.h
scriptMapping = CPP :: cpp/cpp/.c
scriptMapping = COPYBOOK :: dclgenc/copybook/.cpy
scriptMapping = TEXT :: jcl/text/.jcl

Causes build to fail with:
Caught: com.ibm.dbb.build.BuildException: java.io.FileNotFoundException: /u/dohertl/sclmMigration/jazz00/repo/jazz00/hpp/text/incl1.txt
com.ibm.dbb.build.BuildException: java.io.FileNotFoundException: /u/dohertl/sclmMigration/jazz00/repo/jazz00/hpp/text/incl1.txt

There seems to be a lot of mismatch here. In langexts.txt I told it to use .h as the extension, however it imports the file in as INCL1.jcl and then tries to reference it with incl1.txt?

Workaround is to leave the HPP files as .txt or create separate language definitions.

Enhance PackageBuildOutputs Script

I am proposing an update to the PackageBuildOutputs script, which

  • externalises the copyModeMap to a properties file.
  • optionally adds build logs files to the tar file.

// Map of last level dataset qualifier to DBB CopyToFS CopyMode.
// TODO: Customize to your needs.
def copyModeMap = ["COPYBOOK": CopyMode.TEXT, "COPY": CopyMode.TEXT, "DBRM": CopyMode.BINARY, "LOAD": CopyMode.LOAD]

ISPF Client doesn't set HOME

I ran into the issue that due to the use of bpwxunix, the environment variables from etc/profile are not set. This gives issues when trying to use git commit.
The user.name and user.email are not found.

I solved this in BGZSTART with the following command:
/get home directory/
sh_rc = bpxwunix('pwd',,stdout.,stderr.)
home_dir=stdout.1

and added it as an ENVIR
gitenv.14 = 'export HOME='home_dir

Is this a known issue or just a local disturbance?

build.groovy scripts not running in order

Trying to run the build script with the mortgage application, but we get a link-edit job fail because the compile and cobol compiles have not run. These are marked in the buildOrder in build.groovy as ...

// build programs by invoking the appropriate build script
def buildOrder = ["BMSProcessing", "Compile", "CobolCompile", "LinkEdit"]
// optionally execute IMS MFS builds
if (properties.BUILD_MFS.toBoolean())
buildOrder << "MFSGENUtility"

println("** Invoking build scripts according to build order: ${buildOrder[1..-1].join(', ')}")
buildOrder.each { script ->
	// Use the ScriptMappings class to get the files mapped to the build script
	def buildFiles = ScriptMappings.getMappedList(script, buildList)
	buildFiles.each { file ->
		runScript(new File("${script}.groovy"), ["file":file])
		processCounter++

...but as you can see from the output they are not running and it's doing the BMSProcessing and then the LinkEdit. The first line in the Compile.groovy is a println but this doesn't come out, so what could be stopping them?

Output from the script is below...

_`** Searching for last successful build commit hash for build group MortgageApplication
Could not locate last successful build commit hash for build group MortgageApplication. Building all files.
** Scan the build list to collect dependency data
Scanning bms/epsmlis.bms
Scanning bms/epsmort.bms
Scanning cobol/epsmpmt.cbl
Scanning cobol/epsmlist.cbl
Scanning cobol/epsnbrvl.cbl
Scanning cobol_cics/epscsmrd.cbl
Scanning cobol_cics/epscsmrt.cbl
Scanning cobol_cics_db2/epscmort.cbl
Scanning link/epsmlist.lnk
Scanning mfs/dfsiv1.mfs
Scanning copybook/epsmortf.cpy
Scanning copybook/epsmtcom.cpy
Scanning copybook/epsmtinp.cpy
Scanning copybook/epsmtout.cpy
Scanning copybook/epsnbrpm.cpy
Scanning copybook/epspdata.cpy
** Storing remaining 16 logical files in repository collection 'MortgageApplication'
HTTP/1.1 200 OK
** Invoking build scripts according to build order: Compile, CobolCompile, LinkEdit, MFSGENUtility

  • Building bms/epsmlis.bms using BMSProcessing.groovy script
    Copying /u/home/DEVOPS/build/mainframe-code/MortgageApplication//bms/epsmlis.bms to DEVOPS.BMS(EPSMLIS)
    Processing BMS Map bms/epsmlis.bms
  • Building bms/epsmort.bms using BMSProcessing.groovy script
    Copying /u/home/DEVOPS/build/mainframe-code/MortgageApplication//bms/epsmort.bms to DEVOPS.BMS(EPSMORT)
    Processing BMS Map bms/epsmort.bms
  • Building link/epsmlist.lnk using LinkEdit.groovy script
    Copying /u/home/DEVOPS/build/mainframe-code/MortgageApplication//link/epsmlist.lnk to DEVOPS.LINK(EPSMLIST)
    Link editing link file link/epsmlist.lnk
    *! The return code (12) for link/epsmlist.lnk exceeded the maximum return code allowed (0)`_

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.