jenkinsci / templating-engine-plugin

create tool-agnostic, templated pipelines to be shared by multiple teams

Jenkins Templating Engine

Table of Contents: Jenkins Templating Engine Overview Learn More Participate Adopters Join the Conversation Report a Bug or Request a Feature Contributions are Welcome! Documentation Unit Tests Squash Some Bugs Feature Development

Overview

The Jenkins Templating Engine (JTE) is a plugin originally created by Booz Allen Hamilton enabling pipeline templating and governance.

JTE brings the Template Method Design Pattern to Jenkins pipelines. Users can remove the Jenkinsfile from individual source code repositories in favor of a centralized, tool-agnostic Pipeline Template. This template provides the structure of the pipeline.

Pipeline functionality is provided by Library Steps. For example, if the Pipeline Template references a build() step then the implementation of build() can be deferred to a user-developed library providing that step such as a Maven or Gradle library.

A hierarchical Pipeline Configuration is used for each individual pipeline to determine which libraries to load (among other things).

Learn More

There are many resources available to help you get started:

• Documentation
• Presentations
  • CDF Online Meetup
  • Jenkins Online Meetup
  • JTE at KubeCon 2019
  • Trusted Software Supply Chains with JTE
  • Jenkins World 2018
• Blog Posts
  • Introducing the Jenkins Templating Engine

Participate

There are many ways to get involved. We look forward to hearing from you.

Join the Conversation

JTE has a channel in the Jenkins community's gitter space. It's a great place to ask questions or propose ideas for the future of JTE.

Report a Bug or Request a Feature

Something not quite working right? Have a cool idea for how to make JTE better? Open an Issue on the JTE repository and let's get the conversation started.

Contributions Welcome

No contribution is too small - all are appreciated!

Check out the Contributing Section of the documentation to understand how to get started.

templating-engine-plugin's Issues

Fallback to a heavyweight SCM checkout if SCM plugin does not support SCMFileSystem

Determine if it's possible to take Primitive classes out of the resources directory and recompile classes using CPS transformer so they behave appropriately during pipeline execution

Library Validation: support Arrays

Currently unable to validate a library configuration is of the type ArrayList because it intersects with how enumerations are validated.

The following test case resulted in a warning:

[JTE] Library Validator: Not sure how to handle value class java.util.ArrayList with class class java.lang.Class

library_config.groovy:

fields{
  required{
    someField = ArrayList
  }
}

pipeline_config.groovy:

libraries{
  someLibrary{
    someField = [ "a", "b", "c" ] 
  }
}

Where do I define "custom variables"?

I'm trying to implement a proof-of-concept for my project using the Jenkins Template Engine. However it seems as if I'm still missing something in the concept. Please find the following files as an example:

General pipeline_config.groovy

libraries {
    merge = true // allow individual apps to contribute additional libraries
    docker
}

Docker library step: build_docker.groovy

void call() {
    println "Building and publishing Docker image ${DOCKER_REGISTRY}/${APP_NAME}"
}

Application-specific pipeline_config.groovy

libraries {
    gradle
}

As you might see I'd like to use the variables $DOCKER_REGISTRY and $APP_NAME:

1. $DOCKER_REGISTRY shall be a "global" variable which is valid for all pipeline library steps.
2. $APP_NAME shall be a variable specific for the application.

My problem is: Where do I define these variables so that they are usable in the required scope?

I already tried the following but this doesn't work:

libraries {
    gradle
    docker {
      APP_NAME = "my-app"
    }
}

Guidance on Multibranch Pipelines

With a multibranch or Github/Bitbucket project type, a single Jenkinsfile is usually the entry point for all Jenkins execution. Combined with templates, the benefit is a single scheme can be applied to many projects. The challenge is each branch, repo, or event may be expected to follow a different pipeline. Generally, the approach is to have filtering/routing logic in the initial Jenkinsfile and then load the applicable pipelines/methods. There are some opinionated approaches to accomplish this:

• Jenkins Pipeline - Load: https://www.jenkins.io/doc/pipeline/steps/workflow-cps/#load-evaluate-a-groovy-source-file-into-the-pipeline-script
• Templates via Shared Library: https://www.jenkins.io/blog/2017/10/02/pipeline-templates-with-shared-libraries/

---

What is JTE's opinion on per-branch, per-repo, or per-event pipelines? Is there a JTE recommended way?

Enable build parameters parsing in the config.groovy

As of today while executing a job, we can't really load Jenkins build parameters into the application repository's pipeline_config.groovy. Right now the workaround is to access the parameters as part of your library step by using something like,
def releaseName = env.RELEASE_NAME ?: config.release_name ?: { error “release name not defined” }()

Remove LifecycleHook "context" parameter in favor of an autowired "context" variable

Be able to exclude library steps from being loaded via the pipeline configuration file

GitHub Organization Repository Selection

I am applying the JTE template to my GitHub Organization. In the Projects section I have set the Project Recognizer as Jenkins Templating Engine. However, this adds all the repositories in the Organization. Is it possible to only include repositories that contain the pipeline_config.groovy file in the root directory (Similar to selecting only repositories that contain JenkinsFile)?

Unable to build the plugin

Hello!

I am trying to build the plugin locally but I am getting the following error when executing gradle

> No value has been specified for property 'mainClassName'.

Any help? Thanks!

Any alternative to stash/unstash?

Heys guys, I'm currently doing a few proof of concepts to see if we could use the JTE in our own CI.

Most of our projects are Maven-based and I was wondering if there is a way to avoid the stash/unstash mechanic and just run the whole pipeline on the same node+workspace from start? This would allow to run a stage build where components are compiled and in a different test stage invoke the test phase without having to rebuild everything and then run the static analysis.

We have quite large multi-module projects and I fear stashing several hundred megabytes of data might impact performance...

NotSerializableException: java.lang.reflect.Method

I have been getting intermittent (but frequent) build failures since upgrading to JTE version 1.3 from 1.1.1.

Please find the stack trace below.

So far this appears to happen only inside a parallel() block of my scripted pipeline.

Attaching a debugger, com.cloudbees.groovy.cps.impl.FunctionCallEnv.locals refers to org.boozallen.plugins.jte.hooks.Hooks, specifically the section that iterates through methods:

stepWrappers.each{ step ->
   step.impl.class.methods.each{ method ->
      if (method.getAnnotation(a)){
         AnnotatedMethod am = new AnnotatedMethod(a.getSimpleName(), method.name, step)
         discovered.push(am) 
      }            
   }
}

Adding @NonCPS to discover(Class<? extends Annotation> a, TemplateBinding b) appears to fix the issue, but I don't know if that would be a correct solution.

The stack trace is:

        at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:926)
        at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:844)
(parts skipped here)
at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:1040)
        at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:1019)
        at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:920)
(parts skipped here)
        at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:920)
        at org.jboss.marshalling.AbstractObjectOutput.writeObject(AbstractObjectOutput.java:58)
        at org.jboss.marshalling.AbstractMarshaller.writeObject(AbstractMarshaller.java:111)
        at org.jenkinsci.plugins.workflow.support.pickles.serialization.RiverWriter.lambda$writeObject$0(RiverWriter.java:144)
        at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.GroovySandbox.runInSandbox(GroovySandbox.java:241)
        at org.jenkinsci.plugins.workflow.support.pickles.serialization.RiverWriter.writeObject(RiverWriter.java:143)
        at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.saveProgram(CpsThreadGroup.java:523)
        at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.saveProgram(CpsThreadGroup.java:499)
        at org.jenkinsci.plugins.workflow.cps.CpsStepContext$3.onSuccess(CpsStepContext.java:531)
(parts skipped here)

        in field com.cloudbees.groovy.cps.impl.FunctionCallEnv.locals
        in object com.cloudbees.groovy.cps.impl.FunctionCallEnv@1f3c4ffd
        in field com.cloudbees.groovy.cps.impl.ProxyEnv.parent
        in object com.cloudbees.groovy.cps.impl.BlockScopeEnv@3af6683
        in field com.cloudbees.groovy.cps.impl.CallEnv.caller
        in object com.cloudbees.groovy.cps.impl.FunctionCallEnv@2c34353f
        in field com.cloudbees.groovy.cps.impl.ProxyEnv.parent
        in object com.cloudbees.groovy.cps.impl.BlockScopeEnv@5c4e9fdf
        in field com.cloudbees.groovy.cps.impl.ProxyEnv.parent
        in object com.cloudbees.groovy.cps.impl.LoopBlockScopeEnv@6f00f6be
        in field com.cloudbees.groovy.cps.impl.ProxyEnv.parent
        in object com.cloudbees.groovy.cps.impl.BlockScopeEnv@353af2c6
        in field com.cloudbees.groovy.cps.impl.CallEnv.caller
        in object com.cloudbees.groovy.cps.impl.ClosureCallEnv@638465d3
        in field com.cloudbees.groovy.cps.impl.ProxyEnv.parent
        in object com.cloudbees.groovy.cps.impl.BlockScopeEnv@2249d814
        in field com.cloudbees.groovy.cps.impl.CallEnv.caller
        in object com.cloudbees.groovy.cps.impl.FunctionCallEnv@4ee1edbb
        in field com.cloudbees.groovy.cps.Continuable.e
        in object org.jenkinsci.plugins.workflow.cps.SandboxContinuable@4a1d37ad
        in field org.jenkinsci.plugins.workflow.cps.CpsThread.program
        in object org.jenkinsci.plugins.workflow.cps.CpsThread@546b40f7
        in field org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.threads
        in object org.jenkinsci.plugins.workflow.cps.CpsThreadGroup@746deff9
        in object org.jenkinsci.plugins.workflow.cps.CpsThreadGroup@746deff9

Take static constructors out of the StepWrapper and create a StepWrapperFactory

Add the ability to mark a library deprecated in the library config

Feature Request

In the library_config.groovy file - be able to set deprecated = true and log this to the build log when the library is loaded

additionally - be able to provide a custom message

deprecated{
  message = "use this other library" 
}

New 'Exclude branches without a pipeline configuration file' doesnt allow setting 'Configuration Base Directory'

Jenkins: 2.204
JTE Plugin: 1.7.1

#52 Introduced a feature to skip branches that do not include a pipeline configuration file.

Currently it is only supported to be in root, even though the baseDir setting is available under 'Configuration Base Directory'

We currently use 'Configuration Base Directory' to store multiple templates in a single repository, and cannot enable this yet as it will skip branches that DO include a config file (under baseDir)

Allow to specify the Pipeline Configuration directly in the Jenkins UI in the Folder Configuration

I my organization the jobs of each team are organized around Folders and each team use a different Pipeline Configuration, we have the same configuration between all jobs of team but different configuration between teams. Currently we have a different pipeline configuration repository for each team and we pass these repositories in the Pipeline Configuration of each Folder.

The request is allow to pass a closure in the Pipeline Configuration of the Folder in order to reduce complexity of managing the configuration, we think that having the configuration in a SCM add a extra layer of complexity without to much benefit in our case.

Removing the default checkout ability now clashes with gitlab-branch-source-plugin

Hello,

I'm using both templating-engine-plugin and gitlab-branch-source-plugin.
In gitlab-branch-source-plugin, there's a post-hook on the checkout to send a notification to GitLab (https://github.com/jenkinsci/gitlab-branch-source-plugin/blob/a894e782d62cea3ce53a3374d23b38d2783a1ca3/src/main/java/io/jenkins/plugins/gitlabbranchsource/helpers/GitLabPipelineStatusNotifier.java#L373) which allows to consider the build as RUNNING is GitLab-CI if currentBuild.result is null in Jenkins (https://github.com/jenkinsci/gitlab-branch-source-plugin/blob/a894e782d62cea3ce53a3374d23b38d2783a1ca3/src/main/java/io/jenkins/plugins/gitlabbranchsource/helpers/GitLabPipelineStatusNotifier.java#L209)

Since the version 1.7 of templating-engine-plugin, the ability to have JTE performing the checkout have been removed and I do have to do it manually.
This is fine, but, just below where the checkout happened before, there's a manual assignation

templating-engine-plugin/src/main/resources/org/boozallen/plugins/jte/TemplateEntryPoint.groovy

Line 36 in 5959425

currentBuild.result = "SUCCESS"

Because of that, when the post-hook on the SCM checkout is performed by gitlab-branch-source-plugin, the result is no longer null but SUCCESS resulting in the GitLab-CI pipeline displaying as success while the unit testing did not even started yet.

Maybe I'm missing a use case here but is there a real need to copy the currentBuild.result into context.status while we can access it with currentBuild.result in our pipelines when needed ?
Would it be possible to set it to null at start ?

Context class hudson.FilePath is missing on @CleanUp

As suggested in this message on Gitter I wrapped my template steps in a node statement to make sure that all steps run on the same node.

Jenkinsfile

node {
    build()
    publish_image()
    cleanup()
}

The cleanup() step is basically just this:

cleanup.groovy

@CleanUp
void call(context) {
    cleanWs()
}

When running this pipeline I get the following error:

[Pipeline] // node
[JTE] [@CleanUp - common/cleanup.call]
[Pipeline] cleanWs
Required context class hudson.FilePath is missing
Perhaps you forgot to surround the code with a step that provides this, such as: node
Perhaps you forgot to surround the code with a step that provides this, such as: node
[Pipeline] End of Pipeline
hudson.remoting.ProxyException: org.jenkinsci.plugins.workflow.steps.MissingContextVariableException: Required context class hudson.FilePath is missing
Perhaps you forgot to surround the code with a step that provides this, such as: node

When wrapping the cleanWs() in a node then the pipeline finishes successfully but then it might happen that cleanWs() is running on a different node.

Is it possible to use JTE for declarative pipelines?

Integrate CodeNarc to standardize coding style across the code base

Would like to know if it is support the Jenkins Kubernetes plugin

We have deployed the Jenkins in Kubernetes and use the kubernetes to lunch the agent pod dynamically. Here is our steps:

/libraries/maven/build.groovy

void call(){
def label = "agent-${UUID.randomUUID().toString()}"
podTemplate(
cloud: 'pas-development',
serviceAccount: 'jenkins',
namespace: 'devops-jenkins',
containers: [
containerTemplate(name: 'jnlp', image: "${env.TOOL_AGENT}", args: '${computer.jnlpmac} ${computer.name}')
]){
node(label) {
stage("Maven: Build"){
println "build from the maven library"
}
}
}
}

The pipeline is hanging and the agent pod never lunched.

I would like to know if it support the kubernetes plugin. If so, how we can do it. Thanks

Louie Liu

Plugin does not respect Git Refspec

When changing the Refspec in the advanced features of the Repositories section for Git, the refspec always reverts to +refs/heads/*:refs/remotes/origin/*, making it impossible to checkout tags.

Plugin Parameters:

Actual:

Expected:
The expected git command git fetch --tags --progress --prune -- origin +refs/tags/v0.0.1:refs/remotes/origin/tags/v0.0.1 returns successfully from local

Combination of merge and override behavior for configs

Feature Request

We wanted to be able to provide default library configs in the governance tier, but allow the application layer to override them.

for example, with this setup:

// governance tier
someblock {
  config1 = "g1"
  config2 = "g2"
}

// application tier
someblock {
  config2 = "a2"
  config3 = "a3"
}

we would like to see the following config being applied:

someblock {
  config1 = "g1"
  config2 = "a2"
  config3 = "a3"
}

However this is not currently possible. With merge = true, config2 doesn't get overriden, and with override = true, config1 got lost.

Execute the pipeline in a Jenkins Slave (AWS ECS EC2 based docker)

I tried the JTE and it's working fine in my Master.

How can I specify to run the entire pipeline in a Jeknins slave ?

So that, the code checkout, build , test and deploy etc should be executed in Jenkins Slave (Docker)
I am using AWS ECS -EC2 based containers as my Jenkins slave. So when each job invoked, it will spin-up a docker container, execute every stages and once completed, the slave will be terminated.

In my declarative pipeline I used "agent {label 'ec2-jenkins-slave'}" for that. So the code checkout, build test etc are happened in a Jenkins Slave Docker container.

But in JTE, I couldn't implement it successfully.

I tried the following in my each library

node("my-docker-agent-name"){
--- my commannds---
}

but it ended-up with creating new docker slaved on each steps. Also I coudn't see the source code in the new containers.

So I changed back the libraries in the following format

node{
--- my commannds---
}

and tried, the following in the Jenkins file

node("ec2-jenkins-slave")
{
build()
test()
deploy()
}

Which ended up in creating a new docker container and add the stages executed in my Jenkins MASTER

Can you tell me how can I implement JTE to execute all steps in a Jenkins slave So that, when I trigger the job, the code pull, build, test, deployment trigger etc should be executed in a single container

---

My Files ( used for Testing)

pipeline-configuration/Jenkinsfile

node("ec2-jenkins-slave") { build() s3push() }

Pipeline Config file
pipeline-configuration/pipeline_config.groovy

libraries{ docker s3 }

Test library for docker
docker/build.groovy

`def call(){
stage "Building Docker Image", {
println "Building Docker Image"
}
node{
sh "docker build -t test:test ."
login_to_registry()
sh """
echo 'hostname' ;
hostname ;
docker tag 'test:test' '2xxxxxx8.dkr.ecr.us-east-2.amazonaws.com/myexp:${BUILD_NUMBER}' ;
docker push '2xxxxxxxx8.dkr.ecr.us-east-2.amazonaws.com/myexp:${BUILD_NUMBER}'
"""

}
}`

Test library for S3
S3/s3push.groovy

`def call(){
stage "Pushing Artifacts to S3", {
println "Push Artifacts to S3"
}
node{
sh """
aws s3 cp test.zip s3://myexps3/

   """

}
}`

How to handle conflicting libraries

Would you have an example or guidance on a case where the repository has a stage with parallel steps with the same library functions?

Example
A repo contains both C++ and Java source code. For the build stage, I want to call both a gradle build and make build in parallel.

Questions

1. How do would do you run steps in parallel under JTE?
2. How would JTE resolve the possible library conflict with ../gradle/build.groovy and ../make/build.groovy?

Unable to call a step - that takes an application_environment object as a param - from stage block

In my pipeline_config:

application_environments{
    dev{
        long_name = "Development"
    }
    prod{
        long_name = "Production"
    }
}

I've got a step called project_specific_step.groovy:

void call(environment) {
   echo "environment.short_name: ${environment.short_name}"
}

Calling it from Jenkinsfile:

project_specific_step dev

OUTPUT:

environment.short_name: dev

As expected

Calling it from a stage block defined in pipeline_config.groovy:

stages{
  stage_do_things{
    project_specific_step dev
    common_step_a
    common_step_b
  }
}

OUTPUT:

hudson.remoting.ProxyException: groovy.lang.MissingMethodException: No signature of method: org.boozallen.plugins.jte.binding.injectors.ApplicationEnvironment.call() is applicable for argument types: () values: []
Possible solutions: wait(), any(), wait(long), any(groovy.lang.Closure), each(groovy.lang.Closure), wait(long, int)

Error.

Is passing an application_environment object to a step supported from a stage block?

Use case: Only one of my project need to call project_specific_step step. The others don't. I want to reuse the same Jenkinsfile though. I don't want to have to create another Jenkinsfile just for that one project that is different. So overriding a stage block primitive in pipeline_config of this specific project is ideal.
However my step needs an input, as it is being called a few times, in different stages. Behaviour will change depending on the environment we pass

Suggestions please?

Gitlab-branch-source Merge Request build failed

Issue description

Let say I have a Merge Request (MR) from branch A to branch B. Both branches have pipeline_config.groovy in root directory. But when the MR triggers Jenkins build, templating engine doesn't obtain pipeline_config.groovy of any branch, it only obtains pipeline_config.groovy in Governance repo

If the build is triggered by branch commit, then templating engine works fine (obtain pipeline_config.groovy in both source repo and Governance repo)

Steps to reproduce the issue

1. Governance's pipeline_config.groovy

// restrict individual repository Jenkinsfiles
allow_scm_jenkinsfile = false
// skip the default JTE checkout and do it explicitly
skip_default_checkout = true

libraries{
  merge = true
  gitlab
  sonarqube
}

2. Source repo's pipeline_config.groovy

libraries{
  kubernetes {
    podTemplate = "nodeTemplate"
  }
  npm
}

3. Pipeline template (Jenkinsfile in Governance repo)

startPod {
  clone_source()
  build()
  static_code_analysis()
}

4. JCasc job config

jobs:
  - script: >
      organizationFolder('Gitlab Organization Folder') {
        description("Gitlab folder configured with JCasC")
        displayName('Projects')
        // "Projects"
        organizations {
          gitLabSCMNavigator {
            projectOwner("owner_id")
            credentialsId("ssh_key")
            serverName("gitlab-server")
            // "Traits" ("Behaviours" in the GUI) that are "declarative-compatible"
            traits {
              subGroupProjectDiscoveryTrait() // discover projects inside subgroups
              gitLabBranchDiscovery {
                strategyId(3) // discover all branches
              }
              originMergeRequestDiscoveryTrait {
                strategyId(1) // discover MRs
              }
              gitLabTagDiscovery() // discover tags
            }
          }
        }
        // "Traits" ("Behaviours" in the GUI) that are NOT "declarative-compatible"
        // For some 'traits, we need to configure this stuff by hand until JobDSL handles it
        // https://issues.jenkins.io/browse/JENKINS-45504
        configure { node ->
            def traits = node / navigators / 'io.jenkins.plugins.gitlabbranchsource.GitLabSCMNavigator' / traits
            traits << 'io.jenkins.plugins.gitlabbranchsource.ForkMergeRequestDiscoveryTrait' {
                strategyId('2')
                trust(class: 'io.jenkins.plugins.gitlabbranchsource.ForkMergeRequestDiscoveryTrait$TrustPermission')
            }
            //traits << 'io.jenkins.plugins.gitlabbranchsource.SSHCheckoutTrait' {
            //    credentialsId('ssh_key')
            //}
        }
        // "Project Recognizers"
        projectFactories {
            templateMultiBranchProjectFactory()
        }
        // "Orphaned Item Strategy"
        orphanedItemStrategy {
          discardOldItems {
            daysToKeep(-1)
            numToKeep(-1)
          }
        }
        // "Scan Organization Folder Triggers" : 1 day
        // We need to configure this stuff by hand because JobDSL only allow 'periodic(int min)' for now
        configure { node ->
          node / triggers / 'com.cloudbees.hudson.plugins.folder.computed.PeriodicFolderTrigger' {
            spec('H H * * *')
            interval(86400000)
          }
        }
      }

5. Trigger Jenkins build by Merge Request creation

What's the expected result?

• Templating engine obtains and merges pipeline_config.groovy in both source repo and Governance repo

What's the actual result?

• Templating engine doesn't obtain pipeline_config.groovy in source repo, it only obtains pipeline_config.groovy in Governance repo.

• This lead to build fail with error:

No such DSL method startPod

6. Tool version
gitlab-branch-source: 1.4.3
job-dsl: 1.76
templating-engine: 1.5.1

7. Notes
As @steven-terrana pointed out this bug could related to GitLab SCM doesn't follow SCM API 2.0 spec

link

I'll post an issue on their Jenkins JIRA, but hopefully JTE can provide a work around in the meantime

Thank you.

Add support for Library Resources

Hi , This template engine looks pretty neat and I'm trying to implement this in our organization.

Wondering , whether there is any similar way to organize the supporting shell scripts used by groovy as they tend to become large in number and hard to organize.

Eg abc.groovy calls shells a1.sh , b1.sh for completing its tasks. we usually keep the a1.sh and b1.sh in scripts folder in git repo. Any way of organizing the shell scripts as well along with this engine?

Enable by parameter the branch/tag of the libraries repository

Feature Request

In order to implement new functionalities without affecting an entire organization or having to modify the configuration of the Jenkins Job, a property should be enabled inside the pipeline_config.groovy that allows deciding which branch or tag to use from the libraries.

Example:

A repo could have

libraries {
  version = 'master' // use master branch of libraries repo
  ...
}

Other repo could have

libraries {
  version = '1.0.1' // use 1.0.1 tag of libraries repo
  ...
}

Other repo could have

libraries {
  ... // use default JTE Plugin configuration
}

Error Loading Libraries

Received the following error for all builds using JTE yesterday when it was loading in libraries:
[JTE] hudson.plugins.git.GitException: Command "git config remote.origin.url https://github.com/boozallen/sdp-libraries.git" returned status code 255:
[JTE] stdout:
[JTE] stderr: error: could not lock config file .git/config: File exists

This error resulted in build failures for all of our jobs.

Was able to fix the issue by going to $JENKINS_HOME/caches and running the command:
"find . -type f -name "config.lock""
to find the location of the config.lock file. After finding where the file was, I removed it and this fixed the builds.

Pipeline always fails when Jenkins master is restarted

Hello!!

We are testing how resilient are our pipelines and we have discovered that every build using this plugin fails when restarting the master.

java.lang.ClassNotFoundException: script1580725490369752501149
 	at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:471)
 	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:588)
 	at groovy.lang.GroovyClassLoader.loadClass(GroovyClassLoader.java:677)
 	at groovy.lang.GroovyClassLoader.loadClass(GroovyClassLoader.java:787)
 	at groovy.lang.GroovyClassLoader.loadClass(GroovyClassLoader.java:775)
 	at java.base/java.lang.Class.forName0(Native Method)
 	at java.base/java.lang.Class.forName(Class.java:398)
 	at org.jboss.marshalling.AbstractClassResolver.loadClass(AbstractClassResolver.java:123)
 	at org.jboss.marshalling.AbstractClassResolver.resolveClass(AbstractClassResolver.java:104)
 	at org.jboss.marshalling.river.RiverUnmarshaller.doReadClassDescriptor(RiverUnmarshaller.java:1022)
 	at org.jboss.marshalling.river.RiverUnmarshaller.doReadNewObject(RiverUnmarshaller.java:1355)
 	at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:272)
 	at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:220)
 	at org.jboss.marshalling.river.RiverUnmarshaller.readFields(RiverUnmarshaller.java:1853)
 	at org.jboss.marshalling.river.RiverUnmarshaller.doInitSerializable(RiverUnmarshaller.java:1767)
 	at org.jboss.marshalling.river.RiverUnmarshaller.doInitSerializable(RiverUnmarshaller.java:1715)
 	at org.jboss.marshalling.river.RiverUnmarshaller.doInitSerializable(RiverUnmarshaller.java:1715)
 	at org.jboss.marshalling.river.RiverUnmarshaller.doReadNewObject(RiverUnmarshaller.java:1395)
 	at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:272)
 	at org.jboss.marshalling.river.BlockUnmarshaller.readObject(BlockUnmarshaller.java:149)
 	at org.jboss.marshalling.river.BlockUnmarshaller.readObject(BlockUnmarshaller.java:135)
 	at org.jboss.marshalling.MarshallerObjectInputStream.readObjectOverride(MarshallerObjectInputStream.java:53)
 	at org.jboss.marshalling.river.RiverObjectInputStream.readObjectOverride(RiverObjectInputStream.java:307)
 	at java.base/java.io.ObjectInputStream.readObject(ObjectInputStream.java:424)
 	at java.base/java.util.HashMap.readObject(HashMap.java:1460)
 	at java.base/jdk.internal.reflect.GeneratedMethodAccessor39.invoke(Unknown Source)
 	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
 	at org.jboss.marshalling.reflect.JDKSpecific$SerMethods.callReadObject(JDKSpecific.java:179)
 	at org.jboss.marshalling.reflect.SerializableClass.callReadObject(SerializableClass.java:212)
 	at org.jboss.marshalling.river.RiverUnmarshaller.doInitSerializable(RiverUnmarshaller.java:1746)
 	at org.jboss.marshalling.river.RiverUnmarshaller.doReadNewObject(RiverUnmarshaller.java:1395)
 	at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:272)
 	at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:220)
 	at org.jboss.marshalling.river.RiverUnmarshaller.readFields(RiverUnmarshaller.java:1853)
 	at org.jboss.marshalling.river.RiverUnmarshaller.doInitSerializable(RiverUnmarshaller.java:1767)
 	at org.jboss.marshalling.river.RiverUnmarshaller.doReadNewObject(RiverUnmarshaller.java:1395)
 	at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:272)
 	at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:205)
 	at org.jboss.marshalling.AbstractObjectInput.readObject(AbstractObjectInput.java:41)
 	at org.jenkinsci.plugins.workflow.support.pickles.serialization.RiverReader$SandboxedUnmarshaller.lambda$readObject$0(RiverReader.java:250)
 	at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.GroovySandbox.runInSandbox(GroovySandbox.java:237)
 	at org.jenkinsci.plugins.workflow.support.pickles.serialization.RiverReader$SandboxedUnmarshaller.sandbox(RiverReader.java:237)
 	at org.jenkinsci.plugins.workflow.support.pickles.serialization.RiverReader$SandboxedUnmarshaller.readObject(RiverReader.java:250)
 	at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution$2.onSuccess(CpsFlowExecution.java:783)
 	at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution$2.onSuccess(CpsFlowExecution.java:776)
 	at org.jenkinsci.plugins.workflow.support.concurrent.Futures$1.run(Futures.java:150)
 	at com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:253)
 	at com.google.common.util.concurrent.ExecutionList$RunnableExecutorPair.execute(ExecutionList.java:149)
 	at com.google.common.util.concurrent.ExecutionList.execute(ExecutionList.java:134)
 	at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:170)
 	at org.jenkinsci.plugins.workflow.support.concurrent.ChainingListenableFuture.access$000(ChainingListenableFuture.java:33)
 	at org.jenkinsci.plugins.workflow.support.concurrent.ChainingListenableFuture$1.run(ChainingListenableFuture.java:196)
 	at com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:253)
 	at com.google.common.util.concurrent.ExecutionList$RunnableExecutorPair.execute(ExecutionList.java:149)
 	at com.google.common.util.concurrent.ExecutionList.add(ExecutionList.java:105)
 	at com.google.common.util.concurrent.AbstractFuture.addListener(AbstractFuture.java:155)
 	at org.jenkinsci.plugins.workflow.support.concurrent.ChainingListenableFuture.run(ChainingListenableFuture.java:189)
 	at com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:253)
 	at com.google.common.util.concurrent.ExecutionList$RunnableExecutorPair.execute(ExecutionList.java:149)
 	at com.google.common.util.concurrent.ExecutionList.execute(ExecutionList.java:134)
 	at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:170)
 	at org.jenkinsci.plugins.workflow.support.concurrent.ChainingListenableFuture.access$000(ChainingListenableFuture.java:33)
 	at org.jenkinsci.plugins.workflow.support.concurrent.ChainingListenableFuture$1.run(ChainingListenableFuture.java:196)
 	at com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:253)
 	at com.google.common.util.concurrent.ExecutionList$RunnableExecutorPair.execute(ExecutionList.java:149)
 	at com.google.common.util.concurrent.ExecutionList.add(ExecutionList.java:105)
 	at com.google.common.util.concurrent.AbstractFuture.addListener(AbstractFuture.java:155)
 	at org.jenkinsci.plugins.workflow.support.concurrent.ChainingListenableFuture.run(ChainingListenableFuture.java:189)
 	at com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:253)
 	at com.google.common.util.concurrent.ExecutionList$RunnableExecutorPair.execute(ExecutionList.java:149)
 	at com.google.common.util.concurrent.ExecutionList.execute(ExecutionList.java:134)
 	at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:170)
 	at org.jenkinsci.plugins.workflow.support.concurrent.ListFuture.setOneValue(ListFuture.java:158)
 	at org.jenkinsci.plugins.workflow.support.concurrent.ListFuture.access$000(ListFuture.java:40)
 	at org.jenkinsci.plugins.workflow.support.concurrent.ListFuture$2.run(ListFuture.java:107)
 	at com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:253)
 	at com.google.common.util.concurrent.ExecutionList$RunnableExecutorPair.execute(ExecutionList.java:149)
 	at com.google.common.util.concurrent.ExecutionList.execute(ExecutionList.java:134)
 	at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:170)
 	at org.jenkinsci.plugins.workflow.support.concurrent.ChainingListenableFuture.access$000(ChainingListenableFuture.java:33)
 	at org.jenkinsci.plugins.workflow.support.concurrent.ChainingListenableFuture$1.run(ChainingListenableFuture.java:196)
 	at com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:253)
 	at com.google.common.util.concurrent.ExecutionList$RunnableExecutorPair.execute(ExecutionList.java:149)
 	at com.google.common.util.concurrent.ExecutionList.add(ExecutionList.java:105)
 	at com.google.common.util.concurrent.AbstractFuture.addListener(AbstractFuture.java:155)
 	at org.jenkinsci.plugins.workflow.support.concurrent.ChainingListenableFuture.run(ChainingListenableFuture.java:189)
 	at com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:253)
 	at com.google.common.util.concurrent.ExecutionList$RunnableExecutorPair.execute(ExecutionList.java:149)
 	at com.google.common.util.concurrent.ExecutionList.execute(ExecutionList.java:134)
 	at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:170)
 	at org.jenkinsci.plugins.workflow.support.pickles.TryRepeatedly.access$500(TryRepeatedly.java:48)
 	at org.jenkinsci.plugins.workflow.support.pickles.TryRepeatedly$1.run(TryRepeatedly.java:112)
 	at jenkins.security.ImpersonatingScheduledExecutorService$1.run(ImpersonatingScheduledExecutorService.java:58)
 	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
 	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
 	at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
 Caused: java.io.IOException: Failed to load build state
 	at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution$3.onSuccess(CpsFlowExecution.java:855)
 	at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution$3.onSuccess(CpsFlowExecution.java:853)
 	at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution$4$1.run(CpsFlowExecution.java:907)
 	at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$1.run(CpsVmExecutorService.java:38)
 	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
 	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
 	at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:131)
 	at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
 	at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:59)
 	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
 	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
 	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
 	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
 	at java.base/java.lang.Thread.run(Thread.java:834)

We are running our agents in Kubernetes and the JNLP connection is recovered properly. Any ideas about how we can improve the plugin to avoid this?

Regards!

Expand initialization logging so that each injector logs what it is injecting

Currently logging for JTE initialization is all over the place.

The idea for this issue would be to leverage TemplateLogger in every TemplatePrimitiveInjector so that we log each primitive JTE creates and makes accessible to the pipeline template during runtime

"Exclude repository branches without a pipeline configuration file" seems to be broken

Hi,

I was going through the article https://www.jenkins.io/blog/2019/05/09/templating-engine/ and I've used repositories in the article to at least make it discover repositories with pipeline_config.groovy but failed.

First of all, I have tested the same this twice on Jenkins 2.222.3 and on 2,199, JTE plugin version 1.7 for both cases.

So steps to reproduce are pretty simple:

1. I created forks of library, configuration and gradle app repo from the article repo in my gitHub org.
2. Added a Github organization pipeline with the default behavior - discover branches, pull requests, PR from forks,
3. "Project recognizers" is JTE, I checked "Exclude repository branches without a pipeline configuration file"
4. I scan a repo and I can see in the logs the same error for each branch it discovers:

Proposing example-jte-app-gradle
Examining JTE-sample/example-jte-app-gradle

Checking branches...

Getting remote branches...

Checking branch master

Getting remote pull requests...
ERROR: Failed to create or update a subproject example-jte-app-gradle
groovy.lang.MissingPropertyException: No such property: ScmPipelineConfigurationProvider for class: org.boozallen.plugins.jte.job.TemplateBranchProjectFactory
at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.unwrap(ScriptBytecodeAdapter.java:53)
at org.codehaus.groovy.runtime.callsite.GetEffectivePogoPropertySite.callGroovyObjectGetProperty(GetEffectivePogoPropertySite.java:70)
at org.boozallen.plugins.jte.job.TemplateBranchProjectFactory$1.isHead(TemplateBranchProjectFactory.groovy:71)
at jenkins.scm.api.trait.SCMSourceRequest.process(SCMSourceRequest.java:342)
at jenkins.scm.api.trait.SCMSourceRequest.process(SCMSourceRequest.java:249)
at org.jenkinsci.plugins.github_branch_source.GitHubSCMSource.retrieve(GitHubSCMSource.java:1000)
at jenkins.scm.api.SCMSource._retrieve(SCMSource.java:373)
at jenkins.scm.api.SCMSource.fetch(SCMSource.java:327)
at jenkins.branch.MultiBranchProjectFactory$BySCMSourceCriteria.recognizes(MultiBranchProjectFactory.java:261)
at jenkins.branch.OrganizationFolder$SCMSourceObserverImpl$1.recognizes(OrganizationFolder.java:1407)
at jenkins.branch.OrganizationFolder$SCMSourceObserverImpl$1.complete(OrganizationFolder.java:1422)
at jenkins.scm.api.trait.SCMNavigatorRequest.process(SCMNavigatorRequest.java:254)
at jenkins.scm.api.trait.SCMNavigatorRequest.process(SCMNavigatorRequest.java:204)
at org.jenkinsci.plugins.github_branch_source.GitHubSCMNavigator.visitSources(GitHubSCMNavigator.java:996)
at jenkins.branch.OrganizationFolder.computeChildren(OrganizationFolder.java:490)
at com.cloudbees.hudson.plugins.folder.computed.ComputedFolder.updateChildren(ComputedFolder.java:277)
at com.cloudbees.hudson.plugins.folder.computed.FolderComputation.run(FolderComputation.java:164)
at jenkins.branch.OrganizationFolder$OrganizationScan.run(OrganizationFolder.java:971)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:428)
12:57:51 4 repositories were processed
[Wed Apr 29 12:57:51 CEST 2020] Finished organization scan. Scan took 7 sec

Considering that file is pipline_config.groovy is present, I expect the repo to be discovered.
Could you please advise what I'm doing wrong?

Thank you

Support the Jenkins 'read-only' feature

Your checklist for this issue

🚨 Please review the guidelines for contributing to this repository.

• Link to any upstream changes that might be required (for example Jenkins Core pull request)

Feature Request

Jenkins has released the 'read-only' feature for preview, which allows for restricting configuration UI and API. Update the Jelly templates in JTE so that JTE is compatible with read-only Jenkins.

https://www.jenkins.io/blog/2020/05/25/read-only-jenkins-announcement/

https://www.jenkins.io/doc/developer/views/read-only/

Ability to include groovy steps defined in the application repository

Feature Request

Sometimes an application or team need to implement a different step. Which may not be useful to any other application or team. For instance handling of custom test framework.

Step implementation https://jenkinsci.github.io/templating-engine-plugin/pages/Primitives/default_step_implementation.html#default-step-implementation
sort of allow that. But it is only for simple steps that are docker based. Not suitable for series of commands, try/catch, if conditions, etc.

At the moment, that particular app or team would have to implement that step in a separate repository and add the library inclusion in the definition of the job. They could use the same repository as the application, but even then, the lib needs to be added to the job configuration. And they would have to specify the branch. Which is confusing as multi-branch pipeline is already handling multiple branches.

Can we load libraries defined locally, in a subfolder, alongside the pipeline_config.groovy?

Library Validation: regex match fields

For a pipeline configuration like the following:

libraries{
    someLibrary {
            projects {
                Infrastructure {
                    id = 'My.Project.Infrastructure'
                    threshold = 74
                }
                Api {
                    id = 'My.Project.Api' 
                    threshold = 72
                }
                Application {
                    id = 'My.Project.Application' 
                    threshold = 90
                }
            }
    }
}

it would be ideal to be able to use a regular expression in the library_config.groovy file to match all arbitrary project names and validate their contents like so:

fields{
  required{
      projects{
        ~/.*/ {
          id = String
          threshold = Integer
        }
      }
  }
}

Load library in a pipeline step

I use a global template as below
`set_agent {

stage ('Checkout') {
	git_checkout()
}
stage ('Build') {
	build()
}
stage ('UnitTests') {      
	unit_test()
}
stage('Publish')
{        
	publish()
}

}`

I have organized my libraries as maven, node, docker that implement the build() and publish() steps.
Now, I have a scenario where I have a maven code that needs to be built and containerized as docker.
Is there a way to first load the maven library, use the above template to build and publish the java code. Once successful, I would want to load the docker library and use the same template (or a named template) to build using the docker library.

Now, can we call the templating engine by loading a different library?

Pipeline plugin fails if application project defines new stages with merge or overrides

JTE Plugin version: 1.4

Scenario:

Goverence tier configuration

stages {
    stage_one {
        override = true
    }
}

Application Configuration

stages {
    stage_two {
        merge = true
    }
    stage_three {
        override = true
    }
}

Expected behaviour
Plugin should ignore new stages defined by application configuration

Actual behaviour:

Code fails with following error

Caused by: java.lang.NullPointerException: Cannot set property 'merge' on null object
	at org.boozallen.plugins.jte.config.TemplateConfigDsl.serialize_closure1$_closure4(TemplateConfigDsl.groovy:84)
	at groovy.lang.Closure.call(Closure.java:418)
	at org.boozallen.plugins.jte.config.TemplateConfigDsl.serialize_closure1(TemplateConfigDsl.groovy:79)
	at groovy.lang.Closure.call(Closure.java:418)
	at groovy.lang.Closure.call(Closure.java:434)
	at org.boozallen.plugins.jte.config.TemplateConfigDsl.serialize(TemplateConfigDsl.groovy:67)
	... 1 more

It appears plugin assumes any properties added to merge or override set in the application config is also present in parent configuration and it fails while serializing config object

JTE invokes a node block during initialization that cannot be passed a label

Hi,

I search for a way to force node label in pipeline template.

I tried this but it doesn't work :

properties([
    parameters([
        ...
    ])
])

node('master'){
     stage('Stage 1'){
          fonction1()
     }
     stage('Stage 1'){
          fonction2()
     }
}

Did you implement an other way to do this ?

Thanks

Improve the JTE Hooks documentation to specify when to use which type of hook

The documentation for lifecycle hooks should be improved to specify the idiomatic usage of the hooks.

Validate and Init are functionality equivalent but should be used for different things, etc

Add JavaDoc across the codebase to better communicate software architecture

Support Java 11

Hello!!

I am trying to run the plugin in a Java 11 Jenkins instance and I am getting the following error

java.lang.ClassNotFoundException: @63a800a7)
 	at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:471)
 	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:588)
 	at groovy.lang.GroovyClassLoader.loadClass(GroovyClassLoader.java:677)
 	at groovy.lang.GroovyClassLoader.loadClass(GroovyClassLoader.java:787)
 	at groovy.lang.GroovyClassLoader.loadClass(GroovyClassLoader.java:775)
 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
 	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
 	at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
 	at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
 	at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1213)
 	at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1022)
 	at org.codehaus.groovy.runtime.callsite.PojoMetaClassSite.call(PojoMetaClassSite.java:47)
 	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:125)
 	at org.boozallen.plugins.jte.utils.TemplateScriptEngine.parseClass(TemplateScriptEngine.groovy:76)
 	at org.boozallen.plugins.jte.utils.TemplateScriptEngine$parseClass.call(Unknown Source)
 	at org.boozallen.plugins.jte.binding.injectors.LibraryLoader.getPrimitiveClass(LibraryLoader.groovy:95)
 	at org.boozallen.plugins.jte.binding.injectors.LibraryLoader$getPrimitiveClass.call(Unknown Source)
 	at org.boozallen.plugins.jte.binding.TemplateBinding.setVariable(TemplateBinding.groovy:43)
 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
 	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
 	at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
 	at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
 	at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1213)
 	at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1022)
 	at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.call(PogoMetaClassSite.java:42)
 	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:133)
 	at org.boozallen.plugins.jte.TemplateEntryPointVariable.getValue(TemplateEntryPointVariable.groovy:65)
 	at org.jenkinsci.plugins.workflow.cps.CpsScript.invokeMethod(CpsScript.java:113)
 	at jdk.internal.reflect.GeneratedMethodAccessor1079.invoke(Unknown Source)
 	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
 	at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
 	at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
 	at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1213)
 	at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1022)
 	at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.call(PogoMetaClassSite.java:42)
 	at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
 	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
 	at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:160)
 	at org.kohsuke.groovy.sandbox.GroovyInterceptor.onMethodCall(GroovyInterceptor.java:23)
 	at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxInterceptor.onMethodCall(SandboxInterceptor.java:157)
 	at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxInterceptor.onMethodCall(SandboxInterceptor.java:142)
 	at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:158)
 	at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:162)
 	at com.cloudbees.groovy.cps.sandbox.SandboxInvoker.methodCall(SandboxInvoker.java:17)
 	at WorkflowScript.run(WorkflowScript:1)
 	at ___cps.transform___(Native Method)
 	at com.cloudbees.groovy.cps.impl.ContinuationGroup.methodCall(ContinuationGroup.java:84)
 	at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.dispatchOrArg(FunctionCallBlock.java:113)
 	at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.fixName(FunctionCallBlock.java:78)
 	at jdk.internal.reflect.GeneratedMethodAccessor761.invoke(Unknown Source)
 	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
 	at com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72)
 	at com.cloudbees.groovy.cps.impl.ConstantBlock.eval(ConstantBlock.java:21)
 	at com.cloudbees.groovy.cps.Next.step(Next.java:83)
 	at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:174)
 	at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:163)
 	at org.codehaus.groovy.runtime.GroovyCategorySupport$ThreadCategoryInfo.use(GroovyCategorySupport.java:129)
 	at org.codehaus.groovy.runtime.GroovyCategorySupport.use(GroovyCategorySupport.java:268)
 	at com.cloudbees.groovy.cps.Continuable.run0(Continuable.java:163)
 	at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.access$001(SandboxContinuable.java:18)
 	at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.run0(SandboxContinuable.java:51)
 	at org.jenkinsci.plugins.workflow.cps.CpsThread.runNextChunk(CpsThread.java:186)
 	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:370)
 	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.access$200(CpsThreadGroup.java:93)
 	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:282)
 	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:270)
 	at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:66)
 	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
 	at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:131)
 	at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
 	at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:59)
 	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
 	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
 	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
 	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
 	at java.base/java.lang.Thread.run(Thread.java:834)

I think the problem is related with Java 11.

Thanks in advance!

Ad hoc pipeline jobs to be able to inherit templates

We have many nightly builds configured as 'ad hoc pipeline jobs', and we want to use pipeline inheritance.

Governance pipeline_templates/nightly_build

node{
    stage("Task"){
        echo "task"
    }
}

Jenkins UI Configuration Template

pipeline_template='nightly_build`

Currently the pipeline inheritance only work for something like Multibranch Projects, not for Ad Hoc Pipeline Jobs.

Reading the getTemplate method in the TemplateEntryPointVariable.groovy class, probably the solution to this case is simple. The convention could be, if the flowDefinition.getTemplate() is not empty then use the job pipeline; if it is empty, then inherit template.

For example:

if (flowDefinition instanceof TemplateFlowDefinition){
    TemplateLogger.print "Obtained Pipeline Template from job configuration"
    String template = flowDefinition.getTemplate()
    if (template){
    	return template
    }
    //if flow template is empty, continue to inherit template from governance
}

Best regards

Calling non-groovy script possible?

From a library step, is it possible to make a call to a bash script or a python script which is stored inside the same repository as the groovy step(JTE library)?

Example, from a step

sh "my-custom-script.sh"

I added my-custom-script.sh at root of the repository (the one that contains the librairies)
But I don't see any library related file in the workspace

Need help in adding Build parameters to Jenkins Template Engine

We are using dynamic build parameters in our pipeline script with old Jenkinsfile way. With Jenkins Template Engine , I'm not able to add build parameters in the Template Engine. Please help.

Fail if step is not implemented

The current behavior when a step is not implemented (i.e. if you do not load any library, or do not have any pipeline_config.groovy file at the root of the repository) is to simply output:

Step X is not implemented.

A great addition would be to have some boolean in the configuration file (like allow_scm_jenkinsfile and skip_default_checkout) to manage this behavior to raise an exception instead of creating null step.

Bitbucket Server as a pipeline configuration source fails

Issue
When attempting to use "Bitbucket Server" as a source for pipeline configurations, JTE cannot locate the pipeline template. Testing the connection and other projects using the Bitbucket Server configurtion confirms that Jenkins is able to access the repo.

Workaround
Switching to a "Git" as a source, JTE works properly.

Versions
Jenkins 2.204.4
Templating Engine 1.5.2
Bitbucket Server Integration 1.1.0

Log

> Running in Durability level: MAX_SURVIVABILITY
[Pipeline] Start of Pipeline
[JTE] Obtained Template Configuration File (show)
[JTE] Obtained Repository Jenkinsfile (show)
[Pipeline] End of Pipeline
org.boozallen.plugins.jte.config.TemplateConfigException: Could not determine pipeline template.
	at sun.reflect.GeneratedConstructorAccessor1376.newInstance(Unknown Source)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
	at org.codehaus.groovy.reflection.CachedConstructor.invoke(CachedConstructor.java:83)
	at org.codehaus.groovy.runtime.callsite.ConstructorSite$ConstructorSiteNoUnwrapNoCoerce.callConstructor(ConstructorSite.java:105)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callConstructor(AbstractCallSite.java:247)
	at org.boozallen.plugins.jte.TemplateEntryPointVariable.getTemplate(TemplateEntryPointVariable.groovy:193)
	at org.boozallen.plugins.jte.TemplateEntryPointVariable$getTemplate.call(Unknown Source)
	at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
	at com.cloudbees.groovy.cps.sandbox.DefaultInvoker.methodCall(DefaultInvoker.java:20)
	at script15870935980111752575261.call(script15870935980111752575261.groovy:32)
	at script15870935980111752575261.call(script15870935980111752575261.groovy)
	...

Perform Initialization of JTE prior to the start of the pipeline (before the CpsFlowExecution has begun)

Refactor JTE's package structure to better communicate the software architecture

1.5.2 to 1.6 upgrade reset the Pipeline Configuration

Hello,

The upgrade from 1.5.2 to 1.6 completely reset the Pipeline Configuration defined in the configuration panel of Jenkins resulting in JTE failing to retrieve the configuration.
The jobs needing JTE now fails with this traceback:

org.boozallen.plugins.jte.config.TemplateConfigException: Could not determine pipeline template.
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
	at org.codehaus.groovy.reflection.CachedConstructor.invoke(CachedConstructor.java:83)
	at org.codehaus.groovy.runtime.callsite.ConstructorSite$ConstructorSiteNoUnwrapNoCoerce.callConstructor(ConstructorSite.java:105)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callConstructor(AbstractCallSite.java:247)
	at org.boozallen.plugins.jte.TemplateEntryPointVariable.getTemplate(TemplateEntryPointVariable.groovy:215)
	at org.boozallen.plugins.jte.TemplateEntryPointVariable$getTemplate$0.call(Unknown Source)
	at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
	at com.cloudbees.groovy.cps.sandbox.DefaultInvoker.methodCall(DefaultInvoker.java:20)
	at script15875524396401752575261.call(script15875524396401752575261.groovy:32)
	at script15875524396401752575261.call(script15875524396401752575261.groovy)
	at WorkflowScript.run(WorkflowScript:1)
	at ___cps.transform___(Native Method)
	at com.cloudbees.groovy.cps.impl.ContinuationGroup.methodCall(ContinuationGroup.java:86)
	at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.dispatchOrArg(FunctionCallBlock.java:113)
	at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.fixArg(FunctionCallBlock.java:83)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72)
	at com.cloudbees.groovy.cps.impl.PropertyishBlock$ContinuationImpl.get(PropertyishBlock.java:76)
	at com.cloudbees.groovy.cps.LValueBlock$GetAdapter.receive(LValueBlock.java:30)
	at com.cloudbees.groovy.cps.impl.PropertyishBlock$ContinuationImpl.fixName(PropertyishBlock.java:66)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72)
	at com.cloudbees.groovy.cps.impl.ConstantBlock.eval(ConstantBlock.java:21)
	at com.cloudbees.groovy.cps.Next.step(Next.java:83)
	at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:174)
	at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:163)
	at org.codehaus.groovy.runtime.GroovyCategorySupport$ThreadCategoryInfo.use(GroovyCategorySupport.java:129)
	at org.codehaus.groovy.runtime.GroovyCategorySupport.use(GroovyCategorySupport.java:268)
	at com.cloudbees.groovy.cps.Continuable.run0(Continuable.java:163)
	at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.access$001(SandboxContinuable.java:18)
	at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.run0(SandboxContinuable.java:51)
	at org.jenkinsci.plugins.workflow.cps.CpsThread.runNextChunk(CpsThread.java:185)
	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:400)
	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.access$400(CpsThreadGroup.java:96)
	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:312)
	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:276)
	at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:67)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:131)
	at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
	at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:59)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Finished: FAILURE

Luckily I'm in a process of setting up a new Jenkins instance and I did not add any real projects yet but this could be problematic for a big company with hundreds of projects to have all their build failing.
I think it should be at least more obvious in the release notes that the new feature for defining config directly in Jenkins UI will reset what is existing (or the previous configuration should be kept if one was existing)

Cheers, and thanks for that plugin !