Coder Social home page Coder Social logo

teamcity-pipelines-dsl's Introduction

TeamCity Pipelines DSL

Please note that TeamCity Build Chain DSL Extension (AKA Pipeline DSL) is no longer implemented as a separate library or plugin but included in TeamCity Kotlin DSL implementation and available as such since TeamCity version 2019.2

Kotlin DSL library for TeamCity pipelines

The library provides a number of extensions to simplify creating TeamCity build chains in Kotlin DSL. The main feature of the library is automatic setup of snapshot dependencies.

Usage

  1. Add jitpack repository to your .teamcity/pom.xml
    <repositories>
      <repository>
        <id>jitpack.io</id>
        <url>https://jitpack.io</url>
      </repository>
    </repositories>
  1. Add the library as a dependency in .teamcity/pom.xml
   <dependency>
	 <groupId>com.github.JetBrains</groupId>
	 <artifactId>teamcity-pipelines-dsl</artifactId>
	 <version>[tag]</version>
   </dependency>

Examples

To compose the pipeline with the aid of TeamCity Pipelines DSL library you will be using three main terms: sequence, parallel, and build.

The main block of the pipeline is the sequence. Inside the sequence we may define parallel blocks and the individual build stages. Further on, the parallel blocks may include the individual builds but also sequences.

Simple sequence

The following example demonstrates a simple sequence with three individual build configurations:

//settings.kts

import jetbrains.buildServer.configs.kotlin.v2018_2.BuildType
import jetbrains.buildServer.configs.kotlin.v2018_2.buildSteps.script
import jetbrains.buildServer.configs.kotlin.v2018_2.project
import jetbrains.buildServer.configs.kotlin.v2018_2.version

version = "2018.2"


project {
    sequence {
        build(Compile) {
            produces("application.jar")
        }
        build(Test) {
            requires(Compile, "application.jar")
            produces("test.reports.zip")
        }
        build(Package) {
            requires(Compile, "application.jar")
            produces("application.zip")
        }
    }
}

object Compile : BuildType({
    name = "Compile"
    //...
})

object Test : BuildType({
    name = "Test"
    //...
})

object Package : BuildType({
    name = "Package"
    //...
})

The snapshot dependencies are defined automatically: Package depends on Test, and Test depends on Compile.

    _________         ______         _________
   |         |       |      |       |         |              
   | Compile | ----> | Test | ----> | Package |  
   |_________|       |______|       |_________|   

The additional artifact dependencies are defined explicitly by using requires(...) and produces(...) functions.

The minimal diamond

If there's a need to parallelize part of the sequence we may use parallel {} block define the intent:

    sequence {
        build(Compile) {
            produces("application.jar")
        }
        parallel {
            build(Test1) {
                requires(Compile, "application.jar")
                produces("test.reports.zip")
            }
            build(Test2) {
                requires(Compile, "application.jar")
                produces("test.reports.zip")
            }
        }
        build(Package) {
            requires(Compile, "application.jar")
            produces("application.zip")
        }
    }

All the build configurations declared in the parallel block will have a snapshot dependency on the last build configuration specified before the parallel block. In this case, Test1 and Test2 depend on Compile.

The first build configuration declared after the parallel block will have a snapshot dependency on all the build configrations declared in the parallel block. In this case, Package depends on Test1 and Test2.

                      _______
                     |       |
    _________        | Test1 |        _________
   |         | ----> |_______| ----> |         |             
   | Compile |        _______        | Package |  
   |_________| ----> |       | ----> |_________|
                     | Test2 |
                     |_______|      

Combining different blocks

Even more, we can put a new sequence into the parallel block:

    sequence {
        build(Compile) {
            produces("application.jar")
        }
        parallel {
            build(Test) {
                requires(Compile, "application.jar")
                produces("test.reports.zip")
            }
            sequence {
                build(RunInspections) {
                   produces("inspection.reports.zip")
                }
                build(RunPerformanceTests) {
                   produces("perf.reports.zip")
                }
            }         
        }
        build(Package) {
            requires(Compile, "application.jar")
            produces("application.zip")
        }
    }   

The result is as follows:

                      _______
                     |       |
    _________        | Test  |                                _________
   |         | ----> |_______| ----------------------------> |         |             
   | Compile |        _______         ____________           | Package |  
   |_________| ----> | RunIn | ----> | RunPerform | -------> |_________|
                     | spect |       | anceTests  |
                     | ions  |       |____________|
                     |_______|      

Inline build definitions

There's actually an alternative form of the build() function that allows specifying the build configuration inline:

    sequence {
        val compile = build {
            id("Compile")
            name = "Compile"
            
            //alternatively, we could use produces() function here
            artifactRules = "target/application.jar"
         
            vcs {
              root(ApplicationVcs)
            }
            
            steps {
               maven {
                  goals = "clean package"
               }
            } 
        }
        val test = build {
            id("Test")
            name = "Test"
            
            vcs {
               root(ApplicationTestsVcs)
            }
            
            steps {
               // do something here..
            }
            
            // requires(...) is an alternative to 
            // dependencies { at
            //   artifact(compile) {
            //     artifactRules = ..
            //   }
            // }
            requires(compile, "application.jar")
            
        }
    }   

In the example above, we define two build configurations -- Compile and Test -- and register those in the sequence, meaning that Test build configuration will have a snapshot dependency on Compile.

Arbitrary dependencies

The sequence is an abstraction that allows you to specify the dependencies in build configurations depending on the order in which they are declared within the sequence.

However, sometimes it might be needed to create a dependency on a build configuration that is defined outside of the sequence. In this case, it is possible to use dependsOn method within a block

    build(OtherBuild)
     
    sequence {
        build(Compile) {
            produces("application.jar")
        }
        parallel {
            dependsOn(SomeOtherConfiguration)
            build(Test) {
                requires(Compile, "application.jar")
                produces("test.reports.zip")
            }
            sequence {
                build(RunInspections) {
                   produces("inspection.reports.zip")
                }
                build(RunPerformanceTests) {
                   produces("perf.reports.zip")
                }
            }         
        }
        build(Package) {
            requires(Compile, "application.jar")
            produces("application.zip")
        }
    }   

The dependsOn method invoked in a parallel block adds a dependency to every stage in that block. Hence, Test and RunInspections build configuration will end up having a dependency on OtherBuild:

   ________________
  |                |
  |  OtherBuild    |                 _______
  |________________|--------------->|       |
        |          _________        | Test  |                                _________
        |         |         | ----> |_______| ----------------------------> |         |             
        |         | Compile |        _______         ____________           | Package |  
        |         |_________| ----> | RunIn | ----> | RunPerform | -------> |_________|
        |                           | spect |       | anceTests  |
        --------------------------->| ions  |       |____________|
                                    |_______|      

You can also use dependsOn to declare a dependency for a build configuration on a sequence (or a parallel block):

    var seq = Sequence() 
    
    sequence {
        build(Compile) {
            produces("application.jar")
        }
        parallel {
            build(Test) {
                requires(Compile, "application.jar")
                produces("test.reports.zip")
            }
            seq = sequence { // <--------- assigning a nested sequence 
                build(RunInspections) {
                   produces("inspection.reports.zip")
                }
                build(RunPerformanceTests) {
                   produces("perf.reports.zip")
                }
            }         
        }
        build(Package) {
            requires(Compile, "application.jar")
            produces("application.zip")
        }
    }     

    build(OtherBuild){
        dependsOn(seq) // <----------- depends on nested sequence
    }

The result is as follows:

                      _______
                     |       |
    _________        | Test  |                                _________          ____________
   |         | ----> |_______| ----------------------------> |         |        |            |     
   | Compile |        _______         ____________           | Package |        | OtherBuild |
   |_________| ----> | RunIn | ----> | RunPerform | -------> |_________|   ---->|____________|
                     | spect |       | anceTests  |                       |
                     | ions  |       |____________| ----------------------
                     |_______|      

Snapshot dependency settings

Snapshot dependencies include various settings. For instance, we might choose to run the build on the same agent as the dependency.

    sequence {
        build(Compile) {
            produces("application.jar")
        }
        parallel {
            build(Test) {
                requires(Compile, "application.jar")
                produces("test.reports.zip")
            }
            sequence {
                build(RunInspections) {
                   produces("inspection.reports.zip")
                }
                build(RunPerformanceTests) {
                   produces("perf.reports.zip")
                }
            }   
            dependencySettings {
               runOnSameAgent = true // <--- 'Test' & 'RunInspections' will run on the same agent as 'Compile'
            }
        }
        build(Package) {
            requires(Compile, "application.jar")
            produces("application.zip")
        }
    }   

Also, the dependsOn method allows adding a block with the snapshot dependency settings:

    val other = build { id("other")}

    sequence {
        build(Compile) {
            produces("application.jar")
        }
        parallel {
            build(Test) {
                dependsOn(other){
                   runOnSameAgent = true // <--- 'Test' will run on the same agent as 'other'
                }
                requires(Compile, "application.jar")
                produces("test.reports.zip")
            }
            sequence {
                build(RunInspections) {
                   produces("inspection.reports.zip")
                }
                build(RunPerformanceTests) {
                   produces("perf.reports.zip")
                }
            }         
        }
        build(Package) {
            requires(Compile, "application.jar")
            produces("application.zip")
        }
    }   

The dependsOn method can be used for the individual bulid configuration as well as for parallel and sequence blocks:

    val other = build {id("other")}

    sequence {
        build(Compile) {
            produces("application.jar")
        }
        parallel {
            dependsOn(other){
                runOnSameAgent = true // <--- 'Test' and 'RunInspections' will both run on the same agent as 'other'
            }
            build(Test) {
                requires(Compile, "application.jar")
                produces("test.reports.zip")
            }
            sequence {
                build(RunInspections) {
                   produces("inspection.reports.zip")
                }
                build(RunPerformanceTests) {
                   produces("perf.reports.zip")
                }
            }         
        }
        build(Package) {
            requires(Compile, "application.jar")
            produces("application.zip")
        }
    }   

teamcity-pipelines-dsl's People

Contributors

antonarhipov avatar karanagai avatar logerfo avatar orybak avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

teamcity-pipelines-dsl's Issues

Artifact dependency points to non existing build configuration

Code

project {
   sequence {
      parallel {
         sequence {
            build(Restore) {
               produces("**/*")
            }
            build(Build) {
               requires(Restore, "**/*")
            }
         }
         sequence {
            //...
         }
      }
   }
}

object Restore : BuildType({
   name = "Restore"
   steps {
      msBuild {
         //...
      }
   }
})

object Build : BuildType({
   name = "Build"
   steps {
      msBuild {
         //...
      }
   }
})

Error

Artifact dependency points to non existing build configuration with external id: <PROJECT_ID>_Restore

It actually prints twice in a row, I don't know why.

Solution attemps

I tried changing the build id:

object Restore : BuildType({
   id = "<PROJECT_ID>"
   name = "Restore"
})

But the error changed to:

Artifact dependency points to non existing build configuration with external id: <PROJECT_ID>_<PROJECT_ID>_Restore

Publish new pom.xml that has the https endpoint?

Hi @antonarhipov -

When trying to apply this dependency in our pom.xml, our TeamCity server fails to resolve the dependency with a handshake failure:

Failed to apply changes from VCS to project settings
Received fatal alert: handshake_failure. Please fix the errors in VCS and commit again.

Any chance this is due to the pom.xml having http instead of https? Can you republish 0.8 as 0.8.1 with the new POM you committed to see if this resolves the issue?

dependsOn(parallel) with onDependencyFailure ignored

I'm trying to setup a dependency like this, so that the last build(Package) in the sequence doesn't start if any of its dependencies failed.

Issue: The config is valid but the onDependencyFailure is ignored and doesn't end up in the generated XML.

    var par = Parallel() 
    sequence {
        build(Compile) {
            produces("application.jar")
        }
        par = parallel {
            build(Test) {
                requires(Compile, "application.jar")
                produces("test.reports.zip")
            }
            sequence {
                build(RunInspections) {
                   produces("inspection.reports.zip")
                }
                build(RunPerformanceTests) {
                   produces("perf.reports.zip")
                }
            }   

        }
        build(Package) {
            requires(Compile, "application.jar")
            produces("application.zip")
            dependsOn(par){
                onDependencyFailure = FailureAction.FAIL_TO_START // <--- don't want to Package if any build in parallel failed
            }
        }
    }

By default the dependency is setup with RUN_ADD_PROBLEM

How to configure Android Studio to use .teamcity code files in project?

I am trying to setup a TeamCity project configuration using Kotlin DSL. I've created a sample project in TeamCity pointing to just an empty Android Studio project with empty Activity which I 've published in the github. I've enabled Kotlin DSL configuration on the TeamCity for that project.

After that TeamCity has commited a .teamcity folder to my Android Studio project. However, when I am inspecting the code in settings.kts the IDE is not giving me any hints and I cannot use the autocompletion for the Kotlin DSL configuration code.
For example I cannot navigate to implementation of function triggers, cannot use autocompletion etc.

I was trying to follow https://blog.jetbrains.com/teamcity/2019/03/configuration-as-code-part-1-getting-started-with-kotlin-dsl/#commento-login-box-container but unfortunately I cannot right-click on the pom.xml file and don't see any option to Add as Maven.
In addition I cannot see any External Libraries added to my project related to TeamCity DSL

I am using Android Studio 4.0

Is it possible to use such configuration in my case?
If so, can you please help me how I should properly setup it in Android Studio to have such benefits in Android Studio?
Android Studio project uses Gradle by default

VCSRoot "useMirrors = false" will always be overwritten

Hey there,

i am using the pipeline dsl for some of my projekt an have a issue, where my centos docker container, on my windows agent cant access the path in the ./git/objects/info/alternates (directing somewhere on the windows VM, C:\BuildAgent.... )

i can fix this by enabling the "use mirrors" flag in the VCS root.

But for the pipeline script, it will be overwritten after settings are synced.

my VCS root in my settings.kts looks like this

object MyVCS: GitVcsRoot({
    name = "project name"
    url = "gitlab url"
    branch = "master"
    
    useTagsAsBranches = true
    useMirrors = false

    branchSpec = """
      +:refs/heads/(*)
      +:refs/tags/(*)
    """.trimIndent()

    authMethod ...

})

Any idea why this flag is always set to true?

Will you be adding a license?

I've been building something very similar (since I'm a big fan of the Jenkins DSL and think the TeamCity one has great potential) and was wondering if you either have a roadmap and if you'd be able to add a license so we can extend this repo accordingly to work with our needs.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.