Coder Social home page Coder Social logo

jx3-pipeline-catalog's Introduction

jx3-pipeline-catalog

The default pipeline catalog for Jenkins X 3.x

Jenkins X 3.x comes with its own default pipeline catalog for different languages, tools and frameworks. This catalog contains reusable steps, Tasks, Pipelines and Packs you can use on any project.

For more information check out the Jenkins X 3.x support for Tekton Catalog

Contents

  • tasks a reusable folder of tasks and associated triggers
  • packs contains the language and/or framework specific packs containing tekton pipelines and associated files used by the pipelines such as Dockerfile or helm charts.
  • helm contains reusable helm charts that are imported into the various folders in packs such as packs/javascript/charts to share charts across the different programming languages

Custom Pipeline Catalogs

(Blog article)

To use your own custom pipeline catalog, you can fork this catalog to make changes for your team or share between teams in your company. You can make as many catalogs as you like and put whichever catalogs you want in the extensions/pipeline-catalog.yaml file of your cluster git repository of your Jenkins X 3.x install, like for example:

# Source: <boot-repo>/extensions/pipeline-catalog.yaml
apiVersion: project.jenkins-x.io/v1alpha1
kind: PipelineCatalog
spec:
  repositories:
  - label: Your Pipeline Catalog
    gitUrl: https://github.com/<your-org>/jx3-pipeline-catalog
    gitRef: master

For more detail there's the configuration reference here.

Then when developers create a new quickstart or import a repository developers will be asked to pick the catalog they want from your list if there is more than one, or the configured catalog is silently used.

This gives you complete freedom to configure things at a global, team or repository level while also making it easy to share changes across projects, teams and companies.

Upgrading Charts

Many of these packs contain helm charts such as packs/javascript/charts which are managed via kpt.

If the helm/charts folder is modified you can upgrade the charts in each pack via:

jx gitops upgrade --ignore-yaml-error

jx3-pipeline-catalog's People

Contributors

ajpauwels avatar ankitm123 avatar babadofar avatar bdurrow avatar brandong954 avatar chrismellard avatar dicolasi avatar freper138 avatar gazal-k avatar gbrener avatar gonzalochief avatar haysclark avatar hervelemeur avatar jenkins-x-bot avatar jenkins-x-bot-test avatar joshuasimon-taulia avatar jstrachan avatar laucapgemini avatar m1pl avatar msvticket avatar osamamagdy avatar pow-devops2020 avatar rajatgupta24 avatar rawlingsj avatar skisocks avatar tdcox avatar tomhobson avatar vbehar avatar wrbobby avatar yelhouti avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

jx3-pipeline-catalog's Issues

Javascript quickstarts fail due to git related tasks

I tried to use the react and node-http quickstarts. Both of them failed to create their pipeline. jx pipeline lint reveals:

WARNING: ignoring missing user name in git credentials file: /Users/brad/.git-credentials URL: https://:@github.com
FILE                                   STATUS
.lighthouse/jenkins-x/triggers.yaml    OK
.lighthouse/jenkins-x/pullrequest.yaml failed to unmarshal YAML file .lighthouse/jenkins-x/pullrequest.yaml: failed to validate generated PipelineRun: invalid value: git-clone: spec.pipelinespec.tasks[0].taskSpec.steps[2].name
invalid value: git-merge: spec.pipelinespec.tasks[0].taskSpec.steps[3].name
.lighthouse/jenkins-x/release.yaml     failed to unmarshal YAML file .lighthouse/jenkins-x/release.yaml: failed to validate generated PipelineRun: invalid value: git-clone: spec.pipelinespec.tasks[0].taskSpec.steps[1].name

I am not sure if it is related but my cluster has a clusterTask called git-clone. Removing that git-clone clusterTask does not seem to resolve the problem.

If I remove metadata.annotations.lighthouse.jenkins-x.io/prependStepsURL: https://raw.githubusercontent.com/jenkins-x/jx3-pipeline-catalog/9ef3f3c107abc35b8f49d2e17d3af9ba5c5ef932/tasks/git-clone/git-clone-pr.yaml it sems to resolve the lint issue. I figured this out because I imported a javascript project that didn't have this problem and a diff of their pullrequest.yaml was the only difference. The working copy did not have the offending annotation.

I don't think it is related but I will mention it here: after fixing this linting problem, committing the changes and pushing them to the repo. pipelineRuns are created which in turn create taskRuns but the taskRuns never start and do not get a status block.

Automatically update pipeline steps when versionstream is updated

Pipeline files should receive automatic updates when a newer version of jx is released.

It's important for us to keep to the same major version.
Unsure about minor but would certainly work for patch.

Ie 2.1.7 -> 2.1.8
3.0.1 -> 3.0.3

This will save a lot of headaches when updating microservices pipelines.

Add Makefile,Kptfile and .helmignore in .helmignore

It's a minor thing but I think those files should not be included in helm chart by default. Or maybe it's just me that have no use case for them :).

./README.md
./Chart.yaml
./Kptfile
./Makefile
./templates
./templates/release.yaml
......................................
./templates/hpa.yaml
./.helmignore
./values.yaml

Maven tasks should have mvn-install for the release pipeline

At present, the maven tasks use "mvn deploy" in the release pipeline. This pushes packages to the maven registry, which may be useful in some situations, but in many situations, these packages will never be used and simply take up space. We could add an "mvn install" step, which does the same, but without pushing packages. Then each user could modify the pipeline in the builds and select if they want to use "mvn install" or "mvn deploy".

translating TaskSpec to Pod: could not parse reference: uses:jenkins-x/jx3-pipeline-catalog/tasks/git-clone/git-clone-pr.yaml@versionStream.

Hello,

jx import project failed with error below :

error: failed to wait for repository to be setup in lighthouse: failed to find trigger in the lighthouse configuration in ConfigMap config in namespace jx for repository

After some investigation found out .lighthouse/jenkins-x/pullrequest.yaml was failing due to the error :
Message: failed to create task run pod "pullrequest-from-build-pack-b24bb": translating TaskSpec to Pod: could not parse reference: uses:jenkins-x/jx3-pipeline-catalog/tasks/git-clone/git-clone-pr.yaml@versionStream. Maybe invalid TaskSpec

Much obliged.

same wrong image for all charts

icon: https://raw.githubusercontent.com/jenkins-x/jenkins-x-platform/d273e09/images/nodejs.png

and

# Default values for node projects.

should be modified in all repos to match the project type (gradle: java...)

Docs explaining real world scenarios

It would be great if the docs explain real world scenarios like:

  1. Integrating security scanners like Clair/Trivy/Anchore. I see that the Tekton catalog has a trivy-scanner but how does one go about using it. The only thing I found was "jx pipeline import" which pulled in the trivy-scanner. No clear documentation on the full steps.

  2. Build and include other dependant micro services (not databases and the like) in a preview environment as part of an integration test.

There seems to be no books on JenkinsX v3 or courses. Quite a few using v2. The main docs doesn’t go into detail. Love the approach of JenkinsX but the lack of docs is frustrating.

Evaluating JenkinsX for a new AI SaaS product as JX seems to eliminate a lot of plumbing. This would be deployed on Amazon EKS.

Any advice would be appreciated. Thx

PipelineRun timeout not working as expected

Hi,
I'm trying to reduce the timeout of the tasks (in my case, maven-pullrequest), but if I reduce the timeout for testing, from "12h0m0s" to "0h0m30s", the pipeline didn't stop after 30 secs, it still running for 4min.

image
image

In my case, I have forked this repository and with "image: uses:lighthouse:[myrepository]/tasks/maven-java11/
pullrequest.yaml@versionStream", I'm able to use my forked tasks (this works fine), but I can't set other timeout...

Also, I have tried to use the timeout inside the task steps, to set custom timeouts to each step, with the same result... Example
For me, this didn't work neither example

Tried this format: "options: timeout: [ time: / unit: ]" and didn't works. I suppose it is for jenkinsX 2...

Im using JenkinsX 3.

Thanks you in advance.
Cumal.

implement git shallow clone for the git-clone task

I'm facing issues with a repository that has a very long history, even though it is not heavy by itself.
As the git-clone task is implemented, every time there is a clone action, the cluster clones the entire repository, which is not efficien; especially when there are large repos (due to long histories).
I propose to implement git shallow clone for the git clone task, instead of the regular clone

proposal:
git clone --depth=1 $REPO_URL $SUBDIR

Original:
git clone $REPO_URL $SUBDIR

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.