Coder Social home page Coder Social logo

openshift-pipelines / pipelines-as-code Goto Github PK

View Code? Open in Web Editor NEW
120.0 7.0 75.0 37.39 MB

Pipelines-as-Code for Tekton

Home Page: https://pipelinesascode.com

License: Apache License 2.0

Makefile 0.42% Go 96.20% Shell 1.46% Dockerfile 0.04% Python 1.88%
tekton-pipelines tekton github pipeline kubernetes continuous-delivery pipelines-as-code gitlab ci bitbucket gitops

pipelines-as-code's Introduction

Pipelines-as-Code

Container Repository on GHC codecov Go Report Card E2E Tests

Pipelines-as-Code -- An opinionated CI based on OpenShift Pipelines / Tekton.

Full documentation for the stable version is available from https://pipelinesascode.com Documentation for the development branch is available here

Introduction

Pipelines-as-Code let you use the Pipelines-as-Code flow directly with Tekton and OpenShift Pipelines.

The goal of Pipelines-as-Code is to let you define your Tekton templates inside your source code repository and have the pipeline run and report the status of the execution when triggered by a Pull Request or a Push.

Pipelines-as-Code features:

  • Pull-request status support: When iterating over a Pull Request, status and control is done on the platform.

  • GitHub Checks API support to set the status of a PipelineRun including rechecks

  • GitHub Pull Request and Commit event support

  • Pull-request actions in comments such as /retest

  • Git events filtering and support for separate pipelines for each event

  • Automatic Task resolution in Pipelines (local Tasks, Tekton Hub and remote URLs)

  • Efficient use of GitHub blobs and objects API for retrieving configurations

  • ACL over a GitHub organization or with a Prow style OWNER file.

  • tkn-pac plug-in for Tekton CLI for managing pipelines-as-code repositories and bootstrapping.

  • GitLab, Bitbucket Server, Bitbucket Cloud and GitHub through Webhook support.

Installation Guide

The easiest way to get started is to use the tkn pac CLI and its bootstrap command.

Download and install first the tkn-pac CLI following these instructions.

Connected to your cluster execute the command :

-$ tkn pac bootstrap

and follow the questions and installation methods which will install Pipelines-as-Code on cluster and help you create a GitHub Application.

feel free to look over the --help to see the different options on how to install for example on GitHub Enterprise.

This getting started video will guide you thought the tkn-pac bootstrap flow using Kind :

Getting started to Pipelines-as-Code

For more details on the different installation method please follow this document for installing Pipelines-as-Code on OpenShift.

Getting Started

The flow for using pipelines as code generally begins with admin installing the Pipelines-as-Code infrastructure, creating a GitHub App and sharing the GitHub App URL across the organization for app teams to enable the app on their GitHub repositories.

Start creating a GitHub repository by going to this URL https://github.com/new, you will need to provide a name (eg: pac-demo) and check the "[ ] Add a README file" box before pressing the "Create Repository" button.

You are now able to enable the Pipelines-as-Code GitHub Application as created by the Admin onto your new repository by following this guide here.

Once you have enabled your GitHub App for your GitHub repository, you can use the Tekton CLI pac plug-in to bootstrap pipelines as code:

$ git clone https://github.com/youruser/pac-demo
$ cd pac-demo
$ tkn pac create repository
? Enter the Git repository url containing the pipelines (default: https://github.com/youruser/pac-demo):
? Please enter the namespace where the pipeline should run (default: pac-demo):
! Namespace pac-demo is not found
? Would you like me to create the namespace pac-demo? Yes
✓ Repository youruser-pac-demo has been created in pac-demo namespace
ℹ Directory .tekton has been created.
✓ A basic template has been created in .tekton/pipelinerun.yaml, feel free to customize it.
ℹ You can test your pipeline by pushing the generated template to your git repository

The above command would create a Repository CRD in your demo namespace which is used to determine where the PipelineRuns for your GitHub repository should run. It also generates an example pipeline in the .tekton folder. Commit and push the pipeline to your repo to start using pipelines as code.

Note that even if installing with GitHub application is the preferred installation method, Pipeline As Code supports other methods :

  • GitHub direct Webhook
  • GitLab public and private instances.
  • Bitbucket Cloud
  • Bitbucket Server

You can use the command tkn pac webhook to help you update webhooks on your repository. See the INSTALL guide for more details on each install method.

Usage Guide

The usage guide available here offer a comprehensive documentation on how to use and configure Pipeline As Code.

Contributing

We ❤️ our contributors!

If you want to help and contribute to the pipelines-as-code project, you can see the documentation here to get started: https://pipelinesascode.com/dev/ (and thank you).

Getting in touch

There are several ways to communicate with us:

Videos/Blog Posts

pipelines-as-code's People

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

pipelines-as-code's Issues

Error when applying the Pipelinerun if we are doing a push to tag

[pipelines-as-code : apply-and-launch] Error: unable to parse requirement: values[0][tekton.dev/pipeline-ascode-branch]: Invalid value: "refs/tags/0.2": a valid label must be an empty string or consist of alphanumeric characters, '-', '_' or '.', and must start and end with an alphanumeric character (e.g. 'MyValue',  or 'my_value',  or '12345', regex used for validation is '(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])?')

OWNERS support for running CI

OWNERS support

As a platform administrator I do not want random people being able to start using our infra via Pipeline as Code for bitcoin or others.

See :

https://github.blog/2021-04-22-github-actions-update-helping-maintainers-combat-bad-actors/

Prow

The way it works on prow :

  • If the PR submitted is part of the OWNERS file (or a group), the user would be allowed to run the CI.
  • If the user is not in the OWNERS file, someone else from the OWNERS is able to issue a :
    /ok-to-test command in the comment to let the PR be run in CI.

POC

The way it works in POC (https://github.com/chmouel/tekton-asa-code#ownership):

  • By default all pull request are denied unless the repo owner is submitting them or explictely allowed.
  • if allowed user issue a /tekton ok-to-test the PR will be allowed.
  • Collaborators of a repository are automatically allowed to run the CI.
  • tekton-asa-code will try to find a OWNERS file at the root of the .tekton directory in the main branch (i.e: as defined in Github).
  • If the user who submitted the PR is in that file the PR will be allowed.
  • If there is a line starting with @ (ie: @google) it will query the github organisation membership of the user who submitted the PR and allows it if the user is part of that organisation.
  • Same configuration can be applied directly in tekton.yaml configuration files under the owners sections, i.e:
owners:
    - @tektoncd
    - other_user_outside_of_tektoncd_github_org

Add Commit message in the metadata of pipelinerun

Add commit message to the PLR labels/annotations, so the list of pipeline Runs in the Repository page can have commit message shown on the tooltip, while hovering over the SHA field.

Requirement: SHA will be added to the column and its tooltip will have the commit message.

image

Tekton.yaml format

This is a issue to document the tekton.yaml format.

The file is inside .tekton/tekton.yaml and is optional.

The tasks and namespace section is already implemented

Here is the proposed syntax :

# Versioning is mandatory, follows k8 convention. This allows content
# negotiation and moving forward to new file format version in the future.
version: v1alpha1

# Events filtering, Loosely based on GitHub actions.  pipeline_context is a
# directory with a bunch of tekton yaml insides it.
on:
  # Match on branch push
  branch:
    # main
    main:
      - pipeline_context: ".tekton/main"

    # Greedy matching
    releases/*:
      - pipeline_context: ".tekton/release"

  # Mostly same syntax than on branch  but match tags
  tags:
    - v1:
        - pipeline_context: ".tekton/v1"
        
  pull_request:
    # match by labels
    labels:
      nightly_ci:
        - pipeline_context: ".tekton/nightly-ci"

    # default pipeline context 
    pipeline_context: ".tekton"
      
          
# Optional namespace to make sure all pipelines are started from
# there. (implemented)
namespace: namespace 
          
# Tasks injection, which gets merged inside the pipelinerun via
# resolve. (Already implemented)
tasks:
  - task1 # Fetch from hub the latest version of the task named task1
  - task2:0.2 # Fetch from hub version 0.2 of the task named task2
  - https://domain/task.yaml # Fetch from hub version 0.2 of the task named task2
  - taskdir/file.yaml # Grab the file.yaml inside the repo where is this tekton.yaml

Replace workflow action by pipelines-as-code

When we get event filtering on branch push we can remove the github action by a PAC pipeline.

The action is only here to regenerate the codecov report on main branch so there is comparisons on PR to base on.

Make sure that the token is validated before using it.

or we would crash :

goroutine 1 [running]:
github.com/openshift-pipelines/pipelines-as-code/pkg/webvcs.GithubVCS.CreateStatus(0x1e4d7b0, 0xc000124010, 0xc000692dc0, 0xc000197110, 0x1be850c, 0x9, 0x1be66ba, 0x7, 0xc0000662c0, 0x294, ...)
/home/chmouel/GIT/go/src/github.com/openshift-pipelines/pipelines-as-code/pkg/webvcs/github.go:234 +0x2cf
github.com/openshift-pipelines/pipelines-as-code/pkg/cmd/pipelineascode.runWrap(0x1e4dba0, 0xc000205860, 0xc000197100, 0x7ffd16b0b6ac, 0x1d8)
/home/chmouel/GIT/go/src/github.com/openshift-pipelines/pipelines-as-code/pkg/cmd/pipelineascode/pipelineascode.go:89 +0x25e
github.com/openshift-pipelines/pipelines-as-code/pkg/cmd/pipelineascode.Command.func2(0xc0001b5680, 0xc0007129a0, 0x0, 0xe, 0x0, 0x0)
/home/chmouel/GIT/go/src/github.com/openshift-pipelines/pipelines-as-code/pkg/cmd/pipelineascode/pipelineascode.go:38 +0xf0
github.com/spf13/cobra.(*Command).execute(0xc0001b5680, 0xc0007128c0, 0xe, 0xe, 0xc0001b5680, 0xc0007128c0)
/home/chmouel/GIT/go/src/github.com/openshift-pipelines/pipelines-as-code/vendor/github.com/spf13/cobra/command.go:852 +0x472
github.com/spf13/cobra.(*Command).ExecuteC(0xc0001b5180, 0xc00012a010, 0xf, 0xf)
/home/chmouel/GIT/go/src/github.com/openshift-pipelines/pipelines-as-code/vendor/github.com/spf13/cobra/command.go:960 +0x375
github.com/spf13/cobra.(*Command).Execute(...)
/home/chmouel/GIT/go/src/github.com/openshift-pipelines/pipelines-as-code/vendor/github.com/spf13/cobra/command.go:897
main.main()
/home/chmouel/GIT/go/src/github.com/openshift-pipelines/pipelines-as-code/cmd/pipelines-as-code/main.go:21 +0x1c6

Kustomize support

Investigate how kustomize can fit in here.

Should we just layer on top of it?

Fix SA not being able to cleanup pipeline

There was an issue validating the commit: "pipelineruns.tekton.dev "scratch-my-back-pr-g48r2" is forbidden: User "system:serviceaccount:pipelines-as-code:pipelines-as-code-sa-el" cannot delete resource "pipelineruns" in API group "tekton.dev" in the namespace "scratch-my-back""

Refactoring: Remove the passing by argument keep only payload

I tought it would be a good idea to be able to specify by arguments, but doing this via payload is much nicer and handle much more case while using github go client. So let's remove all the argument parsing stuff and keep only passing by payload file.

BitBucket support

PaC is a great project - exactly what is missing for proper CI/CD with OpenShift Pipelines and Tekton. One thing that would be great to have is a BitBucket integration :-)

rerequest from Github checks interface is not working on push

triggers spits this out :

{"level":"error","ts":"2021-06-02T15:48:55.186Z","logger":"eventlistener","caller":"sink/sink.go:232","msg":"failed to ApplyEventValuesToParams: failed to replace JSONPath value for param pull_request_number: $(body.check_run.check_suite.pull_requests[0].number): array index out of bounds: index 0, length 0","knative.dev/controller":"eventlistener","/triggers-eventid":"a2f50cba-a52a-4c9e-9a91-a4d99b961317","/trigger":"github-check-run-recheck","stacktrace":"github.com/tektoncd/triggers/pkg/sink.Sink.processTrigger\n\t/opt/app-root/src/go/src/github.com/tektoncd/triggers/pkg/sink/sink.go:232\ngithub.com/tektoncd/triggers/pkg/sink.Sink.HandleEvent.func1\n\t/opt/app-root/src/go/src/github.com/tektoncd/triggers/pkg/sink/sink.go:127"}

Pipelines gets created twice on a pull request push when it's not coming from a fork

If a user has a repo and push a branch to a repo for a PR we gets two different PipelineRun created.

Since we intercept the push (for pushing to main nightly branch etc) a pipelinerun gets created before the filtering is done to see if it match to repository.

Since we update (or syncronize) the pull_request another PR gets created.

The problem is that another pipelinerun gets created and update the status even tho it cannot find a match for it (or hopefully it shouldnt)

Update trigger el syntax to latest

Triggers >0.13 changed its syntax, the old syntax is still backward compatible but since openshift-pipelines have now a version that is >0.13 let's switch to it.

Send a codecov report on push too

Codecov seems broken and not reporting properly, since we moved to tekton CI for push instead of workflow we didn't reimport the codecov report. And with that caching go module it should be quick.

Namespaced Repository CRD

Currently the Repositories CRD are Cluster scoped and doesn't check ownership of the namespace where it runs.

Problem

We query all of them and match them to the Payload request. So if the payload comes from the repository URL https://github.com/org/user we go over every CRD on clusters that match that url and launch it in the specified namespace from the CRD.

The problem is when you have a malicious user on cluster who would said :

Match this URL to that namespace of this other user which I want to exploit. And have a pipeline that grabs things in that other user namespace.

Possible Solution

As a possible solution we could switch the Repository CRD to "Namespaced" and only launch the Pipeline in the namespace where the user installed it.

In that way a CRD matching can only be made to a namespace the user belongs too.

Caveats

Since we are running namespaced, tools are not going to be able to say :

Show me all my repositories since they can be to multiple namespaces.

The Project api from OpenShift is here for that, since as user we have the permission to list all our projects/namespace created and tools can check for Repositories in there.

The only drawback is that we would be tight to a OpenShift implementation and would mean it would be hard to support it on Kubernetes unless we would only list the current namespace repositories.

UI Recheck are not working since triggers is being silly while parsing the event from GH

image

Event JSON
{
  "action": "rerequested",
  "check_run": {
    "id": 2528852876,
    "node_id": "MDg6Q2hlY2tSdW4yNTI4ODUyODc2",
    "head_sha": "d5cb4a13749b577f27bdc43166154bb6c2aa48ae",
    "external_id": "",
    "url": "https://api.github.com/repos/chmouel/scratchmyback/check-runs/2528852876",
    "html_url": "https://github.com/chmouel/scratchmyback/runs/2528852876",
    "details_url": "https://console-openshift-console.apps.paac.devcluster.openshift.com/k8s/ns/scratch-my-back/tekton.dev~v1beta1~PipelineRun/scratch-my-back-pr-9trkl/logs",
    "status": "completed",
    "conclusion": "failure",
    "started_at": "2021-05-07T15:47:38Z",
    "completed_at": "2021-05-07T15:47:48Z",
    "output": {
      "title": "😞 CI Run: Failed",
      "summary": "Tekon CI has <b>failed</b>.",
      "text": "\n| Status | Duration | Name |\n| --- | --- | --- |\n✅ Succeeded | 8 seconds | task-from-remote\n✅ Succeeded | 8 seconds | task-from-tektondir\n✅ Succeeded | 7 seconds | task-referenced-internally\n❌ Failed | 8 seconds | task-spec\n\n<hr>\n\n<details>\n <summary>🗒️ More detailed status</summary>\n <pre>Name:              scratch-my-back-pr-9trkl\nNamespace:         scratch-my-back\nService Account:   pipeline\nTimeout:           1h0m0s\nLabels:\n tekton.dev/pipeline=scratch-my-back-pr-9trkl\n tekton.dev/pipeline-ascode-branch=main\n tekton.dev/pipeline-ascode-owner=chmouel\n tekton.dev/pipeline-ascode-repository=scratchmyback\n tekton.dev/pipeline-ascode-sender=chmouel\n tekton.dev/pipeline-ascode-sha=d5cb4a13749b577f27bdc43166154bb6c2aa48ae\n\nStatus\n\nSTARTED         DURATION    STATUS\n9 seconds ago   8 seconds   Failed\n\nMessage\n\nTasks Completed: 4 (Failed: 1, Cancelled 0), Skipped: 0 (\"step-task-spec\" exited with code 1 (image: \"registry.access.redhat.com/ubi8/ubi-minimal@sha256:5cfbaf45ca96806917830c183e9f37df2e913b187aadb32e89fd83fa455ebaa6\"); for logs run: kubectl -n scratch-my-back logs scratch-my-back-pr-9trkl-task-spec-kjg9d-pod-cw5pq -c step-task-spec\n)\n\nResources\n\n No resources\n\nParams\n\n NAME       VALUE\n repo_url   https://github.com/chmouel/scratchmyback\n revision   d5cb4a13749b577f27bdc43166154bb6c2aa48ae\n\nResults\n\n No results\n\nWorkspaces\n\n NAME     SUB PATH   WORKSPACE BINDING\n source   ---        VolumeClaimTemplate\n\nTaskruns\n\n NAME                                                        TASK NAME                    STARTED         DURATION    STATUS\n scratch-my-back-pr-9trkl-task-from-tektondir-n86f4          task-from-tektondir          9 seconds ago   8 seconds   Succeeded\n scratch-my-back-pr-9trkl-task-referenced-internally-bm7p8   task-referenced-internally   9 seconds ago   7 seconds   Succeeded\n scratch-my-back-pr-9trkl-task-spec-kjg9d                    task-spec                    9 seconds ago   8 seconds   Failed\n scratch-my-back-pr-9trkl-task-from-remote-kh9kr             task-from-remote             9 seconds ago   8 seconds   Succeeded\n</pre>\n</details>\n\n",
      "annotations_count": 0,
      "annotations_url": "https://api.github.com/repos/chmouel/scratchmyback/check-runs/2528852876/annotations"
    },
    "name": "Tekton Pipeline as Code CI",
    "check_suite": {
      "id": 2678187975,
      "node_id": "MDEwOkNoZWNrU3VpdGUyNjc4MTg3OTc1",
      "head_branch": "tektonci",
      "head_sha": "d5cb4a13749b577f27bdc43166154bb6c2aa48ae",
      "status": "queued",
      "conclusion": null,
      "url": "https://api.github.com/repos/chmouel/scratchmyback/check-suites/2678187975",
      "before": "b865bbe983f9a0cedd2512122b3756af2edaa3e8",
      "after": "d5cb4a13749b577f27bdc43166154bb6c2aa48ae",
      "pull_requests": [
        {
          "url": "https://api.github.com/repos/chmouel/scratchmyback/pulls/1",
          "id": 617467616,
          "number": 1,
          "head": {
            "ref": "tektonci",
            "sha": "d5cb4a13749b577f27bdc43166154bb6c2aa48ae",
            "repo": {
              "id": 359117785,
              "url": "https://api.github.com/repos/chmouel/scratchmyback",
              "name": "scratchmyback"
            }
          },
          "base": {
            "ref": "main",
            "sha": "4b8bbc927fae0c4dda82de8d69346f268e68e6e5",
            "repo": {
              "id": 359117785,
              "url": "https://api.github.com/repos/chmouel/scratchmyback",
              "name": "scratchmyback"
            }
          }
        }
      ],
      "app": {
        "id": 112592,
        "slug": "pipelines-as-code",
        "node_id": "MDM6QXBwMTEyNTky",
        "owner": {
          "login": "openshift-pipelines",
          "id": 57996262,
          "node_id": "MDEyOk9yZ2FuaXphdGlvbjU3OTk2MjYy",
          "avatar_url": "https://avatars.githubusercontent.com/u/57996262?v=4",
          "gravatar_id": "",
          "url": "https://api.github.com/users/openshift-pipelines",
          "html_url": "https://github.com/openshift-pipelines",
          "followers_url": "https://api.github.com/users/openshift-pipelines/followers",
          "following_url": "https://api.github.com/users/openshift-pipelines/following{/other_user}",
          "gists_url": "https://api.github.com/users/openshift-pipelines/gists{/gist_id}",
          "starred_url": "https://api.github.com/users/openshift-pipelines/starred{/owner}{/repo}",
          "subscriptions_url": "https://api.github.com/users/openshift-pipelines/subscriptions",
          "organizations_url": "https://api.github.com/users/openshift-pipelines/orgs",
          "repos_url": "https://api.github.com/users/openshift-pipelines/repos",
          "events_url": "https://api.github.com/users/openshift-pipelines/events{/privacy}",
          "received_events_url": "https://api.github.com/users/openshift-pipelines/received_events",
          "type": "Organization",
          "site_admin": false
        },
        "name": "Pipelines as Code",
        "description": "Pipelines as code application to validate your repositories with OpenShift Pipelines",
        "external_url": "https://github.com/openshift-pipelines/pipelines-as-code/",
        "html_url": "https://github.com/apps/pipelines-as-code",
        "created_at": "2021-04-27T12:15:44Z",
        "updated_at": "2021-04-27T12:25:57Z",
        "permissions": {
          "administration": "read",
          "checks": "write",
          "contents": "write",
          "issues": "write",
          "members": "read",
          "metadata": "read",
          "organization_plan": "read",
          "pull_requests": "write"
        },
        "events": [
          "commit_comment",
          "issue_comment",
          "pull_request",
          "pull_request_review",
          "pull_request_review_comment",
          "push"
        ]
      },
      "created_at": "2021-05-07T15:47:24Z",
      "updated_at": "2021-05-07T15:53:17Z"
    },
    "app": {
      "id": 112592,
      "slug": "pipelines-as-code",
      "node_id": "MDM6QXBwMTEyNTky",
      "owner": {
        "login": "openshift-pipelines",
        "id": 57996262,
        "node_id": "MDEyOk9yZ2FuaXphdGlvbjU3OTk2MjYy",
        "avatar_url": "https://avatars.githubusercontent.com/u/57996262?v=4",
        "gravatar_id": "",
        "url": "https://api.github.com/users/openshift-pipelines",
        "html_url": "https://github.com/openshift-pipelines",
        "followers_url": "https://api.github.com/users/openshift-pipelines/followers",
        "following_url": "https://api.github.com/users/openshift-pipelines/following{/other_user}",
        "gists_url": "https://api.github.com/users/openshift-pipelines/gists{/gist_id}",
        "starred_url": "https://api.github.com/users/openshift-pipelines/starred{/owner}{/repo}",
        "subscriptions_url": "https://api.github.com/users/openshift-pipelines/subscriptions",
        "organizations_url": "https://api.github.com/users/openshift-pipelines/orgs",
        "repos_url": "https://api.github.com/users/openshift-pipelines/repos",
        "events_url": "https://api.github.com/users/openshift-pipelines/events{/privacy}",
        "received_events_url": "https://api.github.com/users/openshift-pipelines/received_events",
        "type": "Organization",
        "site_admin": false
      },
      "name": "Pipelines as Code",
      "description": "Pipelines as code application to validate your repositories with OpenShift Pipelines",
      "external_url": "https://github.com/openshift-pipelines/pipelines-as-code/",
      "html_url": "https://github.com/apps/pipelines-as-code",
      "created_at": "2021-04-27T12:15:44Z",
      "updated_at": "2021-04-27T12:25:57Z",
      "permissions": {
        "administration": "read",
        "checks": "write",
        "contents": "write",
        "issues": "write",
        "members": "read",
        "metadata": "read",
        "organization_plan": "read",
        "pull_requests": "write"
      },
      "events": [
        "commit_comment",
        "issue_comment",
        "pull_request",
        "pull_request_review",
        "pull_request_review_comment",
        "push"
      ]
    },
    "pull_requests": [
      {
        "url": "https://api.github.com/repos/chmouel/scratchmyback/pulls/1",
        "id": 617467616,
        "number": 1,
        "head": {
          "ref": "tektonci",
          "sha": "d5cb4a13749b577f27bdc43166154bb6c2aa48ae",
          "repo": {
            "id": 359117785,
            "url": "https://api.github.com/repos/chmouel/scratchmyback",
            "name": "scratchmyback"
          }
        },
        "base": {
          "ref": "main",
          "sha": "4b8bbc927fae0c4dda82de8d69346f268e68e6e5",
          "repo": {
            "id": 359117785,
            "url": "https://api.github.com/repos/chmouel/scratchmyback",
            "name": "scratchmyback"
          }
        }
      }
    ]
  },
  "repository": {
    "id": 359117785,
    "node_id": "MDEwOlJlcG9zaXRvcnkzNTkxMTc3ODU=",
    "name": "scratchmyback",
    "full_name": "chmouel/scratchmyback",
    "private": false,
    "owner": {
      "login": "chmouel",
      "id": 98980,
      "node_id": "MDQ6VXNlcjk4OTgw",
      "avatar_url": "https://avatars.githubusercontent.com/u/98980?v=4",
      "gravatar_id": "",
      "url": "https://api.github.com/users/chmouel",
      "html_url": "https://github.com/chmouel",
      "followers_url": "https://api.github.com/users/chmouel/followers",
      "following_url": "https://api.github.com/users/chmouel/following{/other_user}",
      "gists_url": "https://api.github.com/users/chmouel/gists{/gist_id}",
      "starred_url": "https://api.github.com/users/chmouel/starred{/owner}{/repo}",
      "subscriptions_url": "https://api.github.com/users/chmouel/subscriptions",
      "organizations_url": "https://api.github.com/users/chmouel/orgs",
      "repos_url": "https://api.github.com/users/chmouel/repos",
      "events_url": "https://api.github.com/users/chmouel/events{/privacy}",
      "received_events_url": "https://api.github.com/users/chmouel/received_events",
      "type": "User",
      "site_admin": false
    },
    "html_url": "https://github.com/chmouel/scratchmyback",
    "description": "Back scratching",
    "fork": false,
    "url": "https://api.github.com/repos/chmouel/scratchmyback",
    "forks_url": "https://api.github.com/repos/chmouel/scratchmyback/forks",
    "keys_url": "https://api.github.com/repos/chmouel/scratchmyback/keys{/key_id}",
    "collaborators_url": "https://api.github.com/repos/chmouel/scratchmyback/collaborators{/collaborator}",
    "teams_url": "https://api.github.com/repos/chmouel/scratchmyback/teams",
    "hooks_url": "https://api.github.com/repos/chmouel/scratchmyback/hooks",
    "issue_events_url": "https://api.github.com/repos/chmouel/scratchmyback/issues/events{/number}",
    "events_url": "https://api.github.com/repos/chmouel/scratchmyback/events",
    "assignees_url": "https://api.github.com/repos/chmouel/scratchmyback/assignees{/user}",
    "branches_url": "https://api.github.com/repos/chmouel/scratchmyback/branches{/branch}",
    "tags_url": "https://api.github.com/repos/chmouel/scratchmyback/tags",
    "blobs_url": "https://api.github.com/repos/chmouel/scratchmyback/git/blobs{/sha}",
    "git_tags_url": "https://api.github.com/repos/chmouel/scratchmyback/git/tags{/sha}",
    "git_refs_url": "https://api.github.com/repos/chmouel/scratchmyback/git/refs{/sha}",
    "trees_url": "https://api.github.com/repos/chmouel/scratchmyback/git/trees{/sha}",
    "statuses_url": "https://api.github.com/repos/chmouel/scratchmyback/statuses/{sha}",
    "languages_url": "https://api.github.com/repos/chmouel/scratchmyback/languages",
    "stargazers_url": "https://api.github.com/repos/chmouel/scratchmyback/stargazers",
    "contributors_url": "https://api.github.com/repos/chmouel/scratchmyback/contributors",
    "subscribers_url": "https://api.github.com/repos/chmouel/scratchmyback/subscribers",
    "subscription_url": "https://api.github.com/repos/chmouel/scratchmyback/subscription",
    "commits_url": "https://api.github.com/repos/chmouel/scratchmyback/commits{/sha}",
    "git_commits_url": "https://api.github.com/repos/chmouel/scratchmyback/git/commits{/sha}",
    "comments_url": "https://api.github.com/repos/chmouel/scratchmyback/comments{/number}",
    "issue_comment_url": "https://api.github.com/repos/chmouel/scratchmyback/issues/comments{/number}",
    "contents_url": "https://api.github.com/repos/chmouel/scratchmyback/contents/{+path}",
    "compare_url": "https://api.github.com/repos/chmouel/scratchmyback/compare/{base}...{head}",
    "merges_url": "https://api.github.com/repos/chmouel/scratchmyback/merges",
    "archive_url": "https://api.github.com/repos/chmouel/scratchmyback/{archive_format}{/ref}",
    "downloads_url": "https://api.github.com/repos/chmouel/scratchmyback/downloads",
    "issues_url": "https://api.github.com/repos/chmouel/scratchmyback/issues{/number}",
    "pulls_url": "https://api.github.com/repos/chmouel/scratchmyback/pulls{/number}",
    "milestones_url": "https://api.github.com/repos/chmouel/scratchmyback/milestones{/number}",
    "notifications_url": "https://api.github.com/repos/chmouel/scratchmyback/notifications{?since,all,participating}",
    "labels_url": "https://api.github.com/repos/chmouel/scratchmyback/labels{/name}",
    "releases_url": "https://api.github.com/repos/chmouel/scratchmyback/releases{/id}",
    "deployments_url": "https://api.github.com/repos/chmouel/scratchmyback/deployments",
    "created_at": "2021-04-18T10:57:45Z",
    "updated_at": "2021-04-29T10:19:59Z",
    "pushed_at": "2021-05-07T15:47:24Z",
    "git_url": "git://github.com/chmouel/scratchmyback.git",
    "ssh_url": "[email protected]:chmouel/scratchmyback.git",
    "clone_url": "https://github.com/chmouel/scratchmyback.git",
    "svn_url": "https://github.com/chmouel/scratchmyback",
    "homepage": null,
    "size": 21,
    "stargazers_count": 0,
    "watchers_count": 0,
    "language": null,
    "has_issues": true,
    "has_projects": true,
    "has_downloads": true,
    "has_wiki": true,
    "has_pages": false,
    "forks_count": 0,
    "mirror_url": null,
    "archived": false,
    "disabled": false,
    "open_issues_count": 1,
    "license": {
      "key": "gpl-3.0",
      "name": "GNU General Public License v3.0",
      "spdx_id": "GPL-3.0",
      "url": "https://api.github.com/licenses/gpl-3.0",
      "node_id": "MDc6TGljZW5zZTk="
    },
    "forks": 0,
    "open_issues": 1,
    "watchers": 0,
    "default_branch": "main"
  },
  "sender": {
    "login": "chmouel",
    "id": 98980,
    "node_id": "MDQ6VXNlcjk4OTgw",
    "avatar_url": "https://avatars.githubusercontent.com/u/98980?v=4",
    "gravatar_id": "",
    "url": "https://api.github.com/users/chmouel",
    "html_url": "https://github.com/chmouel",
    "followers_url": "https://api.github.com/users/chmouel/followers",
    "following_url": "https://api.github.com/users/chmouel/following{/other_user}",
    "gists_url": "https://api.github.com/users/chmouel/gists{/gist_id}",
    "starred_url": "https://api.github.com/users/chmouel/starred{/owner}{/repo}",
    "subscriptions_url": "https://api.github.com/users/chmouel/subscriptions",
    "organizations_url": "https://api.github.com/users/chmouel/orgs",
    "repos_url": "https://api.github.com/users/chmouel/repos",
    "events_url": "https://api.github.com/users/chmouel/events{/privacy}",
    "received_events_url": "https://api.github.com/users/chmouel/received_events",
    "type": "User",
    "site_admin": false
  },
  "installation": {
    "id": 1234,
    "node_id": "MDIzOkludGVncmF0aW9uSW5zdGFsbGF0aW9uMTY1NTYzODI="
  }
}

Handle /retest

We may have support for re-run from github but we still need /retest for some use case, (and for other vcs which would not have it)

Add a "tkn-pac repo logs" command

Add a tkn-pac repo logs command which :

Get the last run with a pipelinerun logs and automatically send it to "tkn pr logs"

with option -w open a browser to the Web Console Log URL,

Private repository are not working

When the Webhook is coming from a private repository we are all good for API operation and initial check if there is .tekton/ repository since we have a token generated from the private key.

But when doing git-clone from the PipelineRun user would need to have a token or ssh to be able to checkout this git-clone from its pipelinerun.

To make it easier so the user would not need to install a new Secret and use the short live token generated from github-app, we could automatically create a new secret in the target namespace :

pac-git-basic-auth-${OWNER}-${REPOSITORY}

we target OWNER REPOSITORY so there is no mixup when different github user target same namespace but don't have the same rights (which is a edge case)

We need them to document to the user to modify her pipeline/pipelinrun to add the optional basic-auth volume to the git-clone task, like for example :

the modification would look like this :

diff --git a/.tekton/pipeline.yaml b/.tekton/pipeline.yaml
index 99922bd..c8e10f4 100644
--- a/.tekton/pipeline.yaml
+++ b/.tekton/pipeline.yaml
@@ -6,6 +6,7 @@ metadata:
 spec:
   workspaces:
   - name: source
+  - name: basic-auth
   params:
     - name: repo_url
     - name: revision
@@ -44,3 +45,5 @@ spec:
       workspaces:
         - name: output
           workspace: source
+        - name: basic-auth
+          workspace: basic-auth
diff --git a/.tekton/run.yaml b/.tekton/run.yaml
index c09df69..ee2518d 100644
--- a/.tekton/run.yaml
+++ b/.tekton/run.yaml
@@ -26,3 +26,6 @@ spec:
         resources:
           requests:
             storage: 1Gi
+  - name: basic-auth
+    secret:
+      secretName: pac-git-basic-auth-openshift-pipelines-pac-test

git-clone task would automatically pick this up as documented in :

https://github.com/tektoncd/catalog/blob/main/task/git-clone/0.4/README.md#using-basic-auth-credentials

TODO :

  • Add {{repo_owner}} {{repo_name}} as template variable
  • Update serviceaccount to be able to create/update/get secrets in repos.
  • Generate secret pac-git-basic-auth-${REPO_OWNER}-${REPO_NAME}
  • Document it in README
  • Add E2E test on GHE
  • Add an option to disable the behaviour

Fully qualified url in the labels for SHA and url-repository.

Based on the ODC Pipelines meetings, It is decided that SHA should be a clickable link that takes the user to the original commit in the VCS. Right now the labels doesn't have the full qualified URL, so frontend do not have a way to determine if it is from github, gitlab, etc.

labels:
   pipelinesascode.tekton.dev/sha: 1f06540627c00f72a31bcd7182712d447eaee24f
   pipelinesascode.tekton.dev/url-org: karthikjeeyar
   pipelinesascode.tekton.dev/url-repository: demo-app

Request is to add url-repository and sha to have the fully qualified url or new labels that contain the full url.

image

tkn-pac resolve

allow to automatically detect from the current dir the revision and git repo instead of having the user specifying it,

UI rerequest is failing

E2E test failing and experienced as well from the UI :

image

Details
=== RUN   TestPullRerequest
{"level":"info","ts":1628585546.407639,"caller":"repository/create.go:23","msg":"Repository created in pac-e2e-ns-67sgf"}
{"level":"info","ts":1628585547.401167,"caller":"test/pullrequest_rerequest_test.go:85","msg":"Commit e92c10ecb0a481d7ac2c028a7b37f81ad6ba914d has been created and pushed to refs/heads/pac-e2e-test-qldjd"}
{"level":"info","ts":1628585548.1439874,"caller":"github/pr.go:86","msg":"Pull request created: https://ghe.pipelines.devcluster.openshift.com/pac/pac-e2e/pull/109"}
{"level":"info","ts":1628585548.1440327,"caller":"test/pullrequest_rerequest_test.go:92","msg":"Waiting for Repository to be updated"}
coverage: [no statements]
panic: test timed out after 10m0s

goroutine 725 [running]:
testing.(*M).startAlarm.func1()
	/home/linuxbrew/.linuxbrew/opt/go/libexec/src/testing/testing.go:1700 +0xe5
created by time.goFunc
	/home/linuxbrew/.linuxbrew/opt/go/libexec/src/time/sleep.go:180 +0x45

goroutine 1 [chan receive, 8 minutes]:
testing.(*T).Run(0xc000981500, 0x1761e3b, 0x11, 0x17fcfa8, 0x498601)
	/home/linuxbrew/.linuxbrew/opt/go/libexec/src/testing/testing.go:1239 +0x2da
testing.runTests.func1(0xc000381e00)
	/home/linuxbrew/.linuxbrew/opt/go/libexec/src/testing/testing.go:1511 +0x78
testing.tRunner(0xc000381e00, 0xc000187d20)
	/home/linuxbrew/.linuxbrew/opt/go/libexec/src/testing/testing.go:1193 +0xef
testing.runTests(0xc000454348, 0x22aa6e0, 0x5, 0x5, 0xc03cadee71ebac1f, 0x8bb3238f2b, 0x22bd980, 0x151ee60)
	/home/linuxbrew/.linuxbrew/opt/go/libexec/src/testing/testing.go:1509 +0x2fe
testing.(*M).Run(0xc0003eab00, 0x0)
	/home/linuxbrew/.linuxbrew/opt/go/libexec/src/testing/testing.go:1417 +0x1eb
github.com/openshift-pipelines/pipelines-as-code/test.TestMain(0xc0003eab00)
	/home/chmouel/GIT/go/src/github.com/openshift-pipelines/pipelines-as-code/test/init_test.go:87 +0xe5
main.main()
	_testmain.go:95 +0x1f6

goroutine 19 [chan receive]:
k8s.io/klog/v2.(*loggingT).flushDaemon(0x22bdca0)
	/home/chmouel/GIT/go/src/github.com/openshift-pipelines/pipelines-as-code/vendor/k8s.io/klog/v2/klog.go:1169 +0x8b
created by k8s.io/klog/v2.init.0
	/home/chmouel/GIT/go/src/github.com/openshift-pipelines/pipelines-as-code/vendor/k8s.io/klog/v2/klog.go:417 +0xdf

goroutine 204 [select]:
k8s.io/apimachinery/pkg/util/wait.WaitFor(0xc000155140, 0xc000245960, 0xc0008e9740, 0x0, 0x0)
	/home/chmouel/GIT/go/src/github.com/openshift-pipelines/pipelines-as-code/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:539 +0xf1
k8s.io/apimachinery/pkg/util/wait.pollInternal(0xc000155140, 0xc000245960, 0x0, 0x0)
	/home/chmouel/GIT/go/src/github.com/openshift-pipelines/pipelines-as-code/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:427 +0x87
k8s.io/apimachinery/pkg/util/wait.pollImmediateInternal(0xc000155140, 0xc000245960, 0xc000155140, 0xc00003c850)
	/home/chmouel/GIT/go/src/github.com/openshift-pipelines/pipelines-as-code/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:452 +0x74
k8s.io/apimachinery/pkg/util/wait.PollImmediate(0x3b9aca00, 0x8bb2c97000, 0xc000245960, 0xc03cae090895e932, 0xa473803a3f)
	/home/chmouel/GIT/go/src/github.com/openshift-pipelines/pipelines-as-code/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:441 +0x4d
github.com/openshift-pipelines/pipelines-as-code/pkg/kubeinteraction.PollImmediateWithContext(0x19455c8, 0xc00003c840, 0xc0002459c8, 0x19455c8, 0xc00003c840)
	/home/chmouel/GIT/go/src/github.com/openshift-pipelines/pipelines-as-code/pkg/kubeinteraction/wait.go:87 +0x7e
github.com/openshift-pipelines/pipelines-as-code/test/pkg/wait.UntilRepositoryUpdated(0x1945590, 0xc00011e010, 0x191d340, 0xc0001c6420, 0xc000394270, 0x10, 0xc000394270, 0x10, 0x0, 0x8bb2c97000, ...)
	/home/chmouel/GIT/go/src/github.com/openshift-pipelines/pipelines-as-code/test/pkg/wait/wait.go:17 +0x145
github.com/openshift-pipelines/pipelines-as-code/test.TestPullRerequest(0xc000981500)
	/home/chmouel/GIT/go/src/github.com/openshift-pipelines/pipelines-as-code/test/pullrequest_rerequest_test.go:93 +0xb87
testing.tRunner(0xc000981500, 0x17fcfa8)
	/home/linuxbrew/.linuxbrew/opt/go/libexec/src/testing/testing.go:1193 +0xef
created by testing.(*T).Run
	/home/linuxbrew/.linuxbrew/opt/go/libexec/src/testing/testing.go:1238 +0x2b3

goroutine 40 [IO wait]:
internal/poll.runtime_pollWait(0x7ff6c40fd6b0, 0x72, 0xffffffffffffffff)
	/home/linuxbrew/.linuxbrew/opt/go/libexec/src/runtime/netpoll.go:222 +0x55
internal/poll.(*pollDesc).wait(0xc00039e118, 0x72, 0x1200, 0x128b, 0xffffffffffffffff)
	/home/linuxbrew/.linuxbrew/opt/go/libexec/src/internal/poll/fd_poll_runtime.go:87 +0x45
internal/poll.(*pollDesc).waitRead(...)
	/home/linuxbrew/.linuxbrew/opt/go/libexec/src/internal/poll/fd_poll_runtime.go:92
internal/poll.(*FD).Read(0xc00039e100, 0xc00031b900, 0x128b, 0x128b, 0x0, 0x0, 0x0)
	/home/linuxbrew/.linuxbrew/opt/go/libexec/src/internal/poll/fd_unix.go:166 +0x1d5
net.(*netFD).Read(0xc00039e100, 0xc00031b900, 0x128b, 0x128b, 0x1286, 0xc00031b900, 0x5)
	/home/linuxbrew/.linuxbrew/opt/go/libexec/src/net/fd_posix.go:55 +0x4f
net.(*conn).Read(0xc000010010, 0xc00031b900, 0x128b, 0x128b, 0x0, 0x0, 0x0)
	/home/linuxbrew/.linuxbrew/opt/go/libexec/src/net/net.go:183 +0x91
crypto/tls.(*atLeastReader).Read(0xc00000e9c0, 0xc00031b900, 0x128b, 0x128b, 0x1286, 0x22be5c0, 0x0)
	/home/linuxbrew/.linuxbrew/opt/go/libexec/src/crypto/tls/conn.go:776 +0x63
bytes.(*Buffer).ReadFrom(0xc00042c278, 0x1915460, 0xc00000e9c0, 0x40d905, 0x15bb360, 0x16edcc0)
	/home/linuxbrew/.linuxbrew/opt/go/libexec/src/bytes/buffer.go:204 +0xbe
crypto/tls.(*Conn).readFromUntil(0xc00042c000, 0x1917360, 0xc000010010, 0x5, 0xc000010010, 0x326)
	/home/linuxbrew/.linuxbrew/opt/go/libexec/src/crypto/tls/conn.go:798 +0xf3
crypto/tls.(*Conn).readRecordOrCCS(0xc00042c000, 0x0, 0x0, 0x1)
	/home/linuxbrew/.linuxbrew/opt/go/libexec/src/crypto/tls/conn.go:605 +0x115
crypto/tls.(*Conn).readRecord(...)
	/home/linuxbrew/.linuxbrew/opt/go/libexec/src/crypto/tls/conn.go:573
crypto/tls.(*Conn).Read(0xc00042c000, 0xc000262000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/home/linuxbrew/.linuxbrew/opt/go/libexec/src/crypto/tls/conn.go:1276 +0x165
bufio.(*Reader).Read(0xc000372960, 0xc00024e2d8, 0x9, 0x9, 0x92df8b, 0xc00014bc78, 0x409005)
	/home/linuxbrew/.linuxbrew/opt/go/libexec/src/bufio/bufio.go:227 +0x222
io.ReadAtLeast(0x19152c0, 0xc000372960, 0xc00024e2d8, 0x9, 0x9, 0x9, 0xc0006347b0, 0x76b371d9fdb300, 0xc0006347b0)
	/home/linuxbrew/.linuxbrew/opt/go/libexec/src/io/io.go:328 +0x87
io.ReadFull(...)
	/home/linuxbrew/.linuxbrew/opt/go/libexec/src/io/io.go:347
golang.org/x/net/http2.readFrameHeader(0xc00024e2d8, 0x9, 0x9, 0x19152c0, 0xc000372960, 0x0, 0x0, 0x0, 0x0)
	/home/chmouel/GIT/go/src/github.com/openshift-pipelines/pipelines-as-code/vendor/golang.org/x/net/http2/frame.go:237 +0x89
golang.org/x/net/http2.(*Framer).ReadFrame(0xc00024e2a0, 0xc0006891d0, 0x0, 0x0, 0x0)
	/home/chmouel/GIT/go/src/github.com/openshift-pipelines/pipelines-as-code/vendor/golang.org/x/net/http2/frame.go:492 +0xa5
golang.org/x/net/http2.(*clientConnReadLoop).run(0xc00014bfa8, 0x0, 0x0)
	/home/chmouel/GIT/go/src/github.com/openshift-pipelines/pipelines-as-code/vendor/golang.org/x/net/http2/transport.go:1819 +0xd8
golang.org/x/net/http2.(*ClientConn).readLoop(0xc00051a480)
	/home/chmouel/GIT/go/src/github.com/openshift-pipelines/pipelines-as-code/vendor/golang.org/x/net/http2/transport.go:1741 +0x6f
created by golang.org/x/net/http2.(*Transport).newClientConn
	/home/chmouel/GIT/go/src/github.com/openshift-pipelines/pipelines-as-code/vendor/golang.org/x/net/http2/transport.go:705 +0x6c5

goroutine 216 [select]:
k8s.io/apimachinery/pkg/util/wait.poller.func1.1(0xc0008e9800, 0x3b9aca00, 0x8bb2c97000, 0xc0008e97a0)
	/home/chmouel/GIT/go/src/github.com/openshift-pipelines/pipelines-as-code/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:588 +0x135
created by k8s.io/apimachinery/pkg/util/wait.poller.func1
	/home/chmouel/GIT/go/src/github.com/openshift-pipelines/pipelines-as-code/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:571 +0x8c
FAIL	github.com/openshift-pipelines/pipelines-as-code/test	600.015s
FAIL
make: *** [Makefile:45: test-e2e] Error 1

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.