swisscom / gitlab-merge-request-resource

A concourse resource to check for new merge requests on GitLab

GitLab Merge Request Concourse Resource

A concourse resource to check for new merge requests on GitLab and update the merge request status.

Source Configuration

resource_types:
- name: merge-request
  type: docker-image
  source:
    repository: mastertinner/gitlab-merge-request-resource

resources:
- name: repo-mr
  type: merge-request
  source:
    uri: https://gitlab.com/myname/myproject.git
    private_token: XXX
    username: my_username
    password: xxx

• uri: The location of the repository (required)

• private_token: Your GitLab user's private token (required, can be found in your profile settings)

• private_key: The private SSH key for SSH auth when pulling

  Example:

  private_key: |
    -----BEGIN RSA PRIVATE KEY-----
    MIIEowIBAAKCAQEAtCS10/f7W7lkQaSgD/mVeaSOvSF9ql4hf/zfMwfVGgHWjj+W
    <Lots more text>
    DWiJL+OFeg9kawcUL6hQ8JeXPhlImG6RTUffma9+iGQyyBMCGd1l
    -----END RSA PRIVATE KEY-----

• username: The username for HTTP(S) auth when pulling

• password: The password for HTTP(S) auth when pulling

• no_ssl: Set to true if the GitLab API should be used over HTTP instead of HTTPS

• skip_ssl_verification: Optional. Connect to GitLab insecurely - i.e. skip SSL validation. Defaults to false if not provided.

Please note that you have to provide either private_key or username and password.

Behavior

check: Check for new merge requests

Checks if there are new merge requests or merge requests with new commits.

in: Clone merge request source branch

git clones the source branch of the respective merge request.

out: Update a merge request's merge status

Updates the merge request's merge_status which displays nicely in the GitLab UI and allows to only merge changes if they pass the test.

Parameters

• repository: The path of the repository of the merge request's source branch (required)
• status: The new status of the merge request (required, can be either pending, running, success, failed, or canceled)
• build_label: The label of the build in GitLab (optional, defaults to "Concourse")

Example

jobs:
- name: test-merge-request
  plan:
  - get: repo
    resource: repo-mr
    trigger: true
  - put: repo-mr
    params:
      repository: repo
      status: running
  - task: run-tests
    file: repo/ci/tasks/run-tests.yml
  on_failure:
    put: repo-mr
    params:
      repository: repo
      status: failed
  on_success:
    put: repo-mr
    params:
      repository: repo
      status: success

gitlab-merge-request-resource's Issues

jq: error (at <stdin>:0): strptime/1 requires string inputs and arguments

resource script '/opt/resource/check []' failed: exit status 5

stderr:
jq: error (at <stdin>:0): strptime/1 requires string inputs and arguments

Since the latest build 7 hours ago all resources based on this have been receiving this error. Rebuilding the resource in the pipe does not resolve the problem

Incorrect commit status being updated in put

I'm playing around with the example config to learn how to use this resource, but the put step appears to be updating the wrong commit. Here's a sample of my pipeline:

---
resource_types:
- name: merge-request
  type: docker-image
  source:
    repository: mastertinner/gitlab-merge-request-resource

resources:
- name: repo
  type: git
  source:
    uri: [email protected]:my-user/demo.git
    branch: master
    private_key: ((gitlab-private-key))
- name: repo-mr
  type: merge-request
  source:
    uri: [email protected]:my-user/demo.git
    private_token: ((gitlab-personal-token))
    private_key: ((gitlab-private-key))

jobs:
- name: test-merge-request
  plan:
  - get: repo
  - get: repo-mr
    trigger: true
  - put: repo-mr
    params:
      repository: repo
      status: running
  on_failure:
    put: repo-mr
    params:
      repository: repo
      status: failed
  on_success:
    put: repo-mr
    params:
      repository: repo
      status: success

Suppose the latest commit on master of my repo is aaa111. I create a new feature-branch, make a new commit bbb222, and create a new merge request from this. The Concourse pipeline picks up the merge request and correct commit (bbb222) in check and get. However, put updates the commit status of the latest commit on master (aaa111), instead of my merge request commit (bbb222). Should bbb222's commit status be updated instead?

Different timezone causes checks to skip merge requests

I'm on an older version of GitLab - v10.8.3 - and the commits API returns the committed_date using the committer's time zone. e.g. "2018-12-06T17:17:38.214-05:00". However, the latest version of GitLab looks like it's returning the timezone in UTC ("2018-12-06T16:08:35.000Z", src).

check currently checks the first 19 characters of committed_date, which works if all times are returned in UTC. However, this causes the check to miss particular merge requests when the commits are in different zones (with older versions of GitLab). I can't seem to find which release of GitLab changed the committed_date format, but I believe it's related to this issue.

`file not found` after restart

If I restart my pipeline or reboot my server after an upgrade I get a file not found error in the check step of all merge-request resources.
The only solution I could find at the moment is destroying all pipelines using the resource and setting them again.

Add support for merging MR

It'd be nice to be able to merge the MR from this resource.

Resource fails when MR with deleted source branch exists

When in Gitlab a merge request is open but its source branch was deleted, the Gitlab API returns the following json:

{
        "id": 105,
        "iid": 47,
        "project_id": 21,
        "title": "Feature/xxxxx",
        "description": "xxxxx",
        "state": "opened",
        "created_at": "2017-05-03T13:55:50.786Z",
        "updated_at": "2017-05-08T09:36:02.817Z",
        "target_branch": "development",
        "source_branch": "feature/xxxx",
        "upvotes": 0,
        "downvotes": 0,
        "author": {
            "name": "xxxxxx",
            "username": "xxxxx",
            "id": 8,
            "state": "active",
            "avatar_url": "xxxxx",
            "web_url": "xxxxx"
        },
        "assignee": {
            "name": "xxxxxx",
            "username": "xxxxx",
            "id": 8,
            "state": "active",
            "avatar_url": "xxxxx",
            "web_url": "xxxxx"
        },
        "source_project_id": 21,
        "target_project_id": 21,
        "labels": [

        ],
        "work_in_progress": false,
        "milestone": null,
        "merge_when_build_succeeds": false,
        "merge_status": "unchecked",
        "sha": null,
        "merge_commit_sha": null,
        "subscribed": false,
        "user_notes_count": 2,
        "should_remove_source_branch": null,
        "force_remove_source_branch": false,
        "web_url": "http://gitlab.company.de/proj/path/merge_requests/47"
    }

Because the sha value is null the resource is failing in the check step and no merge request can be build.

resource script '/opt/resource/check []' failed: exit status 128

stderr:
Already up-to-date.
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100  3440  100  3440    0     0  19355      0 --:--:-- --:--:-- --:--:-- 19545
fatal: ambiguous argument 'null': unknown revision or path not in the working tree.
Use '--' to separate paths from revisions, like this:
'git <command> [<revision>...] -- [<file>...]'

The problematic line is https://github.com/swisscom/gitlab-merge-request-resource/blob/master/scripts/check#L72

Maybe it would be good to check for this case and just skip this merge request with a warning message.

PS:
Thank you for the plugin, it works great! 👍

Rebasing on an open MR

I recently ran into an edge case where one of my team members rebased their commit history on an open merge request (not best practice), resulting in the resource to fail.

What is the expected behaviour if a commit hash is removed from the history while a merge request is open?

Thank you for this resource and time!

HTTPS/HTTP URI

Hi @mastertinner,

I have an issue with the way the HTTP(S) and Gitlab API HTTP(S) requests are made. As it is, https:// is hardcoded! Example:

uri="https://${username}:${password}@${gitlab_host}/${project_path}.git"

or

open_mrs="$(curl \
--header "PRIVATE-TOKEN: ${private_token}" \
"https://${gitlab_host}/api/v3/projects/${project_path_encoded}/merge_requests?state=opened&order_by=updated_at")"

At my organization, we have a local gitlab server, so we don't bother with HTTPS, and use HTTP instead. In order to use your resource, I hardcoded http:// and pushed my own Docker image in order to use the resource. Eventually, I would like use the mastertinner/gitlab-merge-request-resource Docker image, but I cannot do that right now.

My suggestion is to treat the URI similarly to Concourse's official Git resource manages it. I think it would be good to get rid of gitlab_host and project_path and add:

• uri: Gitlab URI

Example:

http://gitlab.example.com/my/project.git

or

[email protected]:my/project.git

• api_uri: Gitlab API URI

Example:

https://gitlab.example.com/api/v3/projects/my%2Fproject

or

http://gitlab.example.com/api/v3/projects/my%2Fproject

Alternatively, I was thinking we could just add:

• http_secured: true or false (default value true)
  And we could use HTTPS or HTTP accordingly to this configuration.

This would allow all users to specify the protocol they need to use. I understand the first option would also break the way the resource is currently used, but it is more in line with how the other resources are implemented. Let me know what you think.

Require image for ARM64

Hi Team,

I am trying to use the Swisscom/GitLab-merge-request-resource image on the ARM64 platform, but it seems it is unavailable. I have tried building the same using this dockerfile, but it is failing as the base image concourse/buildroot is not available for arm64 and git source is archived.

Do you have plans on releasing arm64 images?

Please suggest if it is possible to use any other image in place of buildroot image.

add support to forked repository MR

When I create a MR from a forked repo, this is detected by the MR job, and a error is displayed:

Cloning into '/tmp/build/get'...
fatal: Could not parse object 'b567e4cf897f2f26a4079086de0965576f7b0123'.

this commit belongs to the forked repo and it doesn't exist in the upstream repo what is been monitorized

Could it be possible to detect the project path from the metadata of the commit and execute a git clone for the forked repo??

/opt/resource/out: line 61: ATC_EXTERNAL_URL: unbound variable

resource_types:
- name: merge-request
  type: docker-image
  source:
    repository: mastertinner/gitlab-merge-request-resource

/opt/resource/out: line 61: ATC_EXTERNAL_URL: unbound variable
need this ENV var?
error when put to merge-request with updating merge status

Could not resolve host: 'username'

For some reason, when performing the 'check' step, it does not resolve the host properly with the following error:

resource script '/opt/resource/check []' failed: exit status 128

stderr:
Cloning into '/tmp/repo'...
fatal: unable to access 'https://username:[email protected]/repo/project.git/': Couldn't resolve host 'username'

However, when I perform a git clone in my terminal, it clones the repo. Alternatively, when I clone it from an EC2 server, I get the same error as above.

resource script '/opt/resource/check []' failed: exit status 5

Getting the following error: at repo-mr resource:

resource script '/opt/resource/check []' failed: exit status 5

stderr:
jq: error (at <stdin>:1): Cannot index object with number

I am setting the following pipeline. The pipeline gets created successfully.

resource_types:
- name: merge-request
  type: docker-image
  source:
    repository: mastertinner/gitlab-merge-request-resource
resources:
- name: repo-mr
  type: merge-request
  source:
    uri: ((repo-uri))
    private_token: ((repo-private-token))
    username: ((username))
    password: ((password))
jobs:
- name: test-merge-request
  plan:
  - get: repo
    resource: repo-mr
    trigger: false
  - put: repo-mr
    params:
      repository: repo
      status: running
  on_failure:
    put: repo-mr
    params:
      repository: repo
      status: failed
  on_success:
    put: repo-mr
    params:
      repository: repo
      status: success

A few notes:

• Identical uri, username and password works fine in a separate pipeline with a git resource.
• The uri format is https://git.mycompany.com/username/project.git
• Tried a wrong password deliberately and got the same error
• Didn't have any luck with using private_key either.

Are there any especial configuration needed on the gitlab side? Would appreciate any help.

SSH Uri and private key

Username and password works just fine but using SSH fails with the following exception:

resource script '/opt/resource/check []' failed: exit status 3

Thank you