pulumiverse / pulumi-buildkite

A Pulumi provider for interacting with the Buildkite CI/CD service

Buildkite Resource Provider

The Buildkite Resource Provider lets you manage Buildkite resources.

The provider is built on https://github.com/buildkite/terraform-provider-buildkite.

Installing

Important

The provider version v2.3.1 was built on the Terraform provider v0.25.1 which was a pre-release version. From v3.0.0 onwards this provider is compatible with the Terraform provider v1.x.x. Please note that backwards compatibility might not be given.

This package is available in many languages in the standard packaging formats.

Node.js (JavaScript/TypeScript)

To use from JavaScript or TypeScript in Node.js, install using either npm:

npm install @pulumiverse/buildkite

or yarn:

yarn add @pulumiverse/buildkite

Python

To use from Python, install using pip:

pip install pulumiverse-buildkite

Go

To use from Go, use go get to grab the latest version of the library:

go get github.com/pulumiverse/pulumi-buildkite/sdk/go/...

.NET

To use from .NET, install using dotnet add package:

dotnet add package Pulumiverse.Buildkite

Configuration

The following configuration points are available for the buildkite provider:

• buildkite:api_token (required, environment: BUILDKITE_API_TOKEN) - A Buildkite API Access Token. Must have GraphQL access, as well as the write_pipelines and read_pipelines scopes.
• buildkite:organization (required, environment: BUILDKITE_ORGANIZATION) - The Buildkite organization slug.
• buildkite:graphql_url (optional, environment: BUILDKITE_GRAPHQL_URL) - The Buildkite GraphQL URL.
• buildkite:rest_url (optional, environment: BUILDKITE_REST_URL) - The Buildkite REST URL.

Example

Example for Typescript to create a resource:

import * as buildkite from '@pulumiverse/buildkite';

const args = {};
const vm = new buildkite.agent.AgentToken(
  'token',
  args,
);

Reference

For detailed reference documentation, please visit the upstream Terraform provider's documentation at: https://registry.terraform.io/providers/buildkite/buildkite/latest

Contributors

Thanks goes to these wonderful people (emoji key):

Daniel Mühlbachler-Pietrzykowski 🚧 💻 📖

Christopher Maier 💻 📖

Susan Evans 📖

Paul Stack 📖

This project follows the all-contributors specification. Contributions of any kind welcome!

pulumi-buildkite's Issues

.NET package is not published.

The .NET package is not published, although the Github Actions run didn't error. Reason is this:

pulumi-buildkite/.github/workflows/release.yml

Line 148 in 99cff57

run: find "${{ github.workspace }}/sdk/dotnet/bin/Debug/" -name 'Pulumi.*.nupkg' -exec dotnet nuget push -k "${{ secrets.NUGET_PUBLISH_KEY }}" -s ${{ env.NUGET_FEED_URL }} {} ';'

The find ... -name 'Pulumi.*.nupkg' doesn render any results becuause the prefix for us is Pulumiverse. Preferrably drop the find command. See the command used by another provider:

https://github.com/pulumiverse/pulumi-astra/blob/b7f7bb45d90cde3cc3d7c9061e57fc336433eccb/.github/workflows/release.yml#L121

Can you update the workflow and create a new release?

[bug report] invalid "provider_settings" attribute error when upgrading from v2 to v3

If you stack is on v2, upon upgrading to v3 you would run into following errors:

error: Unable to Read Previously Saved State for UpgradeResourceState: There was an error reading the saved resource state using the prior resource schema defined for version 0 upgrade.

Please report this to the provider developer:

AttributeName("provider_settings"): invalid JSON, expected "[", got "{"

it turns out that the resource of type buildkite:Pipeline/pipeline:Pipeline has different type for inputs.providerSettings in tf-buildkite in post-v1:

The previous version (prior to v1) used an array of provider settings.

ProviderSettings                     []*providerSettingsModel `tfsdk:"provider_settings"`

Whereas the newer model is a pointer to an object/struct.

ProviderSettings                     *providerSettingsModel `tfsdk:"provider_settings"`

The workaround is modifying the statefile:

• Update the provider resource from 2 to 3
• Remove inputs.providerSettings.__defaults

pulumi stack export -s stackName > stateFile.json
# edit the stateFile.json
pulumi stack import -s stackName --file stateFile.json

statefile before:

{
    "urn": "urn:pulumi:platform::buildkite-pipelines::pulumi:providers:buildkite::default_2_2_0_github_/api.github.com/pulumiverse/pulumi-buildkite",
    "custom": true,
    "id": "4c1a2d5b-b292-4eec-ac4a-d4d08cd75be6",
    "type": "pulumi:providers:buildkite",
    "inputs": {
        "pluginDownloadURL": "github://api.github.com/pulumiverse/pulumi-buildkite",
        "version": "2.2.0"
    },
    "outputs": {
        "pluginDownloadURL": "github://api.github.com/pulumiverse/pulumi-buildkite",
        "version": "2.2.0"
    },
    "created": "2023-07-04T07:23:14.026243401Z",
    "modified": "2023-07-04T07:23:14.026243401Z"
},

{
    "urn": "urn:pulumi:platform::buildkite-pipelines::buildkite:Pipeline/pipeline:Pipeline::NAME_HERE",
    "type": "buildkite:Pipeline/pipeline:Pipeline",
    "inputs": {
        "providerSettings": {
            "__defaults": []
        }
    }
}

statefile after:

{
    "urn": "urn:pulumi:platform::buildkite-pipelines::pulumi:providers:buildkite::default_3_1_1_github_/api.github.com/pulumiverse/pulumi-buildkite",
    "custom": true,
    "id": "4c1a2d5b-b292-4eec-ac4a-d4d08cd75be6",
    "type": "pulumi:providers:buildkite",
    "inputs": {
        "pluginDownloadURL": "github://api.github.com/pulumiverse/pulumi-buildkite",
        "version": "3.1.1"
    },
    "outputs": {
        "pluginDownloadURL": "github://api.github.com/pulumiverse/pulumi-buildkite",
        "version": "3.1.1"
    },
    "created": "2023-07-04T07:23:14.026243401Z",
    "modified": "2023-07-04T07:23:14.026243401Z"
},

{
    "urn": "urn:pulumi:platform::buildkite-pipelines::buildkite:Pipeline/pipeline:Pipeline::NAME_HERE",
    "type": "buildkite:Pipeline/pipeline:Pipeline",
    "inputs": {
        "providerSettings": {

        }
    }
}

Ideally we want to provide a seamless way for users to upgrade with a mapper (v2 to v3).

Any thoughts on how to approach this issue @muhlba91

Dependency Dashboard

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

• chore(deps): update anchore/sbom-action action to v0.17.1
• chore(deps): update dependency go to v1.23.0
• chore(deps): update jaxxstorm/action-install-gh-release action to v1.12.0
• chore(deps): update module github.com/buildkite/terraform-provider-buildkite to v1.10.2
• chore(deps): update module github.com/hashicorp/terraform-plugin-framework to v1.11.0
• chore(deps): update module github.com/pulumi/pulumi-terraform-bridge/pf to v0.42.1
• chore(deps): update module github.com/pulumi/pulumi-terraform-bridge/v3 to v3.89.1
• chore(deps): update module github.com/pulumi/pulumi/sdk/v3 to v3.129.0
• chore(deps): update sigstore/cosign-installer action to v3.6.0
• chore(deps): update golangci/golangci-lint-action action to v6
• chore(deps): update goreleaser/goreleaser-action action to v6
• Click on this checkbox to rebase all open PRs at once

Ignored or Blocked

These are blocked by an existing closed PR and will not be recreated unless you click a checkbox below.

• chore(deps): update github.com/pulumi/terraform-plugin-sdk/v2 digest to 7ac578c

Detected dependencies

github-actions

.github/workflows/release.yml

• google-github-actions/release-please-action v4
• actions/checkout v4
• actions/setup-go v5
• jaxxstorm/action-install-gh-release v1.11.0
• pulumi/actions v5
• sigstore/cosign-installer v3.5.0
• anchore/sbom-action v0.15.11
• goreleaser/goreleaser-action v5
• actions/checkout v4
• actions/setup-go v5
• jaxxstorm/action-install-gh-release v1.11.0
• pulumi/actions v5
• actions/setup-dotnet v4
• actions/setup-python v5
• actions/setup-node v4
• JS-DevTools/npm-publish v3
• actions/checkout v4

.github/workflows/verify.yml

• actions/checkout v4
• siderolabs/conform v0.1.0-alpha.27
• actions/checkout v4
• actions/setup-go v5
• jaxxstorm/action-install-gh-release v1.11.0
• pulumi/actions v5
• golangci/golangci-lint-action v5
• actions/checkout v4
• actions/setup-go v5
• jaxxstorm/action-install-gh-release v1.11.0
• pulumi/actions v5
• actions/setup-dotnet v4
• actions/setup-python v5
• actions/setup-node v4

gomod

examples/go.mod

• go 1.20

provider/go.mod

• go 1.21
• go 1.21.1
• github.com/pulumi/terraform-plugin-sdk/v2 v2.0.0-20240229143312-4f60ee4e2975@4f60ee4e2975
• github.com/buildkite/terraform-provider-buildkite v1.5.2
• github.com/hashicorp/terraform-plugin-framework v1.6.1
• github.com/pulumi/pulumi-terraform-bridge/pf v0.32.0
• github.com/pulumi/pulumi-terraform-bridge/v3 v3.80.0
• github.com/pulumi/pulumi/sdk/v3 v3.113.0

sdk/go.mod

• go 1.21
• go 1.21.1
• github.com/blang/semver v3.5.1+incompatible
• github.com/pulumi/pulumi/sdk/v3 v3.113.0

npm

package.json

• all-contributors-cli ^6.24.0

---

• Check this box to trigger a request for Renovate to run again on this repository

"No ID was supplied" error

System info

nodejs 18.16.1
pulumi 3.74.0
pulumi-buildkite 2.2.0 # I can confirm 2.3.0 also suffer the same issue
macos 13.4.1 (22F82) arm64

Steps to re-produce

• Make a new index.ts to create just one example pipeline
• Then run pulumi up then y
• Expected: No error
• Actual:

View in Browser (Ctrl+O): https://app.pulumi.com/compono/buildkite-pipelines/trung/updates/2

     Type                            Name                       Status                  Info
     pulumi:pulumi:Stack             buildkite-pipelines-trung  **failed**              1 error; 1 warning
 +   └─ buildkite:Pipeline:Pipeline  trung-pipeline             **creating failed**     1 error


Diagnostics:
  pulumi:pulumi:Stack (buildkite-pipelines-trung):

  buildkite:Pipeline:Pipeline (trung-pipeline):
    error: 1 error occurred:
    	* No ID was supplied

I did not have this issue last week; it's happened today when I tried to make changes to existing pipelines so I re-create the codes just to do one HelloWorld pipeline to verify the issue.

I am trying to search for keyword No ID was supplied but could not find any in the provider code.

Unsure if it is a bug or not.

Codes

    const stepsFile = "hello_world.yml";
    const repositoryName = "hello_world"; // This is a private GitHub repo
    const organizationName = "runlevel5";
  
    const pipelineConfig = {
      name: pipelineName,
      repository: `[email protected]:${organizationName}/${repositoryName}.git`,
      steps: fs.readFileSync(`./pipelines/${stepsFile}`, "utf8"),
      branchConfiguration: "*",
      defaultBranch: "main",
      description: 'trung',
      allowRebuilds: true,
      providerSettings: {
        buildBranches: true,
        buildTags: false,
        buildPullRequests: true,
        publishCommitStatus: true,
        publishBlockedAsPending: true,
        skipPullRequestBuildsForExistingCommits: false,
        triggerMode: "code",
      },
      teams:
        [
          {
            slug: "everyone",
            accessLevel: "BUILD_AND_READ",
          },
          {
            slug: "admin",
            accessLevel: "MANAGE_BUILD_AND_READ"
          }
        ]
    };

    const pipelineInstance = new buildkite.pipeline.Pipeline("trung-pipeline", pipelineConfig);

Error downloading the provider

Sine a few weeks I receive the following error in my CI pipelines downloading the provider: error downloading provider buildkite to file: failed to download plugin: buildkite-0.2.5: 404 HTTP error fetching plugin from https://github.com/grapl-security/pulumi-buildkite/releases/download/v$%7BVERSION%7D/pulumi-resource-buildkite-v0.2.5-linux-arm64.tar.gz

I believe this could be due to the PluginDownloadURL you set in https://github.com/pulumiverse/pulumi-buildkite/blob/main/provider/resources.go#L71.
Looking at my ProxmoxVE provider this URL should work without the ${VERSION} as well: https://github.com/muhlba91/pulumi-proxmoxve/blob/main/provider/resources.go#LL68C34-L68C34.
Note: I also create release tags following vX.Y.Z as you do.

Example on how to read YAML file into steps property is not correct

In the example, the steps takes in the Buffer of fs.readFileSync which is not correct because steps takes pulumi.Input<string> type instead.

Unless there is a helper that do so that I am not aware of.

Failed to update pipeline

Version: 2.2.0
Description: Failed to update existing pipeline
Steps to reproduce:

• Provision a new pipeline
• Make any change to pipeline attributes
• Then pulumi up

Expected: no errors
Actual:

buildkite:Pipeline:Pipeline (pipelineName):
    error: 1 error occurred:
    	* updating urn:pulumi:stackName::buildkite-pipelines::buildkite:Pipeline/pipeline:Pipeline::pipelineName: 1 error occurred:
    	* input:2: Variable $input of type PipelineUpdateInput! was provided invalid value for buildRetentionEnabled (Field is not defined on PipelineUpdateInput)

Notes:

I have a look into the structure of Pipeline class but could not find the buildRetentionEnabled, however such attribute does exist in the terraform counterpart.

And the PR 291 might be the solution?

Create .NET SDK

We create Python, NodeJS, and Go SDKs, but not one for .NET. The main reason for this is just lack of familiarity with the .NET ecosystem. We should dig into this so we can provide SDKs for all languages that Pulumi supports.

Add `allow_rebuilds` property to `pipeline` resource

When the next version of the Terraform Buildkite provider after 0.8.0 is released, it will have support for setting the allow_rebuilds property of pipeline resources.

Remove `relref` in docs

Hello,

Can you apply the same change in the docs folder?

pulumi/registry@10d968f

Providing API key through config does not work

Bug

I'm trying to programatically create agent tokens using:

    const agentToken = new buildkite.agent.AgentToken("my-token", {});

and keep running into the following error:

non-200 OK status code: 401 Unauthorized body: "{\n  \"errors\": [\n    {\n      \"message\": \"Authentication required. Please supply a valid API Access Token: https://buildkite.com/docs/apis/graphql-api#authentication\"\n    }\n  ]\n}\n"

I have set buildkite:api_token to a valid API token and even setup a Provider manually:

export const BK_PROVIDER = new buildkite.Provider("bk-provider", {
  apiToken: new Config("buildkite").requireSecret("apiToken"),
  organization: "bint",
});

Workaround

Setting the BUILDKITE_API_TOKEN and clearing out the buildkite:api_token seems to work.

Create Java SDK

Pulumi supports Java SDKs now, so it would be nice to publish a Java version.

The main work here is setting up the CI/CD pipeline to build and test Java packages, as well as creation of a repository account for publication.

[Bug report] Failed to provision team pipeline objects with version 2.3.0 and 2.3.1

In v2.2.0:

I could simply do:

const pipeline = new buildkite.pipeline.Pipeline("pipelineName", {
  teams: [
    {
      slug: "everyone",
      accessLevel: "BUILD_AND_READ",
    }
  ]
}, opts);

However the code no longer works in v2.3.1, the same code yields errors:

error: buildkite:Pipeline/pipeline:Pipeline resource 'pipelineName' has a problem: Block Deprecated. This block is deprecated. Please use `buildkite_pipeline_team` instead.. Examine values at 'pipelineName.teams'.

So I removed the pipelineName.teams attribute and add the buildkite.pipeline.Team as instructed:

const pipeline = new buildkite.pipeline.Pipeline("pipelineName", {
  teams: []
}, opts);

new buildkite.pipeline.Team(`${pipeline.pipelineName}-${team.slug}`, {
  teamId: "the GraphQL ID of the team",
  pipelineId: pipeline.id,
  accessLevel: "BUILD_AND_READ",
})

However I got errors:

error: Failed to create pipeline: input: Please ensure at least one team is added to this pipeline

So it's to me it is likely a bug?

[Bug] - Few missing modules from NodeJs module

Steps to reproduce

Follow instructions here: https://www.pulumi.com/registry/packages/buildkite/api-docs/pipeline/team/ to set up pipeline permissions for a team.

Notice error:

Just tried setting up a testsuite and it also appears this is missing from the node module.

I'm excited to manage these resources via Pulumi instead of by hand, thanks for the help!