pulumi / actions-pulumify

Pulumify - A GitHub Action to continuously deploy static website previews

Pulumify - Easy Static Website Previews for GitHub 🍹

Pulumify automatically builds and publishes static website previews to AWS S3 in response to GitHub Pull Requests. This lets you rapidly iterate on website changes that are easily reviewable with your team.

Just configure it using the steps below, open a new PR, and voila!

Pulumify is powered by Pulumi and GitHub Actions. Pulumi supports other clouds and based on interest, we'd be happy to add support for Azure, GCP, etc.

Configuring Pulumify in Your Repo

To enable Pulumify in your repo, do the following:

1. Enable GitHub Actions in your account.

2. Configure your GitHub secrets. This must include your PULUMI_ACCESS_TOKEN in addition to your cloud credentials. For example, to deploy to AWS, you'll need AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY values with appropriate IAM permissions to deploy an S3 website to your account.

3. Commit the following file as .github/workflows/pulumify.yml:

   name: Pulumify
   on: [pull_request, delete]
   jobs:
     updateLivePreview:
       name: Update Live Preview
       runs-on: ubuntu-latest
       steps:
       - uses: actions/checkout@v2
       - uses: docker://pulumi/pulumify
         env:
           AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
           AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
           GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
           PULUMI_ACCESS_TOKEN: ${{ secrets.PULUMI_ACCESS_TOKEN }}
           PULUMIFY_BUILD: make ensure && hugo --buildFuture -e $GITHUB_SHA
           PULUMIFY_ROOT: public
   

   Feel free to customize the PULUMIFY_BUILD or PULUMIFY_ROOT settings as appropriate. This example uses Hugo. It's also possible to deploy to a Pulumi organization using PULUMIFY_ORGANIZATION.

After these three steps, the Pulumify GitHub Action will comment on your PRs automatically with URLs to your websites.

actions-pulumify's Issues

Avoid synchronizing repo if it has already been pulled

Hi! I am running through some errors trying to use this action. The reason its because it is trying to init and fetch an already fetched repo with actions/checkout@v1 and outputting the following error: fatal: remote origin already exists..

Suggested solutions

1. Specify on the readme that actions/checkout@v1 should be run first and dont try to fetch the repo within the action (this is kind of standard for other actions)
2. Add a SKIP_FETCH flag if the user wants to skip the fetching.

Any thoughts? Happy to put a PR if needed :).

Failure to POST the PR comment to GH API does not fail the workflow

When the POST request to GH API fails, it doesn't seem to fail the workflow execution of Pulumify. The JSON response looks like this:

{
  "message": "Server Error"
}

It would be good to check the response and fail the workflow, too, if the request itself has failed, which is the important part for Pulumify.

Automate publishing releases to Docker Hub

Retry on 409s

If you run two Pulumify actions in parallel, we fail with a 409.

Updating (pulumi/pulumify-pulumi-docs-joeduffy-1591-improve-autonaming-docs):
error: [409] Conflict: Another update is currently in progress.

Let's just keep retrying for a finite (configurable) period of time until it succeeds.

Running `pulumi config refresh -f` fails if existing stack doesn't have a deployment

I just ran into the following edge case bug: if Pulumify had previously created a stack, but failed to update, subsequent runs of Pulumify will attempt to run pulumi config refresh -f (since the stack exists), but this will fail because there is no previous deployment.

+ PULUMI_STACK_NAME=pulumi/pulumify-pulumi-docs-justin-pulumify-upgrade
+ echo '# PR stack name: pulumi/pulumify-pulumi-docs-justin-pulumify-upgrade'
+ pulumi stack select pulumi/pulumify-pulumi-docs-justin-pulumify-upgrade
# PR stack name: pulumi/pulumify-pulumi-docs-justin-pulumify-upgrade
+ PULUMI_STACK_EXISTS=0
+ '[' 0 -eq 0 ']'
+ pulumi config refresh -f
error: no previous deployment

An error occurred (NoSuchKey) when calling the GetObject operation: The specified key does not exist.

Pulumify failed for this PR: pulumi/docs#2485

2020-02-20T04:43:26.8338490Z  +  pulumi:pulumi:Stack pulumify-pulumify-pulumi-docs-justin-purge creating Error: Invoking sync function 'arn:aws:lambda:us-east-1:894850187425:function:content-syncer-copyfunc-8c64fd3' failed [Unhandled]: "{\"errorMessage\": \"An error occurred (NoSuchKey) when calling the GetObject operation: The specified key does not exist.\", \"errorType\": \"NoSuchKey\", \"stackTrace\": [\"  File \\\"/var/task/index.py\\\", line 42, in handler\\n    archive_bytes = io.BytesIO(s3.meta.client.get_object(Bucket=bucket, Key=archive_key)['Body'].read())\\n\", \"  File \\\"/var/task/botocore/client.py\\\", line 357, in _api_call\\n    return self._make_api_call(operation_name, kwargs)\\n\", \"  File \\\"/var/task/botocore/client.py\\\", line 661, in _make_api_call\\n    raise error_class(parsed_response, operation_name)\\n\"]}"
2020-02-20T04:43:26.8339512Z  +  pulumi:pulumi:Stack pulumify-pulumify-pulumi-docs-justin-purge creating     at Object.<anonymous> (<anonymous>:54:23)
2020-02-20T04:43:26.8341791Z  +  pulumi:pulumi:Stack pulumify-pulumify-pulumi-docs-justin-purge creating     at Generator.next (<anonymous>)
2020-02-20T04:43:26.8342772Z  +  pulumi:pulumi:Stack pulumify-pulumify-pulumi-docs-justin-purge creating     at fulfilled (<anonymous>:11:58)
2020-02-20T04:43:26.8343814Z  +  pulumi:pulumi:Stack pulumify-pulumify-pulumi-docs-justin-purge creating     at processTicksAndRejections (internal/process/task_queues.js:86:5)
2020-02-20T04:43:26.8357568Z  +  pulumi-nodejs:dynamic:Resource content-syncer creating error: Invoking sync function 'arn:aws:lambda:us-east-1:894850187425:function:content-syncer-copyfunc-8c64fd3' failed [Unhandled]: "{\"errorMessage\": \"An error occurred (NoSuchKey) when calling the GetObject operation: The specified key does not exist.\", \"errorType\": \"NoSuchKey\", \"stackTrace\": [\"  File \\\"/var/task/index.py\\\", line 42, in handler\\n    archive_bytes = io.BytesIO(s3.meta.client.get_object(Bucket=bucket, Key=archive_key)['Body'].read())\\n\", \"  File \\\"/var/task/botocore/client.py\\\", line 357, in _api_call\\n    return self._make_api_call(operation_name, kwargs)\\n\", \"  File \\\"/var/task/botocore/client.py\\\", line 661, in _make_api_call\\n    raise error_class(parsed_response, operation_name)\\n\"]}"
2020-02-20T04:43:26.8359562Z  +  pulumi-nodejs:dynamic:Resource content-syncer **creating failed** error: Invoking sync function 'arn:aws:lambda:us-east-1:894850187425:function:content-syncer-copyfunc-8c64fd3' failed [Unhandled]: "{\"errorMessage\": \"An error occurred (NoSuchKey) when calling the GetObject operation: The specified key does not exist.\", \"errorType\": \"NoSuchKey\", \"stackTrace\": [\"  File \\\"/var/task/index.py\\\", line 42, in handler\\n    archive_bytes = io.BytesIO(s3.meta.client.get_object(Bucket=bucket, Key=archive_key)['Body'].read())\\n\", \"  File \\\"/var/task/botocore/client.py\\\", line 357, in _api_call\\n    return self._make_api_call(operation_name, kwargs)\\n\", \"  File \\\"/var/task/botocore/client.py\\\", line 661, in _make_api_call\\n    raise error_class(parsed_response, operation_name)\\n\"]}"
2020-02-20T04:43:26.9955277Z  +  pulumi:pulumi:Stack pulumify-pulumify-pulumi-docs-justin-purge creating error: update failed
2020-02-20T04:43:27.0318459Z  +  pulumi:pulumi:Stack pulumify-pulumify-pulumi-docs-justin-purge **creating failed** 1 error; 6 messages
2020-02-20T04:43:27.0319216Z  
2020-02-20T04:43:27.0319425Z Diagnostics:
2020-02-20T04:43:27.0320216Z   pulumi-nodejs:dynamic:Resource (content-syncer):
2020-02-20T04:43:27.0321342Z     error: Invoking sync function 'arn:aws:lambda:us-east-1:894850187425:function:content-syncer-copyfunc-8c64fd3' failed [Unhandled]: "{\"errorMessage\": \"An error occurred (NoSuchKey) when calling the GetObject operation: The specified key does not exist.\", \"errorType\": \"NoSuchKey\", \"stackTrace\": [\"  File \\\"/var/task/index.py\\\", line 42, in handler\\n    archive_bytes = io.BytesIO(s3.meta.client.get_object(Bucket=bucket, Key=archive_key)['Body'].read())\\n\", \"  File \\\"/var/task/botocore/client.py\\\", line 357, in _api_call\\n    return self._make_api_call(operation_name, kwargs)\\n\", \"  File \\\"/var/task/botocore/client.py\\\", line 661, in _make_api_call\\n    raise error_class(parsed_response, operation_name)\\n\"]}"
2020-02-20T04:43:27.0321596Z  
2020-02-20T04:43:27.0321959Z   pulumi:pulumi:Stack (pulumify-pulumify-pulumi-docs-justin-purge):
2020-02-20T04:43:27.0322110Z     (node:134) ExperimentalWarning: queueMicrotask() is experimental.
2020-02-20T04:43:27.0322241Z  
2020-02-20T04:43:27.0323337Z     Error: Invoking sync function 'arn:aws:lambda:us-east-1:894850187425:function:content-syncer-copyfunc-8c64fd3' failed [Unhandled]: "{\"errorMessage\": \"An error occurred (NoSuchKey) when calling the GetObject operation: The specified key does not exist.\", \"errorType\": \"NoSuchKey\", \"stackTrace\": [\"  File \\\"/var/task/index.py\\\", line 42, in handler\\n    archive_bytes = io.BytesIO(s3.meta.client.get_object(Bucket=bucket, Key=archive_key)['Body'].read())\\n\", \"  File \\\"/var/task/botocore/client.py\\\", line 357, in _api_call\\n    return self._make_api_call(operation_name, kwargs)\\n\", \"  File \\\"/var/task/botocore/client.py\\\", line 661, in _make_api_call\\n    raise error_class(parsed_response, operation_name)\\n\"]}"
2020-02-20T04:43:27.0323719Z         at Object.<anonymous> (<anonymous>:54:23)
2020-02-20T04:43:27.0323860Z         at Generator.next (<anonymous>)
2020-02-20T04:43:27.0323989Z         at fulfilled (<anonymous>:11:58)
2020-02-20T04:43:27.0324128Z         at processTicksAndRejections (internal/process/task_queues.js:86:5)
2020-02-20T04:43:27.0324246Z  
2020-02-20T04:43:27.0324361Z     error: update failed

@cnunciato, @stack72, any ideas since we made some changes here today?

Note: If the fix involves tweaking the Python lambda code in https://github.com/pulumi/actions-pulumify/blob/master/infra/lambda/index.py, remember to run ./make.sh to copy the changes into the bin directory.

Rebase or merge with the target branch before building

Occasionally, we see build errors related to branches being a more than a few commits behind master. It's easily fixed with a merge or rebase, but we should probably be doing that merge/rebase in Pulumify on every commit in order to generate previews that more closely reflect what'll eventually be deployed, or use one of GitHub's checkout actions.

Handle large sites

We've started seeing the following errors on Pulumify deployments of https://github.com/pulumi/docs.

I would bet this is due to hitting the 512MB hard limit on disk space in Lambda, and that the site above uncompressed just recently crossed this size.

Error: Plan apply failed: Invoking sync function 'arn:aws:lambda:us-east-1:894850187425:function:content-syncer-copyfunc-60c6e0b' failed [Unhandled]: "{\"errorMessage\": \"[Errno 28] No space left on device\", \"errorType\": \"OSError\", \"stackTrace\": [\"  File \\\"/var/task/index.py\\\", line 40, in handler\\n    s3.meta.client.download_file(bucket, archive_key, tmp_archive)\\n\", \"  File \\\"/var/task/boto3/s3/inject.py\\\", line 172, in download_file\\n    extra_args=ExtraArgs, callback=Callback)\\n\", \"  File \\\"/var/task/boto3/s3/transfer.py\\\", line 307, in download_file\\n    future.result()\\n\", \"  File \\\"/var/task/s3transfer/futures.py\\\", line 106, in result\\n    return self._coordinator.result()\\n\", \"  File \\\"/var/task/s3transfer/futures.py\\\", line 265, in result\\n    raise self._exception\\n\", \"  File \\\"/var/task/s3transfer/tasks.py\\\", line 126, in __call__\\n    return self._execute_main(kwargs)\\n\", \"  File \\\"/var/task/s3transfer/tasks.py\\\", line 150, in _execute_main\\n    return_value = self._main(**kwargs)\\n\", \"  File \\\"/var/task/s3transfer/download.py\\\", line 602, in _main\\n    fileobj.close()\\n\", \"  File \\\"/var/task/s3transfer/utils.py\\\", line 371, in close\\n    self._fileobj.close()\\n\"]}"
    pulumi:pulumi:Stack pulumify-pulumify-pulumi-docs-zchase-kubernetes-landing-page running error: update failed
    pulumi:pulumi:Stack pulumify-pulumify-pulumi-docs-zchase-kubernetes-landing-page **failed** 1 error; 9 messages
    awsx:s3:BucketDirectory content  

Updates sometimes don't update