percy / exec-action

A (deprecated) GitHub action to run Percy Agent `percy exec` commands

Percy

Deprecated

⚠️ This action is only required if you're using an older SDK not using Percy CLI. In Percy CLI, GitHub actions will work out of the box.

Example for how to migrate

Before

name: CI
on: [push, pull_request]
jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/checkout@master
      - name: Install
        run: yarn
      - name: Percy Test
        uses: percy/[email protected]
        with:
          command: "[your-test-command]"
        env:
          PERCY_TOKEN: ${{ secrets.PERCY_TOKEN }}

After

name: CI
on: [push, pull_request]
jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/checkout@master
      - name: Install
        run: yarn
      - name: Percy Test
        run: npx percy exec -- [your-test-command]
        env:
          PERCY_TOKEN: ${{ secrets.PERCY_TOKEN }}

---

A GitHub action to run percy exec CLI commands for visual testing. Full API docs for this action can be found here

Quick start

To use the Percy exec GitHub action you will need to add a new step to your actions config using the percy/exec-action action. exec-action has one required input: the command to run your tests. You will also need to set your PERCY_TOKEN in your GitHub projects settings.

Below is a sample config that runs Cypress with Percy:

name: CI
on: [push, pull_request]
jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/checkout@master
      - name: Install
        run: yarn
      - name: Percy Test
        uses: percy/[email protected]
        with:
          command: "cypress run"
        env:
          PERCY_TOKEN: ${{ secrets.PERCY_TOKEN }}

exec-action's Issues

Very little info on how to migrate to Percy CLI

From the readme:

⚠️ This action is only required if you're using an older SDK not using Percy CLI. In Percy CLI, GitHub actions will work out of the box.

From the docs:

Now you can run your tests with Percy (typically percy exec -- [test command]) and send builds to Percy from CI.

So from that I take, I shouldn't be using this anymore:

- name: Percy Upload
        uses: percy/[email protected]
        with:
          custom-command: "npx percy upload ./images"
        env:
          PERCY_TOKEN: ${{ secrets.PERCY_TOKEN }}

But it doesn't really say what I should do instead...
From the docs I take I understand I can just do something like:

- name: Percy Upload
        run: percy upload ./images
        env:
          PERCY_TOKEN: ${{ secrets.PERCY_TOKEN }}

But that doesn't work...
So what's the snippet I should use these days for the newest Percy CLI?

Build not triggering on GitHub PRs

I have installed the Percy in my repo and also using the GitHub action to get the check-in the PR itself -

My yaml script -

name: CI

on:
  pull_request:

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - name: Percy Test
        uses: percy/[email protected]
        with:
          command: 'cypress run'
        env:
          PERCY_TOKEN: ${{ secrets.PERCY_TOKEN }}

Error log -

23s
Run percy/[email protected]
  with:
    command: cypress run
    working-directory: ./
    passthrough: false
    silence: false
    verbose: false
  env:
    PERCY_TOKEN: ***
/usr/local/bin/npx percy exec -- cypress run

[percy] Error: command not found "cypress"
Error: The process '/usr/local/bin/npx' failed with exit code 127

update node LTS requirement?

The current active LTS is 14, with maintenance-only LTS being 12, but if I update the action.yml to use node14 instead of node12, I get the error System.ArgumentOutOfRangeException: Specified argument was out of the range of valid values. (Parameter ''using: node14' is not supported, use 'docker' or 'node12' instead.')

Does this really need using: 'node12' or will this work if the task it is itself used in already uses actions/setup-node@v2 for node 14 or 15?

How to use with finalize --all?

Hi, I'm currently trying to run percy finalize --all to complete my parallel tests:

Relevant section:

      - name: Finish batch
        uses: percy/[email protected]
        with:
          command: "npx percy finalize --all"
          working-directory: "./e2e"
        env:
          PERCY_TOKEN: ${{ secrets.PERCY_TOKEN }}
          PERCY_PARALLEL_NONCE: ${GITHUB_ACTION}

Currently, the command will result in an error so there must be a syntax issue with it.
I tag all the parallel tests with the nonce ${GITHUB_ACTION} which is a unique ID for each GitHub Action.

What is the right way to do this?

Investigate querying GH API for needed info

The info you get from webhook payloads is all over the place. Since you can trigger actions from almost any GH event, we won't be able to gather the info we need (branch name, PR #, commit SHA).

What if we just queried the API always, no matter the build type, for this info? Can we? It would make things a whole lot easier to maintain.

`set-env` command is deprecated and will be disabled soon

Hi Percy!

Ever since GitHub deprecated set-env, we've been getting deprecation warnings from exec-action due, I think, to its dependence on an old version of @actions/core (1.2.0 vs. 1.2.6 or greater).

Error: Unable to process command '::set-env name=PERCY_GITHUB_ACTION,::exec/0.1.1' successfully

I'm trying to setup

I have added a github secret named PERCY_TOKEN and pushed the following github action workflow:

name: Percy (Visual Regression Testing)
on: [push, pull_request]
jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/checkout@master
      - name: Install
        run: yarn
      - name: Set environment variable(s)
        run: |
          echo "PERCY_TOKEN=${{ secrets.PERCY_TOKEN }}" >> $GITHUB_ENV
      - name: Percy Test
        uses: percy/[email protected]
        with:
          command: 'cypress run'

but it keeps failing with

Error: Unable to process command '::set-env name=PERCY_GITHUB_ACTION,::exec/0.1.1' successfully.
Error: The `set-env` command is disabled. Please upgrade to using Environment Files or opt into unsecure command execution by setting the `ACTIONS_ALLOW_UNSECURE_COMMANDS` environment variable to `true`. For more information see: https://github.blog/changelog/2020-10-01-github-actions-deprecating-set-env-and-add-path-commands/
Error: Unable to process command '::set-env name=PERCY_BRANCH,::implement-css' successfully.
Error: The `set-env` command is disabled. Please upgrade to using Environment Files or opt into unsecure command execution by setting the `ACTIONS_ALLOW_UNSECURE_COMMANDS` environment variable to `true`. For more information see: https://github.blog/changelog/2020-10-01-github-actions-deprecating-set-env-and-add-path-commands/
Error: Unable to process command '::set-env name=PERCY_PULL_REQUEST,::30' successfully.
Error: The `set-env` command is disabled. Please upgrade to using Environment Files or opt into unsecure command execution by setting the `ACTIONS_ALLOW_UNSECURE_COMMANDS` environment variable to `true`. For more information see: https://github.blog/changelog/2020-10-01-github-actions-deprecating-set-env-and-add-path-commands/

I'm setting the env according the the guidelines:

https://docs.github.com/en/free-pro-team@latest/actions/reference/workflow-commands-for-github-actions#environment-files

false negatives with nx:affected and cache

If you run percy with nx and cypress and only those tests that were affected by your changes. It can be that the following happens:

• cache results are pulled from nx cloud (tests aren't rerun, only the result is returned from the previous run)
• the e2e test doesn't generate a percy snapshot (mono-repo not all tests use percy)

Both cases cause percy action to "fail" although both could result in successful cypress runs.

exec-action creating 2 builds and giving errors Error: listen EADDRINUSE: address already in use :::5338

Hey

I have been trying to integrate percyScript tests with Github Actions, here is the output of exec-action

  with:
    command: percy exec -- node homepage-snapshot.js
    working-directory: ./tests/percy/tests
    verbose: true
    passthrough: false
    silence: false
  env:
    PERCY_TOKEN: ***
/usr/local/bin/npx percy exec -- percy exec -- node homepage-snapshot.js
[percy] Current config file path: /home/runner/work/dir/tests/percy/.percy.yml
[percy] Using config: { version: 1, snapshot: { widths: [ 375, 1280, 1920 ] } }
[percy] created build #47: https://percy.io/example/builds/4964845
[percy] -> assetDiscoveryService.puppeteer.launch
[percy] -> assetDiscoveryService.createPagePool
[percy] -> assetDiscoveryService.setup
[percy] percy has started.
[percy] -> assetDiscoveryService.browser.newPage
[percy] Current config file path: /home/runner/work/dir/tests/percy/.percy.yml
[percy] Using config: { version: 1, snapshot: { widths: [ 375, 1280, 1920 ] } }
[percy] created build #48: https://percy.io/example/builds/4964846
events.js:288
      throw er; // Unhandled 'error' event
      ^

Error: listen EADDRINUSE: address already in use :::5338
    at Server.setupListenHandle [as _listen2] (net.js:1309:16)
    at listenInCluster (net.js:1357:12)
  C

My workflow file looks like this:

name: Percy Workflow
on: 
  pull_request:
    branches:
      - develop
jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/[email protected]
      - name: Install dependencies 
        working-directory: ./tests/percy
        run: npm install
      - name: Percy Test
        uses: percy/[email protected]
        with:
          command: percy exec -- node homepage-snapshot.js
          working-directory: ./tests/percy/tests
          verbose: true
        env:
          PERCY_TOKEN: ${{ secrets.PERCY_TOKEN }}

My homepage-snapshot.js looks like this

const PercyScript = require('@percy/script');

PercyScript.run(async (page, percySnapshot) => {
  await page.goto('URL');
  // ensure the page has loaded before capturing a snapshot
  await page.waitFor('body');
  await percySnapshot('homepage');
});

Why this is initiating 2 builds, because of 2 build I think getting this issues of "Address is already in use"

Looking forward.
Thanks!

Typo with version in README.md

The README.md references the following:

uses: percy/[email protected]

The version should be v0.1.0.

Same issue here: https://github.com/marketplace/actions/exec-action

Add the ability to pass any custom command

This would be the ultimate escape hatch, just in case it's needed

Which files are required to make this work "in someone's own repo"?

Due to business requirements, some repos can only run github-sanctioned or even only local actions: which files are required in order to make this action work by placing it in a repo's own .github/actions/ dir?

(and ideally, can that information be added as a README.md section, to have that knowledge concisely available in a single, obvious place =)

Support parallel builds

This would set the NONCE as well as the parallel total to -1 (and use finalize all).

It's unknown if another action will be needed for finalize all, or if the user could just run npx finalize --all (probably will become an action)

Run with puppeter

Hi
we are using percy and puppeter for visual tests, as your sample you just wrote the way to run tests with cypress
but I want to run it with puppeter
but I don't know what it the correct way

it is my yaml file:

name: CI
on: push
jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/[email protected]
      - name: Install
        run: npm i
      - name: Percy
        uses: percy/[email protected]
        with:
          command: "npm run test-visuals"
        env:
          PERCY_TOKEN: ${{ secrets.PERCY_TOKEN }}

and it is the result of test:

it fails because I should start the server to run project then run tests, but it gives me different error on that situation
my test script is this:

"test-visuals": "percy exec -p 40300 -- jest tests/screenshots/analytics/QuickStats.test.ts"

exec-action fails when triggered by schedule in GitHub Actions

We are executing Percy visual tests within our Cypress end-to-end test framework and are using the Percy exec-action in our GitHub Actions workflow. Tests run fine when the workflow is triggered on a pull request, but not on a schedule.

Here's our GitHub workflow setup for the trigger:

name: E2E Testing

on:
  schedule:
    # Run at 6am, 10am, 2pm, and 6pm every day
    - cron: "0 6,10,14,18 * * *"

And here is the log from when GitHub Actions executes percy/[email protected]

##[group]Run percy/[email protected]
with:
  command: cypress run --browser chrome --record --parallel --ci-build-id=$CYPRESS_CI_BUILD_ID
  working-directory: ./
  silence: false
  verbose: false
env:
  pythonLocation: /opt/hostedtoolcache/Python/3.7.5/x64
  CI: 1
  CYPRESS_CACHE_FOLDER: /home/runner/.cache/Cypress
  PERCY_TOKEN: ***
  CYPRESS_RECORD_KEY: ***
  CYPRESS_CI_BUILD_ID: 245c5acac86c06f269991fca3e900eef43bea5d1-27344
##[endgroup]
##[error]Cannot read property 'replace' of undefined
##[error]Node run failed with exit code 1

The issue might be here: https://github.com/percy/exec-action/blob/master/src/index.js#L17