cypress-io / cypress-example-docker-circle

Cypress + Docker + CircleCI = ❤️

cypress-example-docker-circle

Cypress + Docker + CircleCI = ❤️

Running your Cypress E2E tests on Circle CI v2.0 is very simple. See circle.yml for the current build commands. You can use our base image cypress/base with all dependencies pre-installed.

Then check out the code and call cypress run command. That is it! See test runs for this example at circleci.com

Note: if you want to use CircleCI Workflows feature to run multiple tests in parallel, check out our example repo cypress-example-docker-circle-workflows.

Artifacts

You can save generated videos and screenshots as CircleCI artifacts

steps:
  - checkout
  - run:
      name: Running E2E tests
      command: cypress run
  - store_artifacts:
      path: cypress/videos
  - store_artifacts:
      path: cypress/screenshots

Test summary

Generate just XML report

CircleCI can store test results from a large number of test reporters. Cypress can output test results with custom reporters, including using built-in junit format. Just add the following options to the CI command to generate and store test results.

- run:
    name: Running E2E tests
    command: cypress run --reporter junit --reporter-options "mochaFile=results/my-test-output.xml"
- store_test_results:
    path: results

The generated file will be placed in folder results and the folder will be uploaded to CircleCI storage. This summary will be really helpful when a test fails. For example, I have introduced a different label into the test, the word testing never appears on the page, yet the test is looking for it.

// a-spec.js
it('has h2', () => {
  cy.contains('h2', 'testing')
})

See the failed CI test run at https://circleci.com/gh/cypress-io/cypress-example-docker-circle/10.

The CircleCI test summary shows failed test and user-friendly message.

Switching to the artifacts tab, we can find the screenshot PNG image taken at the failure moment.

Finally, we can open either the video, or the screenshot artifact

The failure is now easy to see and fix.

In this repository take a look at test-junit script inside package.json

Spec + XML reports

You can generate XML junit report and see spec output while CI is running using mocha-multi-reporters.

npm install --save-dev mocha mocha-multi-reporters mocha-junit-reporter
cypress run --reporter mocha-multi-reporters --reporter-options configFile=config.json

File config.json has

{
  "reporterEnabled": "spec, mocha-junit-reporter",
  "mochaJunitReporterReporterOptions": {
    "mochaFile": "multiple-results/results.xml"
  }
}

The standard output will have spec names during the test run, and the final result will be saved in JUnit XML format in file multiple-results/results.xml.

Take a look at test-multiple script inside package.json for example.

Happy testing

If you find problems with Cypress and CI, please

• consult the documentation
• ask in our Gitter channel
• find an existing issue or open a new one

cypress-example-docker-circle's Issues

Render Process Crashed

https://docs.cypress.io/guides/references/error-messages.html#The-Chromium-Renderer-process-just-crashed

I can't figure out a way to add the --ipc=host option to docker in CircleCI.

Only reports on last spec run

I ran both the test-junit and test-multiple commands.

JUnit output:

<?xml version="1.0" encoding="UTF-8"?>
<testsuites name="Mocha Tests" time="0.814" tests="1" failures="0">
  <testsuite name="Root Suite" timestamp="2018-07-17T21:43:58" tests="0" failures="0" time="0">
  </testsuite>
  <testsuite name="b-spec" timestamp="2018-07-17T21:43:58" tests="0" failures="0" time="0">
  </testsuite>
  <testsuite name="page" timestamp="2018-07-17T21:43:58" tests="1" failures="0" time="0.814">
    <testcase name="b-spec page has h2" time="0.814" classname="has h2">
    </testcase>
  </testsuite>
</testsuites>

Multiple output:

<?xml version="1.0" encoding="UTF-8"?>
<testsuites name="Mocha Tests" time="0.225" tests="1" failures="0">
  <testsuite name="Root Suite" timestamp="2018-07-17T21:45:13" tests="0" failures="0" time="0">
  </testsuite>
  <testsuite name="b-spec" timestamp="2018-07-17T21:45:13" tests="0" failures="0" time="0">
  </testsuite>
  <testsuite name="page" timestamp="2018-07-17T21:45:13" tests="1" failures="0" time="0.225">
    <testcase name="b-spec page has h2" time="0.225" classname="has h2">
    </testcase>
  </testsuite>
</testsuites>

In both cases, only the results for spec b are reported. @valscion reported this here, but I'm pretty sure it's a separate issue from #6.

multi-reporter example fails

Hi,

I'm trying to have spec output and JUnit report while running Cypress from command line. But having some issues.

First, it seems that Cypress cannot load mocha-multi-reporters if it is installed globally on the system:

Could not load reporter by name: mocha-multi-reporters

We searched for the reporter in these paths:

• /path/to/project/mocha-multi-reporters
• /path/to/project/node_modules/mocha-multi-reporters

Learn more at https://on.cypress.io/reporters

Second, when running these exact commands:

npm install --save-dev mocha mocha-multi-reporters mocha-junit-reporter
cypress run --reporter mocha-multi-reporters --reporter-options configFile=config.json

Cypress fails just after running the tests with the following stack trace (twice):

TypeError: test.titlePath is not a function
    at /path/to/project/node_modules/mocha/lib/reporters/base.js:227:10
    at Array.forEach (native)
    at Function.exports.list (/path/to/project/node_modules/mocha/lib/reporters/base.js:167:12)
    at Spec.Base.epilogue (/path/to/project/node_modules/mocha/lib/reporters/base.js:344:10)
    at emitOne (events.js:101:20)
    at Runner.emit (events.js:188:7)
    at Reporter.emit (/usr/local/lib/node_modules/cypress/dist/Cypress.app/Contents/Resources/app/packages/server/lib/reporter.js:168:55)
    at Object.server.startWebsockets.onMocha (/usr/local/lib/node_modules/cypress/dist/Cypress.app/Contents/Resources/app/packages/server/lib/project.js:224:22)
    at Socket.<anonymous> (/usr/local/lib/node_modules/cypress/dist/Cypress.app/Contents/Resources/app/packages/server/lib/socket.js:237:36)
    at emitOne (events.js:96:13)
    at Socket.emit (events.js:188:7)
    at /usr/local/lib/node_modules/cypress/dist/Cypress.app/Contents/Resources/app/packages/socket/node_modules/socket.io/lib/socket.js:503:12
    at _combinedTickCallback (internal/process/next_tick.js:67:7)
    at process._tickCallback (internal/process/next_tick.js:98:9)

I can't really tell if this is an Cypress or Mocha issue, so I made the issue on this repository because here are the steps documented and they seem not to work..

Upgrade to latest Cypress

As dev dependency

Running PHP app in CircleCI with Cypress

I try to run Cypress tests on CircleCI. Tests are super simple and run fine on local.

My .circle/config.yml file

version: 2.0
jobs:
  build:
    docker:
      - image: circleci/php:7.2.9-apache-node-browsers
        environment:
          - MYSQL_HOST=127.0.0.1
      - image: mariadb:10.2.12
    working_directory: ~/web
    steps:
      - checkout
      [..Removed the php parts..]
      - run: curl http://circeci.test // To test that php app is running fine.
      - run:
          name: Install cypress.io and dependencies.
          command: npm install
      - run:
          name: Run E2E tests
          command: $(npm bin)/cypress run

E2E test fails with error:

Uncaught Error: Script error.

Cypress detected that an uncaught error was thrown from a cross origin script.

We cannot provide you the stack trace, line number, or file where this error occurred.

Check your Developer Tools Console for the actual error - it should be printed there.

It's possible to enable debugging these scripts by adding the 'crossorigin' attribute and setting a CORS header.

https://on.cypress.io/cross-origin-script-error

This error originated from your application code, not from Cypress.

Grab test results as JSON / XML on Circle

How to generate test results file compatible with CircleCI test results?

https://circleci.com/docs/2.0/configuration-reference/#store_test_results

Mocha-allure-Reporter Fails

Hi,

I am trying to generate mocha-allure-reporter with cypress. The report is generating but

1. allure global variable is not accessible across the repo(which is helpful to pass step description
   set Severity etc.).
2. I am blocked getting the screenshot after each test case. below is my code in cypress.json

"reporter": "mocha-allure-reporter",
"reporterOptions": {
"targetDir": "e2e/report"
}

Someone please assist me how to overcome with this two issues.

Update to use Cypress 0.20.0+

Add multiple reporters

To see both spec output while the test runs and generate junit XML file CircleCI can ingest

Update for use with Cypress 3.0

• cache ~/.npm and ~/.cache
• don't cache node_modules