Coder Social home page Coder Social logo

percy / percy-ember Goto Github PK

View Code? Open in Web Editor NEW
73.0 14.0 44.0 5.16 MB

Ember addon for visual regression testing with Percy

Home Page: https://docs.percy.io/docs/ember

License: MIT License

JavaScript 86.96% HTML 10.14% TypeScript 2.64% Handlebars 0.26%
ember-percy percy ember ember-addon percy-sdk

percy-ember's Introduction

@percy/ember

Version Test

Percy visual testing for Ember applications.

Installation

$ npm install --save-dev @percy/cli @percy/ember

Usage

This is an example using the percySnapshot function.

import percySnapshot from '@percy/ember';

describe('My ppp', () => {
  // ...app setup

  it('about page should look good', () => {
    await visit('/about');
    await percySnapshot('My Snapshot');
  });
});

Running the test above directly will result in the following logs:

$ ember test
...
[percy] Percy is not running, disabling snapshots
...

When running with percy exec, and your project's PERCY_TOKEN, a new Percy build will be created and snapshots will be uploaded to your project.

$ export PERCY_TOKEN=[your-project-token]
$ percy exec -- ember test
[percy] Percy has started!
[percy] Created build #1: https://percy.io/[your-project]
[percy] Running "ember test"
...
[percy] Snapshot taken "My Snapshot"
...
[percy] Stopping percy...
[percy] Finalized build #1: https://percy.io/[your-project]
[percy] Done!

Configuration

percySnapshot(name[, options])

Automatic snapshot names

The name argument can optionally be provided as QUnit.assert or an instance of Mocha.Test which will automatically generate a snapshot name based on the full test name.

Important: Snapshot names must be unique. If you have multiple tests with the same title, or call percySnapshot multiple times inside a single test, you must provide a unique name.

QUnit

import { module, test } from 'qunit';
import { setupApplicationTest } from 'ember-qunit';
import { visit, currentURL } from '@ember/test-helpers';

module('Acceptance: My app', function(hooks) {
  setupApplicationTest(hooks);

  test('About page should look good', async function(assert) {
    await visit('/about');
    assert.equal(currentURL(), '/about');
    await percySnapshot(assert);
    // => Snapshot taken: "Acceptance: My app | About page should look good"
  });
});

Mocha

describe('Acceptance: My app', () => {
  // ...app setup

  describe('about page', () => {
    it('should look good', () => {
      await visit('/about');
      await percySnapshot(assert);
      // => Snapshot taken: "Acceptance: My app about page should look good"
    });
  });
});

Upgrading

Automatically with @percy/migrate

We built a tool to help automate migrating to the new CLI toolchain! Migrating can be done by running the following commands and following the prompts:

$ npx @percy/migrate
? Are you currently using @percy/ember? Yes
? Install @percy/cli (required to run percy)? Yes
? Migrate Percy config file? Yes
? Upgrade SDK to @percy/[email protected]? Yes

This will automatically run the changes described below for you.

Manually

Installing @percy/cli

If you're coming from a pre-3.0 version of this package, make sure to install @percy/cli after upgrading to retain any existing scripts that reference the Percy CLI command.

$ npm install --save-dev @percy/cli

Migrating Config

If you have a previous Percy configuration file, migrate it to the newest version with the config:migrate command:

$ percy config:migrate

percy-ember's People

Contributors

artemgurzhii avatar backspace avatar bennettmt avatar bstack-security-github avatar cadeparade avatar ctjhoa avatar dependabot-preview[bot] avatar dependabot[bot] avatar dfreeman avatar dhaulagiri avatar djones avatar erikkessler1 avatar fotinakis avatar gaurav0 avatar itsjwala avatar jon-hall avatar krasnoukhov avatar kratiahuja avatar maprules1000 avatar mfeckie avatar mike-north avatar prklm10 avatar robdel12 avatar samarsault avatar snyk-bot avatar timhaines avatar tomocoop avatar turbo87 avatar twokul avatar wwilsman avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

percy-ember's Issues

Create test helper that works in new ember test framework

The current percySnapshot test helper should be updated to work in the new ember testing framework http://rwjblue.com/2017/10/23/ember-qunit-simplication/

One of the changes is that the way test helpers are created has changed. registerAsyncHelper is no longer used. Rather test helpers are just plain functions that use import { getContext } from '@ember/test-helpers'; to get access to the container.

Take a look here for an example of an async test helper using the new test framework:
https://github.com/ember-animation/ember-animated/blob/f9e4722ca54ffc163f0adfdb4f8fe2fb5a4dc949/addon-test-support/index.js#L11-L18

Build failed

I run ember tests with Percy , before it was working, recently the build always fail with no error showing,
scrnn

on the platform i get this message
image

it is working when using percy snapshot snapshots.json but in my case i need to use percySnapshot during Ember tests,

i am using :

  • ember: 3.24
  • node: 14
  • @percy/cli": 1.24.0
  • @percy/ember: 4.2.0

thank you.

remove @percy/agent as a dependency

Problem

@percy/agent is quite a large download. Chromium itself is 130.3 Mb. This download is incurred by everyone who installs ember-percy, even though, in our case at least, the Percy agent is only ever invoked in ci. We would prefer to install the Percy agent either globally at each ci run, or as part of our docker container.

Solution

Require users of ember-percy to also either add @percy/agent to their package.json or install it globally.

Note

This may also be a use case for peer dependencies, but it seems that this package does not require specific versions of the Percy agent.

ember-component-css compatiblity

For users using ember-component-css, integration tests apparently don't include the component styles, so the snapshot that Percy captures does not contain styles. This is not a Percy bug, but maybe there is a way we can make this easier / or documented?

Apparently the fix is needing import 'dummy/initializers/component-styles'; in integration tests.

If anyone has an example addon/app that uses ember-component-css and is using Percy, would love to work with you to get this officially supported.

Percy running for each Ember Try scenario

Just started using Percy (thanks so much!) and I'm wondering if there's a way to avoid diffing each scenario that Ember Try runs?

I can see the benefit of running and diffing in every scenario but I can also see wanting to only run in the default scenario. Any thoughts?

percy build only hangs

Trying to add percy to try it out here

Could be because I'm using ember-exam to parallelize tests across travis jobs.

Can't link my GitHub project

I know that this is probably the wrong place to report this, but I couldn't find a contact link on the website and the percy-web project seems to have "Issues" disabled.

I was trying to link a project on Percy to the corresponding project on GitHub, but it seems that the website only loads the first ~50 projects from GitHub. Since my GitHub account has a few more projects, the one that I was looking for did not appear in the list and searching for it also did not reveal it. 😒

Fastboot support

Looks like we're starting two percy builds and finishing once, due to recent changes in the way an ember "fastboot build" is created

# was
ember fastboot:build
# is now
ember build

Guards:

// node land
EMBER_CLI_FASTBOOT === 'true' // in fastboot mode
// browser land
typeof FastBoot === 'undefined' // not in fastboot mode

Remove jQuery dependency

Seems like the snapshot.js file is currently dependent on jQuery. We are in the process of removing jQuery from our Ember add ons and applications and it would be nice to be able to run tests without jQuery as well.

`percySnapshot` + `percyCSS` does not work as expected

Hello!

I have been trying to use percyCSS option as outlined in the documentation, and I cannot get it to work.

Here is an example:

// This is a schematic example with QUnit and Ember
test('An amazing test', async function(assert) {
  await visit('/some-url');
  await percySnapshot(assert, {
    percyCSS: `
      .an-existing-css-class {
        display: none !important;
      }
    `
  });
});

I have tried specifying the same CSS in Percy configuration and that did not work:

// .percy.js
module.exports = {
  version: 2,
  snapshot: {
    widths: [1440],
    // match testing container
    minHeight: 900,
    percyCSS: `
      .an-existing-css-class {
        display: none !important;
      }
    `
  },
  discovery: {
    allowedHostnames: [],
    // https://git.io/JZLF4
    // Increased asset discovery timeout to help with font loading
    networkIdleTimeout: 250,
  },
  static: {
    baseUrl: '/',
    files: '**/*.{html,htm}',
    ignore: '',
  },
  upload: {
    files: '**/*.{png,jpg,jpeg}',
    ignore: '',
  },
};

I have tried using the media query, like so and that did not work:

@media only percy {
  .an-existing-css-class {
    display: none !important;
  }
}

The only way I got it to work is via a new class that I have to apply to every element of the DOM I want hidden which is subpar:

@media only percy {
  .hide-in-percy {
    display: none !important;
  }
}

I feel like I'm missing something obvious but cannot seem to see it, unfortunately.

I am using the latest published versions of both @percy/percy-ember & @percy/cli.

Skipping large build resource

I'm running into a an issue with large build resources: [percy][WARNING] Skipping large build resource: /assets/vendor.js.

What's the file size limit internally? Also, do you have any recommendations for this situation? I'm getting this on our vendor.js file soβ€”is our app just too big for Percy?

SVGs Not Showing On Snapshots

  • Our SVGs are located in the assets directory.
  • The SVGs load correctly when serving in all environments including the test environment.
  • Apparently the SVGs have never loaded in Percy snapshots.
  • We use ember exam and testem to run the tests.

What I've tried so far:

  • I've tried upping to the latest version of @percy/ember.
  • I've tried renaming the svg file name to check is it was a caching problem.
  • I've tried using the ember test helper waitFor.

Please let me know what else I should try or if you need anymore information.

`cb() never called!` or `Error: Cannot find module 'percy-client'`

// index.js
const ep = require('ember-percy')

results in:

β–΅  node index.js
module.js:487
    throw err;
    ^

Error: Cannot find module 'percy-client'
    at Function.Module._resolveFilename (module.js:485:15)
    at Function.Module._load (module.js:437:25)
    at Module.require (module.js:513:17)
    at require (internal/module.js:11:18)
    at Object.<anonymous> (/Users/bgonzalez/code/_scratch/ember-percy-demo/node_modules/ember-percy/index.js:9:19)
    at Module._compile (module.js:569:30)
    at Object.Module._extensions..js (module.js:580:10)
    at Module.load (module.js:503:32)
    at tryModuleLoad (module.js:466:12)
    at Function.Module._load (module.js:458:3)

Percy forever waiting on snapshot to be uploaded

Building a ember 2.5 app. Testing with travis with parallel test suits. I've configured pretender to let percy through this.post('/_percy/:path', this.passthrough); and added the environment variable PERCY_PARALLEL_TOTAL=2 in travis.

I've only added one snapshot to a test. All tests pass on travis but percy is hanging.
The "Waiting for snapshot uploads... " message is displayed.

Test also run locally without error.

ember-exam auto discovery

Right now you have to manually set PERCY_PARALLEL_TOTAL if you use ember-exam to split testsβ€”we could probably grab a split number from exam and combine it with things like CIRCLE_NODE_TOTAL to make using Percy's support for parallel suites a bit easier.

Invalid URL or path when running separate build and test steps

In our CI setup, we have separate build + test steps: https://percy.io/docs/clients/javascript/ember#separate-build-and-test-steps

When actually running the test steps, I am seeing the following error:

[percy][ERROR] API call failed, Percy has been disabled for this build.
StatusCodeError: 400 - {"errors":[{"status":"bad_request","detail":"Invalid URL or path: undefined/<FILENAME>"}]}

Tracked this down to the following:

https://github.com/percy/ember-percy/blob/master/index.js#L51

normalizedRootUrl is undefined at runtime when running build + tests separately. It appears that the config method is not called before in this scenario.

Difference in screenshots: Github Actions run vs. Local run

Hello! πŸ‘‹

I came across the difference in screenshots which I cannot explain. Maybe someone could land a hand or could help debug the issue?

I have created a fake change to trigger the build (change in the README file, does not impact the functionality, at all.)

When percy exec runs on Github Actions, it shows that the newly taken screenshots are different from the baseline. Build Link.

When I run percy exec locally, there is no difference. Build Link.

I have compared the environment, head & base commit and browser version - they are all the same.

In both cases, the same command is used yarn percy exec yarn test:visual; yarn test:visual is a shortcut for ember test --filter='visual acceptance'.

The only thing that is different is the $PERCY_BRANCH variable which I do not set in Github Actions. If I read the documentation correctly, I do not need to set it, it defaults to the current git branch.

Textarea content not being captured by snapshots

Textareas in our snapshots show up as <!-- --> due to a combination of how percy snapshots the DOM and how Ember.TextArea updates its content (by setting HTMLTextAreaElement.value rather than updating Node.textContent).

On further investigation we're also experiencing problems due to Ember.TextArea not actually working when run in PhantomJS (2.1.7), but we still see the same output (<!-- --> in any textareas) when taking percy snapshots from Chrome.

Recent `@percy/sdk-utils` change breaks AMD import

The problem

Starting 2 days ago, our nightly "forget the lockfile and see what happens" compatibility tests for our design system package started failing due to test loader errors. I've tracked it down to @percy/ember failing to load @percy/sdk-utils due to the change that landed in percy/cli#1000

the Chrome debugger paused while attempting to call the string '@percy/sdk-utils' as a function

Environment

  • Node version: 16.15.0
  • @percy/cli version: 1.6.4

Code to reproduce issue

This should be reproducible in this repo by refreshing the @percy/sdk-utils entry in the lockfile here.

Percy not working for <input>

We've implemented percy for many editors in our application. As it says, it's an editor, so it contains many input fields. For that, we use the html tag with our components:

 export default Component.create({
  tagName:    'input',
  attributeBindings: ['type', '_value:value'],
  type: 'text',
  ...
});

Unfortunately, the inputs are shown on the snapshots, but without the value. With the same construction, text-areas aren't shown with the value either. The text-area just contains ''.

Isn't it possible to snapshot inputs?

(Also, because of those empty inputs, I implemented percy on our own addon we use for the inputs. There the percy build wasn't finalized. It creates a build, sends the first 4 snapshots and then... nothing. Even if I run it locally. When I run it locally and just run 1 test, it will be finalized. Also with the tests that was not in the first 4 snapshots. Any ideas on this?)

HTML/CSS inconsistency when running percy on Travis CI

We're using Percy.io to test https://github.com/travis-ci/travis-web and we're seeing an inconsistency between what's displayed when running tests and what gets uploaded to Percy. At first my thought was that there's a difference between the VM on which we're running the tests, but today I ran the tests in debug mode in Chrome running inside Xvfb and I got a screenshot, which indicates that the tests look OK. The problem exists only in the profile page. Here's how it looks like in production right now:

Here's what percy snapshots:

And here's a snapshot from the Travis CI's VM: https://transfer.sh/117ixF/foo.jpg

I'll be looking more into this issue, but if you want to try to reproduce this, you can clone the travis-web repo and run tests with ember test --filter "view profile". Let me know if you need any more details.

ember-responsive

Having responsive diffs is quite awesome!

With ember-responsive or flexi one might render different DOM for each device.

Do you have any solution for this scenario in mind?

With ember-response it might be quite easy to overwrite the service and do multiple tests/snapshots

new test API

With the new testing API, usage would change to

import { percySnapshot } from 'ember-percy';

test('foo', async function(assert) {
  await percySnapshot('bar');
});

Correct?

Error fetching DOM library, disabling

Getting this error in an Ember app. Both locally and in Github Actions.

{
    "ember-cli": "~3.24.0",
    "ember-source": "~3.24.0",
    "@percy/ember": "^2.1.4"
}

Capture scrollTop value for DOM elements

It's possible for multiple elements in an HTML document to have a scrollTop attribute set. When Percy creates a DOM snapshot and serializes the HTML, the scrollTop position is not included.

Currently we're facing an issue where some of our tests will scroll to different parts of the page, whereupon we want to take a screenshot w/ Percy. Right now these screenshots don't take scroll position into account and end up taking the same screenshot multiple times (default scroll position).

Current workaround is to use the scope argument to force percySnapshot to focus on a particular section. Would love if Percy could take scrollTop into account and depict scroll position in its snapshots

Duplicate copies of the same resource fail to upload

This is a subtle issue which took a little bit of tracking down.

In our assets we have two images which are the same but at two different paths (the original was copied to a second location).

Because ember-percy uses a sha of the image content as the key to build up the resources map the resource at the last location ALWAYS wins and the others are not uploaded to Percy. This means we get 404-ed images in our snapshots.

Could the resourceUrl be used as a key instead of the content sha?

As an example. We have two assets which are identical in content but just stored at different locations, e.g.

/images/logos/hipchat.png
/assets/images/integrations/hipchat.png

In the final resource payload uploaded to Percy, there's only an entry to /assets/images/integrations/hipchat.png because the second sha key clobbers the first one.

Typescript definitions?

As I'm picking up percy for the first time, I came across this issue: #79

which is very important (and related to importing in general), but also, I was wondering, how would I know what I can important, once I guess at the import path?

CSS is applied in one test but not the other

This screenshot appears to be artificially darker in colour than it is supposed to be. When we compare it to a screenshot generated with our previous tool, BackstopJS, you can see that the Percy screenshot is noticeably darker: the font colour is closer to black than grey, and even the border around the search input, and the magnifying glass icon, are all darker than they are in the backstop image.

I have also reached out to support@ and provided the link to the build. Posting here, as well. I have also provided the screenshots since I cannot share them publicly.

Our stack is Ember + QUnit. @percy/cli is "^1.0.0-beta.54" and @percy/ember is "^3.0.0".

Let me know if there is any other information I can provide.

Capture scroll position

Hello!

I have found a similar issue percy/percy-cypress#2 on the topic. I was wondering what would it take to implement something like this.

After looking at @percy/cli, it seems like we could potentially use prepareDom to denote the elements that have a scrollTop value that is not equal to 0 and store that value as an attribute. Later, the attribute could be used during the (re)hydration.

I'm happy to land a hand (if at all feasible) to implement something like that. I will need some guidance though πŸ˜„

Let me know what you think! 🍻

Edit 1: We could potentially use PercyScript for that but it seems to be deprecated.

Log a more descriptive message when running tests without Percy

Similar to #186 but rather than silence the logs, we might want to elaborate on them. That issue specifically mentions "for example, if the agent server isn't running" which is the most likely reason for that error. We should state so in the error message.

Current:

`[percy] Error fetching DOM library, disabling: ${err}`

Possible suggestion?:

// We know the dom library should be fetchable as long as percy is running.
// Should we still log the error reason? introduce debug logs for the error?
'[percy] Percy is not running - skipping snapshots'

Open to other suggestions as well πŸ‘‚

"Error POSTing DOM" happening regularly

Thanks to the merge of #234 we've been able to upgrade to the most recent Percy stack. However, I've noticed that our builds fail to create all snapshots with the log message originating here:

https://github.com/percy/ember-percy/blob/40cb769830e5688238c34ffa7b9f03b2d9e53d15/addon-test-support/%40percy/ember/index.js#L137

The message is like this:

LOG: [percy] Error POSTing DOM, disabling: TypeError: Failed to fetch

This happened 3-4 times already on different builds since yesterday, including the baseline builds. This makes things really problematic since it results in "bad" baseline build, which in turn results in a lot of diffs for branch builds.

Here's a most recent example: https://percy.io/simplepractice/simplepractice/builds/7002777

Not sure where that error is coming from (percy agent?) but perhaps this addon can add a retry logic of sorts? Any update would be appreciated.

Autogenerated name for QUnit tests

We've been using percySnapshot(assert); to autogenerate the snapshot name in our tests and it's been working great. We're currently trying to upgrade to Ember 2.10 and our CI tests are filled with this error:

StatusCodeError: 400 - {"errors":[{"status":"bad_request"},{"source":{"pointer":"/data/attributes/base"},"detail":"The name of each snapshot must be unique, and this name already exists in the build: ...

Has anyone seen this before?

Multiple Snapshots in One Test

Is there a recommended approach to taking multiple snapshots in a single test?

I would be great to be able to pass an additional label when creating a snapshot in addition to assert

await percySnapshot(assert, 'Sub-State')

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    πŸ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. πŸ“ŠπŸ“ˆπŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❀️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.