On registry https://registry.npmjs.org/, the "latest" version (v6.24.1) of dependency babel-preset-es2016 has the following deprecation notice:

🙌 Thanks for using Babel: we recommend using babel-preset-env now: please read babeljs.io/env to update!

Marking the latest version of an npm package as deprecated results in the entire package being considered deprecated, so contact the package author you think this is a mistake.

Affected package file(s): package.json

If you don't care about this, you can close this issue and not be warned about babel-preset-es2016's deprecation again. If you would like to completely disable all future deprecation warnings then add the following to your config:

"suppressNotifications": ["deprecationWarningIssues"]

I'm trying to set up reg-cli for Copay, an Ionic app we build for iOS, Android, and desktop (macOS, Windows, and Linux).

I'm having some trouble avoiding false positives from very slight rendering differences between Chrome screenshots taken on macOS and those taken on Linux.

Almost everything is the same, but some font and shadow antialiasing seems to be different, and it's causing false positives. There are also some platform differences causing problems (scroll bars, file upload buttons).

Here's the comparison report from our CI: https://1043-18141084-gh.circle-artifacts.com/0/home/circleci/copay/test/visual/report.html

There are a few real issues mixed in with the false positives here, but I wanted to open this issue since it may be possible to solve a lot of these with improvements to reg-cli.

1: Antialiasing issues

For this case:

I really don't know the best way to handle this issue. We're currently using the reg-cli settings: --enableAntialias --thresholdRate 0.005.

Any recommendations for improving this? I'm afraid if I raise the thresholdRate more, we'll start missing real issues.

It would also be helpful to understand what the diff is showing. What is yellow vs. red in the diff?

2: Scrollbars

I'm also trying to figure out the best way to handle platform rendering differences like scrollbars:

Our options here:
A. Avoid any cross platform differences – e.g. render our own custom scrollbar on both platforms
- we would prefer to use the native scrollbar, so this is not really an option
B. Hide conflicting elements for e2e tests – we could inject CSS like * { ::-webkit-scrollbar { display: none; } just to take screenshots

• this is also not ideal, since we'd like screenshots to match the real application as best as possible
  C. Give reg-cli both a macOS and Linux version, so the test passes if either matches.

What are your thoughts on option C? If reg-cli could accept one or more variants for each image, I think this might also offer a better solution for our Antialiasing issues above.

Proposal

For images with lots of antialiasing problems, we can provide a version for both macOS and Linux.

Maybe a new command line option like:

--variantRegex the regex used to identify variants for each image. If omitted, (.*)-variant-[-0-9].

With the default setting, if my previous image list looked like:

test-image.png
test-image-variant-1.png
another-test.png
another-test-variant-1.png
another-test-variant-2.png

test-image-variant-1.png would match the default variantRegex, with a capturing group value of test-image. And both another-test-variant-1.png and another-test-variant-2.png would both have a captured value of another-test.

When testing a new test-image.png, reg-cli would first test against the existing test-image.png.

If it passes we're done. If it fails, test against each variant for test-image (removing the .png to match the capturing groups).

If any variant passes, pass the test. If all variants fail, we use the closest variant for the diff. This allows us to specifically "pass" certain cross-platform differences, without missing real, unintentional changes.

Any thoughts on this idea?

This is a similar issue to #124. In a project like:

https://github.com/bokuweb/reg-example

If instead of test/approved/sample.png, I have subdirectories inside of approved, like:

test/approved/app1/sample.png
test/approved/app2/sample.png

When I run the update operation, I'll get the error:

Error: EPERM: operation not permitted, unlink 'test/visual/approved/app1'

It appears that cleanupExpectedDir expects a flat folder. Everything else seems to work fine with subdirectories.

Hi,

I'm using this tool standalone in my visual regression set up. The report it generates is quite nice.

I was looking to improve the report by adding more image comparison tools. For an example see github's image diff in commits: cameronmcefee/Image-Diff-View-Modes@8e95f70?short_path=eda75e0#diff-eda75e03d7033f3550356ddb8363ed56

Would you be interested in having something like that? I could try implementing something like that myself and put up a pull request. I was just wondering if you already have plans for doing something like that.

For some reason (sometimes) when i run the visual tests (using storycap for capturing the images) reg-cli logs a certain diff (which is a valid change) but when i run the update script, it logs that 2 files were updated. The issue here is that the 2nd file was not even listed in the report.

When i push the changes to GitLab i can see a diff, it seems like the file size is the only thing that is different (i'm not sure why though).

It doesn't happen all the time so its hard to understand what is causing it, maybe storycap has a bug or a config i am missing that makes it generate a different image size.

Note that i am running the tests in a docker, so i doubt its an environment issue.

This is how i run it:

yarn reg-cli somePath somePath somePath -R somePath/report.html --extendedErrors --matchingThreshold 0.1

image-diff is no longer maintained, I ended up rewriting it with the latest node standard.
image-difference should run x4 faster based on the benchmark.

There is an error with this repository's Renovate configuration that needs to be fixed. As a precaution, Renovate will stop PRs until it is resolved.

Error type: undefined. Note: this is a nested preset so please contact the preset author if you are unable to fix it yourself.

This is the process how we use reg-cli:

• /path/to/expected-dir folder is git commited
• whenever developer instatiates/installs the current repository ( e.g. yarn install), it will also cleanup temporary build files,
  therefore it will delete all files of /path/to/actual-dir generated from last test run and reg-cli
• developer knows e.g. by CI or the component he touches, which test he needs to run to update and verify image
• when a developer performs a single test, reg-cli will write the images related to this test to /path/to/actual-dir with --update operation,
• this results in situation that git commited /path/to/expected-dir is not just updating the single test run related img diff, but also the non existing images from other tests to be deleted.

Suggested Solution

• add reg-cli option of --ignoreDeleted which will not update non existing images, but only changed ones from actual to expected dir

Alternative is to manual git revert all images which are not related to the test he was running

if actual / expected have different height, .markers-area height is not image height...

Hi,

it would be helpful to create git tags for each new release. The latest git tag is from early 2018.

Best

I simply ran this command

$ reg-cli actual/ expected/ diff -R index.html

and found some links are broken.

The directly structure I'm seeing is like this.

Another attempt is like this. I added / at the end of diff

$ reg-cli actual/ expected/ diff/ -R index.html

and got this.

And results in this directly structure.

• macOS
• node v8.9.4
• reg-cli (globally installed today)

Describe the bug

When running reg-cli with a bunch of rendered images I get an UnhandledRejection error and the application freezes. I realized that empty pngs trigger this, which I was able to solve by first deleting those files. But it also seems that corrupt files also trigger the error. I'm not sure which file is triggering the error.

Reproduced step

Steps to reproduce the behavior:

1. Add an empty png file to the source folder.

Expected behavior

Invalid image files should not crash or hand the process.

Actual behavior

(node:49896) UnhandledPromiseRejectionWarning: Error: Unexpected end of input
    at module.exports.ChunkStream._end (/usr/local/lib/node_modules/reg-cli/node_modules/pngjs/lib/chunkstream.js:92:24)
    at module.exports.ChunkStream.end (/usr/local/lib/node_modules/reg-cli/node_modules/pngjs/lib/chunkstream.js:81:10)
    at exports.PNG.PNG.end (/usr/local/lib/node_modules/reg-cli/node_modules/pngjs/lib/png.js:100:16)
    at ReadStream.onend (_stream_readable.js:677:10)
    at Object.onceWrapper (events.js:421:28)
    at ReadStream.emit (events.js:327:22)
    at endReadableNT (_stream_readable.js:1224:12)
    at processTicksAndRejections (internal/process/task_queues.js:84:21)
(Use `node --trace-warnings ...` to show where the warning was created)
(node:49896) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag `--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 1)
(node:49896) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

details

A vulnerability has been reported in trim-newlines in the following: GHSA-7p7h-4mm5-852v
So, it looks like reg-cli is increasing the version of trim-newlines in yarn resolutions:

However, the latest version of meow, which reg-cli depend on, has resolved the version of trim-newlines: https://github.com/sindresorhus/meow/blob/fd0bc62ce47781e11da506b8e38e8668eb78a584/package.json#L84

Therefore, I thought that we could eliminate the resolutions by updating the version of meow.
If you need to check the behavior, etc., I will be glad to assist you.

thanks.

Describe the bug

Search input does not filter the results

Reproduced step

Steps to reproduce the behavior:

1. npm i -g reg-cli
2. reg-cli visreg/reference visreg/test rvisreg/diff -R ./visreg/report.html
3. open report
4. filter name with '05_'
5. everything is still shown

Expected behavior

I'm expecting one or more images to disappear when filtering.

Actual behavior

Filtering shows all screenshots regardless of the name (might be related to this reg-viz/reg-cli-report-ui#28) ?

Screenshots

🚧

Should

• Spinner or progress bar when calculating x-img-diff
• Legends for each rectangle

Better

• Mouse interaction for each rectangle
• Y-dimension scrollable

Describe the bug

The comparison report doesn't work correctly since 0.17.2. The Viewer won't open after clicking a card when the image files are placed in a sub-directory.

Reproduced step

Steps to reproduce the behavior:
https://github.com/ysgk/reg-suit-ui-repro

git clone [email protected]:ysgk/reg-suit-ui-repro.git
cd reg-suit-ui-repro
yarn
yarn reg-suit compare
npx http-server .reg

You can click to open the first item but the second won't open.

Expected behavior

Regardless of the hierarchy the images are placed, the viewer ui can be shown.

Actual behavior

Nothing happens after clicking a card if the image is placed in a sub-directory.

The problem

After examining the code, I found that this line was the problem.
https://github.com/reg-viz/reg-cli-report-ui/blob/465b91363112eb4f7d26f70328596809763b993e/src/containers/viewer/ViewerContainer.ts#L117

// rawId: new-sub/sample.png
// id:    new-sub%2Fsample.png

const id = encodeURIComponent(rawId);
const index = allItems.findIndex((entity) => entity.id === id);

If the id has a slash, the findIndex always returns -1 because id is URI-encoded while entity.id is not. Actually, entity.id is encoded here but the slashes are left unencoded.

I didn't understand the intent of the encoding so can you take a look at that?

When i am comparing the images of two directiories using

reg-cli /path/to/actual-dir /path/to/expected-dir /path/to/diff-dir -R ./report.html

i have given the actual directory, expected directory and diff directory path.
Getting error like reg-cli command not found.

When I try to generate an HTML report I get an error that style.css wan’t found. This worked on 0.18.1 but broke on 0.18.2

Reproduction

1. Run reg-cli a b c -R r

Then I see the error

Error: ENOENT: no such file or directory, open '...node_modules/.pnpm/[email protected]/node_modules/reg-cli/report/ui/dist/style.css'

Version 2.3.4 of vue-template-compiler just got published.

This version is covered by your current version range and after updating it in your project the build failed.

As vue-template-compiler is “only” a devDependency of this project it might not break production or downstream projects, but “only” your build or test tools – preventing new deploys or publishes.

I recommend you give this issue a high priority. I’m sure you can resolve this 💪

Your Greenkeeper Bot 🌴

Thanks for the excellent tool!

The CLI seems to expect activity to happen in the current working directory. This npm script causes several errors:

{
    "visual:approve": "reg-cli test/latest test/approved test/diffs --report test/report.html --json test/report.json"
}

But this one works perfectly:

{
    "visual:approve": "cd test && reg-cli latest approved diffs --report report.html --json report.json"
}

A sampling of the errors:
Error: ENOENT: no such file or directory, unlink './approved/[IMG]'
Error: ENOENT: no such file or directory, open './latest/[IMG]

It's also possible I'm just misunderstanding the usage.

Naive difference detection engines (e.g. pixel-matching) sometimes don't work well.

For instance, someone committed with message, "modify header component size". He or she changed only header component and never touched any other components. However the naive difference detector unfortunately misunderstands "the entire page changed!" because it can't recognize the simple Y-dimension translation.

Goal

• provide alternative way to recognize difference

Not goal

• replacement pixel-match: pixel-match is still great solution. It's simple and fast enough. The new way should be optional.

Algorithm

Using local features matching based region recognition. See https://github.com/reg-viz/improve_reg_diff

TODO

• Implement difference detector
• Invoke new engine from reg-cli/cli
• Invoke new engine from reg-cli/report. (OnDemand detection)
• Introduce CLI option to tell reg-cli "enable smart diff"
  • Oops, I missed to write this option to README
• Hoist the above option to reg-suit/core option. User can set this using regconfig.json
• Add new view components to review the new comparison result

It seems a little strange ...

I expect that the "changed items" toggle button expands items under this button itself.

Right now, reg-cli highlights any different pixels in 2 images, regardless of how similar they look. It would be good if there was an option to ignore differences if they are similar colors.

For example the following 2 colors are almost indistinguishable to the human eye, but reg-cli will flag them the same as if they were 2 completely opposite colors:

Sometimes it can be useful to highlight these small differences, but other times it would be good to have a tolereance for similar colors. To achieve this, we can use pixelmatch's threshold option. Right now reg-cli always sets it to 0, but it would be good to change it via a command line argument.

Currently when tests are failing for any reason we get this message in terminal:

Inspect your code changes, re-run with -U to update them

The issue with this message is that in some cases running the reg-cli command is used with an npm/yarn script.
For example - yarn test:visual.

So developers that are using the repo think they need to run yarn test:visual -U but this will just run the test again, what they really need to run is another dedicated script for updating e.g: yarn test:visual:update.

It would be great to pass reg-cli a fail message (we can also pass success message if you think its needed) that will be used in case of a fail. The default message can be the message we have today.

Describe the bug

Up until releasing new version of ReportUI I could simply open generated report via file protocol (e.g. file:///path/to/report/report.html). Yesterday I upgraded from 0.18.1 to latest version 0.18.6 and when I tried to open the report in my browser I got only white page and an error message in console Uncaught DOMException: Failed to construct 'Worker': Script at 'file:///.../.storybook/image-snapshots/worker.js' cannot be accessed from origin 'null'.

When I use http server to serve the report everything goes fine, but we are storing the report as an artifact in our CI and I cannot run server there to see the results.

Reproduced step

Steps to reproduce the behavior:

1. Generate report with [email protected]
2. Right click on report.html and Copy path
3. Paste path into URL bar in browser
4. See error

Expected behavior

Report can be opened via file protocol as before

Actual behavior

Report can be opened only if it's served with http server.

Screenshots

With the latest reg-cli's report, I found some icons gets disappearing 🤔

There is an import of mkdirp in report.js, but mkdirp is not included in package.json.

The result is that reg-cli doesn't work for my application, as I use mkdirp in a newer major version (v3) there. As the dependency is missing in reg-cli's depedency list, NPM doesn't install a local version for reg-cli but uses the one installed for my application (v3). v3 and v2 are not compatible.

The solution would add mkdirp to the dependency list.

Hi, is it possible to test various mobile devices?
If so, how can I do that?
Thank you!

Describe the bug

A clear and concise description of what the bug is.

Reproduced step

Steps to reproduce the behavior:

1. use npm i -D reg-cli to install

Expected behavior

should install the module

Actual behavior

seeing message 'found 1 high severity vulnerability'

Screenshots

If applicable, add screenshots to help explain your problem.

I met the following error:

TypeError: Cannot read property 'passed' of undefined
    at /Users/yosuke/workspaces/javascript/reg-simple-demo/node_modules/reg-cli/dist/index.js:157:15
    at Array.filter (native)
    at /Users/yosuke/workspaces/javascript/reg-simple-demo/node_modules/reg-cli/dist/index.js:156:26
    at process._tickDomainCallback (internal/process/next_tick.js:135:7)
    at Module.runMain (module.js:606:11)
    at run (bootstrap_node.js:393:7)
    at startup (bootstrap_node.js:150:9)
    at bootstrap_node.js:508:3

I think it's thrown when newItems is not empty.

reg-viz/reg-suit#124

reg-cli currently only returns a non-zero exit code only if an image has changed. For usage in CI it would be helpful if a non-zero exit code is also returned also if an image is delete or appended.

use case

Expected folder is part of git repo. CI pipeline generates actual folder. CI pipeline should not only ensure that generated images are matching expected ones but also that there aren't any additional test cases not covered (missing expected image) and no expected test cases are skipped (missing actual image).

This is currently only supported by dirty hacks. One approach would be passing output of reg-cli. Another one is comparing the actual and expected folders using custom bash script. An example for the later one would look like:

reg-cli actual expected diff
RESULT=$?

if [ ${RESULT} -eq 0 ] &&
   [ `comm -3 <(ls actual|sort) <(ls expected|sort) | wc -l` -ne 0 ];
then
  RESULT=1
fi

Would be great to not require that complexity.

On registry https://registry.npmjs.org/, the "latest" version (v6.24.1) of dependency babel-preset-es2017 has the following deprecation notice:

🙌 Thanks for using Babel: we recommend using babel-preset-env now: please read babeljs.io/env to update!

Marking the latest version of an npm package as deprecated results in the entire package being considered deprecated, so contact the package author you think this is a mistake.

Affected package file(s): package.json

If you don't care about this, you can close this issue and not be warned about babel-preset-es2017's deprecation again. If you would like to completely disable all future deprecation warnings then add the following to your config:

"suppressNotifications": ["deprecationWarningIssues"]

Is your feature request related to a problem? Please describe.

The demo site (https://reg-viz.github.io/reg-cli/) provides old version's output.
It's no longer valid UI demo and new version would be more attractive.

Describe the solution you'd like

Update with latest version's output.

Describe the bug

Tooling that produces the images shouldn't be producing invalid png images, but, it seems, pngjs is hypersensitive in some regard. When this happens, details about the error are displayed (stack trace & error message), but, the reg-cli tool hangs instead of completing.

/node_modules/pngjs/lib/parser.js:48
      this.error(new Error("Invalid file signature"));
                 ^

Error: Invalid file signature
    at Parser._parseSignature (/Users/loginname/gitlab/tool/node_modules/pngjs/lib/parser.js:48:18)
    at ChunkStream._processRead (/Users/loginname/gitlab/tool/node_modules/pngjs/lib/chunkstream.js:160:13)
    at ChunkStream._process (/Users/loginname/gitlab/tool/node_modules/pngjs/lib/chunkstream.js:175:14)
    at ChunkStream.write (/Users/loginname/gitlab/tool/node_modules/pngjs/lib/chunkstream.js:57:8)
    at PNG.write (/Users/loginname/gitlab/tool/node_modules/pngjs/lib/png.js:95:16)
    at ReadStream.ondata (node:internal/streams/readable:809:22)
    at ReadStream.emit (node:events:517:28)
    at addChunk (node:internal/streams/readable:368:12)
    at readableAddChunk (node:internal/streams/readable:341:9)
    at Readable.push (node:internal/streams/readable:278:10)

Node.js v18.20.1

Reproduced step

Steps to reproduce the behavior:

1. Images for comparison are created, with one or more having details that pnpjs appears to consider invalid
2. Run the reg-cli tool, specifying the new image folder path, the old image folder path, a path for the temp folder, --json {path to report.json} --report {path to report.html} --ignoreChange --matchingThreshold=0.05
3. Observe the error report shown above
4. Observe that the command never terminates

Expected behavior

The command should finish up with a non-zero return code and a message indicating the cause of the problem.

Actual behavior

The tool never completes. Output does show the error being encountered, but app hangs instead of finishes.

Screenshots

......

Current behavior

threshold stands for 2-points pixels color distance ratio. (It's derived from pixelmatch lib.)

Expected behavior

threshold stands for ratio between differential and matched pixels.
Imagemagick uses threthold term in this meaning.

This value can be thresholded using a -fuzz setting to only count pixels that have a larger then the threshold.

Use i.e. child_process.spawn to improve performance.

Is your feature request related to a problem? Please describe.

Some similar tools like Chromatic, Percy or Happo support approving/declining particular snapshot changes and integrate with GitHub status checks.

Describe the solution you'd like

Generated report should have button to approve/decline all or some snapshots that should post GitHub status check update. This web hook url can be something that developer should pass to the tool manually as a parameter and these buttons could simply call these urls to publish update. When all snapshots are approved status is updated with success.
It would be nice also to be able to filter by approved/declined PRs but that would require some sort of persistence for the state of snapshot.

Describe alternatives you've considered

Right now users would have to request changes on PRs and give a feedback manually.

• reg-cli: version: 0.10.1

When turning -X client, the following error is thrown:

Error: ENOENT: no such file or directory, open '/Users/yosuke/workspaces/javascript/reg-suit/node_modules/reg-cli/report/assets/cv-wasm_browser.js'
    at Object.fs.openSync (fs.js:651:18)
    at Object.fs.readFileSync (fs.js:553:33)
    at createXimdiffWorker (/Users/yosuke/workspaces/javascript/reg-suit/node_modules/reg-cli/dist/report.js:92:35)
    at exports.default (/Users/yosuke/workspaces/javascript/reg-suit/node_modules/reg-cli/dist/report.js:104:22)
    at /Users/yosuke/workspaces/javascript/reg-suit/node_modules/reg-cli/dist/index.js:188:33
    at tryCatcher (/Users/yosuke/workspaces/javascript/reg-suit/node_modules/bluebird/js/release/util.js:16:23)
    at Promise._settlePromiseFromHandler (/Users/yosuke/workspaces/javascript/reg-suit/node_modules/bluebird/js/release/promise.js:512:31)

And the following is log of node_modules/reg-cli/report/assets:

node_modules/reg-cli/report/assets
├── favicon_failure.png
├── favicon_success.png
├── image.png
└── reglogo.svg

0 directories, 4 files

It seems that cv-wasm_browser.js and cv-wasm_browser.wasm are missing.

On registry https://registry.npmjs.org/, the "latest" version (v6.24.1) of dependency babel-preset-es2015 has the following deprecation notice:

🙌 Thanks for using Babel: we recommend using babel-preset-env now: please read babeljs.io/env to update!

Marking the latest version of an npm package as deprecated results in the entire package being considered deprecated, so contact the package author you think this is a mistake.

Affected package file(s): package.json

If you don't care about this, you can close this issue and not be warned about babel-preset-es2015's deprecation again. If you would like to completely disable all future deprecation warnings then add the following to your config:

"suppressNotifications": ["deprecationWarningIssues"]

Version 2.3.4 of vue just got published.

This version is covered by your current version range and after updating it in your project the build failed.

As vue is “only” a devDependency of this project it might not break production or downstream projects, but “only” your build or test tools – preventing new deploys or publishes.

I recommend you give this issue a high priority. I’m sure you can resolve this 💪

Your Greenkeeper Bot 🌴

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Datasource	Name	Replacement PR?
npm	cryptiles
npm	hoek

Rate-Limited

These updates are currently rate-limited. Click on a checkbox below to force their creation now.

• chore(deps): update dependency debug to v4.3.6
• chore(deps): update dependency glob-parent to v6.0.2
• chore(deps): update dependency yargs-parser to v21.1.1
• chore(deps): update dependency puppeteer to v23
• chore(deps): update dependency rimraf to v6
• chore(deps): update dependency tar to v7
• chore(deps): update dependency trim-newlines to v5
• chore(deps): update dependency ubuntu to v22
• fix(deps): update dependency del to v7
• fix(deps): update dependency glob to v11
• fix(deps): update dependency make-dir to v5
• fix(deps): update dependency meow to v13
• 🔐 Create all rate-limited PRs at once 🔐

Edited/Blocked

These updates have been manually edited so Renovate will no longer make changes. To discard all commits and start over, click on a checkbox.

• chore(deps): update dependency node-forge to v1.3.1
• chore(deps): update dependency serialize-javascript to v6.0.2
• chore(deps): update dependency flow-bin to v0.243.0
• chore(deps): update dependency ansi-regex to v6
• fix(deps): update dependency chalk to v5
• fix(deps): update dependency md5-file to v5

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

• chore(deps): update babel monorepo (@babel/cli, @babel/core, @babel/plugin-transform-block-scoping, @babel/plugin-transform-flow-strip-types, @babel/preset-env, @babel/preset-flow)
• chore(deps): update dependency ajv to v8.17.1
• chore(deps): update dependency dot-prop to v9
• chore(deps): update dependency flow-typed to v4
• Click on this checkbox to rebase all open PRs at once

Detected dependencies

• Check this box to trigger a request for Renovate to run again on this repository