dgrebb / backstopjs Goto Github PK

View Code? Open in Web Editor NEW

This project forked from garris/backstopjs

0.0 0.0 0.0 61.42 MB

Catch CSS curve balls.

Home Page: http://backstopjs.org

License: MIT License

Shell 1.43% JavaScript 93.00% CSS 1.36% HTML 3.93% Dockerfile 0.28%

backstopjs's Introduction

Howdy! I'm Dan.

I love computers. They've been a major part of my life for over 30 years.

Building websites since 1999, I never stop experimenting with new technology.

I currently work at Comcast, and fight for the users.

Be excellent to each other.

backstopjs's People

Contributors

backstopjs's Issues

Requirements

The server should be able to host static report files from base path rather than several levels deep.

Maintain backwards comparability to also host static assets several levels deep.

add new static paths to express
add CORS support to super simple webserver
add refs endpoint
enhance tests endpoint with filters
ensure report path can remain hosted from current sever-levels deep implementation
test
update documentation

[CHORE]: Update Dockerfile and test with Node 20 (#34)

The BackstopJS Dockerfile should be updated to and adjust for any Node 20 compatibility issues.

Update Dockerfile
Adjust any related configuration
Test all scenarios

Docker Validations Performed

Note: a npm run docker-load script was added to load local buildx multi-platform image without publishing

cd test/configs/ && node ../../cli/index.js test --docker
npm run sanity-test-docker
npm run smoke-test-docker - exits without indication of passing smoke tests; report opens in browser. expected?
npm run sanity-test-playwright-docker

garris#15 (comment)

Closed with garris#1530

[FEAT]: CI - Add Playwright binary caches to GitHub Actions

Set up caching so browser installation takes less time.

microsoft/playwright#7249 (comment)

[FEAT]: Layer Dockerfile to Improve Build Time

Problem

Docker builds take a long time.

Solution

Layer the docker image so a majority is cached before building the BackstopJS meat.

The layering should be easily overridden with a new npm script via docker buildx build --no-cache or whatever.

Layers

Each of these will be cacheable except the last, which will always get latest backstop, playwright, and puppeteer.

set up the OS and deps
set up npm deps
install backstop, playwright, and docker fresh

[PR]: #34 - Node >= 18 Support

[FEAT] - Watch Mode

Let's add a "watch-mode" to BackstopJS, so it will run automatically on when file changes are detected.

possibly use nodemon
use another method of file-watching
can be passed configuration with files to watch
can be passed a filter to only run certain tests on file change

[FEAT]: CI - Add `report.json` fixtures for each variety GH Actions tests

garris#1533

We need more reliable pass/fail criteria for GitHub actions. Because mismatch errors trigger exit 1, GitHub will always fail.

add continue-on-error to all tests
generate test fixtures
add "if test fixture does not match output" conditional to hard-fail tests

[FEAT] - Refactor Dockerfile

Ensure proper configuration set for

user: does it need to be root? node? something else?
deps: npx playwright install --with-deps in the Dockerfile itself to reduce build times
speed: set up container layering so all dependencies are cached prior to setting up backstop in the RUN command
any other nice-to haves from open issues

[FEAT] - Compare - Dependency Refresh

Requirements

Update compare ecosystem dependencies and build src to output with latest React, Babel, and Webpack.

Tasks

[FEAT] - Updates and Refactoring for Node 20

Requirements

The project should be installed and run with Node 20. As gaps are identified, create issues for larger lifts, or add tasks below to upgrade specific dependencies one at a time.

Tasks

Compatibility Tests

test Node 14
test Node 16
test Node 18

[CHORE]: Convert project to ESM

Requirements

A number of dependencies are now 100% ESM, and do not work with CommonJS/require().

The project should be converted, entirely, to ESM. This conversion is backwards compatible with CommonJS, but CommonJS is not compatible with ESM.

`p-map` and `chalk` for example

`p-map`

❯ npm run sanity-test

> [email protected] sanity-test
> cd test/configs/ && node ../../cli/index.js test

/BackstopJS/core/util/createBitmaps.js:4
const pMap = require('p-map');
             ^

Error [ERR_REQUIRE_ESM]: require() of ES Module /BackstopJS/node_modules/p-map/index.js from /BackstopJS/core/util/createBitmaps.js not supported.
Instead change the require of index.js in /BackstopJS/core/util/createBitmaps.js to a dynamic import() which is available in all CommonJS modules.
    at Object.<anonymous> (/BackstopJS/core/util/createBitmaps.js:4:14)
    at Object.<anonymous> (/BackstopJS/core/command/reference.js:1:23)
    at requireCommand (/BackstopJS/core/command/index.js:49:26)
    at Array.map (<anonymous>)
    at Object.<anonymous> (/BackstopJS/core/command/index.js:46:4)
    at Object.<anonymous> (/BackstopJS/core/runner.js:1:24)
    at Object.<anonymous> (/BackstopJS/cli/index.js:6:16) {
  code: 'ERR_REQUIRE_ESM'

Node.js v20.8.1

`chalk`

❯ npm run sanity-test

> [email protected] sanity-test
> cd test/configs/ && node ../../cli/index.js test

/BackstopJS/core/util/logger.js:1
const chalk = require('chalk');
              ^

Error [ERR_REQUIRE_ESM]: require() of ES Module /BackstopJS/node_modules/chalk/source/index.js from /BackstopJS/core/util/logger.js not supported.
Instead change the require of index.js in /BackstopJS/core/util/logger.js to a dynamic import() which is available in all CommonJS modules.
    at Object.<anonymous> (/BackstopJS/core/util/logger.js:1:15)
    at Object.<anonymous> (/BackstopJS/core/command/index.js:3:16)
    at Object.<anonymous> (/BackstopJS/core/runner.js:1:24)
    at Object.<anonymous> (/BackstopJS/cli/index.js:6:16) {
  code: 'ERR_REQUIRE_ESM'
}

Node.js v20.8.1

[FEAT]: Report - Add Next and Previous buttons to Diff Modal

When in the modal diff scrubber, allow user to navigate to previous/next test without exiting modal.

[FEAT]: Report - Add Approve Button to Modal

When a user is inspecting a diff with the scrubber, allow test acceptance directly in the modal.

[PLAN] - Add Webhooks and Progress Bar in Remote

Requirements

Using Webhooks, report progress (test "X" out of "N" tests) in reports UI.

[FEAT]: CI - Add GitHub Workflows

Let's add GitHub workflows to test all the things!

[FEAT]: runner - add/enhance callback API for custom reporters

This is built into playwright, and could potentially be included as an abstraction API for the config file.

Request: garris#1521

Research & Notes

[FEAT] - Multiple Browser Reporting and Switching

[FEAT] - Update backstop-twentytwenty Dependencies

Requirements

Update backstop-twentytwenty to latest Webpack, Babel, and React.

Tasks

reconfigure webpack as needed
adjust React where needed
test in Node 20 via sanity-test in main project
test in Node 18 via sanity-test in main project
test in Node 16 via sanity-test in main project (nice to have)
add issues for larger lifts

Required for 7.0.0 Release

[FEAT] - add test hash and timestamp to `<title>` and UI to avoid interpreting old tests as current

[FEAT/FIX]: Core/Report - CSP Refactoring (garris#1171)

Problem

Content-Security-Policy is beginning to grow in popularity, and CSS/Script injections will increasingly be blocked.

This applies to:

frame-src: report viewing inside frames in CI/CD like Jenkins (garris#1171)
style-src: injecting styles and inline styles in HTML Report
script-src: injecting scripts and inline scripts in HTML Report

Solution

do more research into CSP thread: puppeteer/puppeteer#1229 (comment)
design feature implementations
- create a loader for Report
- create a loader for test injection (unique to Playwright/Puppeteer?)

Puppeteer

https://pptr.dev/api/puppeteer.page.setbypasscsp/

Important

CSP bypassing happens at the moment of CSP initialization rather than evaluation. Usually, this means that page.setBypassCSP should be called before navigating to the domain.

Playwright

https://playwright.dev/docs/api/class-testoptions#test-options-bypass-csp
we likely want this: https://playwright.dev/docs/api/class-browser

[FEAT] - Capture - Add `page.screenshot` API options to BackstopJS-configuration options

Summary

Problem

Reducing motion is a very necessary challenge when taking screen shots. The Playwright screenshot API has a parameter, animation, that could be used to achieve faster capture.

Page-level APIs aren't currently exposed, so this cannot currently be added by Backstop configuration.

Solution

Add a configuration API for page-level function params that can be processed and passed into the Puppeteer and Playwright runners.

Notes

Playwright screenshot API
API documentation for the page.emulateMedia argument that doesn't ever seem to work
API docuementation for the colorScheme argument that also never seems to work
Feature Request: garris#1461

[FEAT]: CI - Add Docker Workflow to Build with Current Branch

A separate workflow should be included, which will run when labeled "docker". This will use ``${{ github.ref }}to pass in and run withnpm install -g ...` which building Docker and installing `backstopjs`.

Set Playwright and Puppeteer to use `new` headless mode by default

Let's make the full change in source to use new headless mode as the default.

Test regressions, test fallbacks, and ensure support for old mode is supported when new becomes vendor standard.

Notes

garris#1525 (comment)
The Chromium team was supposed to default to new by EOY 2023
there are reports that Playwright runs 40% slower with new headless

[FEAT]: Add Labels to Dockerfile

https://docs.github.com/en/packages/working-with-a-github-packages-registry/working-with-the-container-registry#labelling-container-images

[FEAT]: CI - Pass full backstopjs package name/version to Docker

Summary

Let's allow the backstopjs package installation to be passable in args during Docker build.

Why?

This will allow GitHub actions to pass the current backstopjs branch to Docker, so build, publish, and tests are run with the latest changes this branch has introduced.

How?

rename the Dockerfile ARG "BACKSTOPJS_VERSION" → "BACKSTOP_INSTALL_SOURCE"
adjust the ENV BACKSTOPJS_VERSION=$BACKSTOPJS_VERSION → BACKSTOP_INSTALL_SOURCE=$BACKSTOP_INSTALL_SOURCE
adjust npm scripts as needed
create GitHub Actions var BACKSTOP_INSTALL_SOURCE to concatenate installation source path
update GitHub Actions to pass the BACKSTOP_INSTALL_SOURCE to Docker builds
check if any $TAGs need to be adjusted

Default view state via configuration (or save to localStorage)

Hey, thanks for a great tool :)

I was wondering if you would consider showing all the information by default rather than having it hidden? I appreciate the clean interface, but I didn't notice that the information was hidden until I went to click to copy the file name.

I'm not so confident in getting this spun up to make an MR, but I think the changes would be:

Change this styling to

display: flex;
flex-wrap: wrap;

And then remove these lines and everything pertaining to settings.textInfo

Originally posted by @mikestreety in garris#1519

[BUG] - Update Compare application, dependencies, and webpack configuration

Requirements

Currently npm run build-compare fails due to webpack/babel configuration conflicts. This may be due to differences in how Node handles blobs in > 14.x.x.

Possible Solutions

see dgrebb/dg-backstop@aa47497 for a starting point

Details

❯ npm run build-compare

> [email protected] build-compare
> cp ./node_modules/diverged/src/diverged.js ./compare/output/ && cp ./node_modules/diff/dist/diff.js ./compare/output/ && webpack --config ./compare/webpack.config.js && npm run lint

node:internal/process/promises:289
            triggerUncaughtException(err, true /* fromPromise */);
            ^

Error: Plugin/Preset files are not allowed to export objects, only functions. In /BackstopJS/node_modules/babel-preset-es2015/lib/index.js
    at createDescriptor (/BackstopJS/node_modules/@babel/core/lib/config/config-descriptors.js:159:11)
    at createDescriptor.next (<anonymous>)
    at step (/BackstopJS/node_modules/gensync/index.js:261:32)
    at /BackstopJS/node_modules/gensync/index.js:273:13
    at async.call.result.err.err (/BackstopJS/node_modules/gensync/index.js:223:11)

Node.js v20.8.1

dgrebb / backstopjs Goto Github PK

backstopjs's Introduction

Howdy! I'm Dan.

backstopjs's People

Contributors

backstopjs's Issues

Requirements

Docker Validations Performed

Problem

Solution

Layers

Requirements

Tasks

Requirements

Tasks

Compatibility Tests

Requirements

p-map and chalk for example

p-map

chalk

Requirements

Research & Notes

Requirements

Tasks

Required for 7.0.0 Release

Problem

Solution

Puppeteer

Playwright

Summary

Problem

Solution

Notes

Notes

Summary

Why?

How?

Requirements

Possible Solutions

Details

Recommend Projects

Recommend Topics

Recommend Org

`p-map` and `chalk` for example

`p-map`

`chalk`