replayio / playwright-tests Goto Github PK
View Code? Open in Web Editor NEWPlaywright harness and tests used by replay CI
License: BSD 3-Clause "New" or "Revised" License
Playwright harness and tests used by replay CI
License: BSD 3-Clause "New" or "Revised" License
We should upload some metadata about the new recording to our database so we can view the tests after the fact.
Fields:
[PASS|FAIL] ${title}
We should have a test that
similar to this replay https://app.replay.io/recording/8a34da01-a7a1-40fe-aac0-34fb2d79e2de?point=6814889627197216239043040440746101&time=3005.21325648415&hasFrames=true
We have a utils.js file (separate from the other utils.ts!) that is used by a single test for launching playwright and supports testing with chromium. This file should be removed, and we should support using chromium and a headful browser for tests that use the existing example() runner.
When I click on a video in youtube with the cli, then skip ad does not skip the ad.
working on checkly reminded me that it is fine to have an authentication step
Lets test adding an item to the cart
Currently failing to record pretty regularly
https://www.loom.com/share/38f5ecbbbc5d4ca4846887aa20c90e2f
The logic for downloading and updating browsers predates when we created and started using the @recordreplay/playwright
package. The latter package takes care of downloading browsers, and the custom logic in this repo duplicates that effort, puts the browsers in a different place, has to download the recording driver by itself (also no longer necessary), and doesn't recognize using the RECORD_REPLAY_DIRECTORY env var to set another directory with browsers to use. This logic should be removed, so we can fully rely on the playwright package to manage the installed browsers.
Marko is a JS framework https://markojs.com/
It might be fun to use their playground to edit some of the code and see it re-render
https://markojs.com/try-online/?file=%2Fcomponents%2Fdynamic-tagname%2Fdynamic-component.marko
Several examples use chromium instead of firefox, which we can't record yet. Also, many of the examples set { headless: false }
, which doesn't work with the linux container when we run our automated tests.
Refactor existing tests to use smarter selectors and example
helper.
We'll be talking with their QA team so it would be nice to have a couple playwright tests
Some of these examples will make more sense than others. We want the simplest example of a react app with minimal webgl re-renders
This is related to this issue too
replayio/gecko-dev#325
Login to discord
Visit a public server
This error occurs frequently running running our playwright tests against chromium, causing the test to fail. It doesn't seem like there is a process crash anywhere and that we should be robust enough to recover if this is thrown in pageLog.
/home/ubuntu/recordreplay/playwright-tests/node_modules/playwright/lib/utils/stackTrace.js:48
const stack = new Error().stack;
^
page.evaluate: Execution context was destroyed, most likely because of a navigation.
Error:
at Object.captureStackTrace (/home/ubuntu/recordreplay/playwright-tests/node_modules/playwright/lib/utils/stackTrace.js:48:19)
at Connection.sendMessageToServer (/home/ubuntu/recordreplay/playwright-tests/node_modules/playwright/lib/client/connection.js:69:48)
at Proxy.<anonymous> (/home/ubuntu/recordreplay/playwright-tests/node_modules/playwright/lib/client/channelOwner.js:64:61)
at /home/ubuntu/recordreplay/playwright-tests/node_modules/playwright/lib/client/frame.js:179:42
at Frame._wrapApiCall (/home/ubuntu/recordreplay/playwright-tests/node_modules/playwright/lib/client/channelOwner.js:77:34)
at Frame.evaluate (/home/ubuntu/recordreplay/playwright-tests/node_modules/playwright/lib/client/frame.js:178:21)
at /home/ubuntu/recordreplay/playwright-tests/node_modules/playwright/lib/client/page.js:385:60
at Page._attributeToPage (/home/ubuntu/recordreplay/playwright-tests/node_modules/playwright/lib/client/page.js:231:20)
at Page.evaluate (/home/ubuntu/recordreplay/playwright-tests/node_modules/playwright/lib/client/page.js:385:21)
at pageLog (/home/ubuntu/recordreplay/playwright-tests/src/helpers.js:82:17)
undefined
/home/ubuntu/recordreplay/playwright-tests/utils.ts:55
throw new Error(`Spawned process failed with exit code ${rv.status}`);
^
Error: Spawned process failed with exit code 1
at Object.spawnChecked (/home/ubuntu/recordreplay/playwright-tests/utils.ts:55:11)
at /home/ubuntu/recordreplay/playwright-tests/run.ts:205:3
at step (/home/ubuntu/recordreplay/playwright-tests/run.ts:44:23)
at Object.next (/home/ubuntu/recordreplay/playwright-tests/run.ts:25:53)
at /home/ubuntu/recordreplay/playwright-tests/run.ts:19:71
at new Promise (<anonymous>)
at __awaiter (/home/ubuntu/recordreplay/playwright-tests/run.ts:15:12)
at runTest (/home/ubuntu/recordreplay/playwright-tests/run.ts:163:12)
at /home/ubuntu/recordreplay/playwright-tests/run.ts:230:11
at step (/home/ubuntu/recordreplay/playwright-tests/run.ts:44:23)
The playwright test suite stopped running recently we should bring it back and ensure that we have telemetry wired up so we can be alerted in honeycomb if it stops running again
We have one telemetry event for when the live tests fail to create a recording
It'd be nice to also track attempts so that we can measure uptime and alert when the service is down and we stop trying
Our backend currently needs a manifest for knowing about the new tests. Can we update it to run all of the tests in the qa_wolf directory?
It would be nice to have a log of how many tests reached the end
https://www.loom.com/share/a1001c137f78427b867e290ab32cb440
It would be nice to see a dashboard of the runlive examples:
https://github.com/ryansolid/solid
This playground might be a good example to start with.
also the hn news site
Currently we store the list of playwright tests we use during automated testing in our backend repository, to avoid exposing the credentials used for the facebook test. I don't know how much that really matters but it does make it awkward to keep the list of tests in sync with the repository itself. It would be better if we had a manifest in the playwright-tests repository that includes all the other tests, in any case.
The yelp test occassionally get stuck finding the first restaurant
The facebook fuzzing test almost always gets stuck when trying to add a post. It either doesn't type anything into the new post box, or it doesn't click on the "post" button afterwards. Examples:
https://replay.io/view?id=936f06b8-923c-4a3a-9635-12be994b10f6
https://replay.io/view?id=2d6d6669-322e-434a-b022-c223911fee0b
It should be easy to write a test that goes to a simple replit and views the code / output.
We might be able to do more if we copy their selenium test
https://replit.com/@sergeichestakov/selenium
Our automated tests need to be able to update to the latest chromium browser.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.