Coder Social home page Coder Social logo

Comments (5)

ryanjduffy avatar ryanjduffy commented on August 24, 2024 1

I think the tests are running correctly again. I'm not sure precisely what was broken but documenting what I saw and the resolution steps for posterity:

  1. Running d livetest-gecko initially failed because the browser wasn't found. It was looking in .replay-playwright-browsers instead of .replay. It should have been using .replay based on my reading of the playwright installer code and the RECORD_REPLAY_DIRECTORY wasn't set to override it to something else. My next suspicion was that we had an out of date dependency. Running npm ls --depth=0 in the backend showed a lot of dependency errors so I ran npm ci to wipe and reinstall the pinned versions. This resolved finding the browser directory.
  2. Once running the tests were all failing with tab crashes. I used VNC to screenshare into the macstadium box and saw several instances of replay still running (including some seemingly old ones with our old logo). I killed all of those instances and started up the livetests again (while still in VNC) and most of them appear to be running successfully. I saw a couple tab crashes but they seem to be running better now.

Going to look into adding more telemetry next so we have better insight into these failures.

from playwright-tests.

ryanjduffy avatar ryanjduffy commented on August 24, 2024 1

The playwright browser was out of date (by a couple months) so it was unable to authenticate. Merged the latest webreplay-release into replay-playwright, ran d deploy-playwright and d release-playwright on macstadium, killed all runlive processes and open replay browsers, and started them again via d livetest-gecko and we're getting sample recordings again!

https://ui.honeycomb.io/replay/datasets/backend/result/er82JjoAcGy

from playwright-tests.

jasonLaster avatar jasonLaster commented on August 24, 2024

Yeah a telemetry event {name: "live-test", success: true, testName: "airbnb" }

from playwright-tests.

ryanjduffy avatar ryanjduffy commented on August 24, 2024

Opened #89 to improve error handling. We already have telemetry for runlive tests but we lacked the test info so I've added that in https://github.com/RecordReplay/backend/pull/3198.

from playwright-tests.

ryanjduffy avatar ryanjduffy commented on August 24, 2024

I restarted runlive on mac and we're getting the example in honeycomb now but none of the tests are passing. Investigating ...

from playwright-tests.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.