getsentry / sentry-replay

Moved to http://github.com/getsentry/sentry-javascript

Sentry Replay

Attention: This project was moved to the getsentry/sentry-javascript monorepo.

While moving, we introduced some breaking changes, along with a version bump from 0.x to 7.x. Please take a look at the migration guide to find out what changed.

The project's README was updated along the way, too. Feel free to take a look.

Please use the getsentry/javascript repo to open new issues and PRs.

sentry-replay's Issues

Investigate Sentry replays not fully working

Sentry replays are not fulling working (at least with yarn dev-ui). Some pages work fine (e.g. if you load Settings), but most do not seem to replay.

It's possible this could be a dev vs prod build issue (e.g. dev build + emotion might be producing a lot of diffs).

Will have to investigate further.

Looks like our stitched together replays do not work well. We have lots of warnings when replaying:

Finalize compression setup for Sentry SDK

Just need to finish the work on this PR.

Memory Capture Events show the incorrect timestamp.

Whenever a memory capture event is submitted, the timestamps are shown incorrectly. This is because of the call to getAbsoluteTime() adds window.performance.timeOrigin to the time passed in. And since memory events have a timestamp of now, we fling all the memory events far into the future.

Terminate recording on extended inactivity

Reduce duplicated breadcrumb events.

https://sentry.io/organizations/sentry-emerging-tech/replays/sourcemapsio-replays:777c390d5e734a2580fbbc40bc322af7/

This replay event had duplicate event chapters. I confirmed they're being generated on the SDK side.

Improve accuracy of LCP

Our current method of capturing LCP is not 100% correct. For one, we are capturing all LCP events from the performance API, but not all of them are valid. See https://web.dev/lcp/#differences-between-the-metric-and-the-api for more information.

Move plugin to web worker

We will the plugin to be able to do work in a web worker as to minimize its impact on the user.

Look into 3rd party scripts getting loaded in replays, potentially due to preload attribute

See this replay and look at the network tab: https://sentry.io/organizations/sentry-emerging-tech/replays/billy-test:20052048ae0c4323922af0b26e086182/

Root replay event does not have user meta data (e.g. browser, device, etc)

Compare this one that has no user tags to this one that does

Breadcrumb sidebar title using static background colour

Problem

Solution
Use surface200 from the theme object

Should integration keep separate list of breadcrumbs?

We are currently using the SDK's breadcrumbs that are saved to every event. The SDK does not clear breadcrumbs after capturing an event so the most recent event will have all breadcrumbs that have occurred since the initialization of the SDK. I believe there is a upper bound on the number of breadcrumbs stored, which would cause issues if we were to aggregate breadcrumbs across many events.

Bug: Possible duplicate breadcrumbs captured

Here's a replay with some duplicates. It looks weird on the front-end but i don't know for certain that the issue is really in the SDK, it's possible that we need to group events because that's what the browser api's return...

This example has:

• 2 Page Load crumbs at 0:00
• 3 LCP crumbs at 0:00
• 2 Reload crumbs at 0:51
• 2 LCP at 0:51

https://dev.getsentry.net:7999/organizations/sentry-emerging-tech/replays/ben-test:cd7824017b744d3b88473be9c2f72ebc/#console

I'm pretty sure the Pageload and Reload crumbs are from redirects or something, they should be easier to group on the front end (unless we don't have to!)

The LCP crumbs are more of a mystery to me though.
The payloads for the first three are:

'{"type":"debug","timestamp":"2022-06-09T00:50:22.916Z","level":"info","category":"default","data":{"action":"largest-contentful-paint","size":1024,"nodeId":-1,"label":"LCP"},"id":25,"color":"purple300","description":"Debug"}'

'{"type":"debug","timestamp":"2022-06-09T00:50:22.975Z","level":"info","category":"default","data":{"action":"largest-contentful-paint","size":12654,"nodeId":-1,"label":"LCP"},"id":26,"color":"purple300","description":"Debug"}'

'{"type":"debug","timestamp":"2022-06-09T00:50:23.034Z","level":"info","category":"default","data":{"action":"largest-contentful-paint","size":39204,"nodeId":-1,"label":"LCP"},"id":27,"color":"purple300","description":"Debug"}'

notice that timestamp is slightly different, so they're not really duplicates. And data.size is increasing each time.

V2 Backend

Following on from #41:

https://www.notion.so/sentry/Session-Replay-V2-Ingest-Backend-e83f89cdadd14a4896f93602ecd45a54

Support only recording replays when an error occurs

When an error occurs, we capture and upload a replay to Sentry. This means that we need to buffer the replay events and only upload on an error event. We need to place an upper bound on the events we buffer, in order to do so, we configure rrweb to checkout the full DOM on a time interval (alternatively we could do it based on an event count, or replay size). Otherwise end users will face memory issues having to store all of the incremental updates.

After an error occurs, we should continue to record the session. The SDK currently is easily setup to continue recording forever, however we may want to limit the amount of time after the error. The important parts of the replay should be what immediately happens before and after the error.
- If we record only up to a specific timeframe after the error happens, what should we do if a new error occurs? Should it always become a new session? Should it be part of the previous replay (given that it happens within the session lifetime).

Index UI mocks

Designs

https://www.figma.com/file/ULqGN1mpdvUkiWJ8PCuo0z/Index?node-id=45%3A7162

AC

• Can filter the session's table by project & time range
• Can filter the session's table by tags we ingest(smart search)
• Can sort the session's table by a specific session column
• Have a way to get to a session "detail" view
• Can see each session's URL slug from where it started
• Can see a user's Gravatar, username, email or IP address if we can associate a user to a session
• Users can see a session's timestamp and duration
• Users can show more sessions in a single view
• Users can paginate the session's table

Do not send breadcrumbs with `replay_event`

No need to send core SDK breadcrumbs when we create a replay_event

What is a session?

What should we consider as a session? Things to consider:

• Refreshing the current tab
• Navigating to an external site
• Opening a new tab to current app
• non-SPAs
• User goes idle/inactive
• User changes tabs or hides browser
• User changes tabs and goes back to app
• Usage of cookies/localStorage

Publish a pre-release version so that it can be used in Sentry SaaS

Prevent Breadcrumb close events from wrapping

Problem
I assume these breadcrumb events are happening because they are happening at the exact point of time(?).

Solution Sketch (Open to suggestions)

Adapt SDK to send to new backend (using replay events)

Determine "inactivity" metrics and transmit with event

Look into measuring performance impact of replays

We need an automated way to measure performance impact of replays. Rough order of importance:

• Long tasks/FPS/jankyness
• CPU performance
• Network transmission size
• Memory?
• Bundle size

Capture results of network requests (fetch/xhr/scripts/images/etc)

window.performance does not give us the results of network requests. we'll need to listen to the core SDK for fetch/xhr, and need to derive a status based on the performance entry for the other resource types.

SDK instrumentation handlers do not flush events

Currently we do not create replay events after receiving new updates from core SDK (e.g. breadcrumbs) -- they only get flushed after an rrweb event.

This causes breadcrumbs to be attached to a new session instead of the previous session.

e.g. old session -> breadcrumbs -> (no upload happens)
new session -> rrweb checkout -> upload (with old breadcrumbs)

Browser Extension which will mask/replace DOM nodes on a live page, to match what the SDK will mask

The SDK is configured to treat DOM notes with the classname of: sr-block sr-ignore & sr-mask as PII, and hide the data.

See: https://github.com/getsentry/sentry-replay/blob/main/src/index.ts#L110-L112 for the latest classname configs.

I think it would be useful for developers to have an extension available where:

• Run the extension on any site
• Extension finds any DOM notes with one of those classnames
• Extension implements the same replacement logic as rrweb, so developers can see the black boxes, or * before they enable the replay sdk.
  The benefit would be that developers can audit their site to make sure that PII information is not sent to the Sentry backend before they deploy their site to production.

If the developer could configure the extension to have different classnames, that'd be awesome too because the SDK allows it already.

---

Side note: the browser extension would be useful to anyone who has integrated rrweb into their site, but we should care about the README just like we do for SDK's and maybe get signups from the extension itself!

Use start time based on earliest event when creating root replay event

It's possible to capture events before the SDK creates the root replay event - we will want the timestamp of root replay event to be the same as the earliest recording event.

Capture memory usage as spans

Currently, we capture performance metrics right before we send an rrweb attachment. Let's capture JS memory usage as well in addPerformanceMetrics. We can get memory information via window.performance.memory.

Needed for getsentry/sentry#33623

Explore attaching a mutation observer very early on so that we can buffer events for rrweb

This would be good to explore, otherwise we will not be able to show a replay from the very beginning and we'll lose out on some events like showing LCP rendering.

Use a ServiceWorker to try and capture fetch/img status codes and timing details

Send attachments as envelopes through ingestion

See https://develop.sentry.dev/sdk/envelopes/#attachment

Finishing a transaction on page blur does not get captured until it is focused again

This is annoying to reproduce reliably, but I think I still see some replays where the idle time has been reached and we should have a new session created, but it doesn't.

Bug: Timeline first event pushing the rest to the bottom

The first event seems to be misplaced thus pushing the rest of the events to the bottom.

Replay example: https://sentry.io/organizations/sentry-emerging-tech/replays/billy-test:5c7177578ca842f5ac699c9768d04525/

Terminate recording on refresh/external navigation

Add client side sampling

Add an option to enable sampling. This will be a very basic sampling: if not sampled, do not initialize plugin at all, otherwise proceed as normal.

Note: We will want to sample based on the session, otherwise, in non-SPA, we would have fragments of a user's journey as they navigate throughout a site as sampling would occur on every pageload.

Implement multi-page sessions

Referencing this discussion: #38, we should implement the ideas discussed there.

Brower Compatibility

Currently there has not been any work done in regards to researching browser compatibility (e.g. for Worker, PerformanceObserver, MutationObserver). We'll want to dive deeper into what browsers we want to support and how to support them.

Fix timestamps on replay events to send in seconds (not ms)

sequenceId logic is currently broken

need to save the session after each sequence id increment, or les when updateSessionActivity is called, it blows the current in-memory session away

Collect error ids that occur and send with replay updates

Collect trace ids that occur and send with replay updates

Similar to #125, we want to collect trace ids to send with replay updates.

V1 Backend

https://www.notion.so/sentry/Session-Replay-V1-Architecture-ae068d1e1d514221b6c3ea2233f360f4

Compress snapshots

Snapshots can be quite large, we will want them to be compressed before uploading to Sentry. This needs to be done in a webworker as it is a CPU intensive process and we do not want to block UI.

Blocked by #36

Determine behavior when tab/window loses focus

• Determine current rrweb behavior when tab/window blurs
• Determine appropriate action on blur and focus: Do we keep recording? Do we create a blur/focus event? Both?

Add @sentry/replay to Sentry SaaS active only for internal users

Do we replace existing rrweb integration?

• Create a new feature flag in sentry repo. This should be flagr-supported.
• Configure flagr to enable this for all Sentry employees
• Add @sentry/replay (pre-release pkg-name TBD) to Sentry SaaS

Explore the possibility of correlating the LCP from `window.performance` to rrweb's serialized elements.

window.performance provides an element reference. We can use rrweb's mirror to find the meta data for the element (e.g. the serialized node's id).

On the frontend, we can maybe wait until we see that the element is added to the player and then add some CSS styling to it to highlight the element.

Create initial set of integration tests

Defer creation of the "root" replay event until we send the first set of recording data

The root replay event is created almost immediately (when we create the session). This needs to be deferred until we have recording data and need to upload it.

Record `window.performance` metrics to replay event

This should include fetch events, loading resources, paint metrics.

Don't associate SDK network events with user/app activity

This is the sentry-replay version of the ticket: "Network waterfall should not show all the sentry-replay beacon/post requests"

The network waterfall is full of beacon and fetch requests from the sentry-replay sdk.

Example:

This is really meta, and not part of the page/product itself. I think we should hide it in the Network waterfall so that customers are not confused by all this traffic that they are not in control of.

Retry failed uploads

It's possible due to network conditions that a replay upload fails. We should add a way to retry the upload after a delay and up to a maximum number of attempts.