zivl / sentry-testkit

A Sentry plugin to allow Sentry report interception and further inspection of the data being sent

Sentry is an open-source JavaScript SDK published by Sentry to enable error tracking that helps developers monitor and fix crashes in real time.
However, when building tests for your application, you want to assert that the right flow-tracking or error is being sent to Sentry, but without really sending it to Sentry servers. This way you won't swamp Sentry with false reports during test running and other CI operations.

Sentry Testkit - to the rescue

Sentry Testkit enables Sentry to work natively in your application, and by overriding the default Sentry transport mechanism, the report is not really sent but rather logged locally into memory. In this way, the logged reports can be fetched later for your own usage, verification, or any other use you may have in your local developing/testing environment.

Installation

npm:

npm install sentry-testkit --save-dev

yarn:

yarn add sentry-testkit --dev

Usage

// some.spec.js
const sentryTestkit = require('sentry-testkit')

const {testkit, sentryTransport} = sentryTestkit()

// initialize your Sentry instance with sentryTransport
Sentry.init({
    dsn: 'some_dummy_dsn',
    transport: sentryTransport,
    //... other configurations
})

test('collect error events', function () {
  // run any scenario that eventually calls Sentry.captureException(...)
  expect(testkit.reports()).toHaveLength(1)
  const report = testkit.reports()[0]
  expect(report).toHaveProperty(...)
});

// Similarly for performance events
test('collect performance events', function () {
  // run any scenario that eventually calls Sentry.startTransaction(...)
  expect(testkit.transactions()).toHaveLength(1)
});

Testkit API

See full API description and documentation here: https://zivl.github.io/sentry-testkit/

Running in browser

sentry-testkit relies on express and http packages from NodeJS. We have separated entry sentry-testkit/browser where we not include any NodeJS-related code.

const sentryTestkit = require('sentry-testkit/browser')

const { testkit } = sentryTestkit()
// Your code for browser

Raven-Testkit

The good old legacy raven-testkit documentation can be found here. It it still there to serve Raven which is the old legacy SDK of Sentry for JavaScript/Node.js platforms

Change Log

We're constantly and automatically updating our CHANGELOG file, so its always a good spot to checkout what have we been up to...

Contribution

We'd love any kind of contribution, to get better, improve our capabilities, fix bugs and bring more features as Sentry expanding their tools as well. Please check our CONTRIBUTING guidelines for more info and how to get started.

License

Sentry Testkit is MIT licensed.

sentry-testkit's Issues

move to travis CI

3.1.0. Module not found: Can't resolve 'net' in 'xxxx/node_modules/express/lib'

I just upgraded to 3.1.0 encountered an error

Module not found: Can't resolve 'net' in 'xxxx/node_modules/express/lib'

auto publish to npm once there is a version release

[BUG]: Testkit.reset doesn't clear event state and fingerprints

Describe the bug
With the latest Sentry version 6.8.0 and the dedupe integration (getsentry/sentry-javascript#3730), it's not possible to have a duplicated error. Testkit.reset() should allow to duplicate an error by resetting state and fingerprints as well.

To Reproduce

  test('should testkit reset clear state and fingerprints', async function() {
    const error = new Error('simple error')

    Sentry.captureException(error)
    await waitForExpect(() => expect(testkit.reports()).toHaveLength(1))
    expect(testkit.isExist(error)).toBe(true)

    testkit.reset() // Should remove state and fingerprints

    Sentry.captureException(error)
    await waitForExpect(() => expect(testkit.reports()).toHaveLength(1))
    expect(testkit.isExist(error)).toBe(true)
  })

Sentry Versions (please complete the following information):

• Sentry: 6.8.0
• Sentry-Testkit 3.3.4

update sentry-testkit documentation on docs.sentry.io

There's a section of Sentry-Testkit in official Sentry Docs pages.

We need to enrich the the very initial readme documentation we have there, as it might be outdated and not informal enough.

The sections we need to add are:

• Integration with Puppeteer
• Network Interception

It may be copied from our docs pages. The purpose of this task is so it will be published there as well.

Sentry Docs repo: https://github.com/getsentry/sentry-docs

support in @sentry/node version

docs: help page

as part of #115 task force

[Docs]: homepage + getting started with docusaurus

as part of #115 task force

[Question]: Support for @sentry/react-native v4

Hi,

I use this library to test sentry in a react-native project, but I start having issues after upgrading @sentry/react-native from v3 to v4. Apparently @sentry/react-native v4 is using sentry-javascript v7+ which has breaking changes that haven't been dealt with in this library. Is there a plan to support @sentry/react-native v4 and sentry-javascript v7?

Many thanks

Body not parsed

This code seems to only parse body for requests with content-type application/json or application/x-sentry-envelope:

https://github.com/wix/sentry-testkit/blob/a210f7389ae414b0cdc707682bfea70f672138a2/src/localServerApi.js#L25

But in practice Sentry seems to be sending payloads with Content-Type: text/plain;charset=UTF-8 header so the payload is not parsed and fails when handling the request:

https://github.com/wix/sentry-testkit/blob/a210f7389ae414b0cdc707682bfea70f672138a2/src/localServerApi.js#L32-L33

generate automatic CHANGELOG.md file

We need to investigate what's the effort to create and generate a CHANGELOG.md automatically for every release

add sentry-testkit documentation to expo

In the following page of Expo and Sentry Integration there's a section used to describe running tests with Sentry, but its pretty empty.
adding a few works about sentry-testkit could be a great way to expose it to new areas

[BUG]: items other than transform sent in an envelope request cause an error

Describe the bug
Some requests generated by Sentry fail in transformTransaction because they are missing the contexts property. It appears that it's trying to use the transformTransaction function for session data.

    TypeError: Cannot read property 'trace' of undefined

      at transformTransaction (../node_modules/sentry-testkit/dist/index.js:42:28)
      at ../node_modules/sentry-testkit/dist/index.js:74:28
      at ../node_modules/puppeteer/lib/cjs/vendor/mitt/src/index.js:51:62
          at Array.map (<anonymous>)
      at Object.emit (../node_modules/puppeteer/lib/cjs/vendor/mitt/src/index.js:51:43)
      at Page.emit (../node_modules/puppeteer/lib/cjs/puppeteer/common/EventEmitter.js:72:22)
      at ../node_modules/puppeteer/lib/cjs/puppeteer/common/Page.js:143:100
      at ../node_modules/puppeteer/lib/cjs/vendor/mitt/src/index.js:51:62
          at Array.map (<anonymous>)
      at Object.emit (../node_modules/puppeteer/lib/cjs/vendor/mitt/src/index.js:51:43)

Here is an example of the request to the envelope enpoint

   {"sent_at":"2021-08-17T14:27:12.489Z","sdk":{"name":"sentry.javascript.react","version":"6.11.0"}}
    {"type":"session"}
    {"sid":"<removed>","init":false,"started":"2021-08-17T14:27:11.361Z","timestamp":"2021-08-17T14:27:12.489Z","status":"ok","errors":1,"attrs":{"release":"<removed>","environment":"<removed>","user_agent":"<removed>"}}

To Reproduce
Use @sentry/react to generate data which includes a session item in an envelope request

Expected behavior
Session, and other types of items other than transactions, do not cause an error

Sentry Versions (please complete the following information):

• Sentry: @sentry/react @ 6.11.0
• Sentry-Testkit 3.3.4

[BUG]: TS types broken in v5

Describe the bug
Ran tsc against our project after upgrading sentry-testkit to 5.0.0:

packages/modules/application/__tests__/core/application/reportError.test.ts:3:52 - error TS7016: Could not find a declaration file for module 'sentry-testkit'. '/Users/liam.jones/code/proj/node_modules/sentry-testkit/dist/index.js' implicitly has an 'any' type.
  Try `npm i --save-dev @types/sentry-testkit` if it exists or add a new declaration (.d.ts) file containing `declare module 'sentry-testkit';`

3 import sentryTestkit, { Report, ReportError } from 'sentry-testkit'
                                                     ~~~~~~~~~~~~~~~~

To Reproduce
Try and type check a project against 5.0.0

Expected behavior
Type check passes

Sentry Versions (please complete the following information):

• Sentry: 4.6.1
• Sentry-Testkit 5.0.0

update Sentry docs with latest changes

So this will be added to their docs

update docs with new Sentry

move to yarn

[Docs]: network page in docusaurus

as part of #115 task force

[Docs]: API reference in docusaurus

as part of #115 task force

support in new Sentry API

Add TypeScript Declaration File

[Question]: Sentry Testkit and Expo

Hi all,

While attempting to test my Sentry implementation on a React Native app, I've been struggling to capture the errors and events logged by Sentry.

My React Native app is built on top of Expo (for cross-platform distribution) and I haven't been able to find any docs out there showing support for Sentry with Expo. Expo maintains their own package to facilitate working with Sentry (https://github.com/expo/sentry-expo) and the imports look a little different.

For example the import looks like this:
import * as Sentry from 'sentry-expo'

And logging an error requires a .Native postfix:
Sentry.Native.captureException(error)

My test case looks something like this (Logger.error is a wrapper on Sentry.Native.captureException):

const mockError = jest.spyOn(Logger, 'error')
Logger.error(new Error('hello'))
expect(mockError).toBeCalledWith(new Error('hello'))
expect(testkit.reports().length).toBeGreaterThan(0)

document browser-only support

Is your feature request related to a problem? Please describe.
We use Karma in a browser to run our tests. The local server that is created uses Node.js dependencies that can't be provided ('http', 'net', 'path', etc.) so the tests fail to run. I thought I wanted a new feature of browser-only support that appears to already exist 🎉 https://github.com/zivl/sentry-testkit/blob/v5.0.5/src/browser.ts

Describe the solution you'd like
Document this feature! (Sorry if I overlooked it 🙈 ). Our code ended up looking something like:

import sentryTestkit from 'sentry-testkit/dist/browser';

import { ErrorLogger } from './index';

const { sentryTransport, testkit } = sentryTestkit();

// abridged for clarity
const errorLogger = new ErrorLogger({
  transport: sentryTransport,
});

errorLogger.captureException(new Error('testing'));

testkit.reports()[0].error.message == 'testing'

Describe alternatives you've considered
I considered writing my own stripped-down transport but thankfully won't have to 😅

Additional context
Thanks for building this!

update sentry-testkit to support Sentry v.6

Sentry has been released its version 6.x.x.
We need to make sure sentry-testkit has compatibility for it.

In case there's no capability - then we will need to create a new major version and support sentry@^6 as well as write proper migration guide

Move docs to docusaurus

Motivation
Make the docs site more modern and up-to-date

Describe the solution you'd like
docusaurus is a modern docs generator with a lot more plugins and higher flexibility of frontend frameworks.

Describe alternatives you've considered
Upgrade better docify and try to acheive as many as features available to make the docs site more modern and up-to-date.

[Question]: Does the DSN have to be a real DSN?

Hi! I was wondering whether the DSN you specify to Sentry while using sentry-testkit has to be a real DSN?
I'm asking because my tests are failing because of the reports list being empty even after firing captureException, and I was wondering if it might have to do with this.

Edit: I'm using Jest under node

Please add `close` method to transport

When flushing sentry using await Sentry.flush() sentry attempts to call close on it, this method isn't defined and an error is thrown.

SEO: remake the header level on the documentation

In side bar, the default heading level it h1 and there's another h1 header in some of the pages.
We need to have only single h1 tag.

reset() does not reset the breadcrumbs

When creating exceptions with breadcrumbs in 2 tests, the second test will have the breadcrumbs from the first test.

describe('debug', () => {
    beforeEach(() => {
      testkit.reset();
    });
    it('should log a message 1', () => {
      Sentry.addBreadCrumb({message: 'messageText'});
      Sentry.captureException(new Error('test 1'));

      expect(testkit.reports()).toHaveLength(1);
      const report = testkit.reports()[0];

      expect(report.breadcrumbs).toHaveLength(1);
    });

    it('should log a message 2', () => {
      Sentry.addBreadCrumb({message: 'messageText'});
      Sentry.captureException(new Error('test 2'));

      expect(testkit.reports()).toHaveLength(1);
      const report = testkit.reports()[0];

      expect(report.breadcrumbs).toHaveLength(1);
    });
});

The second test fails as the breadcrumbs array contains 2 items:

consider jest mock of `sentry-X` as example docs or module

as suggested by @dcramer in https://github.com/getsentry/sentry/pull/16116/files#diff-aa34cb9acf9a36938078d3c80d01a25a
it might be a good pattern to mock sentry TOGETHER with the testkit

jest.mock('@sentry/browser', () => {
    ...Sentry,
   testkit,
   init: options =>
       SentryBrowser.init({ ...options, transport: sentryTransport  })
})

Using `scope.addEventProcessor` is...not working

I'm still digging into it, but I've encountered an issue when using a scope and scope.addEventProcessor where neither captureEvent nor sendEvent seem to never get called. (We're using it to parse Koa requests via Sentry.Handlers.parseRequest in our implementation.)

If anyone happens to have an inkling of what might be going on, any insight would be greatly appreciated! I'll continue to try sorting it out, but in the mean time, as a starting point here's a test case that fails if it happens to be useful for anyone wandering into this as well. Simply removing the call to addEventProcessor allows it to pass:

  test('plays well with event processors', async function () {
    const err = new Error('sentry test kit is awesome!');
    Sentry.withScope(async scope => {
      scope.addEventProcessor(event => Sentry.Handlers.parseRequest(event, koaCtx.request));
      Sentry.captureException(err);
    });
    await waitForExpect(() => expect(testkit.reports()).toHaveLength(1));
    expect(testkit.reports()[0].exception).toMatchObject({
      values: [{ type: 'Error', value: 'sentry test kit is awesome!' }]
    });
  })

add more examples for the api docs

Add support for sending the CORS header from localServer

Is your feature request related to a problem? Please describe.
I'm using the localServer export to create a server.

import sentryTestkit from 'sentry-testkit'

const TEST_DSN = 'http://[email protected]/000001'
const { testkit, localServer } = sentryTestkit()

async function run () {
  await localServer.start(TEST_DSN)
  const dsn = localServer.getDsn()
  console.info({ dsn })
}

run()

The server's port is different from my app's port (for example 56838 vs 8000) so the browser (Firefox) raises a CORS error when my app is trying to report Sentry events.

Describe the solution you'd like
To be able to specify the CORS origin when starting localServer which would then be sent with the response in the Access-Control-Allow-Origin header.

[BUG]: 4.0.1 not compatible with @sentry/react-native 4.1.2

Describe the bug
Trying to upgrade to Sentry RN 4+ following the merging of PR #109. Typing now seems to be broken for the transport property of Sentry.init() and it fails to work.

To Reproduce

import sentryTestkit from "sentry-testkit";
import * as Sentry from "@sentry/react-native";

const { testkit, sentryTransport } = sentryTestkit();

Sentry.init({
  dsn: "https://[email protected]/123",
  enableNative: false,
  tracesSampleRate: 1,
  transport: sentryTransport // This line shows a type error
});

Type error:

TS2322: Type 'new (options: any) => Transport' is not assignable to type '(transportOptions: BrowserTransportOptions) => Transport'.   Type 'new (options: any) => Transport' provides no match for the signature '(transportOptions: BrowserTransportOptions): Transport'. options.d.ts(212, 5): The expected type comes from property 'transport' which is declared here on type 'ReactNativeOptions'

Upon running:

Test suite failed to run

    TypeError: Cannot read properties of undefined (reading 'bind')

      19 | const { testkit, sentryTransport } = sentryTestkit()
      20 |
    > 21 | Sentry.init({
         |        ^
      22 |     dsn: 'https://[email protected]/123',
      23 |     enableNative: false,
      24 |     tracesSampleRate: 1,

      at Object.getNativeFetchImplementation (../../../node_modules/src/transports/utils.ts:77:1)
      at makeFetchTransport (../../../node_modules/src/transports/fetch.ts:12:1)
      at Object.transport (../../../node_modules/@sentry/react-native/src/js/client.ts:32:14)
      at new BaseClient (../../../node_modules/@sentry/src/baseclient.ts:108:1)
      at ReactNativeClient._createSuperInternal (../../../node_modules/@sentry/react-native/dist/js/client.js:34:311)
      at new ReactNativeClient (../../../node_modules/@sentry/react-native/src/js/client.ts:36:6)
      at initAndBind (../../../node_modules/@sentry/src/sdk.ts:34:1)
      at Object.init (../../../node_modules/@sentry/react-native/src/js/sdk.tsx:115:3)

Expected behavior
No typing error on the transport line, bootstrapping of Sentry.init() works.

Sentry Versions (please complete the following information):

• Sentry: 4.1.2
• Sentry-Testkit 4.0.1

[Question]: Support for Sentry 7

Hey, I´m having problems testing the latest release of Sentry. Does this library support Sentry 7?

move sentry-testkit code to TypeScript

auto deploy documentation site from travis CI

update eslint and prettier

update dependencies

Expose events

Is your feature request related to a problem? Please describe.
I'd like to test that a custom integration like rrweb is sending its data on error but those events are not captured and not exposed through sentry-testkit.

rrweb triggers POST requests like https://sentry.io/api/000001/events/6fd7cc13658e4ade9b7217b748144f41/attachments/?sentry_key=acacaeaccacacacabcaacdacdacadaca&sentry_version=7&sentry_client=rrweb

where the content type is Content-Type: multipart/form-data; boundary=....

Describe the solution you'd like
An API like testkit.events() that collects all such events.
I'm not sure how would the payload be exposed and whether the content type is something that the integrations gets to choose or it's somewhat standardized in the API.

Just to clarify, I'd be using localServer for this.

[Feature Request]: Return Typed `sentryTransport` function

Typed sentryTransport function

After the code base move to TypeScript, we got rid of the manual *.d.ts files which caused issues with the implicit returned type of sentryTransport function.

The main cause that we cannot use the straitforward retuned type is because we want to support multiple Sentry versions, and that results in multiple conflicted interfaces (some of them are breaking changes).

We need to come up with something we have done on our manual definition file back then:

  type V6TransportClass = {
    new (): Transport
  }

  type V7TransportFunction = () => Transport

But the above is also not straitforward for the TS compiler to digest (for some reason).
For me personally, I'm a TypeScript newbe so I'm looking for the right solution here... any kind of help is welcomed!

add tests to cover @sentry/react-native

in addition to @sentry/browser and @sentry/node test suites.

Describe the solution you'd like
add another test suite file which imports the commonTests and initialized with @sentry/react-native

Additional context
to make sure this does not break: getsentry/sentry-react-native#2397

[Question]: Is this confirmed working with sentry-react-native 4.11.0?

I updated sentry-react-native from version 4.8.0 and the tests using sentry-testkit failed due to no reports being returned from the testkit.

I tried updating to the newest version of testkit from version 4.1.0 to 5.0.4 and it didn't make a difference.

I had a look to see if the testkit versions corresponded to particular versions of sentry, but I couldn't find anything like that.

Could you let me know if there is a specific version of testkit I should use for sentry-react-native version 4.11.0?

`testkit.reset()` doesn't reset Sentry scope

Summary

If Sentry.configureScope(...) is ever called and some scopes are being set (such as scope.setUser or scope.setTag, it cannot be reset, is it expected behaviour?

How to reproduce

describe('testkit.reset()', () => {
  it('does not reset Sentry scope', () => {
    Sentry.init({
      dsn: 'https://[email protected]/000001',
      transport: sentryTransport,
    });

    Sentry.configureScope(scope => {
      scope.setUser({ id: 'test-id' });
      scope.setTag('tagName', 'test-tag');
    });

    Sentry.captureException(new Error('first exception'));
    expect(testkit.reports()).toHaveLength(1);
    expect(testkit.reports()[0].tags).toMatchObject({ tagName: 'test-tag' });
    expect(testkit.reports()[0].user).toMatchObject({ id: 'test-id' });

    testkit.reset();
    expect(testkit.reports()).toHaveLength(0);

    Sentry.captureException(new Error('second exception'));
    // Expect to be empty, but it persists the previous scope values
    expect(testkit.reports()[0].tags).toBe({});
    expect(testkit.reports()[0].user).toBe(undefined);
  });
});

• Result

.close(timeout) method is not defined on transport

Sentry internally calls .close(transport) method when Sentry.flush() is called. This causes my app to crash because it uses sentry-testing in development. Could you add at least mock for this method?

Deployment failing in @sentry/utils/esm

WARNING in .//@sentry/utils/esm/misc.js
Module not found : error : Can't resolve 'perf_hooks' in '/opt/atlassian/pipelines/agent/build/node_modules/@sentry/utils/esm' [/opt/atlassian/pipelines/agent/build/project.csproj]
@ .//@sentry/utils/esm/misc.js 270:28-45
@ .//@sentry/utils/esm/index.js
@ .//@sentry/browser/esm/index.js
@ ./ClientApp/app.js
@ ./ClientApp/boot-app.js

Skip `createLocalServerApi()` for browsers

Is your feature request related to a problem? Please describe.
I'm having a hard time loading this in the browser tests due to the creation of the local express server.

Describe the solution you'd like
Skip running createLocalServerApi() when detecting running in the browser.

Describe alternatives you've considered
Downgrading to 3.0.2 works for my usecase, but having a longterm maintained solution would be better.

Question: can you use sentry-testkit to extract tracing / performance data?

Not sure if sentry-testkit only provides exception reports or if I just haven't configured Sentry correctly. Seems like the former.
Thank you.

create a stunning documentation site

https://docsify.js.org/#/quickstart

bring back jest-nock support

Unfortunately, due to some race condition or wrong definition, we can't get to run the built-in jest-mock functionality.
To remind, it was possible if you're using Jest for testing, all you have to do in your ***.spec.js file is to import the Jest mock as follows:

// some.spec.js
import { testkit } from 'sentry-testkit/dist/jestMock';

test('something', function () {
    // click
    // clack
    // BOOM!
    expect(testkit.reports().length).toBeGreaterThan(0);
});

But we're having issue with that:

1. We're getting errors Sentry is not defined and Maximum call stack size exceeded (looks like circular dep): https://github.com/wix/sentry-testkit/runs/2477315004
2. re-declaring block-scope variables (looks like es imports helps with this, but then we got into first problem): https://github.com/wix/sentry-testkit/pull/78/checks?check_run_id=2826583066

You may refer to the source code of jestMock.ts and we'll be happy for your help and suggestions.

Thank you @sidoruk-sv for pointing this out! 💙