Coder Social home page Coder Social logo

rust-sourcemap's Introduction

sourcemap

This library implements basic processing of JavaScript sourcemaps.

Installation

The crate is called sourcemap and you can depend on it via cargo:

[dependencies]
sourcemap = "*"

If you want to use the git version:

[dependencies.sourcemap]
git = "https://github.com/getsentry/rust-sourcemap.git"

Basic Operation

This crate can load JavaScript sourcemaps from JSON files. It uses serde for parsing of the JSON data. Due to the nature of sourcemaps the entirety of the file must be loaded into memory which can be quite memory intensive.

Usage:

use sourcemap::SourceMap;
let input: &[_] = b"{
    \"version\":3,
    \"sources\":[\"coolstuff.js\"],
    \"names\":[\"x\",\"alert\"],
    \"mappings\":\"AAAA,GAAIA,GAAI,EACR,IAAIA,GAAK,EAAG,CACVC,MAAM\"
}";
let sm = SourceMap::from_reader(input).unwrap();
let token = sm.lookup_token(0, 0).unwrap(); // line-number and column
println!("token: {}", token);

Features

Functionality of the crate can be turned on and off by feature flags. This is the current list of feature flags:

  • ram_bundle: turns on RAM bundle support

License: BSD-3-Clause

rust-sourcemap's People

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

rust-sourcemap's Issues

[Feat Request]Make `SourceMap` struct is `Sync`.

The sourcemap crate is amazing, it maybe used extensive at rust js tools, but here has a problem. The rust js tools always run at parallel(oxc and swc codegen), but the SourceMap struct is not Sync because SourceView using Refcell. Here has a workaround encode it to String map, but after it need to decode to SourceMap, it has an unnecessary overhead. Hope get your advice, thank you.

Panic when source map contains emojis

Environment

How do you use Sentry?
Sentry SaaS (sentry.io) or self-hosted/on-premise (which version?)

Which SDK and version?
e.g: JavaScript 5.11.1, .NET 1.2.0

n/a

Using sourcemap as a crate

Steps to Reproduce

  1. I got a sourcemap generated from following code:
export default function () {
  return "📣❓";
}
  1. When parsing the source map I get a panic:
fn generate_coverage_report(
  script_coverage: &ScriptCoverage,
  script_source: &str,
  maybe_source_map: &Option<Vec<u8>>,
) -> CoverageReport {
  let maybe_source_map = maybe_source_map
    .as_ref()
    .map(|source_map| SourceMap::from_slice(source_map).unwrap()); // <--- panics here
   ...
}

Expected Result

What you thought would happen.

Source map is parsed and code doesn't panic.

Actual Result

What actually happened. Maybe a screenshot/recording? Maybe some logs?

thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: BadJson(Error("invalid unicode code point", line: 1, column: 39318))', cli/tools/coverage.rs:202:57
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Error: Process completed with exit code 1.

Ref denoland/deno#10936

to_wirter hangs and eats lots of memory

test issue_622 ... test issue_622 has been running for over 60 seconds
thread 'issue_622' panicked at 'attempt to add with overflow', /home/kdy/.cargo/registry/src/github.com-1ecc6299db9ec823/sourcemap-5.0.0/src/encoder.rs:42:17

Stack trace:

stack backtrace:
   0:     0x564eebc37e14 - backtrace::backtrace::libunwind::trace::hb827f4c466253f29
                               at /cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.44/src/backtrace/libunwind.rs:86
   1:     0x564eebc37e14 - backtrace::backtrace::trace_unsynchronized::h9d5f4880ced35a0e
                               at /cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.44/src/backtrace/mod.rs:66
   2:     0x564eebc37e14 - std::sys_common::backtrace::_print_fmt::hb1cddf0dbc858d5e
                               at src/libstd/sys_common/backtrace.rs:78
   3:     0x564eebc37e14 - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::h418b05c8466523f7
                               at src/libstd/sys_common/backtrace.rs:59
   4:     0x564eebc5d63c - core::fmt::write::hd7c4c3ef1c519f9c
                               at src/libcore/fmt/mod.rs:1053
   5:     0x564eebc33f67 - std::io::Write::write_fmt::hea7ee7c2b0b56640
                               at src/libstd/io/mod.rs:1428
   6:     0x564eebc3a665 - std::sys_common::backtrace::_print::hd48ee8e8dcfec814
                               at src/libstd/sys_common/backtrace.rs:62
   7:     0x564eebc3a665 - std::sys_common::backtrace::print::hd560d375c3dd4d06
                               at src/libstd/sys_common/backtrace.rs:49
   8:     0x564eebc3a665 - std::panicking::default_hook::{{closure}}::hbd5f2fc476d36ee7
                               at src/libstd/panicking.rs:204
   9:     0x564eebc3a3a7 - std::panicking::default_hook::h04bf70bf3772f200
                               at src/libstd/panicking.rs:224
  10:     0x564eebc3acc2 - std::panicking::rust_panic_with_hook::h98c3e5a93afaee15
                               at src/libstd/panicking.rs:470
  11:     0x564eebc3a8ab - rust_begin_unwind
                               at src/libstd/panicking.rs:378
  12:     0x564eebc5bd51 - core::panicking::panic_fmt::he958d82119fc3512
                               at src/libcore/panicking.rs:85
  13:     0x564eebc5bc9d - core::panicking::panic::h2182320983324d30
                               at src/libcore/panicking.rs:52
  14:     0x564eeb9c3ccd - sourcemap::encoder::serialize_mappings::he5368c381bf98d54
                               at /home/kdy/.cargo/registry/src/github.com-1ecc6299db9ec823/sourcemap-5.0.0/src/encoder.rs:42
  15:     0x564eeb9d425e - <sourcemap::types::SourceMap as sourcemap::encoder::Encodable>::as_raw_sourcemap::h6f43577b5b0cb976
                               at /home/kdy/.cargo/registry/src/github.com-1ecc6299db9ec823/sourcemap-5.0.0/src/encoder.rs:94
  16:     0x564eea427742 - sourcemap::encoder::encode::hc6722bd8ffafcc03
                               at /home/kdy/.cargo/registry/src/github.com-1ecc6299db9ec823/sourcemap-5.0.0/src/encoder.rs:16
  17:     0x564eea423dfb - sourcemap::types::SourceMap::to_writer::h0c5635f97e1a60b7
                               at /home/kdy/.cargo/registry/src/github.com-1ecc6299db9ec823/sourcemap-5.0.0/src/types.rs:478
  18:     0x564eea2741f3 - swc::Compiler::print::{{closure}}::h7ca634e6010c1591
                               at src/lib.rs:174
  19:     0x564eea272bba - swc::Compiler::run::{{closure}}::{{closure}}::h087de114b333ac69
                               at src/lib.rs:73
  20:     0x564eea4253a8 - scoped_tls::ScopedKey<T>::set::hc190b4c7bb37c6a0
                               at /home/kdy/.cargo/registry/src/github.com-1ecc6299db9ec823/scoped-tls-1.0.0/src/lib.rs:137
  21:     0x564eea272dde - swc::Compiler::run::{{closure}}::h19dcdc2181546ec4
                               at src/lib.rs:71
  22:     0x564eea4254e8 - scoped_tls::ScopedKey<T>::set::hc2cdff0b532954cc
                               at /home/kdy/.cargo/registry/src/github.com-1ecc6299db9ec823/scoped-tls-1.0.0/src/lib.rs:137
  23:     0x564eea272b04 - swc::Compiler::run::hbd5e47b903b96686
                               at src/lib.rs:69
  24:     0x564eea06bb9f - swc::Compiler::print::hc94c69917cd8ed00
                               at src/lib.rs:132
  25:     0x564eea277284 - swc::Compiler::process_js::{{closure}}::hc69470c45ef572d0
                               at src/lib.rs:325
  26:     0x564eea272c0d - swc::Compiler::run::{{closure}}::{{closure}}::h178632b111fc47b3
                               at src/lib.rs:73
  27:     0x564eea4258c8 - scoped_tls::ScopedKey<T>::set::hff32a7a440a593d2
                               at /home/kdy/.cargo/registry/src/github.com-1ecc6299db9ec823/scoped-tls-1.0.0/src/lib.rs:137
  28:     0x564eea272d61 - swc::Compiler::run::{{closure}}::h142cf64906a83caa
                               at src/lib.rs:71
  29:     0x564eea425769 - scoped_tls::ScopedKey<T>::set::hd54f0bac5f538779
                               at /home/kdy/.cargo/registry/src/github.com-1ecc6299db9ec823/scoped-tls-1.0.0/src/lib.rs:137
  30:     0x564eea2729f7 - swc::Compiler::run::h320243fd7a12f915
                               at src/lib.rs:69
  31:     0x564eea276b46 - swc::Compiler::process_js::h21f3a17244ee20b0
                               at src/lib.rs:305
  32:     0x564eea06bf19 - swc::Compiler::process_js_file::h6fc3ce69f2d0d0ef
                               at src/lib.rs:294
  33:     0x564eea000038 - source_map::file::{{closure}}::hd88aaa7625176259
                               at tests/source_map.rs:15
  34:     0x564eea002ac1 - testing::Tester::print_errors::{{closure}}::hfba82fd93183fa04
                               at /home/kdy/projects/swc/testing/src/lib.rs:121
  35:     0x564eea000baf - scoped_tls::ScopedKey<T>::set::h45b840b7d7281ae6
                               at /home/kdy/.cargo/registry/src/github.com-1ecc6299db9ec823/scoped-tls-1.0.0/src/lib.rs:137
  36:     0x564eea002839 - testing::Tester::print_errors::h7c03500bc5d6c45d
                               at /home/kdy/projects/swc/testing/src/lib.rs:121
  37:     0x564eea000d90 - source_map::file::hf02ac7086b00092b
                               at tests/source_map.rs:9
  38:     0x564eea000deb - source_map::issue_622::h4c1bee7e0e1fc348
                               at tests/source_map.rs:55
  39:     0x564eea00092a - source_map::issue_622::{{closure}}::hdfd4a6add5f16595
                               at tests/source_map.rs:54
  40:     0x564eea000ece - core::ops::function::FnOnce::call_once::h914c35de894d1b9d
                               at /rustc/0eb878d2aa6e3a1cb315f3f328681b26bb4bffdb/src/libcore/ops/function.rs:232
  41:     0x564eea00fcbf - <alloc::boxed::Box<F> as core::ops::function::FnOnce<A>>::call_once::h78dc70e80892ee4d
                               at /rustc/0eb878d2aa6e3a1cb315f3f328681b26bb4bffdb/src/liballoc/boxed.rs:1017
  42:     0x564eebc426f7 - __rust_maybe_catch_panic
                               at src/libpanic_unwind/lib.rs:86
  43:     0x564eea02ba36 - std::panicking::try::h1193c460e1718171
                               at /rustc/0eb878d2aa6e3a1cb315f3f328681b26bb4bffdb/src/libstd/panicking.rs:281
  44:     0x564eea02ba36 - std::panic::catch_unwind::h163cbf2d8c71ea4f
                               at /rustc/0eb878d2aa6e3a1cb315f3f328681b26bb4bffdb/src/libstd/panic.rs:394
  45:     0x564eea02ba36 - test::run_test_in_process::hc0ac989c0c877eda
                               at src/libtest/lib.rs:542
  46:     0x564eea02ba36 - test::run_test::run_test_inner::{{closure}}::h16bcc9da299fa080
                               at src/libtest/lib.rs:451
  47:     0x564eea003046 - std::sys_common::backtrace::__rust_begin_short_backtrace::h2d43b07eb58e6fec
                               at /rustc/0eb878d2aa6e3a1cb315f3f328681b26bb4bffdb/src/libstd/sys_common/backtrace.rs:130
  48:     0x564eea007836 - std::thread::Builder::spawn_unchecked::{{closure}}::{{closure}}::h7f845242723d073e
                               at /rustc/0eb878d2aa6e3a1cb315f3f328681b26bb4bffdb/src/libstd/thread/mod.rs:475
  49:     0x564eea007836 - <std::panic::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once::hd0a6ccfb2079bc4a
                               at /rustc/0eb878d2aa6e3a1cb315f3f328681b26bb4bffdb/src/libstd/panic.rs:318
  50:     0x564eea007836 - std::panicking::try::do_call::h97ab63ef8bb63089
                               at /rustc/0eb878d2aa6e3a1cb315f3f328681b26bb4bffdb/src/libstd/panicking.rs:303
  51:     0x564eebc426f7 - __rust_maybe_catch_panic
                               at src/libpanic_unwind/lib.rs:86
  52:     0x564eea0083a6 - std::panicking::try::h8790f980cc12634a
                               at /rustc/0eb878d2aa6e3a1cb315f3f328681b26bb4bffdb/src/libstd/panicking.rs:281
  53:     0x564eea0083a6 - std::panic::catch_unwind::h8447e5cab7f3c687
                               at /rustc/0eb878d2aa6e3a1cb315f3f328681b26bb4bffdb/src/libstd/panic.rs:394
  54:     0x564eea0083a6 - std::thread::Builder::spawn_unchecked::{{closure}}::h08723c2810a9c6d3
                               at /rustc/0eb878d2aa6e3a1cb315f3f328681b26bb4bffdb/src/libstd/thread/mod.rs:474
  55:     0x564eea0083a6 - core::ops::function::FnOnce::call_once{{vtable.shim}}::hb90e59c7caac7f1f
                               at /rustc/0eb878d2aa6e3a1cb315f3f328681b26bb4bffdb/src/libcore/ops/function.rs:232
  56:     0x564eebc2d67f - <alloc::boxed::Box<F> as core::ops::function::FnOnce<A>>::call_once::ha0137aa2056446ec
                               at /rustc/0eb878d2aa6e3a1cb315f3f328681b26bb4bffdb/src/liballoc/boxed.rs:1017
  57:     0x564eebc418a0 - <alloc::boxed::Box<F> as core::ops::function::FnOnce<A>>::call_once::h335d32c504b6aa26
                               at /rustc/0eb878d2aa6e3a1cb315f3f328681b26bb4bffdb/src/liballoc/boxed.rs:1017
  58:     0x564eebc418a0 - std::sys_common::thread::start_thread::h6e1133a4f62ec9db
                               at src/libstd/sys_common/thread.rs:13
  59:     0x564eebc418a0 - std::sys::unix::thread::Thread::new::thread_start::h0a1909c3d640ef9b
                               at src/libstd/sys/unix/thread.rs:80
  60:     0x7fd3181bf6db - start_thread
  61:     0x7fd317cd088f - __clone
  62:                0x0 - <unknown>

It also eats lots of memory.
스크린샷, 2020-03-02 16-35-40

Modernize the crate

This crate is pretty old and has a number of outdated/unidiomatic features. For example:

  • SourceMapBuilder::into_sourcemap should be called build.
  • SourceMap stores tokens as an unordered vector with a separate ordered index. There is no clear reason tokens can't be stored in an ordered way in the first place (as a vector or BTreeMap).
  • SourceMap stores sources as a Vec<String>, but sources may be null in the sourcemap JSON format. This leads to null sources being replaced with "" during parsing, to unclear benefit.
  • SourceMap has a lot of getter and setter methods that could just as well be replaced with mutable access to the corresponding fields.

Make sourcemap mappings editable

In order to make debug id injection work properly in sentry-cli, we will need to be able to edit mappings reliably. This will require some internal restructuring of the Sourcemap type.

Sourcemap parsing incorrectly prefixes sources with sourceRoot

When we parse a sourcemap, we prefix the sources with the sourceRoot under some circumstances:

let x = x.unwrap_or_default();
let is_valid = !x.is_empty()
&& (x.starts_with('/')
|| x.starts_with("http:")
|| x.starts_with("https:"));
if is_valid {
x
} else {
format!("{source_root}/{x}")
}

This is a problem because in the parsed sourcemap each source is now prefixed, but the sourceRoot field is also still around. This means that when we write the sourcemap out, the prefix is duplicated. Reading the sourcemap again then prepends another copy of the prefix to each source, and so on.

For a simple example, consider the test

#[test]
fn test_roundtrip() {
    let sm = br#"{
        "version": 3,
        "file": "foo.js",
        "sources": [
            "./bar.js",
            "./baz.js"
        ],
        "sourceRoot": "webpack:///",
        "sourcesContent": [null, null],
        "names": [],
        "mappings": ""
    }"#;

    let sm = SourceMap::from_slice(sm).unwrap();
    let mut out = Vec::new();
    sm.to_writer(&mut out).unwrap();

    let sm_new = SourceMap::from_slice(&out).unwrap();
    assert_eq!(sm_new.sources, sm.sources);
}

It fails with

assertion `left == right` failed
  left: ["webpack:///webpack:///./bar.js", "webpack:///webpack:///./baz.js"]
 right: ["webpack:///./bar.js", "webpack:///./baz.js"]

RFC: Scope (fn) name inference

Over at https://github.com/Swatinem/js-source-scopes I have been experimenting with extracting scopes, and their names from minified source; and re-mapping the individual name components using a sourcemaps names, with surprisingly great success.

We should make up our minds how we expect this inference to work.

This inference works differently in the rendered stack traces of various browsers, which is the primary use-case we are after.

Discussion

Function instance name

The Function instance name is defined like this:

The value of the "name" property is a String that is descriptive of the function. The name has no semantic significance but is typically a variable or property name that is used to refer to the function at its point of definition in ECMAScript code.

For named functions, this is trivial, but for anonymous functions, the spec defines a whole section on how the name should be inferred:

In particular the runtime semantics for NamedEvaluation is being "called" with the name as defined using various ambient syntax.

For example const a = () => {}; is clearly defined by the spec as the following:

LexicalBinding : BindingIdentifier Initializer

    1. Let bindingId be StringValue of BindingIdentifier.
    2. ...
    3. If IsAnonymousFunctionDefinition(Initializer) is true, then
        a. Let value be ? NamedEvaluation of Initializer with argument bindingId.

The final .name property of functions is evaluated at runtime and does support dynamic constructs like so:

$ a = "abc", b = {[a]: () => {}}, b.abc.name
> "abc"

Names for property Assignments

The spec only infers name from relatively simple constructs. Neither Firefox nor Chrome infer a name for this snippet:

$ a = {}, a.abc = () => {}, a.abc.name
> ""

Interesting though that at least the firefox devtools internally do infer the name, and the stacktrace even has the complete object path to the function:

Bildschirmfoto 2022-05-25 um 15 24 55

My prototype infers the name from the complete expression the function was assigned to.

Class/Prototype/Constructors

Another interesting case is classes, constructors, and methods either on the class directly or on the prototype.

class X {
  static y() {}
  z() {}
}
X.prototype.w = () => {};

As before, the .name property for all of these is their name, except w for which it is empty.
However, when throwing an Error, the stack trace will say X.w in chrome/node, and X.prototype.w in Firefox for the method defined on the prototype, but just z in Firefox for the class method.
In stack traces, Chrome/Node will prefix this with the class name of that specific instance, not the name of the class defining the method if you use inheritance!

My personal intuition says that I would like to have the names X.y for the static function, and X.prototype.z for the instance function; as that is how you would refer to them via code.

Anonymous Callbacks

The most complicated case would be "completely" anonymous functions used as callbacks.

const myCallback = useCallback(() => {}, []);

Per spec, this function has no name itself. In stack traces, Chrome/Node do not infer a name at all. Firefox however lists all of the named scopes in which the callback is defined.

Options that we have in this case are:

  • Anonymous callback to useCallback.
  • Anonymous callback which results in myCallback.

The second option is very much specific to the use-case in React, as the framework returns a wrapper that is then being called. We can’t infer from pure syntax if that is the case, or if the callback is immediately invoked, such as with Array.prototype.map, in which case the first option would make more sense.

Sourcemap locations may be resolved incorrectly in Firefox

Please consider this issue an FYI for other Sentry users as much as anything else, since I think the fix will happen in Firefox.

We recently ran into a problem where JavaScript crash reports from Firefox were resolved to the wrong location in the original code. The issue turned out to be that Firefox expresses column numbers in stack traces differently than other browsers. Chrome and Safari report the column number in UTF-16 code units, whereas Firefox reports them in Unicode code points. From what I can tell, this package assumes UTF-16 code units. If a minified source bundle contains many characters (eg. emojis) that require multiple UTF-16 characters to encode, this can result in incorrectly resolved source locations when processing crash reports in Firefox.

I filed an issue at tc39/proposal-error-stacks#42 and https://bugzilla.mozilla.org/show_bug.cgi?id=1746374 to try to find an upstream resolution.

Improving the `SourceMapBuilder` API

This is part of #71.

In general I think the "nice" style of builder API is along the lines of

Builder::new()
    .add_something()
    .add_more()
    []
    .build()

By contrast, SourceMapBuilder has a bunch of of get and set methods. Unfortunately the "flow" style won't work for e.g. add_source because it needs to return the source ID.

I would advocate for getting rid at least of the following methods:

  • get_file
  • get_source_root
  • get_source, set_source
  • get_source_contents
  • strip_prefixes (just do that before adding the sources)

I'm not entirely sure about set_source_contents and load_local_source_contents. Being able to set source contents after the fact might come in handy. But there's also something to be said for only setting everything once.

add/add_with_id/add_raw/add_token should probably condensed and/or cleaned up—it's not easy to understand the difference between them and why e.g. add takes both a source and a source_id parameter.

Overflow Panic in VLQ

The following test inserted into src/vlq.rs causes an overflow panic in debug mode:

#[test]
fn test_overflow() {
    parse_vlq_segment("00000000000000");
}

causes

thread 'vlq::test_overflow' panicked at 'attempt to shift left with overflow', src/vlq.rs:34

results from SourceMap::lookup_token is not correct

you can find a playground project in https://github.com/Brooooooklyn/sourcemap-issue

and yarn && yarn start to see the follow result:

let s = sm.lookup_token(1, 113946).unwrap()
println!("content: {}, column: {}", s.get_source().unwrap(), s.get_src_line());
// '../static/js/main.4a1cd49f.js', 2357

and the result from https://www.npmjs.com/package/source-map:

const jsDecoder = new SourceMapConsumer(sourcemap)
const jsResult = jsDecoder.originalPositionFor({
  line: 1,
  column: 113946
})

console.log(jsResult)
// { source: 'service.ts', line: 10, column: 0, name: null }

Eagerly remove useless mappings

In a sourcemap, consecutive mappings to the same location are redundant, because a mapping always applies until the end of the line or the start of the next mapping. For example, consider a sourcemap containing the following mappings:

[…]
(17, 23) -> ("foo.bar", 95, 8)
(17, 31) -> ("foo.bar", 95, 8)
(18, 5)  -> ("foo.bar", 95, 8)
[…]

The middle mapping is redundant—it adds no information that isn't already covered by the first mapping. By contrast, the third mapping is relevant because it's on a new line.

Removing these redundant mappings could for example be done in SourceMapBuilder::into_sourcemap and/or as part of Sourcemap::rewrite.

We might want to consider doing this as part of #71.

impl DoubleEndedIterator for TokenIter

In some case, I want to search token in reverse to find the last match token, but TokenIter impl Iteractor trait only, so sm.tokens().rev() will not compile.

Implement hermes source maps

We currently cannot handle hermes source maps but it should be possible with some larger changes in the library (refs getsentry/sentry-react-native#649)

We probably want to normalize x_facebook_sources somehow because that contains scope mapping information which is exactly what we need. This means we no longer need to access minified sources for figuring out the function name.

Rough Overview

Hermes internally compiles javascript sources to bytecode directly which means there is actually no minified javascript behind the scenes. Currently we use the minified source to figure out a few things that are crucial for processing:

  • we use the comment in the minified file to locate the source map (
    pub fn locate_sourcemap_reference<R: Read>(rdr: R) -> Result<SourceMapRef> {
    for line in BufReader::new(rdr).lines() {
    let line = line?;
    if line.starts_with("//# sourceMappingURL=") || line.starts_with("//@ sourceMappingURL=") {
    let url = str::from_utf8(&line.as_bytes()[21..])?.trim().to_owned();
    if line.starts_with("//@") {
    return Ok(SourceMapRef::LegacyRef(url));
    } else {
    return Ok(SourceMapRef::Ref(url));
    }
    }
    }
    Ok(SourceMapRef::Missing)
    }
    )
  • we use the minified file to reverse tokenize from the error location to the function scope (heuristic) (

    rust-sourcemap/src/types.rs

    Lines 592 to 601 in a086b92

    pub fn get_original_function_name<'a>(
    &self,
    line: u32,
    col: u32,
    minified_name: &str,
    sv: &'a SourceView<'a>,
    ) -> Option<&str> {
    self.lookup_token(line, col)
    .and_then(|token| sv.get_original_function_name(token, minified_name))
    }
    )

These two things are generally used by sentry and assume some behavior already. In particular we assume that the reported function name is the minified file.

With Hermes both of those assumptions are invalidated. First of all because there is no minified file we cannot use our heuristics. Thankfully though Hermes source maps have a second mapping in them called x_facebook_sources which contains mappings from a token location to the enclosing function. The metro symbolicator already uses this to map back to a function name.

Secondly because there is just one huge source map for all files in a react native app we need a separate algorithm to map from the reported filename in a stack to the section in the source map. In the past (with RAM bundles) we rewrote them to virtual files. (see the ram bundle code). Here we probably want to add an abstraction at a higher level where we provide a file name and it returns where in the source map it would be and which source index in it (as there are multiple virtual files in it).

This means we want to change the "get original function name" logic to make the minified source view optional for the hermes case where the source map has enough information to get back to the original name.

Changes in Sentry

After we fixed this here we also need to do some changes in sentry obviously. Currently we rewrite source maps in sentry-cli on the way up and if they have not been rewritten then, they are rewritten again by sentry on loading. We probably do not want to do anything like this with hermes source maps.

The changes there will likely be these:

  1. use the 0 based indexing for hermes instead of 1 indexed like we do for normal source maps
  2. add a layer of indirection for selecting source maps for hermes / react-native to not look for the minified source file but navigate to the source map directly (assume it always comes with the same fixed name?)
  3. not pass the minified file when looking for the original function name.

Bug in `SourceMapIndex::flatten`

index-map.zip

The attached zip file contains an index sourcemap (index.android.bundle.map) and its flattened form (flattened.map). Looking up line 154, column 1938 in the original map correctly yields

lookup line: 153, column: 1937:
  name: "Error"
  source file: "/Users/krystofwoldrich/random/ram-bundle-sentry-test/sentry-test/App.tsx"
  source line: 92
  source column: 42
  minified line: 0
  minified column: 1932
  original function: not found
  source line:
    Sentry.captureException(new Error('First error'));

but looking up the same line and column in the flattened map yields

lookup line: 153, column: 1937:
  name: "white"
  source file: "/Users/krystofwoldrich/random/ram-bundle-sentry-test/sentry-test/App.tsx"
  source line: 87
  source column: 64
  minified line: 153
  minified column: 1935
  original function: not found
  source line:
    backgroundColor: isDarkMode ? Colors.black : Colors.white,

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.