getsentry / symbolicator Goto Github PK

Native Symbolication as a Service

Home Page: https://getsentry.github.io/symbolicator/

License: MIT License

Rust 98.69% Makefile 0.19% Dockerfile 0.26% Shell 0.30% Lua 0.14% Jsonnet 0.42%

symbolicator's Introduction

A symbolication service for native stacktraces and minidumps with symbol server support. It's a flexible frontend for parts of the symbolic library.

Documentation

Contained Binaries

This repository contains several binary crates that can be run with cargo run -p <PACKAGE>:

symbolicator: Runs a symbolicator server. Calling cargo run without a package will run this binary.
symsorter: Sorts debug files into a structure usable by symbolicator.
wasm-split: Adds build IDs to wasm files.
symbolicli: Performs symbolication on the CLI. See the readme.

Compiling

Symbolicator is written in Rust and requires the latest stable Rust compiler.

To compile, run:

cargo build --release

The resulting binaries ends up in target/release/ along with debug information files. On Linux, debug information is part of the executable and might need to be stripped using objcopy.

Usage with Sentry

The following requires a recent git version of Sentry.

While Symbolicator aims to not be tied to Sentry's usecases, Sentry has a hard dependency on Symbolicator to process native stacktraces. To get it running for local development:

In your ~/.sentry/sentry.conf.py:

# Allow Symbolicator's request IP to fetch debug files from Sentry.
INTERNAL_SYSTEM_IPS = ["127.0.0.1"]

In your ~/.sentry/config.yml:
```
symbolicator.enabled: true
```

Then run sentry devservices up to download and start Symbolicator.

Development

To build Symbolicator, we require the latest stable Rust.

We use VSCode for development. This repository contains settings files configuring code style, linters, and useful features. When opening the project for the first time, make sure to install the Recommended Extensions, as they will allow editor assist during coding.

The root of the repository contains a Makefile with useful commands for development:

make check: Runs code formatting checks and linters. This is useful before opening a pull request.
make test: Runs unit tests, integration tests and Python package tests (see below for more information).
make all: Runs all checks and tests. This runs most of the tasks that are also performed in CI.
make clean: Removes all build artifacts, the virtualenv and cached files.

Building and Running

The easiest way to rebuild and run Symbolicator is using cargo. Depending on the configuration, you may need to have a local instance of Sentry running.

# Rebuild and run with all features
cargo run -- run

For quickly verifying that Symbolicator compiles after making some changes, you can also use cargo check:

cargo check

Local configuration

By default, Symbolicator listens on port 3021. To override this and other configuration values, create a file local.yml in the project root folder. It is excluded from version control by default. Then, start Symbolicator and point to the configuration file:

cargo run -- run --config local.yml

Tests

The test suite comprises unit tests, and integration tests, both of which are implemented as part of the Rust crates and can be run via:

# Tests for default features
make test

Note: On macOS, the default file descriptor limit of 256 is too low and causes test failures. Before running tests, consider to increase is to a higher value:

ulimit -n 4096

Linting

We use rustfmt and clippy from the latest stable channel for code formatting and linting. To make sure that these tools are set up correctly and running with the right configuration, use the following make targets:

# Format the entire codebase
make format

# Run clippy on the entire codebase
make lint

symbolicator's People

Contributors

Stargazers

Watchers

Forkers

pombredanne ktoso dschmidt kingantz slve orfeasz ainopara gregcliffordbitrise embarkstudios 4astra kteofanidis isabella232 samsartor josb mainframeindustries liu3399shuai brancz hamled sentry-arm jaanae benlau bkotsopoulossc yayanet vaartis flub zetarus liuxidev ashwoods ufosky-rust vaind tryweirdier wescarr sonubabukindo409 twcctz200 rreverser zzjpeter tdesveaux sitedata tsondergaard iq-scm simonsan thanosbellos shreyan11 doslin

symbolicator's Issues

Metrics Reporting

Would be useful to report various runtime metrics to (dog)statsd-compatible backend, similar to what semaphore does.

Implement request IDs

Proposal:

/symbolicate: Start symbolication and tie symbolication task to http transaction. No request ID involved. If the request is done, the symbolication is done (downloads and symcaches might stilll have tasks)
/symbolicate/background: Start symbolication and return request ID immediately.
/symbolicate/background/<id>: Poll for status.

Provide with offset for system symbol files

since we cant get lineno for system symbol files
is it possible to return offset - how atos returns offset

unsymbolcated 3 UIKitCore 0x00000001fd2e9890 0x1fcfc8000 + 3283088
Symbolciated -> UIKit -[UICollectionView _performBatchUpdates:completion:invalidationContext:]+80

For a developer when they compare crash reports, it seems to be a useful information to have offset.

see https://developer.apple.com/library/archive/technotes/tn2151/_index.html

Symbol server depricated, but new one is not yet ready.

Hello,

root@server:/opt/symbolserver# RUST_BACKTRACE=1 /usr/bin/cargo run -- --config=symbolserver.yml run
    Finished dev [unoptimized + debuginfo] target(s) in 0.90s
     Running `target/debug/sentry-symbolserver --config=symbolserver.yml run`
thread 'main' panicked at 'called `Option::unwrap()` on a `None` value', libcore/option.rs:335:21
stack backtrace:
   0: std::sys::unix::backtrace::tracing::imp::unwind_backtrace
             at libstd/sys/unix/backtrace/tracing/gcc_s.rs:49
   1: std::sys_common::backtrace::print
             at libstd/sys_common/backtrace.rs:71
             at libstd/sys_common/backtrace.rs:59
   2: std::panicking::default_hook::{{closure}}
             at libstd/panicking.rs:211
   3: std::panicking::default_hook
             at libstd/panicking.rs:227
   4: std::panicking::rust_panic_with_hook
             at libstd/panicking.rs:463
   5: std::panicking::begin_panic_fmt
             at libstd/panicking.rs:350
   6: rust_begin_unwind
             at libstd/panicking.rs:328
   7: core::panicking::panic_fmt
             at libcore/panicking.rs:71
   8: core::panicking::panic
             at libcore/panicking.rs:51
   9: <core::option::Option<T>>::unwrap
             at /checkout/src/libcore/macros.rs:20
  10: openssl::ssl::SslContextBuilder::set_options
             at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/openssl-0.9.7/src/ssl/mod.rs:843
  11: openssl::ssl::connector::ctx
             at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/openssl-0.9.7/src/ssl/connector.rs:35
  12: openssl::ssl::connector::SslConnectorBuilder::new
             at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/openssl-0.9.7/src/ssl/connector.rs:52
  13: native_tls::imp::TlsConnector::builder
             at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/native-tls-0.1.1/src/imp/openssl.rs:166
  14: native_tls::TlsConnector::builder
             at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/native-tls-0.1.1/src/lib.rs:356
  15: hyper_native_tls::NativeTlsClient::new
             at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/hyper-native-tls-0.2.2/src/lib.rs:111
  16: libsymbolserver::s3::new_hyper_client
             at src/s3.rs:67
  17: libsymbolserver::s3::S3::from_config
             at src/s3.rs:90
  18: libsymbolserver::memdb::stash::MemDbStash::new
             at src/memdb/stash.rs:161
  19: libsymbolserver::api::server::ApiServer::new
             at src/api/server.rs:93
  20: libsymbolserver::cli::run_action
             at src/cli.rs:371
  21: libsymbolserver::cli::execute
             at src/cli.rs:277
  22: libsymbolserver::cli::main
             at src/cli.rs:80
  23: sentry_symbolserver::main
             at src/main.rs:4
  24: std::rt::lang_start::{{closure}}
             at /checkout/src/libstd/rt.rs:74
  25: std::panicking::try::do_call
             at libstd/rt.rs:59
             at libstd/panicking.rs:310
  26: __rust_maybe_catch_panic
             at libpanic_unwind/lib.rs:105
  27: std::rt::lang_start_internal
             at libstd/panicking.rs:289
             at libstd/panic.rs:374
             at libstd/rt.rs:58
  28: std::rt::lang_start
             at /checkout/src/libstd/rt.rs:74
  29: main
  30: __libc_start_main
  31: _start

can you please help me with old one server, which is kinda deprecated, but new one is not ready yet?

[Panic] thread 'main' panicked at 'attempted to leave type `linked_hash_map::Node<proto::op::Query, dns_lru::LruValue>` uninitialized

Start up sentry snuba relay symbolicator.
Trigger a crash in iOS client.
Get panic bellow.

RUST_BACKTRACE=full ./symbolicator run --config local.yml

## config file
cache_dir: "/data"
bind: "0.0.0.0:3021"
logging:
  level: "warn"
metrics:
  statsd: null
sentry_dsn: my_sentry_dsn_url

thread 'main' panicked at 'attempted to leave type `linked_hash_map::Node<proto::op::Query, dns_lru::LruValue>` uninitialized, which is invalid', /rustc/b1496c6e606dd908dd651ac2cce89815e10d7fc5/library/core/src/mem/mod.rs:659:9
stack backtrace:
   0: rust_begin_unwind
             at /rustc/b1496c6e606dd908dd651ac2cce89815e10d7fc5/library/std/src/panicking.rs:483:5
   1: core::panicking::panic_fmt
             at /rustc/b1496c6e606dd908dd651ac2cce89815e10d7fc5/library/core/src/panicking.rs:85:14
   2: core::panicking::panic
             at /rustc/b1496c6e606dd908dd651ac2cce89815e10d7fc5/library/core/src/panicking.rs:50:5
   3: core::mem::uninitialized
             at /rustc/b1496c6e606dd908dd651ac2cce89815e10d7fc5/library/core/src/mem/mod.rs:659:9
   4: linked_hash_map::LinkedHashMap<K,V,S>::ensure_guard_node
             at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/linked-hash-map-0.5.2/src/lib.rs:174:52
   5: linked_hash_map::LinkedHashMap<K,V,S>::insert
             at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/linked-hash-map-0.5.2/src/lib.rs:304:9
   6: lru_cache::LruCache<K,V,S>::insert
             at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/lru-cache-0.1.2/src/lib.rs:119:23
   7: trust_dns_resolver::dns_lru::DnsLru::insert
             at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/trust-dns-resolver-0.10.3/src/dns_lru.rs:168:9
   8: <trust_dns_resolver::lookup_state::InsertCache as futures::future::Future>::poll
             at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/trust-dns-resolver-0.10.3/src/lookup_state.rs:395:41
   9: <trust_dns_resolver::lookup_state::QueryState<C> as futures::future::Future>::poll
             at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/trust-dns-resolver-0.10.3/src/lookup_state.rs:612:24
  10: <alloc::boxed::Box<F> as futures::future::Future>::poll
             at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-0.1.29/src/future/mod.rs:113:13
  11: futures::future::chain::Chain<A,B,C>::poll
             at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-0.1.29/src/future/chain.rs:26:23
  12: <futures::future::then::Then<A,B,F> as futures::future::Future>::poll
             at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-0.1.29/src/future/then.rs:32:9
  13: <alloc::boxed::Box<F> as futures::future::Future>::poll
             at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-0.1.29/src/future/mod.rs:113:13
  14: <trust_dns_resolver::lookup_ip::LookupIpFuture<C> as futures::future::Future>::poll
             at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/trust-dns-resolver-0.10.3/src/lookup_ip.rs:94:25
  15: futures::future::chain::Chain<A,B,C>::poll
             at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-0.1.29/src/future/chain.rs:32:48
  16: <futures::future::and_then::AndThen<A,B,F> as futures::future::Future>::poll
             at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-0.1.29/src/future/and_then.rs:32:9
  17: <futures::future::either::Either<A,B> as futures::future::Future>::poll
             at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-0.1.29/src/future/either.rs:35:37
  18: <trust_dns_resolver::async_resolver::Background<F,G> as futures::future::Future>::poll
             at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/trust-dns-resolver-0.10.3/src/async_resolver/mod.rs:345:9
  19: <actix::actors::resolver::ResolveFut as actix::fut::ActorFuture>::poll
             at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/actix-0.7.9/src/actors/resolver.rs:360:19
  20: actix::fut::chain::Chain<A,B,C>::poll
             at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/actix-0.7.9/src/fut/chain.rs:38:49
  21: <actix::fut::and_then::AndThen<A,B,F> as actix::fut::ActorFuture>::poll
             at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/actix-0.7.9/src/fut/and_then.rs:44:9
  22: <alloc::boxed::Box<F> as actix::fut::ActorFuture>::poll
             at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/actix-0.7.9/src/fut/mod.rs:313:9
  23: actix::fut::chain::Chain<A,B,C>::poll
             at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/actix-0.7.9/src/fut/chain.rs:38:49
  24: <alloc::boxed::Box<F> as actix::fut::ActorFuture>::poll
             at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/actix-0.7.9/src/fut/mod.rs:313:9
  25: <actix::contextimpl::ContextFut<A,C> as futures::future::Future>::poll
             at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/actix-0.7.9/src/contextimpl.rs:355:23
  26: <actix::supervisor::Supervisor<A> as futures::future::Future>::poll
             at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/actix-0.7.9/src/supervisor.rs:147:19
  27: <alloc::boxed::Box<F> as futures::future::Future>::poll
             at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-0.1.29/src/future/mod.rs:113:13
  28: futures::task_impl::Spawn<T>::poll_future_notify::{{closure}}
             at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-0.1.29/src/task_impl/mod.rs:329:45
  29: futures::task_impl::Spawn<T>::enter::{{closure}}
             at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-0.1.29/src/task_impl/mod.rs:399:27
  30: futures::task_impl::std::set
             at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-0.1.29/src/task_impl/std/mod.rs:83:13
  31: futures::task_impl::Spawn<T>::enter
             at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-0.1.29/src/task_impl/mod.rs:399:9
  32: futures::task_impl::Spawn<T>::poll_fn_notify
             at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-0.1.29/src/task_impl/mod.rs:291:9
  33: futures::task_impl::Spawn<T>::poll_future_notify
             at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-0.1.29/src/task_impl/mod.rs:329:9
  34: tokio_current_thread::scheduler::Scheduled<U>::tick
             at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-current-thread-0.1.7/src/scheduler.rs:351:25
  35: tokio_current_thread::scheduler::Scheduler<U>::tick::{{closure}}
             at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-current-thread-0.1.7/src/scheduler.rs:330:47
  36: tokio_current_thread::Borrow<U>::enter::{{closure}}::{{closure}}
             at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-current-thread-0.1.7/src/lib.rs:800:40
  37: tokio_current_thread::CurrentRunner::set_spawn
             at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-current-thread-0.1.7/src/lib.rs:841:9
  38: tokio_current_thread::Borrow<U>::enter::{{closure}}
             at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-current-thread-0.1.7/src/lib.rs:800:13
  39: std::thread::local::LocalKey<T>::try_with
             at /rustc/b1496c6e606dd908dd651ac2cce89815e10d7fc5/library/std/src/thread/local.rs:272:16
  40: std::thread::local::LocalKey<T>::with
             at /rustc/b1496c6e606dd908dd651ac2cce89815e10d7fc5/library/std/src/thread/local.rs:248:9
  41: tokio_current_thread::Borrow<U>::enter
             at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-current-thread-0.1.7/src/lib.rs:798:9
  42: tokio_current_thread::scheduler::Scheduler<U>::tick
             at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-current-thread-0.1.7/src/scheduler.rs:330:24
  43: tokio_current_thread::Entered<P>::tick
             at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-current-thread-0.1.7/src/lib.rs:624:9
  44: tokio_current_thread::Entered<P>::block_on
             at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-current-thread-0.1.7/src/lib.rs:514:13
  45: tokio::runtime::current_thread::runtime::Runtime::block_on::{{closure}}
             at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-0.1.22/src/runtime/current_thread/runtime.rs:200:23
  46: tokio::runtime::current_thread::runtime::Runtime::enter::{{closure}}::{{closure}}::{{closure}}::{{closure}}
             at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-0.1.22/src/runtime/current_thread/runtime.rs:241:25
  47: tokio_executor::global::with_default::{{closure}}
             at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-executor-0.1.10/src/global.rs:221:9
  48: std::thread::local::LocalKey<T>::try_with
             at /rustc/b1496c6e606dd908dd651ac2cce89815e10d7fc5/library/std/src/thread/local.rs:272:16
  49: std::thread::local::LocalKey<T>::with
             at /rustc/b1496c6e606dd908dd651ac2cce89815e10d7fc5/library/std/src/thread/local.rs:248:9
  50: tokio_executor::global::with_default
             at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-executor-0.1.10/src/global.rs:190:5
  51: tokio::runtime::current_thread::runtime::Runtime::enter::{{closure}}::{{closure}}::{{closure}}
             at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-0.1.22/src/runtime/current_thread/runtime.rs:239:21
  52: tokio_timer::timer::handle::with_default
             at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-timer-0.2.13/src/timer/handle.rs:74:5
  53: tokio::runtime::current_thread::runtime::Runtime::enter::{{closure}}::{{closure}}
             at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-0.1.22/src/runtime/current_thread/runtime.rs:232:17
  54: tokio_timer::clock::clock::with_default
             at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-timer-0.2.13/src/clock/clock.rs:125:5
  55: tokio::runtime::current_thread::runtime::Runtime::enter::{{closure}}
             at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-0.1.22/src/runtime/current_thread/runtime.rs:231:13
  56: tokio_reactor::with_default
             at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-reactor-0.1.12/src/lib.rs:220:5
  57: tokio::runtime::current_thread::runtime::Runtime::enter
             at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-0.1.22/src/runtime/current_thread/runtime.rs:230:9
  58: tokio::runtime::current_thread::runtime::Runtime::block_on
             at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-0.1.22/src/runtime/current_thread/runtime.rs:198:9
  59: actix::system::SystemRunner::run
             at /root/.cargo/registry/src/github.com-1ecc6299db9ec823/actix-0.7.9/src/system.rs:174:26
  60: symbolicator::server::run
             at ./src/server.rs:38:5
  61: symbolicator::cli::execute
             at ./src/cli.rs:85:25
  62: symbolicator::main
             at ./src/main.rs:35:11
  63: core::ops::function::FnOnce::call_once
             at /rustc/b1496c6e606dd908dd651ac2cce89815e10d7fc5/library/core/src/ops/function.rs:227:5

Override environment tag for sentry integration

We're using symbolicator with sentry to track any symbolicator errors. The environment tag is always getting set to release, and I could not find a way to override it. We'd like to set it to our deployment environment using an envar, similar to what we do with our java and python sentry integrations.

Validate that source IDs are distinct

Special case failing to load `linux-vdso.so.1`

With crashpad on Linux we'll always have this error and no actionable way to fix it:

We should hide this error.

Configure IP whitelist instead of boolean flag

I don't understand why source connections are blocked to the LAN, see:

symbolicator/src/utils/http.rs

Lines 14 to 21 in a886e2e

    
           lazy_static::lazy_static! { 
        
               // https://en.wikipedia.org/wiki/Reserved_IP_addresses#IPv4 
        
               static ref RESERVED_IP_NETS: Vec<IpNetwork> = [ 
        
                   "0.0.0.0/8", "10.0.0.0/8", "100.64.0.0/10", "127.0.0.0/8", "169.254.0.0/16", "172.16.0.0/12", 
        
                   "192.0.0.0/29", "192.0.2.0/24", "192.88.99.0/24", "192.168.0.0/16", "198.18.0.0/15", 
        
                   "198.51.100.0/24", "224.0.0.0/4", "240.0.0.0/4", "255.255.255.255/32", 
        
               ].iter().map(|x| x.parse().unwrap()).collect(); 
        
           }

As discussed in #102, I'm looking at building a small Python adapter to serve symbols into Symbolicator via the HTTP interface. I can hack this list out and rebuild Symbolicator, but I don't understand why it's being blocked in the first place. If nothing else, could weconsider only blocking those addresses when they come in via the multipart symbolication request, and not when they are set in the config yaml?

Add Support for elf-debuginfod

Path format documented here: https://sourceware.org/git/?p=elfutils.git;a=blob;f=debuginfod/debuginfod.8;hb=refs/heads/debuginfod

Implement Two Tier Storage Format (index2.txt)

Microsoft Symbol Server can indicate to clients through the presence of an index2.txt file in the root, that two tier storage is used.

Single tier:

D:\SymStore\Symbols\ntdll.dll\...\
D:\SymStore\Symbols\ntdll.pdb\...\
D:\SymStore\Symbols\kernel32.dll\...\
D:\SymStore\Symbols\kernel32.pdb\...\

Multi tier:

D:\SymStore\Symbols\nt\ntdll.dll\...\
D:\SymStore\Symbols\nt\ntdll.pdb\...\
D:\SymStore\Symbols\ke\kernel32.dll\...\
D:\SymStore\Symbols\ke\kernel32.pdb\...\

Fix download metrics to time returned stream instead of future

S3 support

Make LRU auth caches configurable

Currently all the LRU caches for auth info we have (S3 and GCS) use a hardcoded max (of 100 items). We should make this configurable and also evaluate what a reasonable default is.

Add panic guards

symbolicator responds with lineno 0 on few classes

Example

 "status": "symbolicated",
                    "original_index": 1,
                    "instruction_addr": "0x101185e74",
                    "package": "/var/containers/Bundle/Application/A8FBBA07-8521-4414-BEAB-E7DF258A82FD/CrashSample.app/CrashSample",
                    "lang": "swift",
                    "symbol": "$Ss27_allocateUninitializedArrayySayxG_BptBwlFs7CVarArg_p_Tg5",
                    "sym_addr": "0x101185e00",
                    "function": "specialized _allocateUninitializedArray<A>(_:)",
                    "filename": "<compiler-generated>",
                    "abs_path": "<compiler-generated>",
                    "lineno": 0

{
                    "status": "symbolicated",
                    "original_index": 3,
                    "instruction_addr": "0x1fd2e988c",
                    "package": "/System/Library/PrivateFrameworks/UIKitCore.framework/UIKitCore",
                    "symbol": "-[UIViewController loadViewIfRequired]",
                    "sym_addr": "0x1fd2e94a8",
                    "function": "-[UIViewController loadViewIfRequired]",
                    "lineno": 0
                },

what does lineno 0 mean here? Is there a reason we dont respond with non zero values if exists!

0.3.1 Failed to demangle unknown identifier

Today we upgraded from a quite old version of symbolicator to 0.3.1 and have received several dozen errors saying "Failed to demangle unknown identifier". We're processing Android and iOS crashes.

Is this something for us to be concerned about?

Support NixOS Debug Info System

I was not aware of this but NixOS has a pretty good solution for shipping dwarf information. The cache.nixos.org server has an index for looking up debug information files within NAR archives.

To quote the docs on this:

ELF binaries have a unique build ID embedded in them, allowing
separate debug info files to be looked up. For example, here is the
build ID of a particular build of libnixexpr.so:
$ readelf -a /nix/store/c0srp6xfqyrjrmqhd1pgw6hcrhcghg87-nix-1.12pre5619_346aeee1/lib/libnixexpr.so | grep 'Build ID'
   Build ID: bde350fa1f1bbde3649bfce3ae143b87683d8bf9
Given this build ID, gdb and elfutils will look in
/run/dwarffs/.build-id/bd/e350fa1f1bbde3649bfce3ae143b87683d8bf9.debug
to obtain the debug info for this binary. dwarffs will then look in
https://cache.nixos.org/debuginfo/<build-ID>:
$ curl https://cache.nixos.org/debuginfo/bde350fa1f1bbde3649bfce3ae143b87683d8bf9
{"archive":"../nar/0jz52hawk8wida5b2544i484mws9ddkl8jjpjmw6rzrjf83jr5zw.nar.xz","member":"lib/debug/.build-id/bd/e350fa1f1bbde3649bfce3ae143b87683d8bf9.debug"}
That is, the debug info is contained in the NAR file
https://cache.nixos.org/nar/0jz52hawk8wida5b2544i484mws9ddkl8jjpjmw6rzrjf83jr5zw.nar.xz
(along with the debug info for other binaries from the same
package). dwarffs will unpack all these debug info files to a cache
(/var/cache/dwarffs/) and serve them from there.

While you can't directly address the debug information file from this you can get the path to the NAR archive and you also know where in the archive the debug companion file is. What you can't know unfortunately is where the binary itself is. We know it's in the archive but we would have to scan for it.

Supporting this would be interesting but it would likely require us to have another cache of unpacked nix archives so we don't end up unpacking and fetching the same things over and over again.

Better Casing Control for SymStore

SSQP wants the signature to be lowercase except for age which is uppercase. We currently use breakpad which is uppercase except age is lowercase.

I believe we need these options to work in the real world for the signature:

PDB signature:
- all uppercase (unity, case sensitive)
- lowercase with uppercase age (what SSQP wants but doesn't emit)
- all lowercase (what we already have and what SSQP emits -.-)
PE signature:
- uppercase then lowercase (unity, case sensitive)
- lowercase entirely (what SSQP wants and emits)

Maybe these values would make sense in addition to default, lowercase and uppercase:

symstore: PDB signature all uppercase, PE signature uppercase than lowercase
ssqp: all lowercase with uppercase age for PDB

😒

Make timeouts configurable

Right now they are hardcoded to fit our prod env, but we should have those values be configurable.

Question: Grouping support

Hello there,

I have a question about grouping possibilities. As I can see from the documentation, there is no support for grouping (something like, generate a hash from the thread where application crashed). And I wonder how you solve this on your side? In general, we want to produce hashes for similar crashes during the symbolication to be able to group them in another system.

So, what do you think, is it okay to prepare a PR for the symbolicator and extend it with the desired functionality? Or it is out of the scope of the project?

Thanks for the answer. And thank you for this handy service!

Issue with symbolicating with a particular system symbol file CFNetwork

while trying to symbolicate one stacktrace

trace from crash report

5   CFNetwork                           0x0000000185d1fe88 0x185b66000 + 1810056

file got downloaded in my local cache
symbolicator response

{\"status\":\"missing_symbol\",\"original_index\":5,\"instruction_addr\":\"0x185d1fe88\",\"package\":\"/System/Library/Frameworks/CFNetwork.framework/CFNetwork\"}

Not sure why it says missing symbol when all required things are available. Found this issue for this particular system symbol file in multiple stacktraces

Block download requests to internal IPs

Outgoing requests for HTTP sources need to verify that no connection is made to an internal host based on some configurable blacklist.

This does not apply for S3 sources, since those only allow connection directly with Amazon; assuming that the local network is not compromised.

The Sentry source is somewhat special cased, as this is an internal use case. Most likely, there should be configuration whether Sentry sources are even allowed defaulting to false.

Failed to symbolicate big so file

The so file size is 1.1g. I can't find any further detail about why its failed. Please help.

Symbolicator log

symbolicator-0 symbolicator 2020-01-07T12:57:03Z [symbolicator::server] INFO: Starting http server:
 0.0.0.0:3021
symbolicator-0 symbolicator 2020-01-07T12:57:03Z [actix_net::server::server] INFO: Starting 4 worke
rs
symbolicator-0 symbolicator 2020-01-07T12:57:03Z [actix_net::server::server] INFO: Starting server 
on 0.0.0.0:3021
symbolicator-0 symbolicator 2020-01-07T12:57:03Z [symbolicator::server] INFO: Starting system      
symbolicator-0 symbolicator 2020-01-07T13:00:06Z [symbolicator::actors::symcaches] WARN: Failed to 
write symcache: failed to parse object

Error while fetching cficache: computation was canceled internally

Server: sentry, snuba and symbolicator from the latest docker image in k8s.

Client: sentry-native 0.14, offical exmaple_crash.exe build by vs2017 with debug config on win10 1903

Upload debug info by:
sentry-cli upload-dif -o sentry -p native-cc --include-sources --wait -- .

Symbolicator log:

symbolicator-0 symbolicator thread 'tokio-runtime-worker-0' panicked at 'slice index starts at 70397 but ends at 63488', src/libcore/slice/mod.rs:2591:5
symbolicator-0 symbolicator stack backtrace:
symbolicator-0 symbolicator    0: <unknown>
symbolicator-0 symbolicator    1: core::fmt::write
symbolicator-0 symbolicator    2: <unknown>
symbolicator-0 symbolicator    3: <unknown>
symbolicator-0 symbolicator    4: <unknown>
symbolicator-0 symbolicator    5: std::panicking::rust_panic_with_hook
symbolicator-0 symbolicator    6: <unknown>
symbolicator-0 symbolicator    7: rust_begin_unwind
symbolicator-0 symbolicator    8: core::panicking::panic_fmt
symbolicator-0 symbolicator    9: core::slice::slice_index_order_fail
symbolicator-0 symbolicator   10: goblin::pe::exception::UnwindInfo::parse
symbolicator-0 symbolicator   11: goblin::pe::exception::ExceptionData::get_unwind_info
symbolicator-0 symbolicator   12: symbolic_minidump::cfi::CfiCache::from_object
symbolicator-0 symbolicator   13: <unknown>
symbolicator-0 symbolicator   14: <unknown>
symbolicator-0 symbolicator   15: <unknown>
symbolicator-0 symbolicator   16: __rust_maybe_catch_panic
symbolicator-0 symbolicator   17: <unknown>
symbolicator-0 symbolicator   18: <unknown>
symbolicator-0 symbolicator   19: <unknown>
symbolicator-0 symbolicator   20: <unknown>
symbolicator-0 symbolicator   21: <unknown>
symbolicator-0 symbolicator   22: __rust_maybe_catch_panic
symbolicator-0 symbolicator   23: <unknown>
symbolicator-0 symbolicator   24: <unknown>
symbolicator-0 symbolicator   25: tokio_threadpool::worker::Worker::run
symbolicator-0 symbolicator   26: <unknown>
symbolicator-0 symbolicator   27: <unknown>
symbolicator-0 symbolicator   28: <unknown>
symbolicator-0 symbolicator   29: <unknown>
symbolicator-0 symbolicator   30: <unknown>
symbolicator-0 symbolicator   31: <unknown>
symbolicator-0 symbolicator   32: <unknown>
symbolicator-0 symbolicator   33: <unknown>
symbolicator-0 symbolicator   34: __rust_maybe_catch_panic
symbolicator-0 symbolicator   35: <unknown>
symbolicator-0 symbolicator   36: <unknown>
symbolicator-0 symbolicator   37: <unknown>
symbolicator-0 symbolicator   38: start_thread
symbolicator-0 symbolicator   39: __clone
symbolicator-0 symbolicator note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
symbolicator-0 symbolicator 2019-12-23T12:54:02Z [symbolicator::actors::symbolication] INFO: Error while fetching cficache: computation was canceled internally
symbolicator-0 symbolicator
symbolicator-0 symbolicator stack backtrace:
symbolicator-0 symbolicator    0: <unknown> (0x55b28bcad468)
symbolicator-0 symbolicator    1: failure::backtrace::Backtrace::new::hae2649ba15221c47 (0x55b28bcad620)
symbolicator-0 symbolicator    2: <unknown> (0x55b28b549238)
symbolicator-0 symbolicator    3: <unknown> (0x55b28b489b28)
symbolicator-0 symbolicator    4: <unknown> (0x55b28b54d110)
symbolicator-0 symbolicator    5: <unknown> (0x55b28b47fab2)
symbolicator-0 symbolicator    6: <unknown> (0x55b28b2b93c4)
symbolicator-0 symbolicator    7: <unknown> (0x55b28b466223)
symbolicator-0 symbolicator    8: <unknown> (0x55b28b2cc3c9)
symbolicator-0 symbolicator    9: <unknown> (0x55b28b2c1896)
symbolicator-0 symbolicator   10: <unknown> (0x55b28b47951d)
symbolicator-0 symbolicator   11: <unknown> (0x55b28b4c94b3)
symbolicator-0 symbolicator   12: __rust_maybe_catch_panic (0x55b28bce5a1a)
symbolicator-0 symbolicator   13: <unknown> (0x55b28b41f471)
symbolicator-0 symbolicator   14: <unknown> (0x55b28b3dffba)
symbolicator-0 symbolicator   15: <unknown> (0x55b28b762892)
symbolicator-0 symbolicator   16: <unknown> (0x55b28b78e81b)
symbolicator-0 symbolicator   17: <unknown> (0x55b28b777929)
symbolicator-0 symbolicator   18: <unknown> (0x55b28b78f0d6)
symbolicator-0 symbolicator   19: <unknown> (0x55b28b76bab3)
symbolicator-0 symbolicator   20: <unknown> (0x55b28b76b637)
symbolicator-0 symbolicator   21: <unknown> (0x55b28b76848d)
symbolicator-0 symbolicator   22: <unknown> (0x55b28b769199)
symbolicator-0 symbolicator   23: <unknown> (0x55b28b74b090)
symbolicator-0 symbolicator   24: <unknown> (0x55b28b786928)
symbolicator-0 symbolicator   25: <unknown> (0x55b28b786eb6)
symbolicator-0 symbolicator   26: __rust_maybe_catch_panic (0x55b28bce5a1a)
symbolicator-0 symbolicator   27: <unknown> (0x55b28b7905f6)
symbolicator-0 symbolicator   28: <unknown> (0x55b28bccc95f)
symbolicator-0 symbolicator   29: <unknown> (0x55b28bce4d80)
symbolicator-0 symbolicator   30: start_thread (0x7f188d8334a4)
symbolicator-0 symbolicator   31: __clone (0x7f188e7bdd0f)
symbolicator-0 symbolicator   32: <unknown> (0x0)

Related file:

root@symbolicator-0:/data/cficaches/7# head -n 10 sentry_project_95_sentryinternal
STACK CFI INIT aac90 4c .cfa: $rsp 8 + .ra: .cfa 8 - ^
STACK CFI aac90 .cfa: $rsp 256 +
STACK CFI INIT aacf0 4c .cfa: $rsp 8 + .ra: .cfa 8 - ^
STACK CFI aacf0 .cfa: $rsp 256 +
STACK CFI INIT aad50 4c .cfa: $rsp 8 + .ra: .cfa 8 - ^
STACK CFI aad50 .cfa: $rsp 256 +
STACK CFI INIT aadb0 4c .cfa: $rsp 8 + .ra: .cfa 8 - ^
STACK CFI aadb0 .cfa: $rsp 256 +
STACK CFI INIT aae10 4c .cfa: $rsp 8 + .ra: .cfa 8 - ^
STACK CFI aae10 .cfa: $rsp 256 +
root@symbolicator-0:/data/cficaches/7# tail -n 10 sentry_project_95_sentryinternal
STACK CFI INIT 1f1320 34 .cfa: $rsp 8 + .ra: .cfa 8 - ^
STACK CFI 1f1320 .cfa: $rsp 256 +
STACK CFI INIT 1f1370 34 .cfa: $rsp 8 + .ra: .cfa 8 - ^
STACK CFI 1f1370 .cfa: $rsp 256 +
STACK CFI INIT 1f13c0 34 .cfa: $rsp 8 + .ra: .cfa 8 - ^
STACK CFI 1f13c0 .cfa: $rsp 256 +
STACK CFI INIT 1f1410 34 .cfa: $rsp 8 + .ra: .cfa 8 - ^
STACK CFI 1f1410 .cfa: $rsp 256 +
STACK CFI INIT 1f1460 15 .cfa: $rsp 8 + .ra: .cfa 8 - ^
STACK CFI 1f1460 .cfa: $rsp 48 +

Sentry:

Check how fatObject is handled

[Build Fail] cargo build --release

Environment

System Env

Linux version 4.19.0-6.el7.ucloud.x86_64 (root@2d6b517b2939) (gcc version 4.8.5 20150623 (Red Hat 4.8.5-16) (GCC))

cargo --version
cargo 1.49.0-nightly (79b397d72 2020-10-15)

Steps to reproduce

cargo build --verbose
    Updating git repository `https://github.com/getsentry/symbolic`
    Updating git submodule `https://github.com/getsentry/breakpad`
error: failed to get `symbolic` as a dependency of package `symbolicator v0.2.0 (/app/sentry/symbolicator)`

Caused by:
  failed to load source for dependency `symbolic`

Caused by:
  Unable to update https://github.com/getsentry/symbolic?branch=fix/demangle-fixes

Caused by:
  failed to update submodule `minidump/third_party/breakpad`

Caused by:
  object not found - no match for id (de33482a9d3b28cd0013813ce0e39c57eb79b0ee); class=Odb (9); code=NotFound (-3)

Change Cargo.toml

change

symbolic = { git = "https://github.com/getsentry/symbolic", branch = "fix/demangle-fixes", features = ["common-serde", "demangle", "minidump-serde", "symcache"] }

to 

symbolic = "7.5.0"

cargo update 
success

cargo build 

error: failed to run custom build command for `openssl-sys v0.9.58`

Caused by:
  process didn't exit successfully: `/app/sentry/symbolicator/target/debug/build/openssl-sys-1f28d14d28a5cca7/build-script-main` (exit code: 101)
  --- stdout
  cargo:rustc-cfg=const_fn
  cargo:rerun-if-env-changed=X86_64_UNKNOWN_LINUX_GNU_OPENSSL_LIB_DIR
  X86_64_UNKNOWN_LINUX_GNU_OPENSSL_LIB_DIR unset
  cargo:rerun-if-env-changed=OPENSSL_LIB_DIR
  OPENSSL_LIB_DIR unset
  cargo:rerun-if-env-changed=X86_64_UNKNOWN_LINUX_GNU_OPENSSL_INCLUDE_DIR
  X86_64_UNKNOWN_LINUX_GNU_OPENSSL_INCLUDE_DIR unset
  cargo:rerun-if-env-changed=OPENSSL_INCLUDE_DIR
  OPENSSL_INCLUDE_DIR unset
  cargo:rerun-if-env-changed=X86_64_UNKNOWN_LINUX_GNU_OPENSSL_DIR
  X86_64_UNKNOWN_LINUX_GNU_OPENSSL_DIR unset
  cargo:rerun-if-env-changed=OPENSSL_DIR
  OPENSSL_DIR unset
  cargo:rerun-if-env-changed=OPENSSL_NO_PKG_CONFIG
  cargo:rerun-if-env-changed=PKG_CONFIG
  cargo:rerun-if-env-changed=OPENSSL_STATIC
  cargo:rerun-if-env-changed=OPENSSL_DYNAMIC
  cargo:rerun-if-env-changed=PKG_CONFIG_ALL_STATIC
  cargo:rerun-if-env-changed=PKG_CONFIG_ALL_DYNAMIC
  cargo:rerun-if-env-changed=PKG_CONFIG_PATH_x86_64-unknown-linux-gnu
  cargo:rerun-if-env-changed=PKG_CONFIG_PATH_x86_64_unknown_linux_gnu
  cargo:rerun-if-env-changed=HOST_PKG_CONFIG_PATH
  cargo:rerun-if-env-changed=PKG_CONFIG_PATH
  cargo:rerun-if-env-changed=PKG_CONFIG_LIBDIR_x86_64-unknown-linux-gnu
  cargo:rerun-if-env-changed=PKG_CONFIG_LIBDIR_x86_64_unknown_linux_gnu
  cargo:rerun-if-env-changed=HOST_PKG_CONFIG_LIBDIR
  cargo:rerun-if-env-changed=PKG_CONFIG_LIBDIR
  cargo:rerun-if-env-changed=PKG_CONFIG_SYSROOT_DIR_x86_64-unknown-linux-gnu
  cargo:rerun-if-env-changed=PKG_CONFIG_SYSROOT_DIR_x86_64_unknown_linux_gnu
  cargo:rerun-if-env-changed=HOST_PKG_CONFIG_SYSROOT_DIR
  cargo:rerun-if-env-changed=PKG_CONFIG_SYSROOT_DIR
  run pkg_config fail: "`\"pkg-config\" \"--libs\" \"--cflags\" \"openssl\"` did not exit successfully: exit code: 1\n--- stderr\nPackage openssl was not found in the pkg-config search path.\nPerhaps you should add the directory containing `openssl.pc\'\nto the PKG_CONFIG_PATH environment variable\nNo package \'openssl\' found\n"

  --- stderr
  thread 'main' panicked at '

  Could not find directory of OpenSSL installation, and this `-sys` crate cannot
  proceed without this knowledge. If OpenSSL is installed and this crate had
  trouble finding it,  you can set the `OPENSSL_DIR` environment variable for the
  compilation process.

  Make sure you also have the development packages of openssl installed.
  For example, `libssl-dev` on Ubuntu or `openssl-devel` on Fedora.

  If you're in a situation where you think the directory *should* be found
  automatically, please open a bug at https://github.com/sfackler/rust-openssl
  and include information about your system as well as this message.

  $HOST = x86_64-unknown-linux-gnu
  $TARGET = x86_64-unknown-linux-gnu
  openssl-sys = 0.9.58

  ', /root/.cargo/registry/src/github.com-1ecc6299db9ec823/openssl-sys-0.9.58/build/find_normal.rs:157:5
  note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
warning: build failed, waiting for other jobs to finish...
error: build failed

Too many open files

On 0.3.1 when under heavy load we've been getting "too many open files" errors on downloads and when writing to symcache. Our current open files limit is 1024. Do you have a recommended limit for production?

Dependabot couldn't fetch the branch/reference for https://github.com/jan-auer/cpp_demangle

Your dependency file specified a branch or reference for https://github.com/jan-auer/cpp_demangle, but Dependabot couldn't find it at the project's source. Has it been removed?

View the update logs.

Prefer PDB for unwind info on 32-bit platforms

Building symbolicator from source for use with Sentry

I have symbolicator running on my system, and it's returning reasonable-looking results when I curl minidumps to it, however I can't seem to get a pip install of Sentry 9.1.2 talking to it. I did the required conf setup (https://github.com/getsentry/symbolicator#usage-with-sentry), but it appears that symbolication is occurring when I send the same minidumps to my test Sentry instance (for example, I get errors in the UI about symbol files which I know are missing).

But the stdout from my Symbolicator instance is not showing it having serviced any requests, so it seems like Sentry is doing its own thing internally. Do I need to build the master branch of Sentry in order to get Symbolicator support, or is there some other piece I'm missing here?

Failed to fetch symbol file from S3 if existing file on docker container is 0 bytes

When symbol file at S3 is missing, symbolicator creates a 0bytes file locally on container. Hence it never updates with new file(if uploaded to S3). Only workaround seems to be restart the docker container. Can we not always looks for S3 files if existing is 0 bytes?

Prepare debug files for a custom Symbol Server

Some of our debug files exceed 2GB limit and unable to be uploaded directly. So now we are testing Symbol Server (S3) and it looks fine.

But the problem is a directory layout. We have 3 platforms and all of them require own paths and utils to generate them.

Is there any way to simplify layout generation? Via CLI or proxy (does https://github.com/getsentry/symbolicator/ may help)?

Add WASM Support

Add support for WASM based on the outcomes of getsentry/symbolic#290.

Since WASM does not map object files into memory out current idea of our protocol likely needs to change. At the moment we use instruction_addr to map into a unified memory space. We then use the information of the debug_meta to find the right images for those instruction addresses. Later in the Sentry UI (for instance) we map the instruction addresses into relative addresses for display purposes only.

With WASM the concept of instruction addresses becomes a bit weird. First of all there are really no absolute addresses and addresses themselves are not unique. One way would be to encode a module ID into the instruction address, the other option would be to add a second value (image_idx for instance) which switches the addr to be relative within the image (as identified by the index in the debug image array).

Given what we the reality is on the ground we likely use offsets within the entire WASM file as input, not offsets within the Code section only. While this is a bit odd as the DWARF file is based on Code offsets it's more consistent with what browsers are sending and how users are likely to be interfacing with it.

Dependabot couldn't fetch the branch/reference for https://github.com/jan-auer/cpp_demangle

Your dependency file specified a branch or reference for https://github.com/jan-auer/cpp_demangle, but Dependabot couldn't find it at the project's source. Has it been removed?

View the update logs.

Symbolicator Should Use AWS Credentials Provided to Running Container

Instead of api/applecrashreport/ requiring credentials to be passed, it would be great if we could provide an AWS role + credentials on service start up for it to use with all S3 requests. Additionally, this should allow non-user tokens that use a session token.

Fix path traversal security bugs in cache actor

ObjectError: failed to look into cache

Sentry Issue: SYMBOLICATOR-D

Custom: Timeout while waiting for response
PayloadError: Timeout while waiting for response
ArcFail: failed to download
  ?, in <F as alloc::boxed::FnBox<T>>::call_box
  File "boxed.rs", line 759, in call_once<T>
  ?, in start_thread
  ?, in __clone
  ?, in <unknown>
...
(33 additional frame(s) were not displayed)

ObjectError: failed to look into cache
  ?, in <F as alloc::boxed::FnBox<T>>::call_box
  File "boxed.rs", line 759, in call_once<T>
  ?, in start_thread
  ?, in __clone
  ?, in <unknown>
...
(33 additional frame(s) were not displayed)

Service 'symbolicator-cleanup' failed to build

when I run ./install.sh , get this error:
Service 'symbolicator-cleanup' failed to build: Get https://us.gcr.io/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)

Run integration tests against Sentry

While symbolicator is an independent service, its main use is in Sentry. Since changes to symbolicator may break integration tests in Sentry, it could be useful to run a subset of them on PR builds.

There is prior art. In Relay, we run the onpremise test suite on the master branch, which creates a full Sentry setup (without test mocks), and ingests an event.

For symbolicator, this would mean:

Run the (or an equivalent of the) "symbolicator_integration" test suite from Sentry.
This test suite ships snapshots, which we must be able to update to get tests passing.
Ideally, this does not use a mocked Sentry setup, but proper configuration like onpremise has.

Symbolicator responds with line 0 for all system files

For all system files, line number is returned as 0. Example:

        {
                    "status": "symbolicated",
                    "original_index": 6,
                    "instruction_addr": "0x1fd8e86cc",
                    "package": "/System/Library/PrivateFrameworks/UIKitCore.framework/UIKitCore",
                    "symbol": "-[UIWindow _setHidden:forced:]",
                    "sym_addr": "0x1fd8e85c0",
                    "function": "-[UIWindow _setHidden:forced:]",
                    "lineno": 0
                },

{
                    "status": "symbolicated",
                    "original_index": 9,
                    "instruction_addr": "0x1fd8aee50",
                    "package": "/System/Library/PrivateFrameworks/UIKitCore.framework/UIKitCore",
                    "symbol": "-[UIApplication _runWithMainScene:transitionContext:completion:]",
                    "sym_addr": "0x1fd8ae844",
                    "function": "-[UIApplication _runWithMainScene:transitionContext:completion:]",
                    "lineno": 0
                },

Make symbolicator not use a global arch for instruction heuristic

Remove request.meta.arch from protocol

Empty frames list in symbolicator response

When trying to symbolicate an apple crash report, received empty frame list with status : "completed".

"stacktraces\":[{\"thread_id\":0,\"is_requesting\":false,\"frames\":[]}],

I do have system symbol files uploaded. so the api response should be with unsymbolicated/symbolicated frame status.

whats the scenario when we get empty frames list? or was it fixed in later versions. Am using december commit 3040b43

SymCaches are invalidated when underlying object did not change

The cache key of a symcache currently contains the IDs of all sources. The idea was that symcaches are invalidated because the user added new sources, and that might yield a new selected object.

symcaches are also invalidated when the underlying object didn't change at all though.

Ideally we would cache symcaches based on the object used. That means we would have to move the branching logic for finding multiple objects a level up.

Symsorter overwrites "meta" file

Let's say I have a directory with 2 Windows files:

PS C:\debug> dir ./unsorted


    Directory: C:\debug\unsorted


Mode                LastWriteTime         Length Name
----                -------------         ------ ----
-a----         3/9/2020   4:12 AM      126614528 chrome.dll
-a----         3/9/2020   4:13 AM     2476789760 chrome.dll.pdb

If I will sort chrome.dll only:

PS C:\debug> ./symsorter.exe  -bundle-id win --output ./sorted-mini --prefix win_release_x64 ./unsorted/chrome.dll
chrome.dll (x86_64) -> ./sorted-mini\win_release_x64\26/020e92257781134c4c44205044422e1/executable

Done: sorted 1 debug files

then 26/020e92257781134c4c44205044422e1/meta will contain:

{"name":"chrome.dll","arch":"x86_64","file_format":"pe"}

But if I will specify chrome.dll.pdb instead of chrome.dll:

PS C:\debug> ./symsorter.exe  -bundle-id win --output ./sorted-mini --prefix win_release_x64 ./unsorted/chrome.dll.pdb
chrome.dll.pdb (x86_64) -> ./sorted-mini\win_release_x64\26/020e92257781134c4c44205044422e1/debuginfo

Done: sorted 1 debug files

then the same file 26/020e92257781134c4c44205044422e1/meta will contain:

{"name":"chrome.dll.pdb","arch":"x86_64","file_format":"pdb"}

======

We using symsorter for preparing debug symbols S3 repository and this "swallow" problem sometimes (depending on sort order) makes some .pdbs unavailable.

Dependabot couldn't fetch the branch/reference for https://github.com/jan-auer/gimli

Your dependency file specified a branch or reference for https://github.com/jan-auer/gimli, but Dependabot couldn't find it at the project's source. Has it been removed?

View the update logs.

Document configuring symbolicator for use with Sentry 10

Hi— I'm using Symbolicator with Sentry in conjunction with a small external service which supplies debug symbols stored elsewhere. I can get it working fine when I run symbolicator standalone and directly curl my minidumps at it, but I can't figure out how to get the onpremise deployment to take the config. I thought initially that maybe the symbolicator.options dict from the main config.yml was extracted and passed through somehow:

https://github.com/getsentry/onpremise/blob/a549ab1300f94149bf347df9277cf7bc50f0e5d7/sentry/config.example.yml#L69-L71

However, upon looking at the running sentry_onpremise_symbolicator_1 container, it appears that there's no config at all in there, and the process is /bin/symbolicator run with no -c flag, which per the docs here means it is running entirely with the default settings.

It seems the intent is for symbol servers to be configured on a per-project basis in Sentry, and the relevant ones are passed to Symbolicator on every request. This is not ideal for me as it makes it more difficult to test Symbolicator's behaviour in isolation as the request is more complicated and there are unknown layers of caching going on in the middle. It's also likely not to be a great fit for my org as we may end up with many similarly-configured minidump projects, all using the same source, where we would not want to have to set it up each time.

Is there a plan to support/document a mechanism for injecting symbolicator config with the onpremise setup method?

Sentry returns empty list when requesting symbols file

I am running into an issue working with symbolicator where sentry returned an empty list of debug files when symbolicator reached out of the debug files list. The Microsoft files had no issue. Sentry seems to be able to communicate otherwise with symbolicator.

I set the two config options in Sentry that are documented in the repo (whitelisted symbolicator in sentry and enabled it). I am not using dev services up but instead running symbolicator on another host in the same network. Is there a config I'm missing?

Implement symbolication error types

Symbol source from apt packages

I'd like to be able to have Symbolicator grab symbols from remote apt endpoints. I have two use-cases for this:

The dbgsym packages available from Debian, which are named lib[foo]-dbgsym_[version]_[arch].deb, eg: http://debug.mirrors.debian.org/debian-debug/pool/main/b/boost1.67/
Some internally-generated symbol packages stored in an an apt repo which are simply named debug-[buildid]_0_[arch].deb.

Both of these could be covered by an external server which exposed the Microsoft Symbol Server API to Symbolicator, and that was my first instinct for how to address this, but then it occurred to me that it might work to have an apt source built right in. Basically, you would give the source information (URL, component, etc), and then supply one or more printf-style templates for package names to look for symbols in. The packages would be expected to contain a single .debug file, optionally compressed, which is the gdb symbols to use.

I'd imagine the configuration looking something like:

  {
    "id": "debian-sid-symbols",
    "type": "apt",
    "source": {
      "url": "http://debug.mirrors.debian.org/debian-debug",
      "dist": "sid-debug",
      "components": ["main"]
    },
    "schemas": "%{libname}-dbgsym_%{libver}_%{arch}"
  },

I would expect the implementation to use libapt bindings (eg https://docs.rs/apt-pkg-native), and then download and unpack locally with dpkg in a temporary directory.

I'm happy to work on this if there's interest from the maintainers and I can get a bit more design guidance. If not, I can definitely achieve what I need to with a small Flask server sitting outside of Symbolicator.

	lazy_static::lazy_static! {
	// https://en.wikipedia.org/wiki/Reserved_IP_addresses#IPv4
	static ref RESERVED_IP_NETS: Vec<IpNetwork> = [
	"0.0.0.0/8", "10.0.0.0/8", "100.64.0.0/10", "127.0.0.0/8", "169.254.0.0/16", "172.16.0.0/12",
	"192.0.0.0/29", "192.0.2.0/24", "192.88.99.0/24", "192.168.0.0/16", "198.18.0.0/15",
	"198.51.100.0/24", "224.0.0.0/4", "240.0.0.0/4", "255.255.255.255/32",
	].iter().map(\|x\| x.parse().unwrap()).collect();
	}

getsentry / symbolicator Goto Github PK

symbolicator's Introduction

Contained Binaries

Compiling

Usage with Sentry

Development

Building and Running

Local configuration

Tests

Linting

symbolicator's People

Contributors

Stargazers

Watchers

Forkers

symbolicator's Issues

Environment

Steps to reproduce

Change Cargo.toml

Recommend Projects

Recommend Topics

Recommend Org