Coder Social home page Coder Social logo

rschlaikjer / erlang-atrace-flamegraphs Goto Github PK

View Code? Open in Web Editor NEW
41.0 4.0 12.0 557 KB

Generate flamegraphs from Android method trace files

License: Apache License 2.0

Erlang 46.92% HTML 10.46% Perl 42.10% Dockerfile 0.51%
android performance tools flamegraph

erlang-atrace-flamegraphs's Introduction

AFlame

Visualizer for android profiling information (try it!)

AFlame is a tool for extracting data from Android method trace files without having to use Traceview.

In order to generate some trace files, wrap your code under test like so:

Debug.startMethodTracing("traceName", 128 * 1024 * 1024);
// Code under test
Debug.stopMethodTracing();

If you are testing a particularly intensive section, you can use the sampling tracing method available on newer APIs that considerably ameliorates the tracer's performance impact.

This project relies on Brendan Gregg's excellent Flamegraph project to generate the graphs themselves.

To test out this tool without running it yourself, you can upload your trace file to https://aflame.rhye.org/ and get a result pretty quickly. If you don't have any traces yet, you can take a look at a sample trace from a recipe app here

I have posted in depth about the Android trace format here.

onCreate

To run a local copy of AFlame (with rebar):

git clone [email protected]:rschlaikjer/erlang-atrace-flamegraphs.git aflame
cd aflame
# Update files/app.config to use a port and trace output location that suit
ERL_FLAGS="-config files/app.config" rebar3 shell
# Open it up in your browser
xdg-open http://localhost:8192/

This repository also contains a Dockerfile that can be used to run the system without installing all the necessary dependencies on the host. To do so,

# From within the top level of this repo
docker build .
# Take the successfully built hash from the docker build output as $HASH
# For very large traces, you may need to increase the memory the container
# is allowed to use by passing --memory=3g (or more, as necessary).
docker run --rm -it $HASH

Android Trace file format

The information here is based on the ART implementation of tracing, which can be viewed here: https://android.googlesource.com/platform/art/+/master/runtime/trace.cc https://android.googlesource.com/platform/art/+/master/runtime/trace.h

*version section

Plain text section of key=value pairs. Includes general info about the trace.

  • data-file-overflow: Whether or not this trace overflowed its buffer and so is missing events
  • clock: One of thread-cpu (Clock is CPU time), wall (Clock is wall clock) or dual (Both are present)
  • elapsed-time-usec: total time elapsed during the trace
  • num-method-calls: total number of calls contained in the trace
  • clock-call-overhead-nsec: The time it takes to actually check the clock.
  • vm: Art or Dalvik
  • pid: PID of the process under trace

For traces with GC tracking, the header can also contain:

  • alloc-count: Number of allocated objects
  • alloc-size: Size of allocated objects (bytes)
  • gc-count: Number of GCs completed during trace

*threads section

Plain text section of tab separated pairs. Each pair is a base 10 thread ID, and a thread name. This numeric thread ID is referenced from the trace data section.

*methods section

Plain text section of tab separated pairs. Each line is a 5-tuple of:

  • Method ID (in hex). A primary key for referring to a method entry.
  • Class name
  • Method name
  • Type signature
  • Source file
  • Source line (not always present)

The method ID is referenced from the trace data section. The number here is actually the real method ID left shifted two bits - this is explained in the data section.

Raw trace data section

All numbers are stored as little-endian.

Header:

  • Magic (4 bytes): 'SLOW'
  • Version (2 bytes): Most traces seem to be version 3.
  • Data Offset (2 bytes): The number of bytes after the header that the data section begins.
  • Start Time (8 bytes): The time (microseconds) that the trace began
  • Recordsize (2 bytes): The number of bytes in each trace record.

Data records (v3):

  • ThreadId (2 bytes): References the threads table above
  • MethodId (4 bytes): Compound method ID and method Action (enter 0x0, exit 0x1 and unwind 0x2)
  • TimeDelta (4 bytes): Offset into the trace at which this record was logged (microsecs)
  • Wall clock delta (4 bytes) Offset into the trace (wall time) at which this record was logged

Each method record denotes an event that occurred in the given method. The method itself is identified by the high 14 bits of the compound method ID, which matches against the methods table higher in the file. The action is one of three currently defined:

  • 0x0 (Method enter): Recorded when the runtime enters a method call
  • 0x1 (Method exit): Recorded when the runtime exits a method call
  • 0x2 (Unwind): Recorded when the runtime unwinds the stack. Seems to be functionally equivalent to a return.

I interpret the unwind as a return for the purposes of the graphs; this is based on what I can glean from reading up on when an unwind is issued by the interpreter here , and the part of the interpreter that then checks for kDexNoIndex here

By pushing and popping the records onto a virtual stack per thread, you can recreate the trace, and use the offsets to work out how much time was spent in each function.

All parsing code can be seen here.

License

Copyright 2018 Ross Schlaikjer

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

erlang-atrace-flamegraphs's People

Contributors

rschlaikjer avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

erlang-atrace-flamegraphs's Issues

Unable to build docker image on M1 Mac

Has anyone successfully built the docker image on an M1 Mac?

Result of docker build . for me:

 => ERROR [stage-1 2/5] RUN apt-get update && apt-get install -y openssl daemontools                                                                                                                             1.6s
------
 > [stage-1 2/5] RUN apt-get update && apt-get install -y openssl daemontools:
#6 0.307 Ign:1 http://deb.debian.org/debian stretch InRelease
#6 0.327 Ign:2 http://security.debian.org/debian-security stretch/updates InRelease
#6 0.332 Ign:3 http://deb.debian.org/debian stretch-updates InRelease
#6 0.338 Ign:4 http://security.debian.org/debian-security stretch/updates Release
#6 0.349 Ign:5 http://security.debian.org/debian-security stretch/updates/main all Packages
#6 0.367 Ign:6 http://deb.debian.org/debian stretch Release
#6 0.390 Ign:7 http://deb.debian.org/debian stretch-updates Release
#6 0.413 Ign:8 http://deb.debian.org/debian stretch/main all Packages
#6 0.443 Ign:9 http://deb.debian.org/debian stretch/main arm64 Packages
#6 0.488 Ign:10 http://deb.debian.org/debian stretch-updates/main arm64 Packages
#6 0.491 Ign:11 http://security.debian.org/debian-security stretch/updates/main arm64 Packages
#6 0.507 Ign:5 http://security.debian.org/debian-security stretch/updates/main all Packages
#6 0.532 Ign:12 http://deb.debian.org/debian stretch-updates/main all Packages
#6 0.580 Ign:8 http://deb.debian.org/debian stretch/main all Packages
#6 0.609 Ign:9 http://deb.debian.org/debian stretch/main arm64 Packages
#6 0.636 Ign:10 http://deb.debian.org/debian stretch-updates/main arm64 Packages
#6 0.668 Ign:12 http://deb.debian.org/debian stretch-updates/main all Packages
#6 0.685 Ign:11 http://security.debian.org/debian-security stretch/updates/main arm64 Packages
#6 0.698 Ign:8 http://deb.debian.org/debian stretch/main all Packages
#6 0.805 Ign:9 http://deb.debian.org/debian stretch/main arm64 Packages
#6 0.836 Ign:10 http://deb.debian.org/debian stretch-updates/main arm64 Packages
#6 0.861 Ign:12 http://deb.debian.org/debian stretch-updates/main all Packages
#6 0.892 Ign:8 http://deb.debian.org/debian stretch/main all Packages
#6 0.917 Ign:5 http://security.debian.org/debian-security stretch/updates/main all Packages
#6 0.919 Ign:9 http://deb.debian.org/debian stretch/main arm64 Packages
#6 0.942 Ign:10 http://deb.debian.org/debian stretch-updates/main arm64 Packages
#6 0.968 Ign:12 http://deb.debian.org/debian stretch-updates/main all Packages
#6 0.992 Ign:8 http://deb.debian.org/debian stretch/main all Packages
#6 1.017 Ign:9 http://deb.debian.org/debian stretch/main arm64 Packages
#6 1.040 Ign:10 http://deb.debian.org/debian stretch-updates/main arm64 Packages
#6 1.060 Ign:11 http://security.debian.org/debian-security stretch/updates/main arm64 Packages
#6 1.065 Ign:12 http://deb.debian.org/debian stretch-updates/main all Packages
#6 1.073 Ign:5 http://security.debian.org/debian-security stretch/updates/main all Packages
#6 1.089 Ign:8 http://deb.debian.org/debian stretch/main all Packages
#6 1.132 Err:9 http://deb.debian.org/debian stretch/main arm64 Packages
#6 1.132   404  Not Found
#6 1.156 Err:10 http://deb.debian.org/debian stretch-updates/main arm64 Packages
#6 1.156   404  Not Found
#6 1.180 Ign:12 http://deb.debian.org/debian stretch-updates/main all Packages
#6 1.217 Ign:11 http://security.debian.org/debian-security stretch/updates/main arm64 Packages
#6 1.233 Ign:5 http://security.debian.org/debian-security stretch/updates/main all Packages
#6 1.382 Ign:11 http://security.debian.org/debian-security stretch/updates/main arm64 Packages
#6 1.402 Ign:5 http://security.debian.org/debian-security stretch/updates/main all Packages
#6 1.545 Err:11 http://security.debian.org/debian-security stretch/updates/main arm64 Packages
#6 1.545   404  Not Found [IP: 151.101.194.132 80]
#6 1.550 Reading package lists...
#6 1.558 W: The repository 'http://security.debian.org/debian-security stretch/updates Release' does not have a Release file.
#6 1.558 W: The repository 'http://deb.debian.org/debian stretch Release' does not have a Release file.
#6 1.558 W: The repository 'http://deb.debian.org/debian stretch-updates Release' does not have a Release file.
#6 1.558 E: Failed to fetch http://security.debian.org/debian-security/dists/stretch/updates/main/binary-arm64/Packages  404  Not Found [IP: 151.101.194.132 80]
#6 1.558 E: Failed to fetch http://deb.debian.org/debian/dists/stretch/main/binary-arm64/Packages  404  Not Found
#6 1.558 E: Failed to fetch http://deb.debian.org/debian/dists/stretch-updates/main/binary-arm64/Packages  404  Not Found
#6 1.558 E: Some index files failed to download. They have been ignored, or old ones used instead.
------
executor failed running [/bin/sh -c apt-get update && apt-get install -y openssl daemontools]: exit code: 100

Error while parsing trace

Error processing A45AE0568CD949FC1C8DB04987A79E18

Please open an issue if this is not what you had hoped for.
Guru meditation:
{{case_clause,[<<>>]},
 [{aflame_trace_parser,parse_method_line,1,
                       [{file,"/src/_build/prod/lib/aflame/src/aflame_trace_parser.erl"},
                        {line,401}]},
  {aflame_trace_parser,'-parse_methods/2-fun-2-',2,
                       [{file,"/src/_build/prod/lib/aflame/src/aflame_trace_parser.erl"},
                        {line,390}]},
  {lists,map,2,[{file,"lists.erl"},{line,1239}]},
  {aflame_trace_parser,parse_methods,2,
                       [{file,"/src/_build/prod/lib/aflame/src/aflame_trace_parser.erl"},
                        {line,388}]},
  {aflame_trace_parser,parse,1,
                       [{file,"/src/_build/prod/lib/aflame/src/aflame_trace_parser.erl"},
                        {line,276}]},
  {aflame_trace_parser,handle_cast,2,
                       [{file,"/src/_build/prod/lib/aflame/src/aflame_trace_parser.erl"},
                        {line,98}]},
  {gen_server,try_dispatch,4,[{file,"gen_server.erl"},{line,637}]},
  {gen_server,handle_msg,6,[{file,"gen_server.erl"},{line,711}]}]}

AS 3.2 Beta 4

compilation error

I've been trying to run this project inside my WSL. When compiling, I get the following mistake :
image

As I am not familiar with Erlang, I was wondering if you could help me figure this out.

SVG has 0 byte size (empty)

Hi @rschlaikjer , I follow the instructions to run on my local machine (Mac OS), and the website already launched successfully. However, when i uploaded my trace file, it shows empty page with a title on top of it (A6E3FB43121C7F8C76948513BF121785). Do you know what might be wrong here ? i just checked the file is generated for index.html and temporary .trace file, as well as dozen of SVG files. However all of the SVG files are empty (zero byte).
Thank you in advance

local setup error while uploading

16:33:37.821 [error] Caught {badmatch,{error,enoent}} error:[{aflame_fs,get_temp_file,0,[{file,"/home/ubuntu/erlang-atrace-flamegraphs/_build/default/lib/aflame/src/aflame_fs.erl"},{line,48}]},{aflame_web,upload_trace_multipart,1,[{file,"/home/ubuntu/erlang-atrace-flamegraphs/_build/default/lib/aflame/src/aflame_web.erl"},{line,146}]},{aflame_web,handle,2,[{file,"/home/ubuntu/erlang-atrace-flamegraphs/_build/default/lib/aflame/src/aflame_web.erl"},{line,41}]},{cowboy_handler,handler_handle,4,[{file,"/home/ubuntu/erlang-atrace-flamegraphs/_build/default/lib/cowboy/src/cowboy_handler.erl"},{line,111}]},{cowboy_protocol,execute,4,[{file,"/home/ubuntu/erlang-atrace-flamegraphs/_build/default/lib/cowboy/src/cowboy_protocol.erl"},{line,442}]}]

Any quick fix for this? Thanks.

Could not run on windows with docker!

Actually I have mailed you about this. I could not run this project using docker. It builds successfully and run command executed also. But I could not see any output on 127.0.01:2003 or 127.0.01:8192 or 127.0.01:80 ! Is there anything where I need to change the path or anything before running it? I am stuck here for two day! would you please guide me on this?
SS of output

Docker container exited(137)

The docker container is exiting on very big trace file sizes (128.9MB), probably for an OutOfMemory exception.

Is there a fast solution to avoid this?

Dockerized version failing

Hi, I've tried running the dockerized version as instructed in the readme, but I get the following:

❯ docker run --rm -it a851c0de6ffc
Exec: /deploy/erts-10.0.1/bin/erlexec -noshell -noinput +Bd -boot /deploy/releases/20160514.051731/start -mode embedded -boot_var ERTS_LIB_DIR /deploy/lib -config /deploy/releases/20160514.051731/sys.config -args_file /deploy/releases/20160514.051731/vm.args -pa -- foreground
Root: /deploy
/deploy
=CRASH REPORT==== 19-May-2020::06:54:57.903138 ===
  crasher:
    initial call: lager_handler_watcher:init/1
    pid: <0.585.0>
    registered_name: []
    exception exit: noproc
      in function  gen:do_for_proc/2 (gen.erl, line 228)
      in call from gen_event:rpc/2 (gen_event.erl, line 239)
      in call from lager_handler_watcher:install_handler2/3 (/src/_build/default/lib/lager/src/lager_handler_watcher.erl, line 117)
      in call from lager_handler_watcher:init/1 (/src/_build/default/lib/lager/src/lager_handler_watcher.erl, line 51)
      in call from gen_server:init_it/2 (gen_server.erl, line 374)
      in call from gen_server:init_it/6 (gen_server.erl, line 342)
    ancestors: [lager_handler_watcher_sup,lager_sup,<0.570.0>]
    message_queue_len: 0
    messages: []
    links: [<0.573.0>]
    dictionary: []    trap_exit: false
    status: running    heap_size: 610
    stack_size: 27
    reductions: 298
  neighbours:

=INFO REPORT==== 19-May-2020::06:54:57.908790 ===
estatsd will flush stats to "127.0.0.1":2003 every 10000ms

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.