facebookincubator / profilo Goto Github PK

A library for performance traces from production.

Home Page: https://facebookincubator.github.io/profilo

License: Apache License 2.0

Python 5.48% Java 16.45% C++ 28.18% C 44.99% Shell 0.10% JavaScript 0.30% CSS 0.06% Dockerfile 0.02% Starlark 4.29% AIDL 0.13%

profilo's Introduction

Profilo: an Android performance library

Status

THIS PROJECT IS CURRENTLY IN MAINTENANCE MODE. It will not receive any feature updates, only critical security bug patches. On May 1st, 2023, the repo will be fully archived.

Introduction

Profilo is an Android library for collecting performance traces from production builds of an app.

Index

APIs

Currently, none of the APIs are stable and they will change (mostly so that they're easier to use) before release.

Sample usage

The sample app in java/main/com/facebook/profilo/sample shows the most basic usage of the APIs.

You can also find the prebuilt apk in the Releases section.

Demos

A demo script can be found in python/profilo/workflow_demo.py. It contains examples of simple analysis that can be run on Profilo traces. For more information see the trace processing section of the docs.

License

Profilo is Apache 2 licensed, as found in the LICENSE file.

profilo's People

Contributors

Stargazers

Watchers

Forkers

caabernathy dourgulf tuancoltech dstmath morristech chanjlee attwellbrian simpleton siazhy matrixxun jlf-sky inzkhan matrixy trendingtechnology tejasd xvsdf100 gkmhub bobby0809 dodola ezzatfathy ld2006203 zhangweiqwe pianpian315 oujunli sosojustdo tconnally rajeshpatadiya zhaoxuyang quang0309 qwe00921 kazucocoa ricardorey10 yuhonghong66 jingdq hyyzt daodaoliang wangweijun123 hoangpq popahqiu fcccode thinkhy haozi5205 kronar kakayang2011 forkgits spuermax rahul-a alexhe zhyh329 guolei1130 yidazhang 1260380285 anlutong songjinshi foukation dr-aryone zhenqinrui skcodestack blockspacer lawangle amashoshyna ozgunbal flings amanjeetsingh150 noe-joel zjw-swun freemanzyq cangming666 xcleans theindianappguy jiangtao89 hughcoder wenzhu2018 gkodkod hhhaiai dendisuhubdy blackc03r tigerwood markjian wesely s-pace codingzh mdsami yangboyd xiaogezi wildchen arieg419 5l1v3r1 gmaciolek idaretobe rebeltat new-mike n0texpecterr0r harlber sharrychoo knight-zxw burgess2017 burntbrunch mrleebean meiclin

profilo's Issues

Sample App Crashes

Installed and ran the sample apk for release-fef2654. Clicking the "Start tracing" button causes the following crash on an 8.1.0 device (Pixel 2):

03-14 12:04:14.259 19558 19558 E AndroidRuntime: java.lang.UnsatisfiedLinkError: couldn't find DSO to load: libfbjni.so
03-14 12:04:14.259 19558 19558 E AndroidRuntime: 	at com.facebook.soloader.SoLoader.loadLibraryBySoName(SoLoader.java:442)
03-14 12:04:14.259 19558 19558 E AndroidRuntime: 	at com.facebook.soloader.SoLoader.loadLibrary(SoLoader.java:367)
03-14 12:04:14.259 19558 19558 E AndroidRuntime: 	at com.facebook.soloader.SoLoader.loadLibrary(SoLoader.java:332)
03-14 12:04:14.259 19558 19558 E AndroidRuntime: 	at com.facebook.jni.HybridData.<clinit>(HybridData.java:34)
03-14 12:04:14.259 19558 19558 E AndroidRuntime: 	at com.facebook.profilo.writer.NativeTraceWriter.initHybrid(Native Method)
03-14 12:04:14.259 19558 19558 E AndroidRuntime: 	at com.facebook.profilo.writer.NativeTraceWriter.<init>(NativeTraceWriter.java:37)
03-14 12:04:14.259 19558 19558 E AndroidRuntime: 	at com.facebook.profilo.logger.Logger.startWorkerThreadIfNecessary(Logger.java:377)
03-14 12:04:14.259 19558 19558 E AndroidRuntime: 	at com.facebook.profilo.logger.Logger.postCreateTrace(Logger.java:271)
03-14 12:04:14.259 19558 19558 E AndroidRuntime: 	at com.facebook.profilo.core.TraceControl.startTraceInternal(TraceControl.java:327)
03-14 12:04:14.259 19558 19558 E AndroidRuntime: 	at com.facebook.profilo.core.TraceControl.startTrace(TraceControl.java:285)
03-14 12:04:14.259 19558 19558 E AndroidRuntime: 	at com.facebook.profilo.controllers.external.api.ExternalTraceControl.startTrace(ExternalTraceControl.java:42)
03-14 12:04:14.259 19558 19558 E AndroidRuntime: 	at com.facebook.profilo.sample.SampleAppMainActivity$1.onClick(SampleAppMainActivity.java:63)
03-14 12:04:14.259 19558 19558 E AndroidRuntime: 	at android.view.View.performClick(View.java:6294)
03-14 12:04:14.259 19558 19558 E AndroidRuntime: 	at android.widget.CompoundButton.performClick(CompoundButton.java:134)
03-14 12:04:14.259 19558 19558 E AndroidRuntime: 	at android.view.View$PerformClick.run(View.java:24770)
03-14 12:04:14.259 19558 19558 E AndroidRuntime: 	at android.os.Handler.handleCallback(Handler.java:790)
03-14 12:04:14.259 19558 19558 E AndroidRuntime: 	at android.os.Handler.dispatchMessage(Handler.java:99)
03-14 12:04:14.259 19558 19558 E AndroidRuntime: 	at android.os.Looper.loop(Looper.java:164)
03-14 12:04:14.259 19558 19558 E AndroidRuntime: 	at android.app.ActivityThread.main(ActivityThread.java:6494)
03-14 12:04:14.259 19558 19558 E AndroidRuntime: 	at java.lang.reflect.Method.invoke(Native Method)
03-14 12:04:14.259 19558 19558 E AndroidRuntime: 	at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:438)
03-14 12:04:14.259 19558 19558 E AndroidRuntime: 	at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:807)

How to keep collect stack safety in profiler?

Hi, as i know, if we want to safely walk stack directly in current thread, we must make some situation to satisfy 'walkable' state, such as GC is not active.
I do not find any code to permit for it, so do i missed something in the code?
And i noticed that profilo receive SIGSEGV/SIGBUS signal. Is this just for the exception of memory which produced by something like GC?

Release artifact's name does not match the documentation

In the documentation AAR library is named profilo.aar, but in releases it is named build.aar.

why libprofiloextapi.so not exits ?

Thank you for this job.
I try to assemble it in my app, but it crashed when launch, say

could't find libprofiloextapi.so

I unzipped the release aar and found nothing with name "libprofiloextapi.so"

For help

Unknown IDs in flame graph when stacks are produced

Getting some blocks as IDs only (in flame graph blocks) rather than full method names which we produce through stack output ?

Analyze the demo app log failed

I get the log follow the process in this page https://facebookincubator.github.io/profilo/docs/trace-processing.html, but cause exception when analyze it:

$ python3 -m profilo.workflow_demo b.log stacks b.apk 
Traceback (most recent call last):
  File "/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/Users/dreamtale/develop/nice/profilo/python/profilo/workflow_demo.py", line 255, in <module>
    args.func(tracefile, args)
  File "/Users/dreamtale/develop/nice/profilo/python/profilo/workflow_demo.py", line 176, in stacks
    trace = interpreter.interpret()
  File "/Users/dreamtale/develop/nice/profilo/python/profilo/importer/interpreter.py", line 148, in interpret
    block_entries.end,
  File "/Users/dreamtale/develop/nice/profilo/python/profilo/importer/interpreter.py", line 248, in assign_name
    assert len(self.children[key]) == 1
AssertionError

the log file( the apk file can't upload correctly):

b.log

Run syscounters or block occur error

python3 -m profilo.workflow_demo override-main-27575-2018-3-22T13-18-55-BgeHcWsbAsO-cs-f5457d96.log syscounters outpu
Traceback (most recent call last):
File "/usr/local/Cellar/python/3.6.4_4/Frameworks/Python.framework/Versions/3.6/lib/python3.6/runpy.py", line 193, in _run_module_as_main
"main", mod_spec)
File "/usr/local/Cellar/python/3.6.4_4/Frameworks/Python.framework/Versions/3.6/lib/python3.6/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/Users/Downloads/profilo-master/python/profilo/workflow_demo.py", line 255, in
args.func(tracefile, args)
File "/Users//Downloads/profilo-master/python/profilo/workflow_demo.py", line 124, in syscounters
trace = interpreter.interpret()
File "/Users/Downloads/profilo-master/python/profilo/importer/interpreter.py", line 148, in interpret
block_entries.end,
File "/Users/Downloads/profilo-master/python/profilo/importer/interpreter.py", line 241, in assign_name
assert len(self.children[key]) == 1
KeyError: BytesEntry(id=35950, type='STRING_KEY', arg1=35948, data='__name')

How to remove sample app when building profilo?

I am have no experience using buck so can you show me how to config the buck script to remove the sample app when building profilo?

Memory allocation tracker

Hello
I found this commit, but I can't find the code to show how to track memory allocation，Is this code not open source yet?

Can i ask for the meaning of THREAD_CPU_TIME in Syscounters?

workflow_demo.py stacks and blocks data empty

I run the sample.apk on OnePlus3 Android 8.0, get trace file, run command analyze the data:
override-main-11852-2018-6-16T9-45-5-HELSS7eibyu-cs-24bc196c.log

(env) oujunlideMacBook-Pro:python oujunli$ python3 -m profilo.workflow_demo override-main-11852-2018-6-16T9-45-5-HELSS7eibyu-cs-24bc196c.log stacks sample.apk




(env) oujunlideMacBook-Pro:python oujunli$ python3 -m profilo.workflow_demo override-main-11852-2018-6-16T9-45-5-HELSS7eibyu-cs-24bc196c.log blocks
,IsMain,tid,event,mean,median,std

(env) oujunlideMacBook-Pro:python oujunli$ python3 -m profilo.workflow_demo override-main-11852-2018-6-16T9-45-5-HELSS7eibyu-cs-24bc196c.log syscounters out
(env) oujunlideMacBook-Pro:python oujunli$ ls -l out
total 184
-rw-r--r--  1 oujunli  staff  25646  6 16 11:02 alloc_free_bytes.png
-rw-r--r--  1 oujunli  staff  23876  6 16 11:02 global_alloc_size.png
-rw-r--r--  1 oujunli  staff  15182  6 16 11:02 num_procs.png
-rw-r--r--  1 oujunli  staff  22530  6 16 11:02 proc_cpu_time.png

the stacks and blocks data is empty. Is the trace file only contain syscounters data? If so, how can I config the sample apk to enable stacks and blocks?
Thanks

Attach source code to existing project

Excuse me, for my curiosity, is there any way to use directly provided Profilo's source code to my project without using .aar file? Please give me some information if you can. Thanks.

Missing README in python/demos

The README says

A demo script can be found in python/profilo/workflow_demo.py. It contains examples of simple
analysis that can be run on Profilo traces. For more information see the README
python/demos/README.md

Unfortunately, python/demos is missing the README. Perhaps trace-processing.md is the file the README is intended to link to?

Can't run workflow_demo.py

The following command fails:

$ python3 profilo/workflow_demo.py
Traceback (most recent call last):
  File "profilo/workflow_demo.py", line 18, in <module>
    from .importer.trace_file import TraceFile
ModuleNotFoundError: No module named '__main__.importer'; '__main__' is not a package

When do you plan to support gradle?

I use profilo in my app, but the app has include Facebook other SDK, will cause Java class conflict when build, if gradle support, can exclude the SDK exits. The buck is not Android officially supported build system, I'm not familiar with it, do you plan to support gradle or maybe some other way to solve this problem? Thanks.

Missing framework frames information in stacktraces (pre-lollipop devices)

Dalvik tracer doesn't inherit from the JavaBaseTracer, hence there is no symbolication for framework frames at the time of flushing stack traces when running on Android 4.x or below.

How to handle thread info if thead ID reuse?

Read the code ProcFS.cpp function ThreadCache::forEach and ThreadCache::forThread, the threads info map(cache_ ) get a thread info though the thread ID, but the thread ID can be reused, if so , the same thread ID is different threads, handle this situation or not? Thanks.

How dose this lib impact app performance?

I see lib has loop to trace until stop tracing. So, how dose this lib impact app performance? (or overhead of lib)

Meaning of CPU time in ProcessCounters

Hey,

I am trying to understand why the CPU time is inferred as the sum of user time and system time through getrusage here in ProcessCounters.cpp .

CPU time of process to me is the time when the process is executing instructions on CPU and is not idle. Ideally, the code should not include the time when the CPU is idle due to waiting on I/O or due to lock.

Resources: http://www.brendangregg.com/blog/2017-05-09/cpu-utilization-is-wrong.html

I ran an experiment reading large CSV files which is an IO-bound task (CMIIW) and this number (rusageStats.ru_utime + rusageStats.ru_stime) starts incrementing quickly and according to the definition, it should not grow with quickly large numbers as the CPU will be idle for this process.

Can you suggest some experiments which I can run on my emulators or device to validate that this time will not increment if I keep my CPU idle or stalled due to some external resource?

build error

BUILD FAILED: //deps/procmaps:procmaps: parameter 'srcs': no such file or directory 'deps/procmaps/procmaps.c'

android 7.1.2 arm64 unwind support

Hi @Fortisque, @aandreyeu, I noticed that you've uploaded the 5~8 support for unwind but the 7.1.2 arm64 support, what are the considerations？
And many thanks for your efforts for the support.

Build errors when running `buck build //java/main/com/facebook/profilo/sample/...`

When running buck build //java/main/com/facebook/profilo/sample/... I noticed a few build issues. Some are simple .buckconfig issues. Others I wasn't able to workaround.

Details

When you first try to build samples with buck build //java/main/com/facebook/profilo/sample/... you'll notice a few build errors. By default buck won't fetch the SoLoader. Including a .buckconfig with the following may be a good way to fix this issue.

[download]
  in_build = true
  maven_repo = https://repo1.maven.org/maven2

You may/may-not next encounter a confusing error about aidl parsing. This is fixed by downloading SDK 23. Including the following in the default buckconfig might be a good way to make this issue more obvious (similarly to what the SoLoader repo does).

[android]
  target = Google Inc.:Google APIs:23

One this is changed you'll see some compile issues because <fb/xplat_init.h> can't be found. I was able to fix this by making a small change to the way headers are exported in the xplat BUCK file. And I needed to import profilo_path("deps/xplat_init:xplat_init") in a few BUCK projects.

Next, I saw the following:

deps/fbjni/cxx/fbjni/detail/References-inl.h:119:26: error: exception handling disabled, use -fexceptions to enable
     throw std::bad_alloc{};

This appears to be because all projects that consume deps/fbjni headers don't have -fexceptions flag.

Finally I noticed undefined facebook::profilo::logger::PacketLogger::write errors. Given this wasn't a simple build failure I figured I was getting too deep and had better leave this Issue to the experts.

Is profilo compatible with react-native?

Hello. Hope this finds you well.

Recently I've come accoss a problem that bothers me a lot.
My app already has react-native in it. So when I tried to put profilo into it, I met the following error while starting the app up.

I/AEE_AED: *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
I/AEE_AED: Build fingerprint: 'Meizu/meizu_MX6/MX6:7.1.1/NMF26O/1547836199:user/release-keys'
I/AEE_AED: Revision: '0'
I/AEE_AED: ABI: 'arm'
I/AEE_AED: pid: 5213, tid: 5336, name: MainActivity_Jo >>> com.qiyi.video <<<
I/AEE_AED: signal 11 (SIGSEGV), code 2 (SEGV_ACCERR), fault addr 0xdeadb00c
I/AEE_AED: Abort message: 'Assert (/code/rn-upgrade/android/react-native-0.55/ReactAndroid/src/main/jni/first-party/fb/jni/Environment.cpp:42): g_vm'
I/AEE_AED: r0 0000008c r1 00000000 r2 00000000 r3 deadb00c
I/AEE_AED: r4 e49f9004 r5 00000000 r6 00000001 r7 c4e38158
I/AEE_AED: r8 c4e3817c r9 edc09008 sl c4e381a0 fp c4e38318
I/AEE_AED: ip c4e3704c sp c4e38118 lr e49ec6d9 pc e49ec6dc cpsr 600f0030
I/AEE_AED: backtrace:
I/AEE_AED: #00 pc 000086dc /data/app/com.qiyi.video-2/lib/arm/libfb.so (_ZN8facebook14assertInternalEPKcz+71)
I/AEE_AED: #1 pc 000090af /data/app/com.qiyi.video-2/lib/arm/libfb.so
I/AEE_AED: #2 pc 00009259 /data/app/com.qiyi.video-2/lib/arm/libfb.so (_ZN8facebook3jni11Environment7currentEv+32)
I/AEE_AED: #3 pc 00014a11 /data/app/com.qiyi.video-2/lib/arm/libreactnativejni.so (_ZN8facebook3jni6JClass15registerNativesESt16initializer_listINS0_12NativeMethodEE+22)
I/AEE_AED: #4 pc 000260a1 /data/app/com.qiyi.video-2/lib/arm/libreactnativejni.so
I/AEE_AED: #5 pc 0000d43b /data/app/com.qiyi.video-2/lib/arm/libfb.so (_ZN8facebook3jni10initializeEP7_JavaVMOSt8functionIFvvEE+390)
I/AEE_AED: #6 pc 00025ae1 /data/app/com.qiyi.video-2/lib/arm/libreactnativejni.so (JNI_OnLoad+48)
I/AEE_AED: #7 pc 0023d4c9 /system/lib/libart.so (ZN3art9JavaVMExt17LoadNativeLibraryEP7_JNIEnvRKNSt3__112basic_stringIcNS3_11char_traitsIcEENS3_9allocatorIcEEEEP8_jobjectP8_jstringPS9+1844)
I/AEE_AED: #8 pc 00003233 /system/lib/libopenjdkjvm.so (JVM_NativeLoad+178)
I/AEE_AED: #9 pc 731675c5 /data/dalvik-cache/arm/system@[email protected] (offset 0x2f63000)

It seems to be related to some conflicts on libfb and libfbjni between the two projects: rn and profilo.
As I looked into them, I found that the code path and structure are totally different.
For example, there are
profilo/deps/fb
profilo/deps/fbjni/cxx/fbjni
profilo/deps/fbjni/cxx/lyra
profilo/deps/fbjni/java
&
react-native/ReactAndroid/src/main/jni/first-party/fb/include
react-native/ReactAndroid/src/main/jni/first-party/fb/jni
react-native/ReactAndroid/src/main/jni/first-party/fb/jni/android
react-native/ReactAndroid/src/main/jni/first-party/fb/jni/java
react-native/ReactAndroid/src/main/jni/first-party/fb/lyra

The files in those folders are very similar (same names but slightly different content).
I guess that might be the cause to the crash.

Would you please give some advice on how to resolve it?
Or, it will be perfect if you have plan to support it. XD

Thanks for your time.

Importing profilo code in Android studio.

I want to get insights into working of Profilo.

I cloned the repo and imported it into Android studio. I was able to build the binaries by following through the steps given here. The issue is, even after building the binaries, Android Studio cannot link/index the internals of Java/C++ source files of Profilo.

Any help will be highly appreciated.

Systrace Snooping Supports Less Than Half the Market?

Systrace snooping doesn't work on/above Marshmallow. Is it impossible to setup the hooks on these OS versions? Or just not done yet?

Aside: thanks for open sourcing this :)

NativeTracer

Hi, I noticed that the latest commit with/without native_tracer and want to know if there is an efficient way to get native stacks(cfi on arm32 or framepointer on arm64 maybe), also curious about if the implementation would be open-source in future.

Migration Website to Docusaurus 2 Alpha

Hi there! The Docusaurus team is currently developing Docusaurus 2 and are almost done with an MVP that doesn't include translations and versioning features, which means there's feature parity with your website's current usage of Docusaurus.

Docusaurus 2 brings about many improvements:

Client-side rendering with prerendering (site renders without JavaScript!)
More flexible appearance - Customizable layouts, CSS modules, etc
Embeddable interactive React components within markdown via MDX

I will be your POC in helping you throughout the migration process. Let me know what you think!

cc @yangshun @endiliey

Can not get trace.log on Lolipop devices?

I encounter error "run-as: exec failed for find Error:Permission denied was moved. Retrying..." when using real devices running lolipop. Does profilo only support 6.0 and above?

Python scripts throws errors

I tried to follow the same approach described and reached the point to start / stop tracing but i couldn't pull the tracing file with both approaches
for example
python3 -m profilo pull_traces --last "package name" throws

Traceback (most recent call last):
  File "/usr/local/Cellar/python/3.7.6_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/local/Cellar/python/3.7.6_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/Users/"username"/profilo/python/profilo/profilo.py", line 19, in <module>
    from .device.device import pull_last_trace, pull_all_traces, pull_n_traces
ImportError: attempted relative import with no known parent package

So can u guys modify the description with clear steps how we can run the test app , starting trace . exporting trace files and then checking the result

How mean, median and std of each event are calculated?

For example, i have a method, i place the Start() at the beginning of it and Stop() at the end, does profilo provide me aa inclusive time or exclusive time of this method?

What IDE/editor do you use to develop/debug the project ?

I want to read the source code to figure out what the docs are missing, but I can't use Android Studio/CLion/PyCharm to read Java/C++/Python code, there are unresolved reference everywhere.

So very curious about what IDE/editor you use to develop the project? How do you debug it? Using BUCK build and docker set an barrier for people like me to read the code or even contribute to the project.

Pulling trace file from real device get error

I got this error when pulling trace file from my project deployed on my phone:
File /data/data/com.example.cpu10152_local.threadpool/run-as: exec failed for find Error:Permission denied was moved. Retrying...
But on emulator, I could pull log file successfully.
My device: Asus ZenFone Go

Sample App Crash

install release-96e2c3f and click START TRACING, App crash.
both real device and emulator.

Process: com.facebook.profilo.sample, PID: 20073
    java.lang.UnsatisfiedLinkError: couldn't find DSO to load: libfbjni.so
        at com.facebook.soloader.SoLoader.doLoadLibraryBySoName(SoLoader.java:703)
        at com.facebook.soloader.SoLoader.loadLibraryBySoName(SoLoader.java:564)
        at com.facebook.soloader.SoLoader.loadLibrary(SoLoader.java:500)
        at com.facebook.soloader.SoLoader.loadLibrary(SoLoader.java:455)
        at com.facebook.jni.HybridData.<clinit>(HybridData.java:34)
        at com.facebook.profilo.writer.NativeTraceWriter.initHybrid(Native Method)
        at com.facebook.profilo.writer.NativeTraceWriter.<init>(NativeTraceWriter.java:36)
        at com.facebook.profilo.logger.Logger.startWorkerThreadIfNecessary(Logger.java:181)
        at com.facebook.profilo.logger.Logger.postCreateTrace(Logger.java:110)
        at com.facebook.profilo.core.TraceControl.startTraceInternal(TraceControl.java:344)
        at com.facebook.profilo.core.TraceControl.startTrace(TraceControl.java:300)
        at com.facebook.profilo.controllers.external.api.ExternalTraceControl.startTrace(ExternalTraceControl.java:41)
        at com.facebook.profilo.sample.BaseSampleAppMainActivity$1.onClick(BaseSampleAppMainActivity.java:60)
        at android.view.View.performClick(View.java:6303)
        at android.widget.CompoundButton.performClick(CompoundButton.java:134)
        at android.view.View$PerformClick.run(View.java:24828)
        at android.os.Handler.handleCallback(Handler.java:789)
        at android.os.Handler.dispatchMessage(Handler.java:98)
        at android.os.Looper.loop(Looper.java:164)
        at android.app.ActivityThread.main(ActivityThread.java:6798)
        at java.lang.reflect.Method.invoke(Native Method)
        at com.android.internal.os.Zygote$MethodAndArgsCaller.run(Zygote.java:240)
        at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:767)

Support for x86_64 and arm64 architectures

I'm planning to use Profilo on prod builds, and was wondering if there is a roadmap to support x86_64 and arm64 ABIs?

I was able to build from source for arm64 and x86_64 but I'm getting CLASS_LOAD_FAILED in type for every byte entry being logged from my flows.

Also, would it be advisable to compile for these ABIs at the moment?

workflow_demo.py can't parse APK

I'm having trouble getting the workflow_demo.py script to extract symbols from my test APK.

Here's the stack trace:

Traceback (most recent call last):
File "/usr/lib/python3.5/runpy.py", line 193, in _run_module_as_main
"main", mod_spec)
File "/usr/lib/python3.5/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "profilo/python/profilo/workflow_demo.py", line 255, in
args.func(tracefile, args)
File "profilo/python/profilo/workflow_demo.py", line 174, in stacks
symbols = extract_apk_symbols(args.apk)
File "profilo/python/profilo/symbols/apk_symbols.py", line 235, in extract_apk_symbols
dex = Dex(dex_file)
File "profilo/python/profilo/symbols/apk_symbols.py", line 160, in init
self.strings = self._read_strings(f)
File "profilo/python/profilo/symbols/apk_symbols.py", line 174, in _read_strings
result.append(StringDataItem(f).data)
File "profilo/python/profilo/symbols/apk_symbols.py", line 145, in init
c = f.read(1).decode("ascii")
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc2 in position 0: ordinal not in range(128)

Here's the trace off the device:

override-main-1954-2018-3-28T13-15-6-EsAb7fOK_wy-cs-c8480e9e.log

And the APK:

app-debug.apk.zip

Can any detail about the stacktrace provider implement especially CPP stacktrace?

Failed to run workflow_demo.py

Hi,
I run the sample.apk on Pixel 2 which system is Android P, and get the trace log: override-2018-12-16T12-51-43-E49izqDgqh8.zip.log
](url)
But when running the analysis command, I got the error message:

I tried to run the gunzip command, and got information:

Could you tell me how to fix it, Thanks.

Meaning of Trace.In-Memory

Hi, I noticed that in-memory trace was used to record the information which supposed to be saved into file in-memory. And I want to know that is there anyway to dump the packet buffer(lockfreeringbuffer mmap...) manually? Here we want is to record the stacktrace in memory and compare the diff to upload etc.

There're a lot of unmeaningful tags in the repo

As title mentioned, there're a lot of releases and tags on https://github.com/facebookincubator/profilo/releases

How do I run the project on Android studio?

Some functions I don’t need, I will streamline some functions.

Is there a way to generate file that can be analyze by perfetto (systrace file)?

Can't pull trace from Samsung J5 (Android 5.1.1)

When I run python3 -m profilo.profilo pull_last_trace com.facebook.profilo.sample, I get the following error:

File /data/data/com.facebook.profilo.sample/Function: selinux_compare_spd_ram , priority [2] , priority version is VE=SEPF_SM-J500H_5.1.1_0045 was moved. Retrying...

Same happens with my application when I integrate Profilo into the app.

AGPBI: {"kind":"error","text":"Program type already present: javax.annotation.CheckForNull","sources":[{}],"tool":"D8"}

It happens when i import profilo.aar into my project.How to resolve it.

Stack unwinding for Android 8.0 and above is not supported

Stack unwinding is not supported for android 8.0 and above as of now.
Please, keep this issue open as a placeholder for people to subscribe to.

Crash on API level 28 when SoLoader initialise the .so libraries

Hi, we are facing an issue on API Level 28 while Profilo tries to load .so Libraries using SoLoader. Here are the stack traces

E/SoLoader: couldn't find DSO to load: libgnustl_shared.so caused by: dlopen failed: "/data/data/com.app.telemetry/lib-main/libgnustl_shared.so" has unexpected e_machine: 40 (EM_ARM)
2020-01-16 12:46:39.294 26366-26366/com.app.telemetry W/System.err: com.facebook.soloader.SoLoader$WrongAbiError: APK was built for a different platform
2020-01-16 12:46:39.294 26366-26366/com.app.telemetry W/System.err:     at com.facebook.soloader.SoLoader.loadLibraryBySoName(SoLoader.java:645)
2020-01-16 12:46:39.294 26366-26366/com.app.telemetry W/System.err:     at com.facebook.soloader.SoLoader.loadLibraryBySoName(SoLoader.java:587)
2020-01-16 12:46:39.294 26366-26366/com.app.telemetry W/System.err:     at com.facebook.soloader.DirectorySoSource.loadDependencies(DirectorySoSource.java:111)
2020-01-16 12:46:39.294 26366-26366/com.app.telemetry W/System.err:     at com.facebook.soloader.DirectorySoSource.loadLibraryFrom(DirectorySoSource.java:71)
2020-01-16 12:46:39.294 26366-26366/com.app.telemetry W/System.err:     at com.facebook.soloader.UnpackingSoSource.loadLibrary(UnpackingSoSource.java:458)
2020-01-16 12:46:39.294 26366-26366/com.app.telemetry W/System.err:     at com.facebook.soloader.SoLoader.doLoadLibraryBySoName(SoLoader.java:731)
2020-01-16 12:46:39.294 26366-26366/com.app.telemetry W/System.err:     at com.facebook.soloader.SoLoader.loadLibraryBySoName(SoLoader.java:639)
2020-01-16 12:46:39.294 26366-26366/com.app.telemetry W/System.err:     at com.facebook.soloader.SoLoader.loadLibraryBySoName(SoLoader.java:587)
2020-01-16 12:46:39.294 26366-26366/com.app.telemetry W/System.err:     at com.facebook.soloader.DirectorySoSource.loadDependencies(DirectorySoSource.java:111)
2020-01-16 12:46:39.295 26366-26366/com.app.telemetry W/System.err:     at com.facebook.soloader.DirectorySoSource.loadLibraryFrom(DirectorySoSource.java:71)
2020-01-16 12:46:39.295 26366-26366/com.app.telemetry W/System.err:     at com.facebook.soloader.UnpackingSoSource.loadLibrary(UnpackingSoSource.java:458)
2020-01-16 12:46:39.295 26366-26366/com.app.telemetry W/System.err:     at com.facebook.soloader.SoLoader.doLoadLibraryBySoName(SoLoader.java:731)
2020-01-16 12:46:39.295 26366-26366/com.app.telemetry W/System.err:     at com.facebook.soloader.SoLoader.loadLibraryBySoName(SoLoader.java:639)
2020-01-16 12:46:39.295 26366-26366/com.app.telemetry W/System.err:     at com.facebook.soloader.SoLoader.loadLibrary(SoLoader.java:577)
2020-01-16 12:46:39.295 26366-26366/com.app.telemetry W/System.err:     at com.facebook.soloader.SoLoader.loadLibrary(SoLoader.java:525)
2020-01-16 12:46:39.295 26366-26366/com.app.telemetry W/System.err:     at com.facebook.profilo.logger.Logger.initialize(Logger.java:58)
2020-01-16 12:46:39.296 26366-26366/com.app.telemetry W/System.err:     at com.facebook.profilo.core.TraceOrchestrator.bind(TraceOrchestrator.java:230)
2020-01-16 12:46:39.296 26366-26366/com.app.telemetry W/System.err:     at com.facebook.profilo.core.TraceOrchestrator.initialize(TraceOrchestrator.java:101)

and

java.lang.UnsatisfiedLinkError: couldn't find DSO to load: libprofilo.so caused by: couldn't find DSO to load: libprofilo_fmt.so caused by: APK was built for a different platform
        at com.facebook.soloader.SoLoader.doLoadLibraryBySoName(SoLoader.java:789)
        at com.facebook.soloader.SoLoader.loadLibraryBySoName(SoLoader.java:639)
        at com.facebook.soloader.SoLoader.loadLibrary(SoLoader.java:577)
        at com.facebook.soloader.SoLoader.loadLibrary(SoLoader.java:525)
        at com.facebook.profilo.logger.Logger.initialize(Logger.java:58)
        at com.facebook.profilo.core.TraceOrchestrator.bind(TraceOrchestrator.java:230)
        at com.facebook.profilo.core.TraceOrchestrator.initialize(TraceOrchestrator.java:101)

Specifications of the emulator on which we are getting the issue:
Android version: 28
Device: Pixel XL
Architecture: x86,armeabi-v7a,armeabi(executing adb -s emulator-5556 shell getprop ro.product.cpu.abilist)

We backtracked the releases and figured out that the issue is coming from the following release : https://github.com/facebookincubator/profilo/tree/release-62ca671 in which the SoLoader is updated to the latest version(0.8.0)

Is this a known issue or are we missing something?

Running Stacks and Blocks analyzing return no value

After getting log file from my project deployed on emulator, I run stacks and blocks analyzing but got no result (0 is stacks items size):

Meanwhile, I could get analyzed graph in syscounters:

pull_last_trace get error

i install demo.apk and start trace.
then run python3 -m profilo.profilo pull_traces --last com.facebook.profilo.sample
error msg: Bad magic number for file header. Exiting

Gradle native builds

Let's use this issue to track the discussion/ progress on moving Profilo to Gradle-native builds.

Why?

Profilo depends on commonly used OSS libraries including fbjni, soloader and jsr305. Therefore, people that want to integrate the profilo .aar in their production application are likely to hit duplicated class errors.

java.lang.RuntimeException: Duplicate class com.facebook.jni.CppException found in modules fbjni-0.0.2-runtime.jar (com.facebook.fbjni:fbjni:0.0.2) and sample-release-d369779-runtime.jar (com.facebook.profilo:sample:release-d369779)
Duplicate class com.facebook.jni.CppSystemErrorException found in modules fbjni-0.0.2-runtime.jar (com.facebook.fbjni:fbjni:0.0.2) and sample-release-d369779-runtime.jar (com.facebook.profilo:sample:release-d369779)
Duplicate class com.facebook.jni.DestructorThread found in modules fbjni-0.0.2-runtime.jar (com.facebook.fbjni:fbjni:0.0.2) and sample-release-d369779-runtime.jar (com.facebook.profilo:sample:release-d369779)

Full error and POM gist

What needs to happen?

@BurntBrunch @aandreyeu could you put together a checklist of the high-level steps required before we can replace the buck builds with gradle? I'm happy to collaborate on this and send over PRs under your guidance.

Failed to setup and run workflow_demo.py

Followed the guide, but it failed to install related dependencies

virtualenv -p python3 env
source env/bin/activate
pip3 install -r requirements.txt

Here is an installation log:
install.log

I tried to install NumPy globally, it success, but when invoking workflow_demo:

(env) raymond@Raymonds-MBP:~/workspace/profilo/python$ python3 -m profilo.workflow_demo --help
Traceback (most recent call last):
  File "/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/Users/raymond/workspace/profilo/python/profilo/workflow_demo.py", line 26, in <module>
    import numpy as np
ModuleNotFoundError: No module named 'numpy'

profilo unwind cannot work on 6.0 with arm64 devices

Describe the bug