virtan / eep

Erlang Easy Profiling (eep) application provides a way to analyze application performance and call hierarchy

Erlang Easy Profiling (eep)

Erlang Easy Profiling (eep) application provides a way to analyze application performance and call hierarchy.

Main features:

• no need to modify sources (doesn't need sources at all)
• no need to stop your running system
• start and stop collecting runtime data at arbitrary time
• profile arbitrary module or whole system
• minimal impact on profiled system performance (unlike [fprof] 6)
• very informative visualization of time costs and call stacks ([kcachegrind] 7)
• ability to export call graphs in dot or image format
• optional process separation
• based on [dbg] 8 module and built-in low overhead [trace ports] 9
• optionally route runtime data over network to minimize disk load

Limitations:

• doesn't work with natively compiled code
• doesn't support parent-child links (will appear in future versions)

How to

On target system:

1. Make sure the target system can use eep module (link eep to your rebar project or place compiled eep.beam at any code path)
2. Collect runtime data to local file

1> eep:start_file_tracing("file_name"), timer:sleep(10000), eep:stop_tracing().

3. Copy $PWD/file_name.trace from the target system

Outside the target system:

1. Make sure collected runtime data is in current directory ($PWD/file_name.trace)
2. Convert to callgrind format

1> eep:convert_tracing("file_name").

3. Start kcachegrind

$ kcachegrind callgrind.out.file_name

Also

1. Collect specific module calls only

1> eep:start_file_tracing("file_name", [], [my_module_1, my_module_2]).

2. Include time spent waiting for event (not running)

1> eep:convert_tracing("file_name", [waits]).

3. Dump collected runtime data

1> eep:dump_tracing("file_name").

4. Remove separation by erlang process

$ grep -v "^ob=" callgrind.out.file_name > callgrind.out.merged_file_name

5. Route runtime data to other host, then process trace on that host

 (eep@otherhost) 1> eep:start_net_client("targethost", 1088, "file_name", wait).
(eep@targethost) 1> eep:start_net_tracing(1088).

Useful

• Turn off kcachegrind "cycle detection", eep detects cycles by itself
• Absolute numbers in kcachegrind are microseconds
• ELF Objects in kcachegrind are erlang pids
• By default kcachegrind limits caller depth and node cost (can be changed in call graph context menu in Graph submenu)
• Tail recursion loop within group of functions has incorrect calls and time cost values

Screenshots

• [Overall view] 1
• [Call hierarchy] 2
• [Functions navigator] 3
• [Callees ordered by cost] 4
• [Relative costs view] 5

Author

Igor Milyakov [[email protected]] 10

License

The MIT License (MIT)

Copyright (c) 2013 Igor Milyakov [email protected]

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

eep's Issues

When i convert the file, it reprot a crash.

First, i want trace only one module, eep:start_file_tracing("file_name", [], [b2bAtcfLib]). Then i run eep:stop_tracing(), eep:convert_tracing("file_name").. but it reports an error:

`=ERROR REPORT==== 30-Mar-2015::07:15:29 ===
Error in process <0.6572.6> on node 'ppb1_bs6@esekilvxen245' with exit value: {badarg,[{ets,lookup,[3333325396,<0.2044.0>],[]},{eep,callgrind_convertor,2,[{file,"src/eep.erl"},{line,167}]},{dbg,tc_loop,3,[{file,"dbg.erl"},{line,1241}]}]}

`

Job 1, “kcachegrind callgrind.out.filename…” terminated by signal SIGSEGV (Address boundary error)

Is it a kcachegrind issue?

λ uname -a
Linux 3.13.0-85-generic #129-Ubuntu SMP x86_64 x86_64 x86_64 GNU/Linux
λ kcachegrind -v
Qt: 4.8.6
KDE Development Platform: 4.13.3
KCachegrind: 0.7.4kde

The file conversion is failed

Hello,

eep:convert_tracing("file_name"). is failed

=ERROR REPORT==== 26-Nov-2013::10:32:13 ===
Error in process <0.52.0> on node '[email protected]' with exit value: {function_clause,[{eep,'-pid_item_to_bytes/1-fun-0-',[undefined,{1,1},[]],[{file,"src/eep.erl"},{line,275}]},{orddict,fold,3,[{file,"orddict.erl"},{line,194}]},{eep,pid_item_to_bytes,1,[{file,"src/eep.erl"},{line,275}]},{eep,convertor_unwind...

As I trace id down line 275 from eep.erl is root cause. The SubCalls dict contains undefined {CM, CF, CA}.

The application crashes due to wrong return.

The application crashes due to wrong return.

(eep@127.0.0.1)1> application:start(eep).

=INFO REPORT==== 19-May-2016::11:43:47 ===
    application: eep
    exited: {bad_return,{{eep_app,start,[normal,[]]},ok}}
    type: temporary
{error,{bad_return,{{eep_app,start,[normal,[]]},ok}}}

Inclusion of application into release with rebar3 terminates the node

(typhoon@127.0.0.1)1> {"Kernel pid terminated",application_controller,"{application_start_failure,eep,{bad_return,{{eep_app,start,[normal,[]]},ok}}}"}
Kernel pid terminated (application_controller) ({application_start_failure,eep,{bad_return,{{eep_app,start,[normal,[]]},ok}}})
heart: Thu May 19 11:40:00 2016: Erlang is crashing .. (waiting for crash dump file)
heart: Thu May 19 11:40:00 2016: Would reboot. Terminating.

error when convert trace file

`Error in process <7827.24524.146> on node '...' with exit value:
{problem,[{eep,save_kcachegrind_format,1,[{file,"eep.erl"},{line,315}]}]}

** exception exit: problem
in function eep:save_kcachegrind_format/1 (eep.erl, line 315)`

Error in documentation

In the "Also" section you describe the function for filtering calls to a specific module:

Collect specific module calls only
1> eep:start_file_tracing("file_name", [], my_module).

It looks like it should actually be to a list of modules:

Collect specific module calls only
1> eep:start_file_tracing("file_name", [], [my_module_1, my_module_2]).

Works with hipe?

Can I profile a release with some modules compiled with hipe?

termination of shell process

Hello,

I'm trying to use eep with Erlang/Elixir release which is runned in docker container. I've attached to remote console and executed:

iex(node@ip)1> :eep.start_file_tracing('myapp.trace')
{:ok, [{:matched, :"node@ip", 1843}]}

But after couple of seconds console is closed with error:

iex(node@ip)2> *** ERROR: Shell process terminated! (^G to start new job) ***
command terminated with exit code 137

Any ideas what I'm doing wrong? Thank you.

The same fuction showing up once for every process that's running it in callgrind

Is it normal that if I have 100 gen_servers running, the callgrind output (default, no grouping) contains a separate gen_server:loop/6 line for each of them? (location is something like "<6868.244.12>: gen_server" with varying pids).

This problem doesn't seem to show up on the screenshots attached to the eep doc.

Hosting on the Hex.PM package manager

This looks like an amazing project, that will make profiling a lot easier than using built-in tools like e.g. eprof that only show text-based output.

Would it be possible to host this on Hex.pm? I run projects that are managed by Hex; it would make installing and updating a lot easier.

Thank you!

eep:convert_tracing("file_name") issue

Hello,
I get the error message below when I'll try to convert file to cachegrind format.
OS: Windows 7, Erlang R16B03

=ERROR REPORT==== 3-Aug-2014::13:04:33 ===
Error in process <0.35.0> with exit value: {{badmatch,{error,eacces}},[{eep,save_kcachegrind_format,1,[{file,"src/eep.erl"},{line,298}]}]}

** exception error: no match of right hand side value {error,eacces}
in function eep:save_kcachegrind_format/1 (src/eep.erl, line 298)

Project name is confusing

The project name conflicts with the Erlang Enhancement Proposals. Maybe a rename should be considered?