vandal / prism Goto Github PK

The Valgrind frontend (sigrind) uses a heavily modded Callgrind implementation, which was chosen for its built-in function tracking functionality. For very heavyweight analysis/characterization, like generating SynchroTraces, this is okay because the workload analysis covers up the slow event generation. For simpler prism tools, the sigrind event generation is excessively slow.

A lighter-weight Valgrind frontend (gengrind) has been in progress, with its function-tracking logic also based on Callgrind's. Right now gengrind is not implemented because a dead-end was hit inserting conditional VEX IR for turning instrumentation "on" and "off" without having to flush the instrumented basic blocks.

This is easily done in DynamoRIO by adding branches to labels. However, the support in mainline Valgrind for this is poor. The two options are conditional dirty calls and insert VEX Exit statements.

• Conditional dirty calls would require saving each BB's events in separate buffers, increasing overall memory usage and hurting cache performance
• VEX Exit statements can only target addresses in the original binary, and they appear to generate extra basic blocks, obfuscating the workload.

The 'jit-hacks-2' branch of Valgrind has some dedicated support of IfThenElse VEX IR statements, and should be investigated further.

Dear mikelui,
Hello, I am tring to compile dynamoRIO as frontend on aarch64 platform, but I encounter some compile problems, it seems that a variable's definition cannot be find, have you meet this problem before, could you give me some suggestion?

1)/prism/src/Frontends/DrSigil/dynamorio/clients/drsigil/drsigil.h:69:5: error: unknown type name ‘Sigil2DBISharedData’
Sigil2DBISharedData *shared_mem;
2)error: unknown type name ‘SglMemEv’
SglMemEv *mems;

Regards,
chandler

This issue tracks progress of copying the parser from the SynchroTrace replayer to the STGen example parsers.

Currently example parsers exist for the capnproto format traces.

This is because it's a less common format, so I figured it would be more approachable for peers to use if they could toy around with an already capable parser.

However, the SynchroTrace replayer was originally designed to parse the text format.
In recent changes, the text parser for the replayer also changed. It would be beneficial to fully decouple that text parser for easier testing and integration.

when I use prism to trace the blackscholes of PARSEC, 4 thread, encounter this error:
Sigrind: /home/l00515693/prism/src/Frontends/Gengrind/valgrind/sigrind/sigil2_ipc.c:78 (set_next_buffer): Assertion 'buf_idx == curr_idx' failed.

Both on X86 and ARM64 platform, the error ecountered.

So any bugs in Valgrind for PARSEC benchmark??

Detail information:
bin/prism --backend=stgen -ltextv2 --executable=/home/l00515693/PARSEC-aarch64/bin/parsecmgmt -a run -p blackscholes -i simdev -n 4
[PRISM] [info] executable : /home/l00515693/PARSEC-aarch64/bin/parsecmgmt -a run -p blackscholes -i simdev -n 4
[PRISM] [info] frontend : default
[PRISM] [info] backend : stgen
[PRISM] [info] threads : 1
[PRISM] [info] timed : off
[PRISM] [warning] '/home/l00515693/PARSEC-aarch64/bin/parsecmgmt':
[PRISM] [warning] GCC version 4.9.2 not detected
[PRISM] [warning] GCC version could not be detected
[PRISM] [warning] OpenMP synchronization events may not be captured
[PRISM] [warning] Pthread synchronization events are probably fine
==81125== Sigrind,
==81125== Copyright (C) 2015-2017, by Michael Lui et al.
==81125== Using Valgrind-3.15.0.GIT and LibVEX; rerun with -h for copyright info
==81125== Command: /home/l00515693/PARSEC-aarch64/bin/parsecmgmt -a run -p blackscholes -i simdev -n 4
==81125==
==81125== *********************************************
==81125== Beginning event generation from program start
==81125== *********************************************

Sigrind: /home/l00515693/prism/src/Frontends/Gengrind/valgrind/sigrind/sigil2_ipc.c:78 (set_next_buffer): Assertion 'buf_idx == curr_idx' failed.

host stacktrace:
==81129== at 0x5801B73A: show_sched_status_wrk (m_libcassert.c:369)
==81129== by 0x5801B857: report_and_quit (m_libcassert.c:440)
==81129== by 0x5801B9E9: vgPlain_assert_fail (m_libcassert.c:506)
==81129== by 0x58014B97: vgSigrind_acq_event_slot (sigil2_ipc.c:78)
==81129== by 0x580147A0: vgSigrind_log_2I0D (log_events.c:65)
==81129== by 0x10031EA545: ???
==81129== by 0x1002B26F2F: ???
==81129== by 0x1002048BBF: ???
==81129== by 0x1002B26F17: ???
==81129== by 0x1002B26F2F: ???
==81129== by 0x1002B26F3F: ???
==81129== by 0x14713: ???
==81129== by 0xEC11: ???
==81129== by 0x10031EA4DF: ???
==81129== by 0xEC11: ???

sched status:
running_tid=1

Thread 1: status = VgTs_Runnable (lwpid 81129)
==81129== at 0x5345B22: fork (in /lib/x86_64-linux-gnu/libc-2.27.so)
==81129== by 0x15FA87: make_child (in /bin/bash)
==81129== by 0x169561: command_substitute (in /bin/bash)
==81129== by 0x16E5C2: ??? (in /bin/bash)
==81129== by 0x16F885: ??? (in /bin/bash)
==81129== by 0x172CFA: ??? (in /bin/bash)
==81129== by 0x135A9A: ??? (in /bin/bash)
==81129== by 0x14E0D7: execute_command_internal (in /bin/bash)
==81129== by 0x14E00B: execute_command_internal (in /bin/bash)
==81129== by 0x14FBF1: execute_command (in /bin/bash)
==81129== by 0x14DFD6: execute_command_internal (in /bin/bash)
==81129== by 0x14FBF1: execute_command (in /bin/bash)
==81129== by 0x14DFD6: execute_command_internal (in /bin/bash)
==81129== by 0x14FBF1: execute_command (in /bin/bash)
==81129== by 0x14DFD6: execute_command_internal (in /bin/bash)
==81129== by 0x14FBF1: execute_command (in /bin/bash)
==81129== by 0x14DFD6: execute_command_internal (in /bin/bash)
==81129== by 0x14FBF1: execute_command (in /bin/bash)
==81129== by 0x14DFD6: execute_command_internal (in /bin/bash)
==81129== by 0x14FBF1: execute_command (in /bin/bash)
==81129== by 0x14DFD6: execute_command_internal (in /bin/bash)
==81129== by 0x14FBF1: execute_command (in /bin/bash)
==81129== by 0x14DFD6: execute_command_internal (in /bin/bash)
==81129== by 0x14FBF1: execute_command (in /bin/bash)
==81129== by 0x14DFD6: execute_command_internal (in /bin/bash)
==81129== by 0x14FBF1: execute_command (in /bin/bash)
==81129== by 0x14DFD6: execute_command_internal (in /bin/bash)
==81129== by 0x14FBF1: execute_command (in /bin/bash)
==81129== by 0x14DFD6: execute_command_internal (in /bin/bash)
==81129== by 0x14FBF1: execute_command (in /bin/bash)
==81129== by 0x14DFD6: execute_command_internal (in /bin/bash)
==81129== by 0x14FBF1: execute_command (in /bin/bash)
==81129== by 0x14DFD6: execute_command_internal (in /bin/bash)
==81129== by 0x14FBF1: execute_command (in /bin/bash)
==81129== by 0x14DFD6: execute_command_internal (in /bin/bash)
==81129== by 0x14FBF1: execute_command (in /bin/bash)
==81129== by 0x14DFD6: execute_command_internal (in /bin/bash)
==81129== by 0x14FBF1: execute_command (in /bin/bash)
==81129== by 0x14DFD6: execute_command_internal (in /bin/bash)
==81129== by 0x14FBF1: execute_command (in /bin/bash)
==81129== by 0x14DFD6: execute_command_internal (in /bin/bash)
==81129== by 0x14DF47: execute_command_internal (in /bin/bash)
==81129== by 0x150A78: ??? (in /bin/bash)
==81129== by 0x136207: ??? (in /bin/bash)
==81129== by 0x14E0D7: execute_command_internal (in /bin/bash)
==81129== by 0x14FBF1: execute_command (in /bin/bash)
==81129== by 0x13A273: reader_loop (in /bin/bash)
==81129== by 0x138C7E: main (in /bin/bash)
client stack range: [0x1FFEFFD000 0x1FFF000FFF] client SP: 0x1FFEFFE570
valgrind stack range: [0x1002A27000 0x1002B26FFF] top usage: 6912 of 1048576

Note: see also the FAQ in the source distribution.
It contains workarounds to several common problems.
In particular, if Valgrind aborted or crashed after
identifying problems in your program, there's a good chance
that fixing those problems will prevent Valgrind aborting or
crashing, especially if it happened in m_mallocfree.c.

If that doesn't help, please report this bug to: www.valgrind.org

In the bug report, send all the above text, the valgrind
version, and what OS and version you are using. Thanks.

Sigrind: /home/l00515693/prism/src/Frontends/Gengrind/valgrind/sigrind/sigil2_ipc.c:78 (set_next_buffer): Assertion 'buf_idx == curr_idx' failed.

host stacktrace:
==81125== at 0x5801B73A: show_sched_status_wrk (m_libcassert.c:369)
==81125== by 0x5801B857: report_and_quit (m_libcassert.c:440)
==81125== by 0x5801B9E9: vgPlain_assert_fail (m_libcassert.c:506)
==81125== by 0x58014B97: vgSigrind_acq_event_slot (sigil2_ipc.c:78)
==81125== by 0x58014544: vgSigrind_log_comp_event (log_events.c:114)
==81125== by 0x1002F5649B: ???
==81125== by 0x1002B26F2F: ???
==81125== by 0x1002048BBF: ???
==81125== by 0x1002B26F17: ???
==81125== by 0x1002B26F2F: ???
==81125== by 0x1002B26F3F: ???
==81125== by 0x5846D89F: ???
==81125== by 0xEAA2: ???
==81125== by 0x10046BF30F: ???
==81125== by 0xEAA2: ???

sched status:
running_tid=1

Thread 1: status = VgTs_Runnable (lwpid 81125)
==81125== at 0x52F8A10: free (in /lib/x86_64-linux-gnu/libc-2.27.so)
==81125== by 0x16E5E6: ??? (in /bin/bash)
==81125== by 0x16F885: ??? (in /bin/bash)
==81125== by 0x172CFA: ??? (in /bin/bash)
==81125== by 0x135A9A: ??? (in /bin/bash)
==81125== by 0x14E0D7: execute_command_internal (in /bin/bash)
==81125== by 0x14E00B: execute_command_internal (in /bin/bash)
==81125== by 0x14FBF1: execute_command (in /bin/bash)
==81125== by 0x14DFD6: execute_command_internal (in /bin/bash)
==81125== by 0x14FBF1: execute_command (in /bin/bash)
==81125== by 0x14DFD6: execute_command_internal (in /bin/bash)
==81125== by 0x14FBF1: execute_command (in /bin/bash)
==81125== by 0x14DFD6: execute_command_internal (in /bin/bash)
==81125== by 0x14FBF1: execute_command (in /bin/bash)
==81125== by 0x14DFD6: execute_command_internal (in /bin/bash)
==81125== by 0x14FBF1: execute_command (in /bin/bash)
==81125== by 0x14DFD6: execute_command_internal (in /bin/bash)
==81125== by 0x14FBF1: execute_command (in /bin/bash)
==81125== by 0x14DFD6: execute_command_internal (in /bin/bash)
==81125== by 0x14FBF1: execute_command (in /bin/bash)
==81125== by 0x14DFD6: execute_command_internal (in /bin/bash)
==81125== by 0x14FBF1: execute_command (in /bin/bash)
==81125== by 0x14DFD6: execute_command_internal (in /bin/bash)
==81125== by 0x14FBF1: execute_command (in /bin/bash)
==81125== by 0x14DFD6: execute_command_internal (in /bin/bash)
==81125== by 0x14FBF1: execute_command (in /bin/bash)
==81125== by 0x14DFD6: execute_command_internal (in /bin/bash)
==81125== by 0x14FBF1: execute_command (in /bin/bash)
==81125== by 0x14DFD6: execute_command_internal (in /bin/bash)
==81125== by 0x14FBF1: execute_command (in /bin/bash)
==81125== by 0x14DFD6: execute_command_internal (in /bin/bash)
==81125== by 0x14FBF1: execute_command (in /bin/bash)
==81125== by 0x14DFD6: execute_command_internal (in /bin/bash)
==81125== by 0x14FBF1: execute_command (in /bin/bash)
==81125== by 0x14DFD6: execute_command_internal (in /bin/bash)
==81125== by 0x14FBF1: execute_command (in /bin/bash)
==81125== by 0x14DFD6: execute_command_internal (in /bin/bash)
==81125== by 0x14FBF1: execute_command (in /bin/bash)
==81125== by 0x14DFD6: execute_command_internal (in /bin/bash)
==81125== by 0x14DF47: execute_command_internal (in /bin/bash)
==81125== by 0x150A78: ??? (in /bin/bash)
==81125== by 0x136207: ??? (in /bin/bash)
==81125== by 0x14E0D7: execute_command_internal (in /bin/bash)
==81125== by 0x14FBF1: execute_command (in /bin/bash)
==81125== by 0x13A273: reader_loop (in /bin/bash)
==81125== by 0x138C7E: main (in /bin/bash)
client stack range: [0x1FFEFFD000 0x1FFF000FFF] client SP: 0x1FFEFFE800
valgrind stack range: [0x1002A27000 0x1002B26FFF] top usage: 6912 of 1048576

Note: see also the FAQ in the source distribution.
It contains workarounds to several common problems.
In particular, if Valgrind aborted or crashed after
identifying problems in your program, there's a good chance
that fixing those problems will prevent Valgrind aborting or
crashing, especially if it happened in m_mallocfree.c.

If that doesn't help, please report this bug to: www.valgrind.org

In the bug report, send all the above text, the valgrind
version, and what OS and version you are using. Thanks.

[PRISM] [critical] Unexpected end of fifo

Sigrind, the modified Callgrind tool, uses Callgrind's implementation of differentiating threads which can sometimes miss new threads.

This is because Callgrind uses Valgrind's internal thread id's, which reuses id's of exited threads.

So the following can happen:

Created - tid 1
Created - tid 2
Created - tid 3
Created - tid 4
Exited  - tid 2
Created - tid 2 <-- **Reusing old thread id** should be unique tid 5!

Prism will then conflate events from unique tid 2 and unique tid 5, treating them as coming from the same thread.

Prism's Valgrind (default) does not support launching an script that launches the executable to be profiled.

See #5

The problem lies in the event channel IPC interface.