softsec-kaist / eclipser Goto Github PK

Grey-box Concolic Testing on Binary Code (ICSE '19)

License: MIT License

Shell 9.43% C 23.14% Makefile 0.64% F# 66.09% Dockerfile 0.70%

testcase-generator fuzzer concolic-testing fsharp fuzzing

eclipser's Issues

Please provide a Dockerfile for Eclipser 2.0

So that Eclipser can be used easily on current Linux plus not needing to install all the dependencies required - could you please supply a Dockerfile for Eclipser 2.0?
Based on the fuzzbench builder.Dockerfile + runner.Dockerfile this should not be too much work?
(I would do it myself, but too much on my plate for afl++ at the moment ...)

Understand the output directory

Hi,
I'm trying to fuzz tcpdump with Eclipse and I don't understand the purpose of the testcase directory in the output directory.
After 15 min of fuzzing there is only a testcase tc-0 inside the directory.
The starting input is uninformed, 72 0 (ascii).

Eclipse doesn't log inputs that trigger new coverage?

TCG instrumentation?

I wonder if the TCG instrumentation patches are used in Eclipser (I believe they are not). They seem to give a good performance improvement (3x/4x according to its author).

Missing documentation how to reproduce faults

Please add the documentation (and the feature if it's missing) of how to reproduce crashes and parse the bytes to tell user which are the inputs that made the program crash.

Any chance you can fix qemu to work on ubuntu 20.04

Short version: ubuntu 20.04 uses a later glibc, which leads to a failure to build qemu according to the recipe in the v1.1 ("old Eclipser") release. It looks like QEMU has fixed this, but presumably you are tagged to version 2.10.0 for good reasons. Any chance you can incorporate the fix here https://patchwork.openembedded.org/patch/165581/ to get Eclipser 1.1 runnable on 20.04?

We're trying to make a DeepState (https://github.com/trailofbits/deepstate) release that bases off 20.04 instead of 18.04, and we'd like to keep the "old" Eclipser in the fuzzer stable (and add Eclipser 2.0 when we get a chance; the switch to relying on AFL parallel mode makes this non-trivial). But when we try to build from docker, we run into:

/home/user/Eclipser/Instrumentor/qemu/qemu-2.3.0-pathcov-x64/linux-user/syscall.c:235:16: error: static declaration of 'gettid' follows non-static declaration
  235 | _syscall0(int, gettid)
      |                ^~~~~~
/home/user/Eclipser/Instrumentor/qemu/qemu-2.3.0-pathcov-x64/linux-user/syscall.c:164:13: note: in definition of macro '_syscall0'
  164 | static type name (void)   \
      |             ^~~~
In file included from /usr/include/unistd.h:1170,
                 from /home/user/Eclipser/Instrumentor/qemu/qemu-2.3.0-pathcov-x64/linux-user/syscall.c:27:
/usr/include/x86_64-linux-gnu/bits/unistd_ext.h:34:16: note: previous declaration of 'gettid' was here
   34 | extern __pid_t gettid (void) __THROW;
      |                ^~~~~~
/home/user/Eclipser/Instrumentor/qemu/qemu-2.3.0-pathcov-x86/linux-user/syscall.c:235:16: error: static declaration of 'gettid' follows non-static declaration
  235 | _syscall0(int, gettid)
      |                ^~~~~~
/home/user/Eclipser/Instrumentor/qemu/qemu-2.3.0-pathcov-x86/linux-user/syscall.c:164:13: note: in definition of macro '_syscall0'
  164 | static type name (void)   \
      |             ^~~~
In file included from /usr/include/unistd.h:1170,
                 from /home/user/Eclipser/Instrumentor/qemu/qemu-2.3.0-pathcov-x86/linux-user/syscall.c:27:
/usr/include/x86_64-linux-gnu/bits/unistd_ext.h:34:16: note: previous declaration of 'gettid' was here
   34 | extern __pid_t gettid (void) __THROW;
      |                ^~~~~~
make[2]: *** [/home/user/Eclipser/Instrumentor/qemu/qemu-2.3.0-pathcov-x64/rules.mak:57: linux-user/syscall.o] Error 1
make[1]: *** [Makefile:173: subdir-x86_64-linux-user] Error 2
make[1]: Leaving directory '/home/user/Eclipser/Instrumentor/qemu/qemu-2.3.0-pathcov-x64'
make: *** [Makefile:41: /home/user/Eclipser/Instrumentor/qemu/.compiled_x64] Error 1
make: *** Waiting for unfinished jobs....
/home/user/Eclipser/Instrumentor/qemu/qemu-2.3.0-pathcov-x86/linux-user/syscall.c: In function 'do_syscall':
/home/user/Eclipser/Instrumentor/qemu/qemu-2.3.0-pathcov-x86/linux-user/syscall.c:5962:29: warning: implicit declaration of function 'stime'; did you mean 'utime'? [-Wimplicit-function-declaration]
 5962 |             ret = get_errno(stime(&host_time));
      |                             ^~~~~
      |                             utime
/home/user/Eclipser/Instrumentor/qemu/qemu-2.3.0-pathcov-x86/linux-user/syscall.c:5962:29: warning: nested extern declaration of 'stime' [-Wnested-externs]
make[2]: *** [/home/user/Eclipser/Instrumentor/qemu/qemu-2.3.0-pathcov-x86/rules.mak:57: linux-user/syscall.o] Error 1
make[1]: *** [Makefile:173: subdir-i386-linux-user] Error 2
make[1]: Leaving directory '/home/user/Eclipser/Instrumentor/qemu/qemu-2.3.0-pathcov-x86'
make: *** [Makefile:37: /home/user/Eclipser/Instrumentor/qemu/.compiled_x86] Error 1

There may be other issues preventing Eclipser 1.1 from running on 20.04 but this one seems fixable, if you have time.

Fuzzing rocksdb doesn't work

Another DeepState example: https://github.com/agroce/testleveldb

Install DeepState, leveldb, and rocksdb (no need to follow the instructions to add sanitizers or anything, just compile release versions)
Change the makefile to compile with a recent clang
Try using Eclipser via deepstate-eclipser

Eclipser claims to find a crashing input for the DiffTestDBs executable immediately, but it seems to be just a set of zero bytes that don't actually cause a crash. Incompatibility with something one of the DBs is doing and QEMU?

Fuzzing just leveldb works fine, so it's something RocksDB does. Not high priority, just FYI of an important fuzzing target that seems to break Eclipser.

Fork server died: Success

I run Eclipser in a docker container and running a target results in a crash of Eclipser:

# dotnet /Eclipser/build/Eclipser.dll -t 5 -s eclipser -o out -p fuzz-target --exectimeout 5000 --arg foo -v 2
[00:00:00:00] [*] Fuzz target : /targets/test/fuzz-target
Fork server died: Success

Unhandled Exception: System.Exception: Failed to initialize fork server for coverage tracer
   at Eclipser.Executor.initializeForkServer(FuzzOption opt) in /Eclipser/src/Core/Executor.fs:line 75
   at Eclipser.Executor.initialize(FuzzOption opt) in /Eclipser/src/Core/Executor.fs:line 92
   at Eclipser.Fuzz.main(String[] args) in /Eclipser/src/Fuzz/Fuzz.fs:line 137
Aborted (core dumped)

the target runs without problem when called native or when fuzzed via afl-fuzz -Q.
what could the issue be?

Timeout of -1 does not work in old Eclipser

The argument is passed on to 'System.TimeSpan' and results in:

Parameter name: timeout
   at System.Threading.Thread.ToTimeoutMilliseconds(TimeSpan timeout)
   at [email protected](Unit unitVar) in /root/Eclipser/src/Fuzz/Fuzz.fs:line 122
   at Microsoft.FSharp.Control.AsyncPrimitives.CallThenInvoke[T,TResult](AsyncActivation`1 ctxt, TResult result1, FSharpFunc`2 part2)
   at Microsoft.FSharp.Control.Trampoline.Execute(FSharpFunc`2 firstAction)
--- End of stack trace from previous location where exception was thrown ---
   at [email protected](ExceptionDispatchInfo edi)
   at Microsoft.FSharp.Control.Trampoline.Execute(FSharpFunc`2 firstAction)
   at <StartupCode$FSharp-Core>[email protected](Object o)
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
--- End of stack trace from previous location where exception was thrown ---
   at System.Threading.ThreadPoolWorkQueue.Dispatch()
Aborted

#20 fixes it the same way as it is fixed in master (just not starting a timeout when using -1)

Allow the user to select the arquitecture to compile

Eclipser takes quite a long time to compile, since it recompiles QEMU for i386 and x64, however usually the user needs only one them. Is it possible to add a flag in the Makefile to compile for only one of these?

Sampling input bytes instead of conditional statements in SELECT?

According to the paper:

SELECT randomly selects N conditional statements from the given sequence of conditional statements.

However, instead of selecting N randomly conditional statements, I'm wondering if you have tried to randomly selecting inputs bytes to generate conditional statements. In that case, the results will be (most likely) less or simpler conditions to solve but without the risk of finding unfeasible solutions. You could even select input bytes using different sampling distributions (uniform, or by ranges)

Eclipser doesn't seem to work with LLVM sanitizers

Compiling with -fsanitize=foo seems to cause the fork server to not be able to start, with any of address, undefined, or integer. Be nice to be able to fuzz with sanitizers.

Confirmation on sharing state between multiple instances of Eclipser

I'd like to run multiple instances of Eclipser, is it just as performant to export the entire output directory? Or can I prioritize it so that /.coverage and /.internal (which I think contains the seed queues?) are on a fast-backing, and /crash, and /testcase are on slower disk-type backing?

Changelog from Eclipser 1.0?

I'm wondering if you can give some details regarding the recent changes included in Eclipser 1.0 release since the commit is quite large.

Solving timeout

I am currently running a running a comparison on concolic solvers on fuzzbench:
https://www.fuzzbench.com/reports/experimental/2021-07-03-symbolic/index.html

Everything is set up as comparable as possible, however one important item is how long the solver is tying to solve until it is aborting.
Is this configurable in Eclipser? If yes - how? If not - what is the hard set value?

Using DeepState makes graybox concolic much less effective.

The ICSE paper's example works great as a simple C program, but when I try to fuzz the same logic using DeepState (https://github.com/trailofbits/deepstate), it doesn't work. Graybox concolic testing of DeepState examples doesn't seem to work well, and produces surprisingly few interesting paths.

https://github.com/agroce/tryeclipser has some versions of the code. vulnmix avoids DeepState's more complex parsing of an input file, or need to set CLI arguments, and just reads eclipser.input but Eclipser still doesn't find the vulnerability with that approach.

Any ideas what's going on? Obviously with DeepState the time from program start until interesting things start happening based on input is much, much, longer. Be nice to work out what's going on, and if it can be fixed -- we'd love to have Eclipser as a DeepState backend, the approach is very nice, and results are impressive.

root@a621890e86cf:~/tryeclipser# dotnet ~/Eclipser/build/Eclipser.dll fuzz -p ./vulnmix --timelimit 60 --src file --fixfilepath eclipser.input
Fuzz target: /root/tryeclipser/vulnmix
Time: 60 sec
[00:00:00:00] Forkserver for coverage tracer : 95622
[00:00:00:00] Forkserver for branch tracer : 95624
[00:00:00:00] Fuzzing starts
[00:00:00:00] [*] Found by grey-box concolic (655 new nodes) : ("AAAAAAAA") File[0]=( e1* ...7bytes... (0) (Right))
[00:00:00:07] # of seeds before minimization : 2
[00:00:00:07] [*] (Redundant seed) : ("AAAAAAAA") File[0]=( ...8bytes... (0) (Right))
[00:00:00:07] Total 1 redundant seeds found
^C
root@a621890e86cf:~/tryeclipser# dotnet ~/Eclipser/build/Eclipser.dll fuzz -p ./vulnorig --timelimit 60 --src file --fixfilepath eclipser.input --initarg "eclipser.input"
Fuzz target: /root/tryeclipser/vulnorig
Time: 60 sec
[00:00:00:00] Forkserver for coverage tracer : 10992
[00:00:00:00] Forkserver for branch tracer : 10994
[00:00:00:00] Fuzzing starts
[00:00:00:00] [*] Found by grey-box concolic (225 new nodes) : ("eclipser.input") File[0]=( 34! 3d! ...6bytes... (0) (Right))
[00:00:00:00] Found crash seed : ("eclipser.input") File[0]=( 34! 3d! 00 00 42! 61! 64! 21! (4) (Right))
[00:00:00:00] [*] Found by grey-box concolic (598 new nodes) : ("eclipser.input") File[0]=( 34! 3d! 00 00 42! 61! 64! 21! (4) (Right))
[00:00:00:03] # of seeds before minimization : 2
[00:00:00:03] Total 0 redundant seeds found

softsec-kaist / eclipser Goto Github PK

eclipser's Issues

Please provide a Dockerfile for Eclipser 2.0

Understand the output directory

TCG instrumentation?

Missing documentation how to reproduce faults

Any chance you can fix qemu to work on ubuntu 20.04

Fuzzing rocksdb doesn't work

Fork server died: Success

Timeout of -1 does not work in old Eclipser

Allow the user to select the arquitecture to compile

Sampling input bytes instead of conditional statements in SELECT?

Eclipser doesn't seem to work with LLVM sanitizers

Confirmation on sharing state between multiple instances of Eclipser

Changelog from Eclipser 1.0?

Solving timeout

Using DeepState makes graybox concolic much less effective.

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent