Coder Social home page Coder Social logo

pysymemu's Introduction

PySymEmu

A symbolic execution tool, capable of automatically generating interesting inputs for x86/x64 binary programs.

Ekoparty slides: https://github.com/feliam/pysymemu/blob/master/doc/pysymemu.pdf?raw=true

API documentation: http://feliam.github.io/pysymemu/

Features:

  • Implements most x86/amd64 instructions
  • Loads ELF32 and ELF64 files
  • Enables to recreate specific machine states by API
  • Instruction semantics easy to read and extend
  • Instruction set can operate in concrete or symbolic values
  • Memory modeled so it can be concrete or symbolic (and is COW-enabled)
  • Handles operations on symbolic pointers and indexes
  • Emulation and symbolic states serialiable, meaning that the analisys can be paused/resumed or paralellized(dispy.sourceforge.net)
  • POSIX system calls modeled (Linux32 and Linux64)
  • Automatic generation of instruction testcases
  • API and instruction documentation
  • Automatic generation of intruction unittests
  • Multiple SMT solvers supported through pysmtlib (Z3, YICES, CVC4)

Dependencies:

Quick install of deps?

  
  echo Installing Capstone engine
  sudo pip install capstone
  
  echo Installing pyelftools
  sudo pip install pyelftools
  
  #Install z3 SMT solver 
  echo Go to http://z3.codeplex.com/SourceControl/latest# click Download to download z3 source code
  echo Make a folder. Unzip z3 inside that folder. dos2unix on configure. Then configure;make

Directory structure

 doc/                    Slides and papers
 examples/               Asorted set of small C examples to emulate
 tutorial/               Very simple test cases
 test/                   Unittests
 setup.py                Setuputils/pipy related (not used yet)
 linux.py                The Linux operating system micro model
 memory.py               The symbolic memory model
 smtlibv2.py             Smtlib v2 solver API 
 system.py               A quick command line tool

Tests

You may use the discover command.

$ python -m unittest discover test

Note that cpu.py testcases are generated semi-automatically using tools at test/auto

API Documentation

You may generate a fair amount of API doc using epydoc. epydoc.sourceforge.net/โ€Ž The following command will generate an html/ folder with the api documentation:

$ epydoc cpu.py memory.py linux.py smtlibv2.py system.py

Running it

THIS IS APLHA SOFT. But you may play directly on binary ELF files until you hit an unimplemented instruction or systemcall(2). The commandline gives you a somehow confusing help. :)

 $ python system.py --help
 usage: system.py [-h] [--worspace WORSPACE] [--sym SYM] [--stdin STDIN]
                 [--stdout STDOUT] [--stderr STDERR] [--env ENV]
                 PROGRAM ...

 Symbolically analyze a program

 positional arguments:
   PROGRAM              Program to analyze
   ...                  Program arguments. Need a -- separator. Ex: -- -y 2453
 
 optional arguments:
   -h, --help           show this help message and exit
   --worspace WORSPACE  A folder name fpor temporaries and results. (default pse_?????)
   --sym SYM            Consider a filename as symbolic
   --stdin STDIN        A filename to pass as standard stdin (default: stdin)
   --stdout STDOUT      A filename to pass as standard stdout (default: stdout)
   --stderr STDERR      A filename to pass as standard stderr (default: stderr)
   --env ENV            A environment variable to pass to the program (ex. VAR=VALUE)

Basically you pass a binary file for pysymemu to emulate. Let's try the toy examples:

 $ cd examples
 $ cat toy002-libc.c
int main()
{
    unsigned int cmd;
    
    if (read(0, &cmd, sizeof(cmd)) != sizeof(cmd))
    {
        printf("Error reading stdin!");
        exit(-1);
    }
    
    if (cmd > 0x41)
    {
        printf("Message: It is greater than 0x41\n");
    }
    else 
    {
        printf("Message: It is smaller or equal than 0x41\n");
    }

return 0;
}
 $ make
 gcc -fno-builtin -static -nostdlib -m32  -fomit-frame-pointer  toy001-nostdlib.c  -o toy001-nostdlib
 gcc toy002-libc.c -static -o toy002-libc
 $ cd -

Now run it under the emulator like this. First create 3 dummy files to replace the virtual/emulated stdin, stdout and stderr

 $ touch stderr
 $ touch stdout
 $ echo ++++++++++ > stdin

We'll be considering that the stdin is filled by symbolic data ( marked with '+' (yes, I know)). Also we need to tell pysymemu which part of the environment should be considered symbolic and which concret. We mark the 'stdin' file as symbolic (its '+' will be free 8bit variables) with --sym 'stdin', like this:

$ python system.py --sym stdin examples/toy002-libc

The quick and dirty command line tool will generate something like this..

 $ python system.py  --sym stdin examples/toy002-libc
 [+] Running examples/toy002-libc
 	with arguments: []
 	with environment: []
 [+] Detected arch: amd64
 starting
 Symbolic PC found, possible detinations are:  ['4005ab', '40059d']
 	Saving state dump_00000000004005ab_8452.pkl PC: 0x4005ab
 Program Finnished correctly
 stdin:  '\xc1\x00\x80\x80\x80\x80\x80\x80\x80\x80\x80\x80\x80\x80\x80\x80\x80\x80\x80\x80\x80\x80\x80\x80\x80\x80\x80\x80\x80\x80\x80\x80\x80\x80\x80\x80\x80\x80\x80\x80\x80\x80\x80\x80\x80\x80\x80\x80\x80\x80\n'
 Program Finnished correctly
 stdin:  ' \x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\n'
 Results dumped in  ./pse_xYhZwA
 10392 3038.59649123

And an insanelly verbose system.log file. Also a folder with all intermediate states and results ...

$ ls ./pse_xYhZwA
dump_000000000040059d_8452.pkl  dump_00000000004005ab_8452.pkl  dump_init.pkl  test_2.txt  test_4.txt
 $ tail -n 12341 ./pse_xYhZwA/test*
 ==> ./pse_xYhZwA/test_2.txt <==
 stdin: '\xc1\x00\x80\x80\x80\x80\x80\x80'
 
 ==> ./pse_xYhZwA/test_4.txt <==
 stdin: '\x20\x00\x00\x00\x00\x00\x00\x00'
  1. With a few mods it may accept any smtlibv2 solver that can handle (get-value) command.
  2. In such case you should go to cpu.py or linux.py and add the necesarry code!

pysymemu's People

Contributors

aquynh avatar feliam avatar liumuqing avatar orthographic-pedant avatar pfalcon avatar timgates42 avatar tunz avatar vpxorq avatar wessupermare avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

pysymemu's Issues

example as shown in read me generate only 1 test case

example as shown in read me generate only 1 test case

`
toor@toor-virtual-machine:~/pysymemu$ python system.py --sym stdin examples/toy002-libc

[+] Running examples/toy002-libc
with arguments: []
with environment: []
2016-08-04 03:27:18,653: EXECUTOR:INFO: Loading amd64 ELF program examples/toy002-libc

--- snipt ---

2016-08-04 03:27:18,656: MEMORY:INFO: Initializing Symbolic Memory
2016-08-04 03:27:18,656: CPU:INFO: Cpu Initialized.
2016-08-04 03:27:18,657: LINUX:INFO: Loading examples/toy002-libc as a amd64 elf
2016-08-04 03:27:18,657: LINUX:INFO: Arguments: ['examples/toy002-libc']
2016-08-04 03:27:18,747: LINUX:INFO: Setting argv, envp and auxv.
2016-08-04 03:27:18,780: LINUX:INFO: Setting initial cpu state
2016-08-04 03:27:18,780: LINUX:INFO: Entry point: 0000000000400ce0
2016-08-04 03:27:18,780: LINUX:INFO: Stack start: 00007fffffffdd78
2016-08-04 03:27:18,780: LINUX:INFO: Brk: 00000000006c6000
2016-08-04 03:27:18,780: LINUX:INFO: Mappings:
2016-08-04 03:27:18,781: LINUX:INFO: 0000000000400000-00000000004c2000 r x 00000000 examples/toy002-libc
2016-08-04 03:27:18,781: LINUX:INFO: 00000000006c1000-00000000006c3000 rw 00000000
2016-08-04 03:27:18,781: LINUX:INFO: 00000000006c3000-00000000006c6000 rw 00000000
2016-08-04 03:27:18,781: LINUX:INFO: 00007ffffffde000-00007ffffffff000 rwx 00000000
2016-08-04 03:27:18,781: LINUX:INFO: Found 10 free symbolic values on file stdin
Saving state state_0000000000400ce0_0.pkl
Starting...
Loading state state_0000000000400ce0_0.pkl
Program Finnished correctly
stdin: '\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\n'
Results dumped in ./pse_i2ox9d
5892 4030.23634887
toor@toor-virtual-machine:/pysymemu$ ls pse_i2ox9d/
state_0000000000400ce0_0.pkl test_1.txt
toor@toor-virtual-machine:
/pysymemu/pse_i2ox9d$ cat test_1.txt
stdin: '\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\n'
toor@toor-virtual-machine:~/pysymemu$ cat stdin
++++++++
`

tests error

Hi friend, when I run "python -m unittest discover test", some errors happen.

My linux details: Linux kali 4.0.0-kali1-686-pae #1 SMP Debian 4.0.4-1+kali2 (2015-06-03) i686 GNU/Linux

What's wrong?
Log is here:
root@kali:~/tools/pysymemu# python -m unittest discover test

........EEEEE..................................FEFEFE...........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................F.F...................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................

ERROR: test_load_elf_dynamic_32 (test_linux.LinuxTest)

Traceback (most recent call last):
File "/root/tools/pysymemu/test/test_linux.py", line 69, in test_load_elf_dynamic_32
linux.exe("./binaries/elf-dyn-32")
File "linux.py", line 212, in exe
elf = ELFFile(file(filename))
IOError: [Errno 2] No such file or directory: './binaries/elf-dyn-32'

ERROR: test_load_elf_dynamic_64 (test_linux.LinuxTest)

Traceback (most recent call last):
File "/root/tools/pysymemu/test/test_linux.py", line 60, in test_load_elf_dynamic_64
linux.exe("./binaries/elf-dyn-64")
File "linux.py", line 212, in exe
elf = ELFFile(file(filename))
IOError: [Errno 2] No such file or directory: './binaries/elf-dyn-64'

ERROR: test_load_elf_static_32 (test_linux.LinuxTest)

Traceback (most recent call last):
File "/root/tools/pysymemu/test/test_linux.py", line 51, in test_load_elf_static_32
linux.exe("./binaries/elf-static-32")
File "linux.py", line 212, in exe
elf = ELFFile(file(filename))
IOError: [Errno 2] No such file or directory: './binaries/elf-static-32'

ERROR: test_load_elf_static_64 (test_linux.LinuxTest)

Traceback (most recent call last):
File "/root/tools/pysymemu/test/test_linux.py", line 43, in test_load_elf_static_64
linux.exe("./binaries/elf-static-64")
File "linux.py", line 212, in exe
elf = ELFFile(file(filename))
IOError: [Errno 2] No such file or directory: './binaries/elf-static-64'

ERROR: test_symamd64 (unittest.loader.ModuleImportFailure)

ImportError: Failed to import test module: test_symamd64
Traceback (most recent call last):
File "/usr/lib/python2.7/unittest/loader.py", line 254, in _find_tests
module = self._get_module_from_name(name)
File "/usr/lib/python2.7/unittest/loader.py", line 232, in _get_module_from_name
import(name)
File "/root/tools/pysymemu/test/test_symamd64.py", line 2, in
from smtlibv2 import Solver
ImportError: No module named smtlibv2

ERROR: testmprotecNoReadthenOkRead (test_memory.MemoryTest)

Traceback (most recent call last):
File "/root/tools/pysymemu/test/test_memory.py", line 55, in tearDown
self.assertEqual(self.fds, self.get_open_fds())
AssertionError: Lists differ: [] != [4, 5]

Second list contains 2 additional elements.
First extra element 0:
4

  • []
  • [4, 5]

ERROR: testmprotectFailReading (test_memory.MemoryTest)

Traceback (most recent call last):
File "/root/tools/pysymemu/test/test_memory.py", line 55, in tearDown
self.assertEqual(self.fds, self.get_open_fds())
AssertionError: Lists differ: [] != [4, 5]

Second list contains 2 additional elements.
First extra element 0:
4

  • []
  • [4, 5]

ERROR: testmprotectFailWriting (test_memory.MemoryTest)

Traceback (most recent call last):
File "/root/tools/pysymemu/test/test_memory.py", line 55, in tearDown
self.assertEqual(self.fds, self.get_open_fds())
AssertionError: Lists differ: [] != [4, 5]

Second list contains 2 additional elements.
First extra element 0:
4

  • []
  • [4, 5]

FAIL: testmprotecNoReadthenOkRead (test_memory.MemoryTest)

Traceback (most recent call last):
File "/root/tools/pysymemu/test/test_memory.py", line 1070, in testmprotecNoReadthenOkRead
self.assertRaisesRegexp(MemoryException, "No Access Reading <0x%x>"%addr, mem.getchar, addr)
AssertionError: "No Access Reading <0x10000000>" does not match "No Access Reading <268435456>"

FAIL: testmprotectFailReading (test_memory.MemoryTest)

Traceback (most recent call last):
File "/root/tools/pysymemu/test/test_memory.py", line 1041, in testmprotectFailReading
self.assertRaisesRegexp(MemoryException, "No Access Reading <0x%x>"%addr, mem.getchar, addr)
AssertionError: "No Access Reading <0x10000000>" does not match "No Access Reading <268435456>"

FAIL: testmprotectFailWriting (test_memory.MemoryTest)

Traceback (most recent call last):
File "/root/tools/pysymemu/test/test_memory.py", line 1056, in testmprotectFailWriting
self.assertRaisesRegexp(MemoryException, "No Access Writting <0x%x>"%addr, mem.putchar, addr, 'a')
AssertionError: "No Access Writting <0x10000000>" does not match "No Access Writting <268435456>"

FAIL: testSCAS_2 (test_amd64.CPUTest)

Instruction REPNZ SCASB

Traceback (most recent call last):
File "/root/tools/pysymemu/test/test_amd64.py", line 48290, in testSCAS_2
self.assertEqual(cpu.getRegister(reg_name)&0x8d5, test['pos']['registers'][reg_name]&0x8d5, "%s doesn't match %x vs %x"%(reg_name,cpu.getRegister(reg_name), test['pos']['registers'][reg_name]))
AssertionError: RFLAGS doesn't match 10292 vs 10246

FAIL: testSCAS_4 (test_amd64.CPUTest)

Instruction REPNZ SCASB

Traceback (most recent call last):
File "/root/tools/pysymemu/test/test_amd64.py", line 48328, in testSCAS_4
self.assertEqual(cpu.getRegister(reg_name)&0x8d5, test['pos']['registers'][reg_name]&0x8d5, "%s doesn't match %x vs %x"%(reg_name,cpu.getRegister(reg_name), test['pos']['registers'][reg_name]))
AssertionError: RFLAGS doesn't match 10292 vs 10246


Ran 3075 tests in 5.091s

FAILED (failures=5, errors=8)

error using example

I run command :"python system.py --sym stdin examples/toy002-libc", and meet the following error:

<type 'exceptions.Exception'> 'X86Op' object has no attribute 'size'

stdin: '\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\n'

Traceback (most recent call last):
File "system.py", line 253, in
while current_state.execute():
File "system.py", line 56, in execute
return self.os.execute()
File "/home/ling/Desktop/pysymemu/linux.py", line 1229, in execute
self.current.execute()
File "/home/ling/Desktop/pysymemu/cpu.py", line 950, in execute
instruction = cpu.getInstructionCapstone(cpu.PC)
File "/home/ling/Desktop/pysymemu/cpu.py", line 98, in new_method
value = old_method(obj, *args)
File "/home/ling/Desktop/pysymemu/cpu.py", line 769, in getInstructionCapstone
op.size *= 8

AttributeError: 'X86Op' object has no attribute 'size'

Getting an error using examples

I have tried one of your example, specificallyexamples/toy001-nostdlib, but I receive this error:
Traceback (most recent call last):
File "system.py", line 136, in
linux.exe(args.program, argv, env, stdin=args.stdin, stdout=args.stdout, stderr=args.stderr)
File "/home/badnack/Documents/Projects/Bintrimmer/tools/pysymemu/linux.py", line 1231, in exe
super(SLinux,self).exe(filename, argv, envp, stdin='stdin', stdout='stdout', stderr='stderr')
File "/home/badnack/Documents/Projects/Bintrimmer/tools/pysymemu/linux.py", line 134, in exe
self.files = [ File(stdin,'rb'), File(stdout,'wb'), File(stderr,'wb')]
File "/home/badnack/Documents/Projects/Bintrimmer/tools/pysymemu/linux.py", line 18, in init
self.file = file(args,*kwargs)
IOError: [Errno 2] No such file or directory: 'stdin'

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.