tmetsch / python-dtrace Goto Github PK

View Code? Open in Web Editor NEW

69.0 69.0 21.0 293 KB

A Python DTrace consumer using libdtrace - Now Python can be used as DTrace Provider and Consumer...

Home Page: http://tmetsch.github.io/python-dtrace/

License: MIT License

Python 51.90% C 6.52% Cython 41.58%

dtrace dtrace-consumers python

python-dtrace's People

Contributors

Stargazers

Watchers

Forkers

dizz msabramo szaydel xtao 0x4d4e neostoic engjoy cloudxtreme gcjenkinson cambridge-advancedos slpcat zengkefu jrtc27 gmarler matthewelse tklebanoff rwatson 0mp random-developer

python-dtrace's Issues

Cython version leaks all handles

In the Cython version, the handles are closed in __del__ both for DTraceConsumer and DTraceContinuousConsumer (used by DTraceConsumerThread). However, these are extension types, and Cython does not use __del__ for extension types (for whatever reason), so they will never be called (the explicit del self.consumer in DTraceConsumerThread's destructor, which is called as it's a normal Python object, just ensures the DTraceContinuousConsumer gets deallocated, but the consumer's __del__ still won't be called). I believe using __dealloc__ for the two extension classes would work (though handle must be checked for being non-null first).

This has also exposed a (minor) potential issue: with Cython, __init__ can end up being called multiple times, so technically handle could be initialised twice, leaking the old return from dtrace_open. To be truly correct, either __init__ should guard against this, or __cinit__ should be used instead, which will only ever be called once.

doc too short for understanding

Bug: chewrec_func cannot be set

Hi,

Both DTraceConsumer and DTraceContinuousConsumer constructors fail to correctly set the chewrec_func. As a consequence, the bug is also present in DTraceConsumerThread. The following lines should use chewrec_func instead of simple_chewrec:

python-dtrace/dtrace_cython/consumer.pyx

Line 310 in 56101d6

else simple_chewrec

python-dtrace/dtrace_cython/consumer.pyx

Line 435 in 56101d6

else simple_chewrec

Not sure if other changes are needed.

Aggregates with integer keys

Does python-dtrace support using integer keys in aggregates?

Here is a simple example:

from dtrace import DTraceConsumerThread
from subprocess import check_output, Popen, PIPE, STDOUT
import time

script = """syscall::read:entry /execname == "dd"/
{
    @values[arg2] = count();
}"""
command = ["dd", "if=/dev/zero", "of=/dev/null", "status=none", "bs=1M", "count=10"]

print("Running with dtrace-python")
dtrace_thread = DTraceConsumerThread(script, sleep=1)
dtrace_thread.start()
check_output(command)
dtrace_thread.stop()
dtrace_thread.join()

print("\nRunning with command line dtrace")
with Popen(["dtrace", "-n", script], stderr=STDOUT, stdout=PIPE) as proc:
    time.sleep(0.5) # let dtrace start
    check_output(command)
    proc.terminate()
    print(proc.stdout.read().decode())

python-dtrace prints:

1793 1 [b'/'] 1
1793 1 [b'\x80'] 1
1793 1 [b''] 10

The dtrace command prints

               47                1
              128                1
          1048576               10

Is there some way to recover the integer aggregate keys from the byte strings returned (b'/', b'\x80', b''), or is this functionality not yet supported?
Thanks in advance.

(I'm using version 0.0.12 installed from master, on FreeBSD 13)

Ctypes on Mac OS X

$ python syscall_count_continuous.py
Segmentation fault: 11

$ python -V
Python 2.7.6

xception Type: EXC_BAD_ACCESS (SIGSEGV)
Exception Codes: KERN_INVALID_ADDRESS at 0x0000000000000718

VM Regions Near 0x718:
-->
__TEXT 00000001045b4000-00000001045b5000 [ 4K] r-x/rwx SM=COW /System/Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python

Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
0 libdtrace.dylib 0x00000001049626d5 dtrace_setopt + 253
1 libffi.dylib 0x00007fff90f98f44 ffi_call_unix64 + 76
2 libffi.dylib 0x00007fff90f99781 ffi_call + 853
3 _ctypes.so 0x00000001048c3762 _ctypes_callproc + 874
4 _ctypes.so 0x00000001048bdbad 0x1048bb000 + 11181
5 org.python.python 0x00000001045c92ac PyObject_Call + 99
6 org.python.python 0x0000000104645c00 PyEval_EvalFrameEx + 11370
7 org.python.python 0x0000000104642d62 PyEval_EvalCodeEx + 1413
8 org.python.python 0x00000001045e730a 0x1045bf000 + 164618
9 org.python.python 0x00000001045c92ac PyObject_Call + 99
10 org.python.python 0x00000001045d40cf 0x1045bf000 + 86223
11 org.python.python 0x00000001045c92ac PyObject_Call + 99
12 org.python.python 0x00000001046103f4 0x1045bf000 + 332788
13 org.python.python 0x000000010460bcfb 0x1045bf000 + 314619
14 org.python.python 0x00000001045c92ac PyObject_Call + 99
15 org.python.python 0x0000000104645c00 PyEval_EvalFrameEx + 11370
16 org.python.python 0x000000010464960e 0x1045bf000 + 566798
17 org.python.python 0x00000001046463e3 PyEval_EvalFrameEx + 13389
18 org.python.python 0x0000000104642d62 PyEval_EvalCodeEx + 1413
19 org.python.python 0x00000001046427d7 PyEval_EvalCode + 54
20 org.python.python 0x00000001046627bd 0x1045bf000 + 669629
21 org.python.python 0x0000000104662860 PyRun_FileExFlags + 133
22 org.python.python 0x00000001046623fd PyRun_SimpleFileExFlags + 769
23 org.python.python 0x0000000104673b23 Py_Main + 3051
24 libdyld.dylib 0x00007fff89bd35c9 start + 1

Can't install on Mac OS X 10.10

Version info and build log below:

$ uname -a
Darwin mbp.local 14.0.0 Darwin Kernel Version 14.0.0: Fri Sep 19 00:26:44 PDT 2014; root:xnu-2782.1.97~2/RELEASE_X86_64 x86_64

$ sudo python setup.py install
running install
running build
running build_py
creating build
creating build/lib.macosx-10.10-intel-2.7
creating build/lib.macosx-10.10-intel-2.7/dtrace_ctypes
copying dtrace_ctypes/__init__.py -> build/lib.macosx-10.10-intel-2.7/dtrace_ctypes
copying dtrace_ctypes/consumer.py -> build/lib.macosx-10.10-intel-2.7/dtrace_ctypes
copying dtrace_ctypes/dtrace_structs.py -> build/lib.macosx-10.10-intel-2.7/dtrace_ctypes
running build_ext
cythoning dtrace/dtrace.pyx to dtrace/dtrace.c

Error compiling Cython file:
------------------------------------------------------------
...
    def __init__(self, chew_func=None, chewrec_func=None, out_func=None,
                 walk_func=None):
        '''
        Constructor. Gets a DTrace handle and sets some options.
        '''
        self.chew_func = chew_func or simple_chew
                                                ^
------------------------------------------------------------

dtrace/dtrace.pyx:239:49: Cannot convert 'object (object, int __pyx_skip_dispatch)' to Python object

Error compiling Cython file:
------------------------------------------------------------
...
    def __init__(self, chew_func=None, chewrec_func=None, out_func=None,
                 walk_func=None):
        '''
        Constructor. Gets a DTrace handle and sets some options.
        '''
        self.chew_func = chew_func or simple_chew
                                                ^
------------------------------------------------------------

dtrace/dtrace.pyx:239:49: Cannot convert 'object (object, int __pyx_skip_dispatch)' to Python object

Error compiling Cython file:
------------------------------------------------------------
...
                 walk_func=None):
        '''
        Constructor. Gets a DTrace handle and sets some options.
        '''
        self.chew_func = chew_func or simple_chew
        self.chewrec_func = chewrec_func or simple_chewrec
                                                         ^
------------------------------------------------------------

dtrace/dtrace.pyx:240:58: Cannot convert 'object (object, int __pyx_skip_dispatch)' to Python object

Error compiling Cython file:
------------------------------------------------------------
...
                 walk_func=None):
        '''
        Constructor. Gets a DTrace handle and sets some options.
        '''
        self.chew_func = chew_func or simple_chew
        self.chewrec_func = chewrec_func or simple_chewrec
                                                         ^
------------------------------------------------------------

dtrace/dtrace.pyx:240:58: Cannot convert 'object (object, int __pyx_skip_dispatch)' to Python object

Error compiling Cython file:
------------------------------------------------------------
...
        '''
        Constructor. Gets a DTrace handle and sets some options.
        '''
        self.chew_func = chew_func or simple_chew
        self.chewrec_func = chewrec_func or simple_chewrec
        self.out_func = out_func or simple_out
                                             ^
------------------------------------------------------------

dtrace/dtrace.pyx:241:46: Cannot convert 'object (object, int __pyx_skip_dispatch)' to Python object

Error compiling Cython file:
------------------------------------------------------------
...
        '''
        Constructor. Gets a DTrace handle and sets some options.
        '''
        self.chew_func = chew_func or simple_chew
        self.chewrec_func = chewrec_func or simple_chewrec
        self.out_func = out_func or simple_out
                                             ^
------------------------------------------------------------

dtrace/dtrace.pyx:241:46: Cannot convert 'object (object, int __pyx_skip_dispatch)' to Python object

Error compiling Cython file:
------------------------------------------------------------
...
        Constructor. Gets a DTrace handle and sets some options.
        '''
        self.chew_func = chew_func or simple_chew
        self.chewrec_func = chewrec_func or simple_chewrec
        self.out_func = out_func or simple_out
        self.walk_func = walk_func or simple_walk
                                                ^
------------------------------------------------------------

dtrace/dtrace.pyx:242:49: Cannot convert 'object (object, object, object, object, int __pyx_skip_dispatch)' to Python object

Error compiling Cython file:
------------------------------------------------------------
...
        Constructor. Gets a DTrace handle and sets some options.
        '''
        self.chew_func = chew_func or simple_chew
        self.chewrec_func = chewrec_func or simple_chewrec
        self.out_func = out_func or simple_out
        self.walk_func = walk_func or simple_walk
                                                ^
------------------------------------------------------------

dtrace/dtrace.pyx:242:49: Cannot convert 'object (object, object, object, object, int __pyx_skip_dispatch)' to Python object

Error compiling Cython file:
------------------------------------------------------------
...
    def __init__(self, script, chew_func=None, chewrec_func=None,
                 out_func=None, walk_func=None):
        '''
        Constructor. will get the DTrace handle
        '''
        self.chew_func = chew_func or simple_chew
                                                ^
------------------------------------------------------------

dtrace/dtrace.pyx:354:49: Cannot convert 'object (object, int __pyx_skip_dispatch)' to Python object

Error compiling Cython file:
------------------------------------------------------------
...
    def __init__(self, script, chew_func=None, chewrec_func=None,
                 out_func=None, walk_func=None):
        '''
        Constructor. will get the DTrace handle
        '''
        self.chew_func = chew_func or simple_chew
                                                ^
------------------------------------------------------------

dtrace/dtrace.pyx:354:49: Cannot convert 'object (object, int __pyx_skip_dispatch)' to Python object

Error compiling Cython file:
------------------------------------------------------------
...
                 out_func=None, walk_func=None):
        '''
        Constructor. will get the DTrace handle
        '''
        self.chew_func = chew_func or simple_chew
        self.chewrec_func = chewrec_func or simple_chewrec
                                                         ^
------------------------------------------------------------

dtrace/dtrace.pyx:355:58: Cannot convert 'object (object, int __pyx_skip_dispatch)' to Python object

Error compiling Cython file:
------------------------------------------------------------
...
                 out_func=None, walk_func=None):
        '''
        Constructor. will get the DTrace handle
        '''
        self.chew_func = chew_func or simple_chew
        self.chewrec_func = chewrec_func or simple_chewrec
                                                         ^
------------------------------------------------------------

dtrace/dtrace.pyx:355:58: Cannot convert 'object (object, int __pyx_skip_dispatch)' to Python object

Error compiling Cython file:
------------------------------------------------------------
...
        '''
        Constructor. will get the DTrace handle
        '''
        self.chew_func = chew_func or simple_chew
        self.chewrec_func = chewrec_func or simple_chewrec
        self.out_func = out_func or simple_out
                                             ^
------------------------------------------------------------

dtrace/dtrace.pyx:356:46: Cannot convert 'object (object, int __pyx_skip_dispatch)' to Python object

Error compiling Cython file:
------------------------------------------------------------
...
        '''
        Constructor. will get the DTrace handle
        '''
        self.chew_func = chew_func or simple_chew
        self.chewrec_func = chewrec_func or simple_chewrec
        self.out_func = out_func or simple_out
                                             ^
------------------------------------------------------------

dtrace/dtrace.pyx:356:46: Cannot convert 'object (object, int __pyx_skip_dispatch)' to Python object

Error compiling Cython file:
------------------------------------------------------------
...
        Constructor. will get the DTrace handle
        '''
        self.chew_func = chew_func or simple_chew
        self.chewrec_func = chewrec_func or simple_chewrec
        self.out_func = out_func or simple_out
        self.walk_func = walk_func or simple_walk
                                                ^
------------------------------------------------------------

dtrace/dtrace.pyx:357:49: Cannot convert 'object (object, object, object, object, int __pyx_skip_dispatch)' to Python object

Error compiling Cython file:
------------------------------------------------------------
...
        Constructor. will get the DTrace handle
        '''
        self.chew_func = chew_func or simple_chew
        self.chewrec_func = chewrec_func or simple_chewrec
        self.out_func = out_func or simple_out
        self.walk_func = walk_func or simple_walk
                                                ^
------------------------------------------------------------

dtrace/dtrace.pyx:357:49: Cannot convert 'object (object, object, object, object, int __pyx_skip_dispatch)' to Python object
building 'dtrace' extension
creating build/temp.macosx-10.10-intel-2.7
creating build/temp.macosx-10.10-intel-2.7/dtrace
cc -fno-strict-aliasing -fno-common -dynamic -arch x86_64 -arch i386 -g -Os -pipe -fno-common -fno-strict-aliasing -fwrapv -DENABLE_DTRACE -DMACOSX -DNDEBUG -Wall -Wstrict-prototypes -Wshorten-64-to-32 -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -arch x86_64 -arch i386 -pipe -I/System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c dtrace/dtrace.c -o build/temp.macosx-10.10-intel-2.7/dtrace/dtrace.o
dtrace/dtrace.c:1:2: error: Do not use this file, it is the result of a failed Cython compilation.
#error Do not use this file, it is the result of a failed Cython compilation.
 ^
1 error generated.
error: command 'cc' failed with exit status 1

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.