ggreg / python-synapse

Python package for distributed communications

README for python-synapse

Overview

Synapse is a simple Python module that provides a communication interface through nodes. It abstracts the underlying protocol. The configuration stores protocol specific data and a node is instanciated with the node.makeNode() factory.

The node is the unit of communication. Actor is built on top of Node to provide a distributed communication interface. When an actor joins the network it announces itself to a announce service. The announce is stored in the remote service and broadcast to all other nodes by a pub/sub queue.

Currently it supports Zeromq as the underlying protocol and JSON-RPC to encode and decode messages.

Read doc/quickstart for a practical introduction. As documentation in doc/ is in Sphinx format, type make in the directory to see how to build it in HTML or PDF.

Tests

launch all the tests:

$ python setup.py test

Test a specific feature:

• Message

  $ python setup.py -s synapses.tests.test_message

• All Node features

  $ python setup.py -s synapses.tests.test_node

• Node zmq tests

  $ python setup.py -s synapses.tests.test_node.ZMQTestCase

To test the code coverage

( you need nose, use last version from http://pypi.python.org/pypi/nose )

$ python setup.py nosetests

python-synapse's Issues

zmq.poll in pyzmq>2.0.10.1 is now raising the EINTR

zmq.poll does not anymore handle the EINTR returned by poll() in version pyzmq>2.0.10.1.
Two options:

• this can be a bug in pyzmq, maybe they still want to catch the EINTR to keep the old behavior
• we need to catch the EINTR on our side

zmq.poll fail when the timeout is too large

The zmq.poll methods fails when the timeout is too large.

Exception is:

Traceback (most recent call last):
  File "/usr/local/lib/python2.6/dist-packages/gevent/greenlet.py", line 405, in run
    result = self._run(*self.args, **self.kwargs)
  File "/usr/local/lib/python2.6/dist-packages/synapse/node.py", line 688, in loop
    cont = self.poll()
  File "/usr/local/lib/python2.6/dist-packages/synapse/node.py", line 752, in poll
    actives = self._poller.poll(polling_timeout_ms)
  File "_zmq.pyx", line 1061, in zmq._zmq.Poller.poll (zmq/_zmq.c:8087)
  File "_zmq.pyx", line 942, in zmq._zmq._poll (zmq/_zmq.c:6986)
OverflowError: long int too large to convert to int
<Greenlet at 0x9f71e2c: <bound method ZMQPoller.loop of <synapse.node.ZMQPoller object at 0x9fe8a2c>>> failed with OverflowError

Fails with values: 86400, 3600.
Works with 1800.

But this fairly low value is strange cause it seems to overflow when converting from long to integer, and even if the value (given in seconds) is converted in milliseconds, it would fit into an integer.
If the value is converted in microseconds, it can explain the overflow.

refactor node.ZMQPoller.poll() to call the periodic handler a first time without waiting for the timeout