circuits / circuits Goto Github PK

For the same reasons as openhatch/oh-mainline#1560

See #7

See #7 -- As suggested by @pdurbin

We should built support for the new HTTP 2.0

This could also be done in cooperation with #56

See Also : #48

prologic@daisy
Mon Apr 27 08:55:26 
~/circuits
$ micropython 
Micro Python  on 2015-04-27; linux version
>>> import circuits
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "circuits/__init__.py", line 22, in <module>
  File "circuits/core/__init__.py", line 12, in <module>
  File "circuits/core/bridge.py", line 19, in <module>
ImportError: no module named 'pickle'

I think this can be solved by installing the micropython-pickle module.

pip-micropython install micropython-pickle

It would be nice to wait for the completition of a very specific event instead of just a named event as this name may exists multiple times concurrent.

one can raise SystemExit in a handler. The code then calls self.stop().
but the argument of SystemExit (which may be a int) is not evaluated.

$ python

raise SystemExit(3)
$ echo $?
3

Hi,
after few tests, it seems that circuits consume much more memory when using callEvent, than when using FireEvent with a waiting loop. To be more clear, I have written two script that process the same tasks, and theses scripts measure there memory consumption over iteration.

The two scripts can be found here: https://gist.github.com/y0no/09826e83e643bc0f98c2

One is using "call" and the other "fire". After execution here is the memory consumption for fire:

And this one for call:

As you can see call function use a lot more memory than fire.

In a perfect world, the memory consumption shoulld be the same in each case. And in a really perfect world the memory should not grow over time, because after each task the memory should be released.

After a few investigation, it seems that the issue come from manager who don't correctly stop iterator. It could be great to really stop iterator with a "StopIteration" exception that allow garbage collector to do its job.

See #7

See: badges/pypipins#37

I'm removing them from the README for now.

There should be a Component which is able to drop priviledges.
This is neccessary to e.g. listen on port 80 (as root) and then run as a normal user.

Update README to indicate we've moved the project management and source code of circuits to [Github](https://github.com/circuits]. Related: circuits/org#3 and #7

Refactor the release Fabric task from using Mercurial to Git

prologic@daisy
Mon Apr 27 08:58:13 
~/circuits
$ micropython 
Micro Python  on 2015-04-27; linux version
>>> import circuits
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "circuits/__init__.py", line 22, in <module>
  File "circuits/core/__init__.py", line 12, in <module>
  File "circuits/core/bridge.py", line 21, in <module>
  File "circuits/core/values.py", line 11, in <module>
  File "circuits/six.py", line 5, in <module>
ImportError: no module named 'operator'

There seems to be no port of this for MicroPython :/

• Run the codebase through autopep8

There are several code modules that aren't PEP8 compliant and we are meant to be following the PEP8 style guide as part of ou "coding standards".

The autopep8 tool will pick up most (if not all) non-compliance.

This is important since we (the community at large) lost the wonderful ShginingPanda CI service that supported first-class Python testing/buiding.

Right now circuits is only tested via continous integration on Drone and this points to the old Bitbcuket repo.

See: https://travis-ci.org/circuits/circuits/jobs/64706576

See #7

We should enhance the possibility to write decorators for handler-functions, as @handler copies meta information from the function and does arginspect() to check if the first argument is called event - in that case the event is passed as argument.

If the function gets decorated these meta information doesn't match the real underlying implementation.

We should provide a utility which developers can use so that they don't need to care about these internals when they want to write additional decorators.

Any further ideas on this? I have some ideas in mind which I will implement but I would like to hear what you think.

Let's build a sleep() coroutine that pauses the execution of a coroutine for the given period of time.

We can trivially implement this I think :)

See: https://gist.github.com/prologic/f95e2fe451b5a8eccfa9

See failing test case: 75ff37b

We should discuss how we can redesign circuits.web and improve it's internals and higher level API(s).

We could integrate with HTTOOP
but my only concern with this is feature parity and API compatibility.

See: #48

As I say on IRC to @prologic, the last circuit commit (f22e626) seems to broke some tests after being merged with master branch.

@treemo are you on it ?

See: #78

For almost the same reasons do we need:

• fabfile
• tests

These get packaged up as part of circuits because of the find_packages(".") used in setup.py

See: http://circuits.readthedocs.org/en/latest/dev/index.html

What's missing is a step-by-step guide to setting up a reliable/repeatable development environment with:

• virtualenv
• pip
• installing dev requirements
• running unit tests

BaseComponent/Manager implements bool/nonzero which returns True if there are pending events.
As it is a API to inherit from BaseComponent this is error prone.
I needed a lot time to debug the lines:

if arg:
    foo()

which failed but worked finally with if arg is not None: (where arg is a component, e.g. event.value.manager).

We should not overwrite pythons expected default behavior in the API and instead hide internals like those.

Please add to this list! This will eventually be added to our new website. See: #13

This would be nice to have and we would be the first Python web framework to have this feature (I think) :)

/cc @nathschmidt you want to take this onboard right? :)

circuits could support namespace packages to allow third party components to be easier integrated.

See http://peak.telecommunity.com/DevCenter/setuptools#namespace-packages
See also commit c6b73da where we tried it in the past.

With a little adjustment it works for me with Python2.7 and every combination of the order in sys.path.

See: https://bitbucket.org/circuits/circuits/issue/122/building-a-stronger-circuits-community

The lack of the inspect module is going to cause a lot of issues:

prologic@daisy
Mon Apr 27 08:49:42 
~/circuits
$ micropython 
Micro Python  on 2015-04-27; linux version
>>> import circuits
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "circuits/__init__.py", line 22, in <module>
  File "circuits/core/__init__.py", line 11, in <module>
  File "circuits/core/events.py", line 11, in <module>
ImportError: no module named 'inspect'

See: micropython/micropython-lib#22

See: http://stackoverflow.com/questions/30182663/use-rfc2217-network-serial-ports-with-twisted-python

Many of the tests are failing with:

    @property
    def expose_circuits(self):
>       return self.__expose_circuits
E       AttributeError: 'Server' object has no attribute '_BaseServer__expose_circuits'

I have a proposed fix coming shortly but which also changes the attribute name and keyword argument to display_banner. Is that okay @treemo ? (I think it's a bit more clear this way and more in-line with other things like SMTP Banner, etc)

See also: #39

prologic@daisy
Mon Apr 27 08:54:01 
~/circuits
$ micropython 
Micro Python  on 2015-04-27; linux version
>>> import circuits
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "circuits/__init__.py", line 22, in <module>
  File "circuits/core/__init__.py", line 12, in <module>
  File "circuits/core/bridge.py", line 14, in <module>
ImportError: no module named 'traceback'

I think this can be solved by installing the micropython-traceback module.

pip-micropython install micropython-traceback

A user simply is able to call every event in every channel.

Start a server:

python circuits/examples/web/jsonrpc.py &

Kill the server:

curl http://localhost:8000/ \
    -H 'Content-Type: application/json' \
    -d '{"params": {"signo": 15, "stack": null}, "id": "", "method": "*.signal"}'

It is currently not possible to react on SSL socket errors (which are happening during handshake). The client socket is immediately closed.

I would like to write response data into that socket if the handshake fails. Also I would like to get the broken bytes send as SSL handshake from the client.

See spaceone/circuits.http@6ee3486 for an example on the use case.
Apache HTTPd is capable of this functionality.

Do we need a MANIFEST.in file at all?

AFAIK this controls what "extra" files get packaged up in the source distributions.

I'm not sure that shipped distributions of circuits need to carry:

• docs
• tests
• examples

why do we fire a close() event without an argument in the net/sockets classes when the global event 'stopped' is executed?

IMHO it would be good to prevent this and close all open sockets by fire close(socket) events if it is required at all.

The io.file component also has a close event which is fired. It would be good to give the fd as argument there. Otherwise each file must run in an own channel. There also may be conflicts if there are global event handlers.

unhandled exception occurred while testing test_bridge
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/logilab/common/pytest.py", line 440, in testfile
options=self.options, outstream=sys.stderr)
File "/usr/lib/python2.7/site-packages/logilab/common/pytest.py", line 756, in init
testLoader=NonStrictTestLoader())
File "/usr/lib/python2.7/unittest/main.py", line 77, in init
self.module = import(module)
File "test_bridge.py", line 6, in
if pytest.PLATFORM == "win32":
AttributeError: 'module' object has no attribute 'PLATFORM'

test_bridge.py seems to fail intermitently

Since it's "intermittent" on Travis CI; it's kind of difficulty to determine why it fails.

time python2 slow.py
real 0m6.487s
user 0m0.360s
sys 0m0.030s

from circuits import *

class a(Event): pass
class b(Event): pass

class Slow(BaseComponent):
    @handler('started')
    def started(self, manager):
        self.iter = self.foo()
        self.fire(a())

    def foo(self):
        for d in range(30):
            print d
            yield d

    @handler('a')
    def _on_a(self):
        try:
            d = next(self.iter)
        except StopIteration:
            self.stop()
            return
        yield self.call(b(d))
        yield self.call(a())

    @handler('b')
    def _on_b(self, d):
        print d

Slow().run()

If you replace "yield self.call" by self.fire you will get:
real 0m0.256s

spaceone/circuits.http@ef5d7f6

We have Worker which really wraps around multiprocessing.

Typically you would use this for CPU-bound work in a circuits application.

However there are use-cases where you might want to call I/O blocking functions whereby there is no Asynchronous equivalent and call such "blocking" functions in a thread/threadpool with a Future/Promise.

We kind of used to have this; but it sucked!

I modified circuits/core/manager.py to add in the Debugger() for such a situation to help debug this and this is what's happening:

$ python -i examples/hello_bridge.py 
>>> <registered[*] (<Debugger/* 18273:MainThread (queued=0) [S]>, <App/* 18273:MainThread (queued=4) [R]> )>
<registered[93e2ff71-6991-4a0d-917d-83473043f777] (<UNIXClient/93e2ff71-6991-4a0d-917d-83473043f777 18273:MainThread (queued=0) [S]>, <Bridge/93e2ff71-6991-4a0d-917d-83473043f777 18273:MainThread (queued=0) [S]> )>
<registered[93e2ff71-6991-4a0d-917d-83473043f777] (<Bridge/93e2ff71-6991-4a0d-917d-83473043f777 18273:MainThread (queued=0) [S]>, <App/* 18273:MainThread (queued=4) [R]> )>
<started[*] (<App/* 18273:MainThread (queued=3) [R]> )>
<registered[select] (<Select/select 18273:MainThread (queued=0) [S]>, <UNIXClient/93e2ff71-6991-4a0d-917d-83473043f777 18273:MainThread (queued=0) [S]> )>
<ready[93e2ff71-6991-4a0d-917d-83473043f777] (<UNIXClient/93e2ff71-6991-4a0d-917d-83473043f777 18273:MainThread (queued=0) [S]> )>

>>> x = m.fire(hello())
>>> <_read[93e2ff71-6991-4a0d-917d-83473043f777] (<socket object, fd=4, family=1, type=1, protocol=0> )>
<read[93e2ff71-6991-4a0d-917d-83473043f777] ("(I8759933159977\nccopy_reg\n_reconstructor\np1\n(c__main__\nhello\np2\nc__builtin__\nobject\np3\nNtRp4\n(dp5\nS'uid'\np6\nNsS'args'\np7\n(lp8\nsS'value'\np9\ng1\n(ccircuits.core.values\nValue\np10\ng3\nNtRp11\n(dp12\nS'errors'\np13\nI00\nsS'handled'\np14\nI00\nsS'parent'\np15\ng11\nsS'_value'\np16\nNsS'promise'\np17\nI01\nsS'result'\np18\nI00\nsS'event'\np19\ng4\nsS'notify'\np20\nI00\nsbsS'channels'\np21\n(S'*'\ntp22\nsS'stopped'\np23\nI00\nsS'waitingHandlers'\np24\nI1\nsS'kwargs'\np25\n(dp26\nsS'cancelled'\np27\nI00\nsbt.~~~" )>
<hello[*] ( )>
<read_value_changed[<UNIXClient/93e2ff71-6991-4a0d-917d-83473043f777 18273:MainThread (queued=0) [S]>] (<Value () result=False; errors=False; for <read[93e2ff71-6991-4a0d-917d-83473043f777] ("(I8759933159977\nccopy_reg\n_reconstructor\np1\n(c__main__\nhello\np2\nc__builtin__\nobject\np3\nNtRp4\n(dp5\nS'uid'\np6\nNsS'args'\np7\n(lp8\nsS'value'\np9\ng1\n(ccircuits.core.values\nValue\np10\ng3\nNtRp11\n(dp12\nS'errors'\np13\nI00\nsS'handled'\np14\nI00\nsS'parent'\np15\ng11\nsS'_value'\np16\nNsS'promise'\np17\nI01\nsS'result'\np18\nI00\nsS'event'\np19\ng4\nsS'notify'\np20\nI00\nsbsS'channels'\np21\n(S'*'\ntp22\nsS'stopped'\np23\nI00\nsS'waitingHandlers'\np24\nI1\nsS'kwargs'\np25\n(dp26\nsS'cancelled'\np27\nI00\nsbt.~~~" )> )>
<value_changed[<Bridge/93e2ff71-6991-4a0d-917d-83473043f777 18273:MainThread (queued=0) [S]>] (<Value ('Hello World! (18273)') result=True; errors=False; for <hello[*] ( )> )>
<_write[93e2ff71-6991-4a0d-917d-83473043f777] (<socket object, fd=4, family=1, type=1, protocol=0> )>

>>> x
<Value ('Hello World! (18273)') result=True; errors=False; for <hello[*] ( )>
>>> getpid()
18269
>>> p.terminate()
>>> <signal[*] (15, <frame object at 0x1193b80> )>
Unhandled ERROR: 'NoneType' object has no attribute 'terminate'
Traceback (most recent call last):
  File "/home/prologic/work/circuits/circuits/core/manager.py", line 985, in run
    self.tick()
  File "/home/prologic/work/circuits/circuits/core/manager.py", line 942, in tick
    self._queue and self.flush()
  File "/home/prologic/work/circuits/circuits/core/manager.py", line 601, in flushEvents
    self.root._flush()
  File "/home/prologic/work/circuits/circuits/core/manager.py", line 590, in _flush
    self._dispatcher(event, channels, self._flush_batch)
  File "/home/prologic/work/circuits/circuits/core/manager.py", line 673, in _dispatcher
    self.stop(e.code)
  File "/home/prologic/work/circuits/circuits/core/manager.py", line 806, in stop
    self.__process.terminate()
  File "/usr/lib64/python2.7/multiprocessing/process.py", line 137, in terminate
    self._popen.terminate()
AttributeError: 'NoneType' object has no attribute 'terminate'

@prologic and I have been talking about adding Circuits to http://openhatch.org

Now that openhatch/oh-mainline#1515 I guess I'll go ahead and try per http://irclogs.shortcircuit.net.au/%23circuits/2015-05-05.log.html#t22:59:06

See: https://travis-ci.org/circuits/circuits/jobs/64706579

We should split the graph() function. One function should write the image file (where one can specify also a filename/path). The other one should return the ascii-art. Both use cases exists. Both functions are very nice!

Now that we're on Github and starting to see more contributors (thanks everyone!) -- Let's talk about how we manage Pull Requests and Merging. There's pros/cons to different styles of workflows so let's discuss them here and what we (the community) want to see in terms of circuits future development life-cycle(s).

• RoadMap: https://github.com/circuits/circuits/milestones
• Issues: https://github.com/circuits/circuits/issues
• Releases configuration
• Add Ask a Question (Tag with: circuits-framework)
• Add Google+ Group
• Any old links pointing to the old Bitbucket repos.