support namespacing about circuits

[apollo13]

pep-0420 is the only namespacing to support. setuptools namespaces can break horribly in certain situations (ie removal of packages etc…).

I think this should be closed or at least deferred till python builtin namespaces can be used, at which point there is no work to be done aside from removing __init__.py files in the locations in question.

[apollo13]

@spaceone That would only be an option on py3. That said, I do not think every package needs to be namespacable. I guess if you need it, we can leave the existing support -- though I would recommend moving to circuits_http or so given the problems namespacing can cause.

Btw: what does circuits.http do and why is it not part of circuits ^^

[prologic]

I'm keen to give this a try again and will do some local testing of my own :) Well done :) Let's see if we can move this forward and merge it in for 3.2?

[prologic]

BTW: There's a bug with the implementation of this. If you run anything with circuits under PyPy there's a recursion problem with the namespace code and it blows the stack :/

[prologic]

We should add documentation around this:

• How to add to the circuits namespace
• How to publish new PyPi circuits.* packages

[prologic]

bump

[spaceone]

https://www.python.org/dev/peps/pep-0420/

[prologic]

Can you TL;DR this for us?

[prologic]

Been able to reproduce that recursion bug:

(circuits)
prologic@daisy
Thu Jun 18 00:43:35
~/circuits
$ PYTHONPATH=. pypy ./bin/circuits.bench -e 1000000^C
...
  File "/home/prologic/work/circuits/circuits/six.py", line 331, in exec_
    exec("""exec code in globs, locs""")
  File "<string>", line 1, in <module>
  File "/home/prologic/work/circuits/circuits/__init__.py", line 38, in <module>
    exec_(fd, globals())
  File "/home/prologic/work/circuits/circuits/six.py", line 331, in exec_
    exec("""exec code in globs, locs""")
  File "<string>", line 1, in <module>
RuntimeError: maximum recursion depth exceeded

[spaceone]

Okay, i created PR #101 for this.
It's an error if you have two different circuits in your PYTHONPATH. (or a wrong configured namespacing package).

[prologic]

👍

[spaceone]

Please have a look at: https://github.com/spaceone/circuits/blob/9f1949ec13f0ff2ac2c2b0449c8be95e25aeb9c0/docs/source/dev/namespacing.rst

[prologic]

Looks good so far 👍 -- Incomplete?

[prologic]

Where are we up to with this one?

[spaceone]

Docs for your wished "How to publish new PyPi circuits.* packages" is missing. I don't know what to write there. Otherwise this seems fine.
If we want to support namespacing in circuits.protocols, etc. we have to copy that code into these __init__.py files, too.
Python3.X supports namespacing OOTB for packages which DON'T have a __init__.py. We could add a circuits.thirdparty. namespace for python3 only (bad idea imho).

[prologic]

Yeah let's try to get this over the line I think :)
(I'll try to do some more testing)

I'll see if I can create an example namespace?

Do you think it's a good idea to namespace
app, net, protocols, tools and web?

cheers
James

James Mills / prologic

E: [email protected]
W: prologic.shortcircuit.net.au

On Tue, Jul 14, 2015 at 8:00 PM, spaceone [email protected] wrote:

Docs for your wished "How to publish new PyPi circuits.* packages" is
missing. I don't know what to write there. Otherwise this seems fine.
If we want to support namespacing in circuits.protocols, etc. we have to
copy that code into these init.py files, too.
Python3.X supports namespacing OOTB for packages which DON'T have a init.py.
We could add a circuits.thirdparty. namespace for python3 only (bad idea
imho).

—
Reply to this email directly or view it on GitHub
#20 (comment).

[prologic]

Interesting to read the flask.ext docstring -- Makes me thingk we should not go down this road from lessons learned before :)

[prologic]

See: pallets/flask#1484

[spaceone]

hm. my projects like circuits.http are based on the current way.
if we want to implement pep0420 we need to remove every __init__ file.

[spaceone]

We should leave it as is, it works fine for me and i don't like _ in my own package name, i rather would fork circuits if this gets removed or rename the project into something else.
circuits.http is a web framework which all my websites run on, it's architecture forces you to be HTTP compliant, its API is still in development and it gives you full control over everything, so there are no artificial limitations (e.g. if you want to use unix sockets, if you want to give own error messages or HTML sites if problems with SSL occur, etc.). My goal is that you can really easily build truly RESTful applications with it without the need for code redundancy and workaround code (because HTTP is the uniform interface not the programming language!).
circuits.http is not part of circuits because I didn't speak about this with prologic and it depends on my external library https://github.com/spaceone/httoop which is python2.7 only (because python3 sucks with bytestrings). Prologic wants to enhance circuits.web (or be backwards compatible), I don't want this as I dislike circuits.web much and don't have the time to fix everything there.

[apollo13]

Interesting, re "python3 sucks with bytestrings" -- is that pre or post PEP461?

[spaceone]

well, pre-PEP4.6.1 / python 3.5. But still there are probably some things which I dislike. See spaceone/httoop#1
I will see, maybe I can have a look in the next weeks.

[apollo13]

Btw regarding the current usage of namespacing in circuits (copied from http://peak.telecommunity.com/DevCenter/setuptools#namespace-packages):

You must NOT include any other code and data in a namespace package's __init__.py. Even though it may appear to work during development, or when projects are installed as .egg files, it will not work when the projects are installed using "system" packaging tools -- in such cases the __init__.py files will not be installed, let alone executed.

circuits currently has a few imports in there, so as soon as a namespaced package is installed, those imports can break :/

[prologic]

This is correct and one of the issues I have with supporting namespaces. All of a sudden once you install another package in the same namespace; all the "convenient" imports are gone and you can no longer just do: from circuits import Component, Event James Mills / prologic E: [email protected] W: prologic.shortcircuit.net.au

…

On Sat, Jan 21, 2017 at 12:40 PM, Florian Apolloner < ***@***.***> wrote: Btw regarding the current usage of namespacing in circuits (copied from http://peak.telecommunity.com/DevCenter/setuptools#namespace-packages): You must NOT include any other code and data in a namespace package's __init__.py. Even though it may appear to work during development, or when projects are installed as .egg files, it will not work when the projects are installed using "system" packaging tools -- in such cases the __init__.py files will not be installed, let alone executed. circuits currently has a few imports in there, so as soon as a namespaced package is installed, those imports can break :/ — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#20 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABOv-nGcRqNEFDZpgfvTGl5OF-8JBNcOks5rUfyngaJpZM4ENHDo> .

[spaceone]

hmm, maybe we could hack by assigning sys.modules['circuits'].__dict__['Component'] = Component :D:D:D:D

[prologic]

OH my god! That's brilliant! Also awful :) Is there no way to support namespaces without breaking these imports and not hacking sys.modules? :) James Mills / prologic E: [email protected] W: prologic.shortcircuit.net.au

…

On Sat, Jan 21, 2017 at 1:04 PM, ⁣ Florian Best ***@***.***> wrote: hmm, maybe we could hack by assignign sys.modules['circuits'].*dict*['Component'] = 'Component' :D:D:D:D — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#20 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABOv-oJ216tmSvcMkpaZt8eSwwZ39_JYks5rUgJzgaJpZM4ENHDo> .

[apollo13]

I am sure you are joking, but I do not think

sys.modules['circuits'].__dict__['Component'] = Component

would actually work. You'd need to have some location to actually put that code… circuits/__init__.py is not the one you can use since that is not even guaranteed to get loaded…

[spaceone]

if circuits has a __init__.py file it's not possible to make a namespace package under circuits.contrib, right!?

[spaceone]

yes, it was more a joke, but this was a stupid decision by the python guys...

[prologic]

Why can't we make circuits.contrib the namespace? I'm okay with this approach if we do that? James Mills / prologic E: [email protected] W: prologic.shortcircuit.net.au

…

On Sat, Jan 21, 2017 at 2:05 PM, ⁣ Florian Best ***@***.***> wrote: yes, it was more a joke, but this was a stupid decision by the python guys... — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#20 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABOv-vuR1r9Y6Ta3IGHgrcHgIMODVjiEks5rUhCZgaJpZM4ENHDo> .