Support HTTP2 protocol server/client about circuits

[nathschmidt]

Yep, currently planning to plan something out over the weekend and get started on it pretty quickly.

At the moment, as per our off line discussion, I'm looking at building a framework agnostic library then integrating with circuits. At this point I'm going to dig through the standard python HTTP server lib, the Netty Java HTTP2 server lib and see what the best way to start off is.

One thing I absolutely want, and am yet to see in the couple of server implementations I've played with, is a decent API for handling stream dependancy and prioritisation. One thing those implementations seem to focus on is the stream push aspect, but there's more than just that in the spec.

-N

[prologic]

Since circuits and thus circuits.web is an event-driven architecture; this should be really easy to achieve right? :)

[spaceone]

CC

[spaceone]

I would probably participate in this. I already wrote a HTTP/1.1 library which can be used for server+client+proxy+cache in a generic manner.

Btw: there is already a HTTP/2 client implementation in python: https://github.com/lukasa/hyper
(Well the API is not nice, it looks like all the weak ugly python stdlib HTTP API's but maybe it helps as a starting point for the server implementation)

If I got enough time in the next month I will have a closer look into the HTTP/2 spec and implement this in a generic useable way (server+client).

[prologic]

👍 3 heads are beter than 1 :)

[nathschmidt]

👍 Definitely, 3 > 1.

Yeah, I've had a look at hyper, it's API is pretty weak. As I said at the beginning, I'd really like a nice flexible API that'll let me play with the stream dependancies and priorities etc. I can see your HTTOOP library, are you thinking of extending that library? I was going to just start with a circuits.web.http2 package.

Either way, exciting times - not often a little library like circuits gets the first mover advantage when it comes to something like this.

[spaceone]

Ok, I just finished reading RFC 7540.
I would like to adapt httoop to support HTTP2. HTTP2 is a lot (!)! more to implement.
The HTTP/2 spec are at least very clear and contain a lot of MUST criteria - which are a MUST IMHO.
I don't think that I am able to implement this in only a few weeks.

[prologic]

Can we get httoop on par with circuits.web's API (as much as possible) and re-integarted as the new circuits.web?

[spaceone]

httoop is currently implemented in http://github.com/spaceone/circuits.http and will soon include everything needed to replace all functionality of circuits.web (and even more).
The wrapper classes (Request, Response, Host, ...) are very similar to circuits.web.

circuits.http is already capable of:

• HTTP pipelining
• routing of domains / hosts
• evaluate all caching related HTTP header
• websockets
• HTTP Error handling
• Access logging
• parsing of various request body formats (multipart/formdata, application/json, application/x-www-form-urlencoded, XML)

Missing is currently:

• routing of resources via path
• static file + directory dispatching
  → They aren't commited yet because it is untested
• WSGI → exists in httoop but not in circuits.http yet
• XMLRPC / JSONRPC → I don't like RPC
• the cherrypy-like dispatching → I hate this error-prone and security-leaking dispatching, I won't adapt it
• automatic URL generation (partly exists but not finished yet)
• HTTP digest+basic authentication (currently not implemented in httoop, works manually of course)
• Sessions and Cookies → HTTP is stateless

[prologic]

Yeah this is what I meant by API.

There might be some breaking API(s) changes
but iif we can get htoop integrated into circuits in
the near future that would bd awesome.

I know how much you like RFC compliance and secure code :)

By API too I mean at a high-level. So the same apps
written in circuits.web would mostly still work
(maybe with slight modifications).

A circuits 4.0 if you will?

[spaceone]

I'll add examples how to use things. If you give me circuits.web example web appliactions I can show also how they look ẃith circuits.http.

[prologic]

I guess the main thing here is that we need to compare high-level API(s) of the two and see what the mismatches are.

[spaceone]

one main difference e.g. is that circuits.web uses (request, response) as arguments. circuits.http uses client as arguement (which contains client.request, client.response, client.socket, client.server and client.user). But let's discuss this elsewhere as this issue is for HTTP/2.

[prologic]

(y)

[nathschmidt]

Exciting. @spaceone Yeah, the RFC is pretty beefy. Adding http2 support to httoop seems like the best way to go. Even if it is slightly breaking, circuits 4 sounds good, but it almost seems like we could adapt out most of the breaking behaviour.

Do we want to start a new set of issues for the http2 implementation on httoop? Or rather keep the discussion here for now?

[prologic]

I think we should open two issues and link them here.

Done 👍

[spaceone]

I created https://github.com/spaceone/httoop.http2 were I will start the implementation. I just implemented HTTP Basic and Digest Authentication in httoop. Seems httoop is quite near to the first official version supporting every HTTP/1.1 feature.
I want to implement HTTP/2 in a second repository, as no changes to httoop are required. → resulting in clean and separated code

[prologic]

👍

[prologic]

@nathschmidt @spaceone Either of you have any plans/thoughts on how we are going to achieve HTTP2 support in circuits.web?

[nathschmidt]

Haven't had a chance :( the 3 weeks or so has been wild.

[spaceone]

It's not a thing which is done in one day...

[prologic]

I never said it was/would be :)

Just wondering what our latest thoughts on on this :)