Streaming request body for server about zio-http

[tusharmath]

Thanks for opening this issue @DJLemkes 🙏 I have been thinking about it for a long time and am kinda split between multiple approaches.

One problem that we will face when encoding Content using ZStreams is that — it will be very slow even for smaller Http Requests and Responses where chunking or streaming isn't necessary. HttpObjectAggregator does a really awesome job in terms of performance for small to medium-sized requests (up to a few Kbs). But as soon as we start dealing with file uploads or downloads or requesting using multi-part data etc. the HttpObjectAggregator doesn't scale.

[DJLemkes]

I think that's a valid concern; have been thinking about that myself as well.

So ideally, you'd offer an API that is tailored to your use case I think? That is, if you know you'll only be doing small requests you'd just get the request content as a String. Don't have an idea on how to do that though. And whether that is desirable.

On the other hand, we can tune the chunk size for the Netty Http decoder like here, such that in most cases the HttpObjectAggregator won't have that much aggregation to do.

I can try and see if I can get a small PoC ready this week. It's all speculation until we run some benchmarks. Quite sure that it'll be worse, but the question is how much.

What are the other approaches you were considering?

[tusharmath]

We can split the problem of reading body into two concerns —

1. Request — This is can be done in three ways
   a. Only expose a ZStream based API for reading request body, this is what Http4s is also doing. Performance is going to be really poor here. Especially if you start doing any form of aggregation (based on our internal benchmarks it can be up to 5x slower than Netty's Object Aggregator).

   b. Use a server configuration at startup to decide if you want to expose a ZStream API vs using ObjectAggregator. This is a classical approach and might just solve both use cases. Problem is that the dev will always have to perform a check inside the router to see if body is exposed as a stream or is "completely" available.

   c. Provide a special Http constructor to read requests as a stream eg Http.chunked.collect or something else which guarantees that the data is available as a stream.

2. Response — This is easy to solve, because the developer can choose to use Complete vs Chunked content while writing the response.

May be we can start by at least providing support for streaming responses?

[DJLemkes]

Yes, nice layout of the problem. Actually with this issue I only meant looking at the request side of things. Turns out that might just the hardest part, haha.

I took a small shot at direction 1a for the Request in #147. Very rough around the edges, but please have a look. Yes, it is slower but it's not as bad as I expected actually. One major downside of this streaming approach that provides backpressure to the socket is that you always have to consume the request body to know for sure you're fully processing the request. With GET requests in most cases it'll work without consuming the stream but it's not a guarantee. This makes Http.collect more tricky to work with as consuming/discarding the body will always require a Http.collectM

I do really like idea 1c although then you still have to know at Netty init time if you want to use the HttpObjectAggregator or not.

As for the response side of things; that actually sounds like a nice first step. Are you already working on that?

[amitksingh1490]

@DJLemkes I am working on the Chunked Responses.

[DJLemkes]

Made a small update in #147 that makes server streaming requests configurable like proposed by @tusharmath in 1b. It might not be the prettiest in terms of user experience as he already said. But in this case it definitely helps running benchmarks in the same context :).

[DJLemkes]

Another update in #147 based on the latest work on streaming responses (which are awesome!). Performance of streaming is nearly (~5%) on par of non-streaming.

@amitksingh1490 it also fixes a bug in the non-streaming handler where subsequent requests to the same endpoint were causing Netty to throw exceptions (unexpected message type: DefaultHttpResponse, state: 1). I can trigger that by sending multiple requests to e.g. the /foo endpoint in HelloWordAdvanced app.

[ShrutiVerma97]

Request Streaming is supported in zio-http v1.0.0.0.0-RC27 and v2.0.0.0-RC5+