ctx.Response.Send multiple times per connection? about watsonwebserver

[jchristn]

Here you go! HttpServerLite is live on NuGet.

NuGet v1.0.0.16: https://www.nuget.org/packages/HttpServerLite/1.0.0.16
Commit: jchristn/HttpServerLite@19a3e95

[jchristn]

Hi @winkmichael - first off, thanks for the sponsorship! And it's never a bother!

Regarding your question, right now all of the Send methods are final and will close the connection out. A quick solution would be to use one of the methods that takes a stream as input, but of course, you would have to have the stream built ahead of time.

Would it be of value to you if I researched adding something like a SendPartial method to the server? The only complication I can really see is that the Content-Length header up front might be a problem, but it would otherwise look like SendChunk and SendFinalChunk.

Cheers, Joel

[jchristn]

Another thought - have you given consideration to using a websocket for this use case as well? Something like https://github.com/jchristn/watsonwebsocket could certainly do the trick - it would allow you to stream messages as you like while keeping the connection open long term. (Note: WatsonWebsocket as a server can work with any websocket client implementation, doesn't have to be WatsonWebsocket on both sides)

[winkmichael]

Regarding your question, right now all of the Send methods are final and will close the connection out. A quick solution would be to use one of the methods that takes a stream as input, but of course, you would have to have the stream built ahead of time.

Yah I was nearly certain I'm pretty familiar with Watson (: As I mentioned (and related to my yesterdays comment) I have this implemented using the chunked encoding

                    byte[] jpeg = jpegFile(UrlEntries[2]);
                    ctx.Response.SendChunk(Encoding.ASCII.GetBytes(MJPEGHeader));
                    ctx.Response.SendChunk(Encoding.ASCII.GetBytes("Content-length: " + jpeg.Length.ToString()));
                    ctx.Response.SendChunk(jpeg);
                    ctx.Response.SendChunk(Encoding.ASCII.GetBytes(MJPEGFooter));

This works perfect for the iOS / Safari realm but the Chrome world wants HTTP 1.1 - and iOS will only accept Chunks... Where is that XKCD graphic about standards..

Would it be of value to you if I researched adding something like a SendPartial method to the server? The only complication I can really see is that the Content-Length header up front might be a problem, but it would otherwise look like SendChunk and SendFinalChunk.

I have done some experimenting and it seems most implementations of MJPEG the Content-Length is wishy washing - meaning it can be there or it can't but, but really it is inline to the content (See here https://en.code-bude.net/2013/01/02/wiki-motion-jpeg/ )

If there was a SendPartial that would be killer, or reuse the methods and toggle based on ctx.Response.ChunkedTransfer = true/false.

Thanks for your time and thoughtful responses as always!

[jchristn]

Awesome, thanks @winkmichael I will take a look at the link tomorrow and see what I can do on my end. Cheers!

[jchristn]

Hi @winkmichael looking at the link now, the response shown there is HTTP 1.0. Will this work with HTTP 1.1?

It looks like a simple solution would be for me to exclude the Content-Length header if the Content-Type starts with multipart. Then I could add two new APIs:

SendMultipart (which wouldn't close)
and
SendMultipartFinal (which would close)

Would you agree?

[winkmichael]

Yes sir are correct. I misspoke "typed" when I said said 1.1. What I should have said was the differentiator was between start HTTP and Chunked encoding. I've done a test where I completely skip Content-Length and thus far it works fine. I am pretty much positive your proposal of SendMultipart and SendMultipartFinal would work for this exact situation and many others. Thanks again for your time and help!

[jchristn]

Awesome - I'm doing this today. To confirm, this should use HTTP 1.1, correct?

By the way I'm looking into HttpListenerResponse and not seeing a way other than using ChunkedTransfer that will allow me to ignore the Content-Length header. My concern is that the underlying implementation in http.sys will be demanding that, but we will give it a go anyway.

[jchristn]

Looks like I'm going to have to go the route of having APIs that allow Send without close. Not seeing a good native multipart option. More to come.

[jchristn]

Hi @winkmichael I have the code up and running, but I can't seem to get it to not send the initial Content-Length header. It's streaming the data just fine in chunks (not using chunked transfer, but rather SendWithoutClose APIs) but it's not rendering in either Chrome or Postman. I'm going to put up a StackOverflow post on this today and see what I can get back. I hope this isn't a limitation in either HttpListenerResponse or http.sys. Cheers

[jchristn]

https://stackoverflow.com/questions/62582863/multipart-httplistenerresponse-without-using-chunked-transfer

Will keep you posted.

[jchristn]

Hi @winkmichael it appears that this kind of capability doesn't exist in HTTP 1.1. One of the comments on the SO question I asked pointed to the RFC here: https://tools.ietf.org/html/rfc2616#section-4.4

Do you know for sure the client is the same as the one in the link you sent (i.e. using HTTP 1.0)? Do you know if the client uses HTTP or HTTPS? It wouldn't be too hard to put together a simple TCP-based HTTP server specifically for this use case (probably take me an hour). I don't think WatsonWebserver is going to be able to support it because the HttpListenerResponse.ContentLength64 is not nullable, and if it's set to 0, it won't send any data. Plus the RFC seems to indicate that it would be in clear violation of the protocol spec.

Please let me know your thoughts. I'd be open to putting together a TCP-based HTTP 1.0 server specifically for your use case. It might give me a good start on eliminating reliance on http.sys for WatsonWebserver altogether, which I would need to do in order to build in support for HTTP 2.0.

Cheers, Joel

[winkmichael]

Thank you so very much for all the efforts on this! I was at work very late this evening, but I will setup a test tomorrow morning before I head in. I would say more, but I think it is best I do more testing before commenting further. Thanks again!

[jchristn]

Sounds good @winkmichael - I'm about half way through the TCP-based implementation. Code complete on parsing incoming requests, have a lot of work to do on sending responses. Will keep you posted on it. No other news from the SO question yet.

[jchristn]

Hi @winkmichael I created a private repo called HttpServerLite and invited you to it. I welcome you to check it out. It's waaaay early (hence no NuGet package) but it's a working TCP-based HTTP server with no dependencies on http.sys or HttpListenerResponse etc.

It allows sending whatever you want, whenever you want, with control over closing the connection.

If you check out the Test project, you'll see there are three endpoints:

• img/watson.jpg - picture of Watson as a puppy, sent immediately and in full with no delay. Content-Length header is set
• img-streamed/watson.jpg - same picture, streamed in 1KB blocks with a brief delay between each block. Content-Length header is not set
• html/index.html - simple HTML file, sent immediately and in full with no delay

It should support SSL (it uses CavemanTcp under the hood, another library I wrote, which supports PFX certificate files for SSL), but I haven't tested it.

It has a lot of issues right now, but they are largely in terms of design, efficiency, and completeness. It's working though, so I invite you to clone it and run the test app, using the following URLs:

• http://localhost:9000/img/watson.jpg
• http://localhost:9000/img-streamed/watson.jpg
• http://localhost:9000/html/index.html

Here's a screenshot showing the streamed watson.jpg file. You can see how long it took, and that the content-length header was never set.

Since I'm almost 100% certain we won't be able to accomplish this use case with WatsonWebserver, I'm unfortunately going to close this issue.

However I'm happy to work with you on this prototype to get it going for your needs, and migrate over WatsonWebserver features that you are currently using based on your priority.

I'd like to eventually transition 100% off of http.sys and HttpListenerResponse with WatsonWebserver, and see this as a stepping stone in that direction. As I'm sure you can appreciate, that's a pretty big undertaking because they do a tremendous amount of work under the hood that I have until this point been able to overlook.

Cheers, Joel

[winkmichael]

Thanks man this is awesome! I'm just sneaking away on my lunch break to check email (Strict work VPN) I can't wait to get home and check this out! Thanks again, will post back once I get a chance to dive in!

[jchristn]

Hi @winkmichael any news on your side?

[winkmichael]

Hello Joel, I am sorry for the silence I plan to post back code to you a bit later tonight! So far, everything works great. I thought it might be most logical if you create another project to your offering a WatsonMJPEG server or something to that effect? Anyhow will post you what I got with usage examples etc.

[jchristn]

Hi @winkmichael my only plans are to (eventually) eliminate Watson's dependency on http.sys. If you want to package it and release it please feel free! I can also help you put together a nuget package, etc. If there are any defects (or enhancements) you see in the code please let me know (or submit a PR!)

[winkmichael]

Ok, I'll definitely wait until you make it into a package, and then will release it with the new package as a dependency. Just working a few details on my end with socket reading, but I am pretty sure this is gonna work perfect. I will keep you posted.

[jchristn]

Awesome :) I'll start slowly porting over Watson features and hopefully have something to release in the next week or two. Cheers!

[jchristn]

Just some performance data for reference. WatsonWebserver smokes HttpServerLite in performance, but that's to be expected - http.sys (used by Watson) is in the kernel, whereas HttpServerLite is 100% in user-space.

This is a simple test of 10 clients, 1000 requests, to a loopback endpoint (no content delivered). HttpServerLite is the top set, Watson is the bottom set. Net-net, Watson is 5x faster (average latency) and 3x throughput.

C:\Users\joelc\Downloads>bombardier-windows-amd64.exe -c 10 -n 1000 http://localhost:9000
Bombarding http://localhost:9000 with 1000 request(s) using 10 connection(s)
 1000 / 1000 [======================================================================================] 100.00% 2481/s 0s
Done!
Statistics        Avg      Stdev        Max
  Reqs/sec      2683.78     384.07    3058.27
  Latency        3.71ms   577.70us    12.97ms
  HTTP codes:
    1xx - 0, 2xx - 1000, 3xx - 0, 4xx - 0, 5xx - 0
    others - 0
  Throughput:     1.01MB/s

C:\Users\joelc\Downloads>bombardier-windows-amd64.exe -c 10 -n 1000 http://localhost:9000
Bombarding http://localhost:9000 with 1000 request(s) using 10 connection(s)
 1000 / 1000 [======================================================================================] 100.00% 4954/s 0s
Done!
Statistics        Avg      Stdev        Max
  Reqs/sec     17345.60   15501.15   37627.71
  Latency      598.27us     1.42ms    32.91ms
  HTTP codes:
    1xx - 0, 2xx - 1000, 3xx - 0, 4xx - 0, 5xx - 0
    others - 0
  Throughput:     3.19MB/s

The second test was with some content being sent (JPG image). 1000 requests over 10 clients. Watson (bottom test) was again ~3x faster (latency) and >3x throughput.

C:\Users\joelc\Downloads>bombardier-windows-amd64.exe -c 10 -n 1000 http://localhost:9000/img/watson.jpg
Bombarding http://localhost:9000/img/watson.jpg with 1000 request(s) using 10 connection(s)
 1000 / 1000 [======================================================================================] 100.00% 1243/s 0s
Done!
Statistics        Avg      Stdev        Max
  Reqs/sec      1441.68     211.20    1854.85
  Latency        6.91ms     1.00ms    16.95ms
  HTTP codes:
    1xx - 0, 2xx - 1000, 3xx - 0, 4xx - 0, 5xx - 0
    others - 0
  Throughput:   203.24MB/s

C:\Users\joelc\Downloads>bombardier-windows-amd64.exe -c 10 -n 1000 http://localhost:9000/img/watson.jpg
Bombarding http://localhost:9000/img/watson.jpg with 1000 request(s) using 10 connection(s)
 1000 / 1000 [======================================================================================] 100.00% 2470/s 0s
Done!
Statistics        Avg      Stdev        Max
  Reqs/sec      4774.15    2052.52    7068.31
  Latency        2.12ms     2.11ms    49.48ms
  HTTP codes:
    1xx - 0, 2xx - 1000, 3xx - 0, 4xx - 0, 5xx - 0
    others - 0
  Throughput:   663.02MB/s

I guess living in the kernel has its advantages. Still should be pretty fast (given the fact that it's user space). I only have a few more things left to do to it and will report back here when it's ready. I'm unlikely to make this the replacement for the innards of WatsonWebserver though because of the tremendous performance difference.