Server stops accepting connections after a while about rustless

[s-panferov]

@reem @seanmonstar have you seen something like this before?

@andor44 do you have code to reproduce?

[reem]

I haven't. I'm guessing there's a bug in hyper's ListenerPool that causes it to drop threads.

[seanmonstar]

Logs would also be useful, if possible (RUST_LOG=hyper=trace and usage of env_logger).

[andor44]

@s-panferov I do not have any, yet. I'll see if I can reproduce this with a minimal code example, but like I said, the problem is that this seems to manifest itself over time

@seanmonstar I provided that env var but I see no output whatsoever.

[seanmonstar]

@andor44 you need to set up a logger. The env_logger package on crates.io uses the env var.

[andor44]

@seanmonstar gotcha. I'll keep logging and see if I can catch anything.

[andor44]

Okay, I hit the point where it doesn't accept connections again. I can see a lot of
TRACE:hyper::http: TODO: handle obs-folding (gross!)
But this doesn't seem very relevant. What seems more worrying is that every request seems to end in

DEBUG:hyper::http: read request line
DEBUG:hyper::server: ioerror in keepalive loop = HttpIoError(IoError { kind: EndOfFile, desc: "end of file", detail: None })

And then a new one begins again with DEBUG:hyper::server: Incoming stream. The last request that was processed didn't end up up in that error though.

[seanmonstar]

The EOF error is normal, it's how the server knows the connection was
closed. That's why it's only DEBUG. Unexpected errors log at ERROR.

However, EndOfFile errors were changed to something else with the new IO.
Are you using latest hyper?

On Tue, Mar 10, 2015, 2:23 AM Andor Uhlár [email protected] wrote:

Okay, I hit the point where it doesn't accept connections again. I can see
a lot of
TRACE:hyper::http: TODO: handle obs-folding (gross!)
But this doesn't seem very relevant. What seems more worrying is that
every request seems to end in

DEBUG:hyper::http: read request line
DEBUG:hyper::server: ioerror in keepalive loop = HttpIoError(IoError { kind: EndOfFile, desc: "end of file", detail: None })

And then a new one begins again with DEBUG:hyper::server: Incoming stream.
The last request that was processed didn't end up up in that error though.

—
Reply to this email directly or view it on GitHub
#35 (comment).

[andor44]

I'm on 0.1.13 according to my Cargo.lock. I tried updating earlier last week but was unable to, due to deuterium being out of date IIRC.

[s-panferov]

@andor44 not it's ok, please try the new version.

[reem]

I am still just guessing, but what might be going on is that every thread in the pool is occupied in a keep-alive loop and the connections are never being closed.

[andor44]

I've updated to today's rustc nightly, plus all the highest versions available on crates.io (hyper 0.3.0, rustless 0.5.8). I'll let you know how it goes with this.

FWIW, I'm now seeing

ERROR:hyper::server: request error = HttpMethodError

at the end of requests instead of the IoError

[s-panferov]

@reem maybe we can add some more debugging logging (in temp branch of hyper) to check your idea?

[andor44]

This still happens with hyper 0.3.0

[andor44]

The situation @reem described sounds very plausible, as these are mobile clients that are sending Connection: keep-alive. I'll try bump the number of threads until it's fixed. It'd be probably a good idea to take this over to hyper's issue tracker.

[s-panferov]

@andor44 yes, could you please make an issue there?

I also would be glad if you give me some common feedback about Rustless and maybe some thoughts about additional features you want.

[andor44]

Opened hyperium/hyper#368.

@s-panferov I'll try to collect my thoughts and probably open a separate issue with them in the not too distant future, if that's ok 😃