Connection sharing about http

[ixti]

This depends on #72 and adding of Connection: Keep-Alive support.

[tarcieri]

I had planned to add connection pools ala Net::HTTP::Persistent as well.

@sferik is this the sort of thing you're after?

https://github.com/drbrain/net-http-persistent

not "needed" atm but having something like net-http-persistent in http would give me one more reason to not depend on net/http if i needed to re-use the same socket.

[sferik]

@tarcieri Yes, I’m looking for something functionally similar to net-http-persistent, however, I’m not such a fan of the interface, which reminds me of Net::HTTP. Have you thought about what the HTTP gem’s interface for persistent connections might look like?

imho something like HTTP.persistent("https://www.google.com").get("/foo").get("/bar") could be nice.

[sferik]

@johnny5- I need to be able to make separate, unchained requests using the same connection object, so this would need to work:

connection = HTTP.persistent("https://www.google.com")
connection.get("/foo")
connection.get("/bar")

there's no reason it couldn't work with my example and yours.

i think it would mean response's would end up being stacked in connection though and im not sure if that's good.

[sferik]

@johnny5- I just wanted to make sure it doesn’t require chaining the requests in a single statement.

supporting both might be odd, .get() could return a response that could proxy another .get() through to the same connection object, to implement chaining, but i think it's better if people more familiar with the code weigh in.

[ixti]

I'm totally opposed to idea of proxy objects. IMHO it's confusing magic. Why one would need to call get() without getting response? I imagine API to be something like this:

client = HTTP.persisted # => HTTP::Client::Persisted

users  = client.get("http://example.com/users.json")
groups = client.get("http://example.com/groups.json")

Persisted client will hold connection pool internally, so if one didn't flushed connection before making another one (above) it will create another connection. Once it will have more than one open (and fully flushed) connection to same host, it will close all superfluous ones. Also, if you will get different host - it will add another persisted connection to the pool. Persistence pool will be configurable (total size, per-host size, auto-flush upon next request).

Clarification on "if one didn't flushed connection before":

client = HTTP.persisted # => HTTP::Client::Persisted

# 0 open connections

puts client.get("http://example.com/users.json").to_s
puts client.get("http://example.com/users.json").to_s

# 1 open connection

resources = %w[users groups persmissions].map do |res|
  client.get "http://example.com/#{res}.json"
end

# 3 open connections

resources.each { |r| r.to_s }

# 1 open connection

client.get("http://foobar.com/users.json").to_s

# 2 open connections

[tarcieri]

My ideal API would be transparent... that each HTTP::Client would hold a pool of HTTP::Connection objects per origin (perhaps with a toggle to use a pool or not). I'd define an origin as a tuple of:

[(:http|:https), hostname, port]

i totally agree a proxy on a response back to the original connection would be confusing as hell :)

[sferik]

Resolved via #184.

[sferik]

Hmmm…trying to move this issue into the v0.8.0 milestone but I’m getting an error. Maybe it’s related to the denial of service GitHub was facing earlier today? Anyway, this feature should be released in v0.8.0.

[sferik]

@tarcieri @ixti Are you able to change the milestone to v0.8.0?

[tarcieri]

@sferik GitHub is giving me an error when I try to do that o_O

[sferik]

@tarcieri Me too. I thought it might be a permissions thing. Let’s remember to try again when GitHub isn’t under attack. I’d guess they’ve temporarily disabled some non-critical services like reassigning milestones.

[ixti]

I found the workaround to assign milestones :D via issues index not from particular issue...

[sferik]

@ixti 👍