Can we export the entire Request object from Network.Http.Client? about http-streams

[dmjio]

This problem could also be solved if we created a setHeaders method or exported the deleteHeader method from the http-common repo.

[dmjio]

After importing Network.Http.Internal I was able to access qHeaders

[istathar]

@dmjio Hey there. The library doesn't expose record constructors on purpose, but on the other hand you shouldn't need to use Network.Http.Internal directly either. I've bookmarked this, and will see about whether there would have been a better way to offer this; but you're right; Transfer-Encoding: is special and can't be mucked with by the user without breaking the invariants of HTTP. The main idea is to prevent people from shooting themselves in the foot, so it'd take some special plumbing to work around all the code that prevents you from breaking the internals.

Keep in touch.

AfC

[dmjio]

@afcowie,

Enforcing the spec is definitely a good thing. My biggest fear is that Amazon itself doesn't follow it. After reading through the dynamodb docs it seems like they do not. I've switched to pipes-http and the signature for the request has been calculated correctly on the first try and I'm able to send and receive json. Here is the difference between the two.

pipes-http:

"{\"ExclusiveStartTableName\":null,\"Limit\":null}" -- <-- body
Request {
  host                 = "dynamodb.us-east-1.amazonaws.com"
  port                 = 443
  secure               = True
  requestHeaders       = [("x-amz-date","20141106T185038Z"),("host","dynamodb.us-east-1.amazonaws.com"),("x-amz-target","DynamoDB_20120810.ListTables"),("connection","Keep-Alive"),("content-type","appli\
cation/x-amz-json-1.0"),("Authorization","AWS4-HMAC-SHA256 Credential=AKIAIEIPUFMLQGTUVCPA/20141106/us-east-1/dynamodb/aws4_request, SignedHeaders=connection;content-type;host;x-amz-date;x-amz-target, S\
ignature=ddc19d6ac55f0c3cd7aac4b53d61b02193461afa2eb92e1a2526985650596b40")]
  path                 = "/"
  queryString          = ""
  method               = "POST"
  proxy                = Nothing
  rawBody              = False
  redirectCount        = 10
  responseTimeout      = Just (-3425)
}

{"TableNames":[]}Right ()
()

http-streams

λ> test
"{\"ExclusiveStartTableName\":null,\"Limit\":null}" -- <-- body
POST / HTTP/1.1
Host: <default>
Authorization: AWS4-HMAC-SHA256 Credential=AKIAIEIPUFMLQGTUVCPA/20141106/us-east-1/dynamodb/aws4_request, SignedHeaders=connection;content-type;host;x-amz-date;x-amz-target, Signature=7cf876c8c1f5f17973\
28cf6b2752c958998fefcae1aaafd90a916cbc02409ee9
x-amz-target: DynamoDB_20120810.ListTables
Accept-Encoding: gzip
Transfer-Encoding: chunked
content-type: application/x-amz-json-1.0
host: dynamodb.us-east-1.amazonaws.com
connection: Keep-Alive
x-amz-date: 20141106T185117Z


Right (Object (fromList [("__type",String "com.amazon.coral.service#InvalidSignatureException"),("message",String "The request signature we calculated does not match the signature you provided. Check yo\
ur AWS Secret Access Key and signing method. Consult the service documentation for details.\n\nThe Canonical String for this request should have been\n'POST\n/\n\nconnection:Keep-Alive\ncontent-type:app\
lication/x-amz-json-1.0\nhost:dynamodb.us-east-1.amazonaws.com,dynamodb.us-east-1.amazonaws.com\nx-amz-date:20141106T185117Z\nx-amz-target:DynamoDB_20120810.ListTables\n\nconnection;content-type;host;x-\
amz-date;x-amz-target\nc51be18888ff3ddcd532508cf8dc1587913e0a3a75ef4ba861fa79fd3eee9c4a'\n\nThe String-to-Sign should have been\n'AWS4-HMAC-SHA256\n20141106T185117Z\n20141106/us-east-1/dynamodb/aws4_req\
uest\nfbd5d12a46abfefc52c4c61c472069316be9582bce26b7c35e32efa56637e6a5'\n")]))
λ>

From looking it seems that the only difference between the two is the Transfer-Encoding and Accept-Encoding ... I really want to use http-streams since I use snap and know io-streams as well. So hope I can figure it out soon.

Also, it seems like the host header is getting duplicated, but can't be sure.

[dmjio]

Looks like amazon doesn't like the Transfer-Encoding: chunked header (even though it supports http/1.1). When I issue a request with pipes-http that includes the Transfer-Encoding:chunked header / value it fails.

[istathar]

Oh, ok. I thought you said you were sorted. So the problem is that sendRequest is doing the heavy lifting to work out the transfer encoding. Is it even legal to send an HTTP/1.1 PUT or POST without specifying the encoding of the body?

Anyway, aounds like we need a sendRequestProbablyDoesWhatItShould and a sendRequestRawOnYouGoodLuckWithThat. I'm in the midst of a fairly major rewrite, but it's not yet clear to me what the right way to expose those would be. I hate making clean code ugly just to support corner cases which people only need because other people aren't following the spec. Oh well.

In the mean time, would exposing deleteHeader work for you? Presumably not; what you need is exactly the Request object you specify to be serialized and sent. Hm.

AfC