When using chunked responses carrier returns only the first chunked response about carrier

[nightshade427]

Its almost as if when calling the parser "(funcall parser data)" returned by "fast-http:make-parser" that it expects all data for all chunks to already be in the stream, which i dont think they arent as its streamed in pieces with cl-async?

It looks like "parse-chunked-body" in fast-http reads the chunk size, then the body, then tries to read the next chunk size right away, which I dont think is there yet from cl-async?

Maybe carrier has to read all the chunked responses first, then call " (funcall parser data)"?

[nightshade427]

It looks like maybe fast-http is calling the "message-complete" callback when done reading each chunk.

So what happens is it fast-http reads the first chunk it was sent by carrier, then fast-http calls "message-complete" after it finishes reading the first chunk, which calls carrier's "our-finish-callback" which finishes the request. Maybe?

[orthecreedence]

Yeah, maybe something weird is going on with fast-http. I'll run a few tests...do you have a simple test case you're been using (like a URL you're requesting)? If not I'll try to reproduce via wookie.

[nightshade427]

No sadly, the only uri I have that is even does chunked (and only sometimes) is our authentication system. I googled around to see if I could find a public one that used chunking and couldn't find one.

Let me know what you find out or if I can be of any help ;)

[orthecreedence]

I made a simple chunked test in wookie, and i'm able to reproduce. Looks like a fast-http bug.

[orthecreedence]

Indeed, I wrote a simple test case and it appears to be a fast-http bug: fukamachi/fast-http#10.

[orthecreedence]

Ok, this should be fixed in the latest fast-http, be sure to do a pull there. Let me know if you have problems. Leaving open until I hear back!

[nightshade427]

Thanks again for all the help.

Im getting this error now, here is the code:

(as:with-event-loop ()                                                                                                                                                   
     (asf:alet ((mine (carrier:request "http://localhost:8888/test/test_es.docx" :return-body t)))                                                                          
        (format t "~%mine: ~s~&" mine)))

Expectation failed: NIL                                                                                                                                                           
   [Condition of type FAST-HTTP.PARSER::EXPECT-FAILED]                                                                                                                            

Restarts:                                                                                                                                                                         
 0: [RETRY] Retry SLIME REPL evaluation request.                                                                                                                                  
 1: [*ABORT] Return to SLIME's top level.                                                                                                                                         
 2: [ABORT] Abort thread (#<THREAD "new-repl-thread" RUNNING {1008A85003}>)                                                                                                       

Backtrace:                                                                                                                                                                        
  0: (FAST-HTTP.PARSER::PARSE-CHUNKED-BODY #<unavailable argument> #<unavailable argument> #<unavailable argument> #<unavailable argument> #<unavailable argument>)               
  1: (FAST-HTTP.PARSER:PARSE-RESPONSE #<unavailable argument> #<unavailable argument> #<unavailable argument> :START #<unavailable argument> :END #<unavailable argument>)

But, this works

(as:with-event-loop ()                                                                                                                                                   
     (asf:alet ((mine (carrier:request "http://localhost:8888/test/test_es.txt" :return-body t)))                                                                          
        (format t "~%mine: ~s~&" mine)))

Both appear to be chunked.

[fukamachi]

It may be fixed at fukamachi/fast-http#11. Could you try it again with the latest fast-http?

[nightshade427]

yep that fixed it ;)

[orthecreedence]

👍