Can we stream the response? about anthropic-tools

[nmarwell-anthropic]

Hi @Justus-M , thanks for the feedback. A few notes for you below!

1. Most of what you are hoping to do can be accomplished by setting execution_mode='manual' when invoking tool_user.use_tools(), which will stop execution as soon as a single set of function calls or a message is returned. You can see more about it here:

   anthropic-tools/README.md

   Line 81 in a782267

   You can then make use of your ToolUser by calling its `use_tools()` method and passing in your desired prompt. Setting execution mode to "automatic" makes it execute the function; in the default "manual" mode it returns the function arguments back to the client to be executed there.

   .

2. Re streaming, we know this is a blind spot and hope to address it when this package moves out of beta and likely to our API. If you would like to submit a PR to this repo to support it before then, we would welcome it and you are welcome to do so!

Let me know if you have any follow up questions.
Nick

[Justus-M]

Hi Nick, thanks for your quick response

When I said "generating a function call" I meant claude generating the function arguments, not my system actually executing the function. I am already using the manual option, and the execution tends to be much faster than the function input/text generation anyway.

Unfortunately manual doesn't make a difference in this case.

Regardless, I would argue that it doesn't get people "most" of the way there, as the (non-function call) messages not being streamed is arguably a bigger issue since this is something users can actually get value from instead of waiting, since they can read it. With this library, users will be sitting there wasting their time and being frustrated (especially for long messages) and then reading a long message all at once. It's hard to overstate how important streaming messages are for any applications that display these messages to the user, especially given that this is what people are already used to with chatgpt and claude.ai

Glad to know this is on your radar. I've already implemented an initial version using the raw XML and will consider using this repo when streaming is supported.