Handling submitting tool output back to the agent about superagent

[homanp]

Yeah! I also want to see this in SuperAgent, execute actions is amazing but information should flow back to prompt to be use. If i'm not confuse, it's possible using workflows, but i think should be possible just defining tool and return value from there ( in terms of javascript/typescript )

I agree with you. I will add a way for this.

[joaomirandas]

Yeah! I also want to see this in SuperAgent, execute actions is amazing but information should flow back to prompt to be use. If i'm not confuse, it's possible using workflows, but i think should be possible just defining tool and return value from there ( in terms of javascript/typescript )

[lynchian]

For a direct example of implementation, this is how OpenAI Assistants do it :

https://github.com/openai/openai-python/blob/e41abf7b7dbc1e744d167f748e55d4dedfc0dca7/src/openai/resources/beta/threads/runs/runs.py#L314

[homanp]

@elisalimli let's implement this.

[joaomirandas]

I was wondering about how to do it and i reach 2 thoughts:

1. The way of "Custom Tools" works nowdays it's great, and i don't think that should be removed. It's very usefull in terms of API first service, I didn't see this kind of implementation allowing developer to use "tools" on your own code anywhere else. That's good and should not be removed;

2. We should implement additional something like "Actions" ( just giving another cool name LoL ) which actually is an openAPI spec ( similar how Dify does ) and this will capable developers using Superagent to call endpoints specified in that OpenAPI Specs and receive external data. I think about this because that operation could run inside *Superagent backend, this will enable to use information received from external api as data inside methods and use that data to generate an rich output;

Tips about it:
I already tested dify implementation and it's quite good, but for some pieces sounds like beginner yet, we could have here some like some pattern to receive response from external API's and i also suggest to always require an field with "message" to give generative context about how that request perform. ( eg. if request about get wether succeed should return in response.message "weather acquisition is ok and data is available" - i miss that kind of response context in dify implementation and that make output generation not good enouth. )

Check that part of Dify, it's most similar that i could find.

What did you think about this approach @homanp ? and you @lynchian ?

[homanp]

I was wondering about how to do it and i reach 2 thoughts:

1. The way of "Custom Tools" works nowdays it's great, and i don't think that should be removed. It's very usefull in terms of API first service, I didn't see this kind of implementation allowing developer to use "tools" on your own code anywhere else. That's good and should not be removed;
2. We should implement additional something like "Actions" ( just giving another cool name LoL ) which actually is an openAPI spec ( similar how Dify does ) and this will capable developers using Superagent to call endpoints specified in that OpenAPI Specs and receive external data. I think about this because that operation could run inside *Superagent backend, this will enable to use information received from external api as data inside methods and use that data to generate an rich output;

Tips about it: I already tested dify implementation and it's quite good, but for some pieces sounds like beginner yet, we could have here some like some pattern to receive response from external API's and i also suggest to always require an field with "message" to give generative context about how that request perform. ( eg. if request about get wether succeed should return in response.message "weather acquisition is ok and data is available" - i miss that kind of response context in dify implementation and that make output generation not good enouth. )

Check that part of Dify, it's most similar that i could find.

What did you think about this approach @homanp ? and you @lynchian ?

So you wan to add openapi spec tool calling? Sure we can do that. Would you be willing to make a contribution here? Also that wouldn't really solve the problem for this usecase. But I have a solution for that.

[elisalimli]

@lynchian @joaomirandas

We don't think this would really work with workflows that has multiple agents.

Let's say a workflow has:

• Agent 1
• Agent 2 (this agent has a custom function calling)
• Agent 3

If we run this workflow, and agent 2 decides to call the given custom function, what we will do here? We can not move on to the next agent or stop responding. Because if we stop, we will lost the agent 1's response.

Nevertheless, we do want to support this feature. But for /agents endpoint only.

@lynchian @joaomirandas What are your takes on this?