have ability to ask follow-up questions that remember context of previous result about llm-search

[snexus]

Thanks for the suggestion.

In the chat interaction, the limited context window of the model is filled with previous fragments of conversations, creating the illusion that the chatbot has a memory. In reality, you need to send the entire conversation back and forth.

RAG is different from regular chat. All the context window memory of the model is used to fill it with document extracts to ground the answer, rather than relying on previous conversations.

I am afraid it is not feasible with the current context window lengths, unless you are using expensive, extremely long context window models. While it will remember some of the previous requests, it will have less factual data to provide the answer.

It might become possible in the future as the context length of the models increases.

[Hisma]

thank you for the response. Not much we can do about this at this time then. It's not even a big deal. It's a nice to have, but not really something that can't be mitigated with prompting.
I'll close this for now.