Comments (5)
Hey @nikohann , Great! Could you share the code example as soon as you have a chance to? Even if it's just the outline? I'm up against a deadline, and this would help me out a lot. Thanks.
There is couple serious bugs but I think you will find it out.
https://js.langchain.com/docs/expression_language/streaming#event-reference
I have used streamEvents with streaming output as json format from function calling.
async function submitUserMessage(content: string) {
'use server'
const aiState = getMutableAIState<typeof AI>()
aiState.update({
...aiState.get(),
messages: [
...aiState.get().messages,
{
id: nanoid(),
role: 'user',
content
}
]
})
// Langchain
const prompt = ChatPromptTemplate.fromMessages([
[
"system",
"You are helpful assistant. Be positive and speak about unicorns."
],
["human", "{input}"],
]);
const llm = new ChatOpenAI({
modelName: "gpt-4-0125-preview",
streaming: true,
temperature: 0.4,
});
const chain = prompt.pipe(llm);
let textStream: undefined | ReturnType<typeof createStreamableValue<string>>
let textNode: undefined | React.ReactNode
runAsyncFnWithoutBlocking(async () => {
if (!textStream) {
textStream = createStreamableValue('')
textNode = <BotMessage content={textStream.value} />
}
const response = chain.streamEvents({
input: content,
}, { version: "v1" })
for await (const event of response) {
const eventType = event.event;
if (eventType === "on_chain_stream") {
textStream.update(event.data.chunk.content);
} else if (eventType === "on_llm_end") {
const message = event.data.output.generations[0][0].text;
textStream.done();
aiState.done({
...aiState.get(),
messages: [
...aiState.get().messages,
{
id: nanoid(),
role: 'assistant',
content: message
}
]
})
}
}
})
return {
id: nanoid(),
display: textNode
}
}
from ai-chatbot.
I have used the submitUserMessage method with langchain's streamEvents. I will maybe provide code example later.
from ai-chatbot.
Hey @nikohann ,
Great! Could you share the code example as soon as you have a chance to? Even if it's just the outline? I'm up against a deadline, and this would help me out a lot.
Thanks.
from ai-chatbot.
Hey @nikohann , Great! Could you share the code example as soon as you have a chance to? Even if it's just the outline? I'm up against a deadline, and this would help me out a lot. Thanks.
There is couple serious bugs but I think you will find it out.
https://js.langchain.com/docs/expression_language/streaming#event-reference
I have used streamEvents with streaming output as json format from function calling.
async function submitUserMessage(content: string) { 'use server' const aiState = getMutableAIState<typeof AI>() aiState.update({ ...aiState.get(), messages: [ ...aiState.get().messages, { id: nanoid(), role: 'user', content } ] }) // Langchain const prompt = ChatPromptTemplate.fromMessages([ [ "system", "You are helpful assistant. Be positive and speak about unicorns." ], ["human", "{input}"], ]); const llm = new ChatOpenAI({ modelName: "gpt-4-0125-preview", streaming: true, temperature: 0.4, }); const chain = prompt.pipe(llm); let textStream: undefined | ReturnType<typeof createStreamableValue<string>> let textNode: undefined | React.ReactNode runAsyncFnWithoutBlocking(async () => { if (!textStream) { textStream = createStreamableValue('') textNode = <BotMessage content={textStream.value} /> } const response = chain.streamEvents({ input: content, }, { version: "v1" }) for await (const event of response) { const eventType = event.event; if (eventType === "on_chain_stream") { textStream.update(event.data.chunk.content); } else if (eventType === "on_llm_end") { const message = event.data.output.generations[0][0].text; textStream.done(); aiState.done({ ...aiState.get(), messages: [ ...aiState.get().messages, { id: nanoid(), role: 'assistant', content: message } ] }) } } }) return { id: nanoid(), display: textNode } }
Yea I think I'm running into some real strange bugs. This works totally fine when running locally but as soon as I push it to production it stops working. For some reason production doesn't seem to be streaming the results...
Not sure what's going on
from ai-chatbot.
@AmmarByFar @nikohann Hi guys, have you figured out some stable solution ? Thank you
from ai-chatbot.
Related Issues (20)
- streamUI vs render 😢 HOT 1
- Type error in type checking step in vercel deployment HOT 1
- No response after 6 rounds of dialogues HOT 1
- huggingface support HOT 1
- Error: .update(): Value stream is already closed. HOT 4
- server problem (500) HOT 9
- The theme flashes when switching between modes
- tools : File_search HOT 1
- streamUI & Call Tools in Parallel
- Cannot render UI HOT 1
- Feature request for means to store props to inject into system prompt. HOT 2
- How to handle multistep toll call with streamUI? HOT 6
- When building and using Redis on-premise HOT 1
- Google OAuth HOT 1
- Chat messages disappearing when clicked on New Chat button and coming back to last chat, chats showing after page refresh HOT 3
- All non OpenAI providers giving errors like Type 'GoogleGenerativeAILanguageModel' is not assignable to type 'LanguageModelV1'. HOT 2
- Vercel.json
- When using tools call - double divider is rendered
- Is this project likely to continue? HOT 1
- I got a blank result HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from ai-chatbot.