Seeing a lot of crashes mid thread about slickgpt

[xmoiduts]

Also ran into this issue recently.

[Shackless]

Okay, so when you prompt something, the client calls an Vercel Edge function (middleware) and that will establish the connection to the OpenAI API and "proxy" the stream to the client. If that streams fails for some reason, SlickGPT deletes the latest (incomplete) completion from the history, so that you can re-submit your last prompt easily.
So basically, there are 3 points of failure:

• OpenAI API is down or cuts the connection to Vercel (most likely, happens all the time, especially whenever they update stuff)
• Vercel has problems/downtime and disconnects the client (rather unlikely but not impossible)
• Your internet connection broke

I don't think we can do much on SlickGPT/client side except maybe output some better error messages.

[xmoiduts]

It may not delete the unfinished API response but stuck somewhere. From my usage experience, the disappeared response will show eventually, but may not in the identical session, maybe in another slickgpt chat session. Once the popped up message is so sudden that the entire session context jumped to the other session while I was typing. (hopes it is not some session-switching shortcuts)

[Shackless]

We are about to release a new major release that will change things:

• We'll be offering fixed-price access to the OpenAI API using our Azure infrastructure. It's a lot faster than direct access (probably because Microsoft has special deals with OpenAI?). I haven't seen the problem there yet.
• The Svelte endpoint layer will be completely removed. So even if you don't use that, there's one less point of failure because the client will be talking directly to OpenAI. Maybe that will help as well.

Stay tuned, it's coming soon.