Failed to deserialized about async-openai

[mhuang74]

I ran into someting similar, and adding retries (with exponential backoff) solved my problems.

[mhuang74]

I think openai sometimes respond with empty results that break parsing. Hence retry helps.

…

On Sun, Mar 26, 2023, 7:48 AM Himanshu Neema ***@***.***> wrote: That's strange, the library doesnt retry deserialization errors, one example: https://github.com/64bit/async-openai/blob/main/async-openai/src/client.rs#L288-L290 — Reply to this email directly, view it on GitHub <#61 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACJ6WZ7OBEV6B5R5PZ56CXDW5575JANCNFSM6AAAAAAWEQ43DA> . You are receiving this because you commented.Message ID: ***@***.***>

[64bit]

Hi @federico123579,

It seems that OpenAI API responding with no value where Rust deserializtion was expecting it for one the response types in types.rs. So we might have to change response type to accept Option<T> instead.

Can you provide info about which API call is it?

[federico123579]

Sure, it happened multiple times with a CreateChatCompletionRequest (request_args) sent on client.chat().create(request_args).await.

[64bit]

Thank you, it seems CreateChatCompletionResponse is correctly reflecting openapi spec.
However, looking at the response example here https://platform.openai.com/docs/api-reference/chat/create suggest that model field could be optional.

Also, Chat API is currently in beta, so hard to find a solution here.

What I'm thinking is to update OpenAIError::JSONDeserialize to include actual payload returned by API to make it easy to surface invalid response.

[64bit]

As a first step, I have added tracing::error!( ... ) calls to log actual payload when derserialization fails, that way we get more visibility into what OpenAI is responding with. #62

Please give v0.10.1 a try, you might have to setup the tracing_subscriber to log the errors.

[64bit]

That's strange, the library doesnt retry after deserialization errors, one example:
https://github.com/64bit/async-openai/blob/main/async-openai/src/client.rs#L288-L290

[64bit]

Closing this, hopefully no longer an issue. Please reopen with additional info with debug logs if needed.