Inconsistent binary packet ordering about socketioxide

[kelnos]

I'm running into this too (I think?), using socketioxide 0.8 (I'm using warp 0.3), acting as a server for the openHAB cloud connector (source).

The cloud connector responds to proxy requests from the socketio server by sending a responseHeader event, followed by one or more responseContentBinary events, terminated with a responseFinished event. When making a request, I usually see ordering (from the socketioxide server's perspective) like:

responseHeader
responseContentBinary
responseContentBinary
responseContentBinary
responseFinished
responseContentBinary

I'm not sure this is exactly the same problem: I don't think ACKs are involved at all (none of the events sent require ACKs), and it's the binary events themselves that are received by my application that are out of order, not the ACKs.

If I tell tokio to use the current_thread runtime, the problem goes away.

[Totodore]

The main problem comes from here:

socketioxide/engineioxide/src/socket.rs

Line 198 in 3ab5cd7

internal_tx: mpsc::Sender<Packet>,

. At insertion into the channel, when there is adjacent binary payload that it is needed, an atomic ordering is required.

My two main options for this are:

• Putting a Mutex here but it would be quite bad because most of the time it is single message packet that is sent and therefore it doesn't require any locking / ordering.
• Modifying the channel so that it takes a Vec<Packet> rather than a Packet. Adjacent binary packets would be grouped in each Vec. But most of the Vec<Packet> would be of size 1 and it would cause many new Vec alloc.

[kelnos]

Hm, maybe that's not the same problem I'm seeing then. If I understand this bit of code correctly, that's for packets that are being sent out from the socketioxide server to the client. For me, I'm seeing messages in the other direction that are out of order. Which, now that I read through the engineioxide/socketioxide code some more, I don't quite understand how that could be happening, as it looks like the event is handled and sent up to the application on the same thread as it's received on from the websocket, without any queuing. So maybe the fault is in my code...