[Potential new transport] Streamed xSSE Meek (HTTP/1.1) about xray-core

[PoneyClairDeLune]

Here are my own tests with stdout redirected to /dev/null.

When run as a pure TCP relay of a web server on a mediocre server, Eclipsed Meek can reach a download speed of around 32 Mbps. When run as a TCP relay proxying a VLESS over WebSocket connection, Eclipsed Meek can reach around 22 Mbps in download, 10 Mbps in upload.

[PoneyClairDeLune]

As a proof of concept, I believe Eclipsed Meek is enough to demonstrate the power of Meek. However, the current implementation can be made more efficient.

In terms of console logs, the current Meek still emits too many, which sometimes may overwhelm SSH sessions. JavaScript has its own limits as a non-compiled language, and SSE by itself isn't designed for transmission of large data chunks, as in the Eclipsed tunnel I had to transmit data with OVM43 codec (a faster but incompatible version of Base64, inspired by KORG 7 over 8 codec). Come think of it, the requirement of a codec might be both a blessing rather than a curse.

I'm working on another streamed Meek implementation in the meantime called Ditzy, built on top of binary messages rather than human-readable messages, but that may not come to fruition any time soon. I hope the gurus in this community can provide insights for the current Eclipsed Meek. And who knows, maybe Eclipsed might become a transport in Xray!

[PoneyClairDeLune]

The current Eclipsed implementation cannot penetrate CDNs buffering request bodies, where the client and server would simply time out. Eclipsed itself only works in a streamed manner.

Maybe Ditzy (the next one with binary messages) will fare better with a streaming and buffering combo.

[mmmray]

The current Eclipsed implementation cannot penetrate CDNs buffering request bodies

The selling point of existing Meek implementations so far has been that it can pass arbitrary HTTP "middleboxes", not only CDNs but also CGI servers. If it can't be deployed there, then there is nothing the existing httpupgrade and websocket transports cannot do already.

Do you have a specific CDN in mind that does not buffer request bodies, and also does not support websockets? For this kind of constraint, your Meek implementation could be used.

[RPRX]

@PoneyClairDeLune #2162 (comment)

下行流式上行分包（多次请求）

[PoneyClairDeLune]

Do you have a specific CDN in mind that does not buffer request bodies, and also does not support websockets?

@mmmray No sadly. It just came to my mind when I was implementing SSE from scratch, wondering what would happen if SSE occurred in reverse. It just so happened that it had worked so well even in its current form, I entirely forgot CDNs don't allow streamed request bodies. :(

The problems with Eclipsed don't stop here. In Eclipsed's current form, connections may sometimes get overwhelmed due to bad design, and at least two unidirectional connections have to be established for each duplex connection, resulting in an absurd connection count between the client and the server.

But Eclipsed definitely shattered the myth of Meek being doomed to be slow!

Downstream streaming upstream subcontracting (multiple requests)

@RPRX Actually I had (subjectively) better ideas. The current Eclipsed experiment certainly served as a rewarding experience regarding unidirectional streaming, and while it's my first attempt at implementing Meek, it isn't my first design.

Since Meek has everything regarding the states of connections, I suspected that a full state decouple from underlying transports would be possible, so I designed the binary message format for a draft called Ditzy. If implemented well, Ditzy would work both in a bundled manner and a streamed manner whenever it finds itself capable, and the state of reconstructed connections would be fully decoupled from the underlying HTTP requests altogether, resulting in a much more efficient use of HTTP requests; multiplexing, if you will. You might be able to use it on DNS too.

But right now Ditzy is just a draft, all those advantages currently are just pipe dreams. After I get the binary messages working for request streams, I'll work on bundled requests (one request/response bundling multiple messages at once), possibly also biased stream multiplexing.