High memory usage / large websocket buffer per client remains in memory after message is sent about actix-web

[asonix]

I've narrowed the problem down to two buffers:

• write_buf in actix-http's h1 dispatcher
• buf in actix-web-actors' WebsocketContextFut

These buffers each use BytesMut, which is designed to keep its capacity around as long as possible to optimize for bytes moving in and out.

Unfortunately, this means that once a BytesMut grows to fit a set of continuous bytes, it won't shrink back down. In order to send a message over a websocket, these two buffers are filled with the entire contents of the file, the first after passing through the websocket encoder, and the second after passing through the http encoder.

I verified that if I std::mem::take these buffers rather than write_buf.clear() and buf.split() them, memory use returns to basically nothing after the large file is sent.

Since the default behavior is to reuse existing allocations, i'm hesitant to actually mem::take them as a fix here.

It does seem somewhat unfortunate that these buffers are needed at all when it seems that, at least for the http encoder, the actual message contents are not modified at all, and are simply copied verbatim. The ws encoder sometimes copies verbatim although sometimes there is a 4 byte mask applied at the start.

I don't know if I'd be confident enough to try to eliminate these extra buffers myself, though.

I'd be interested to hear @robjtede 's thoughts on potential solutions here