davidfowl / bedrockframework Goto Github PK

@davidfowl the project has an editorconfig https://github.com/davidfowl/BedrockFramework/blob/master/.editorconfig , I think should be used to enforce some rules.

e.g. this is mine that enforce a specific use of braces, spaces, formatting and for example force you to explicit ConfigureAwait any time you await (for this a couple of analyzers are needed).

I don't mean to use my specifications, but I find this very useful to have a common style that's not dependant on the environment (VS settings) but rely on the editorconfig.

Now that different people try to contribute I think this is quite important and being you the project owner I'd say it's up to you to decide the style to use.

How should I handle this internal exception which is thrown by Client.ConnectAsync? I received this exception while trying to connect to a host which was temporarily down and connection timed out.

This is the entire stacktrace, nothing was removed. I have a handler for unobserved task exceptions which only logs the message (as below).

00:36:51.305 ERROR  [ThrId=02] Unobserved task exception caught in the main AppDomain. System.Net.Internals.SocketExceptionFactory+ExtendedSocketException (60): Operation timed out [::ffff:172.16.x.y]:15019
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw(Exception source)
   at System.Net.Sockets.Socket.EndConnect(IAsyncResult asyncResult)
   at System.Net.Sockets.Socket.<>c.<ConnectAsync>b__274_0(IAsyncResult iar)
--- End of stack trace from previous location where exception was thrown ---
   at Bedrock.Framework.SocketConnection.StartAsync()
   at Bedrock.Framework.Client.ConnectAsync(EndPoint endpoint, CancellationToken cancellationToken)

I am using ServerBuilder UseConnectionHandler to build my Server class. I would like to resolve connection handler class from an external container (Castle Windsor) in order to be able to inject dependencies in my ConnectionHandler. I cannot find a way in the library' ServerBuilder API to resolve an instance of ConnectionHandler externally at client connect time

Is it feasible to use a factory method for resolving connection handlers?

when I DisposeAsync the client connection I see this in the logs:

Object name: 'System.Net.Sockets.Socket'.
at System.Net.Sockets.Socket.Shutdown(SocketShutdown how)
at Bedrock.Framework.SocketConnection.DoSend() in C:\Users\Jan\Source\Repos\MQTTnet\BedrockTransports\Bedrock.Framework\Transports\Sockets\SocketConnection.cs:line 215
at Bedrock.Framework.SocketConnection.ExecuteAsync() in C:\Users\Jan\Source\Repos\MQTTnet\BedrockTransports\Bedrock.Framework\Transports\Sockets\SocketConnection.cs:line 91

Am I supposed to calling something else before disposeasync or is this a bug?

I'd like to know if it's out of scope to have a WEB Api implementation on top of this library, that uses the same concept of mvc to discover Controllers and such so that we can have our custom web server to offer REST API for example.

This comes because of a need I have: I've a project composed of several class libraries (dotnet core 3.1) which are all part of a bigger solution.

It's basically a P2P network where I already use this library to offer listeners and to connect to other peers and it's modular, so I register a component and it will read configuration and do whatever it needs to do. What I want now is a component that exposes some Controllers (using swagger maybe) but I don't want to have a single endpoint that's offered by the core feature because I want to isolate each component and serve on a specific endpoint (that doesn't have to share anything between other components except the DI container that's common).

I tried to configure kestrel on top of bedrock but it's not working properly (may be because of some mistake I did or because Kestrel doesn't support this scenario)

Maybe it is possible to configure kestrel to have different endpoint (so listen to different endpoints) and have a different set of Controller exposed on each different endpoint, in any case I'd like to know if this library scope would cover such kind of scenarios.

Plan B is to wire a single kestrel and use a MiddleWare to expose controllers based on the endpoint, but I'd like more to be able to kick in kestrel servers at will so who wants to implements its own implementation doesn't have to deal with a pre-coocked API feature my software have.

Sorry for the textwall, I realize I'm going quite OT probably, in case just close the issue.

Http1RequestMessageWriter doesn't set the host header when it isn't set as a header. The host header is required, and in HttpClient, we add it directly if it isn't present: https://github.com/dotnet/runtime/blob/4f9ae42d861fcb4be2fcd5d3d55d5f227d30e723/src/libraries/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/HttpConnection.cs#L256.

The URI for the host comes from the connection pool in HttpClient. We either require the host header to be set or pass it into the constructor of the Http1RequestMessageWriter.

Not sure how this snuck in but it shouldn't be there:

@Drawaes As per your suggestion I replaced Pipelines.Sockets.Unofficial with Bedrock for client side connections. Also I cached the instance of ClientBuilder for re-use. However it appears that the Build method is not threadsafe.
If
ClientBuilder clientBuilder = new ClientBuilder(new ServiceCollection().BuildServiceProvider()).UseSockets();

and I execute :

await clientBuilder.Build().ConnectAsync(new IPEndPoint(IPAddress.Loopback, Port), default);

from 20 threads, I have the following often :

System.ArgumentException : Destination array was not long enough. Check the destination index, length, and the array's lower bounds. (Parameter 'destinationArray') Stack Trace: Array.Copy(Array sourceArray, Int32 sourceIndex, Array destinationArray, Int32 destinationIndex, Int32 length, Boolean reliable) List'1.CopyTo(T[] array, Int32 arrayIndex) EnumerableHelpers.ToArray[T](IEnumerable'1 source, Int32& length) Buffer'1.ctor(IEnumerable'1 source) ReverseIterator'1.MoveNext() ConnectionBuilder.Build() ClientBuilder.Build() line 55

If on the other hand I use :

clientBuilder.Build().ConnectAsync(new IPEndPoint(IPAddress.Loopback, Port), default).Result;
no exception.
Am I missing something ?

We'll need an API docs for this project and docfx is the new hotness. We should figure out how to set that up.

The socket transport on the server side is the one used in Kestrel by ASP.NET Core. It's optimized to not allocate and has a memory pool that uses pre-pinned buffers to avoid heap fragmentation. Unfortunately dotnet/aspnetcore#15019 is going to be implemented in .NET 5 but it would be great to have a solution in the mean time that works on 3.1. We should use the same implementation here and have it in the box by default (sockets is by far the most common implementation).

Some options:

1. Copy all of the code for the sockets transport and use it for both the server and the client
2. Copy all of the code for the sockets transport and use it on the client side only (keep the server implementation on the server side).
3. Use private reflection to call into the sockets transport and use it on the client side.
4. Write a new sockets transport and use it for both client and server.

Currently leaning towards 1.

Hi,

I wanted to know if there are any plans to support a UDP transport in this framework?

Regards

Hello,

We started dropping bedrock into our server application and have been getting this:

[20:53:31 ERR] AuthWorker Error processing queue System.AggregateException: One or more errors occurred. ( The WriteAsync method cannot be called when another write operation is pending.) ---> System.NotSupportedException: The WriteAsync method cannot be called when another write operation is pending. at System.Net.Security.SslStream.WriteAsyncInternal[TWriteAdapter](TWriteAdapter writeAdapter, ReadOnlyMemory1 buffer) at System.IO.Pipelines.StreamPipeWriter.FlushAsyncInternal(CancellationToken cancellationToken) --- End of inner exception stack trace --- at System.Threading.Tasks.Task.Wait(Int32 millisecondsTimeout, CancellationToken cancellationToken) at System.Threading.Tasks.Task.Wait() at ArchonPolicyServer.Server.Connections.BedrockConnection.SendAsync(Byte[] msg)

Is there anything we may be doing wrong that may allow us to work around this issue?

We're using https://github.com/redhat-developer/kestrel-linux-transport along with it as the server runs on .NET Core 3 in linux.

Appreciate any insights you may have.

Thanks!

ReadRequestAsync, WriteResponseAsync & SendAsync methods in the protocol classes do not accept CancellationTokens. Is that by design? If so, what is the rationale?

Some protocols take a large stream of bytes and introduce framing to produce a smaller stream of bytes. In some cases those frames don't matter and we want to read them as another stream of bytes. An example of this are chunked response encoding in HTTP/1.1. Each incoming frame is wrapped in a length prefix (in hex) and a new line. Most frameworks ignore this framing and present the user with a stream of bytes.

To make this pattern easier to implement, we should make it possible to take a PipeReader and an IMessageReader<ReadOnlySequence<byte>> and return a new PipeReader that implements that generic de-framing and buffering.

The current prototype is https://github.com/davidfowl/BedrockFramework/compare/davidfowl/pipe-reader

Hi,
I am making use of Bedrock or at least some components of Bedrock in building a small framework for multiprotocol Kestrel based services.
Briefly the design is as follows :
When an app based on the framework is launched it starts up a (parent) Kestrel service bound to a socket.
In addition a pool of AssemblyLoadContexts are spun up.
Each of these ALCs hosts an instance of a (child) Kestrel.
Each child Kestrel however is bound to a (Bedrock) MemoryEndPoint not a socket.
The parent Kestrel is enhanced with a connectionhandler that dispatches each incoming ConnectionContext to an available child Kestrel from the pool.

All of this works and works very well ….. WITH a small modifcation.
Kestrel will only bind to an IPEndPoint not an EndPoint. I therefore modified Bedrock.Framework/Transports/Memory/MemoryEndPoint.cs
to inherit from IPEndPoint and implemented the method Create(SocketAddress) with a no-op.
That’s essentially the design and it meets the requirements.

Further I have also investigated binding the parent Kestrel to a MemoryEndPoint instead of a socket. This was accomplished by adding IConnectionListnerFactory & IConnectionFactory to the service provider.
This ability would certainly be quite useful for testing. However when the ConnectionContext (born in the MemoryTransport) is sent down the kestrel pipeline a NullReferenceException is thrown . It transpires that the feature IMemoryPoolFeature is required and missing. When this feature is set on the ConnectionContext everything appears to work.

What doesn't work however is if you want to simultaneously bind your parent kestrel to a socket AND a MemoryEndPoint. This would again be a very useful feature.

Can it be envisaged that some of these quite modest modifications be added to Bedrock ? Currently I am taking the source files from Bedrock but would prefer to take them and leave them unmodified.

I have spoken with @Drawaes and he told me MemoryEndPoint was never designed to be used with Kestrel. But it could be :-)

Thanks
N

when running my benchmark a large chunk of the cpu is in Socket.SendAsync is this expected?

Im not sure if the vs profiler is accurate.

In protocol handling, I frequently find myself having to deal with potentially recursive payload structures; I'd like to be able to do this in a zero-alloc way.

to set the scene, something like:

    readonly struct SomeFrame
    {
        public readonly int Whatever;
        public readonly ReadOnlySequence<byte> BinaryPayload;
        // sub-elements
        public readonly ReadOnlyMemory<SomeFrame> Children;
    }

This is simple for a class SomeFrame, but gets complex for struct SomeFrame because the runtime / type-loader gets fussy about the size of the element. To be explicit, you can't have a struct Foo that contains SomeGenericStruct<Foo> because it is worried that SomeGenericStruct<T> could (now or at a later date) contain a field T, making it a recursive struct aka "not good".

(assume for this discussion that I can actually construct the tree allocations themselves; that's not a factor here)

The only mechanism in the existing code that works reliably for this is a SomeFrame[] (plus offset/length as needed) - the addition of a ref-type acts as a c-c-c-combo-breaker. It would be nice if this could be abstracted to ReadOnlyMemory<SomeFrame> or ReadOnlySequence<SomeFrame>

I have some mechanisms for doing this, but it gets a little ... messy. In particular, in "Pipelines.Sockets.Unofficial", I have a non-generic Sequence that works a lot like Sequence<T>, and allows type-less storage and retrieval essentially of the start/end position payloads via a simple cast:

private readonly Sequence _items;
internal int ItemsCount => (int)_items.Length;
internal Sequence<RawResult> GetItems() => _items.Cast<RawResult>();

I guess my question here is: do you see this as an interesting scenario for Bedrock, and if so: how should it be addressed? I hacked it by having my own sequence type, but the above isn't supported on ReadOnlySequence<T>, and neither Memory<T> nor ReadOnlySequence<T> have a suitable "give me the data in an untyped form, that I can reconstruct back from later".

In particular, the SequencePosition start/end-based constructor on ReadOnlySequence<T> is not exposed, so we can't just store the .Start and .End as SequencePosition and rehydrate from there.

Thoughts? Or is this just not an interesting scenario in general?

(for immediate context; this came up while trying to write a RESP protocol handler based on Bedrock where I'm essentially building a map of the received payload as lookups into the "live" pipe data, i.e. before we call Advance)

HTTP/2 support will take significantly more effort to implement and prototype than HTTP/1. There is some experimental work that has been copied from Kestrel that we can start with, but figuring out the model for reading and writing messages will be tricky.
Here is the current state of the world:

• Http2Protocol.cs a shared place for Http2 protocol concerns between the client and server
• Http2ClientProtocol.cs Http2 client protocol concerns.
• Http2ServerProtocol.cs Http2 server protocol concerns.

The expected usage will usually come from creating either an HttpClientProtocol or HttpServerProtocol, where it will seemlessly use Http2 if it can or listen for Http2 if configured.

To move this out of experimental, a lot of work needs to be done. Fortunately, both Kestrel and HttpClient have working implementations, so we can have a verification matrix between both implementations to make sure our implementation works.

• Implement Http2ServerProtocol and verify against HttpClient
• Implement Http2ClientProtocol and verify against Kestrel
• Implement both a Http2Connection and Http2Stream type which are shared between client and server. These may end up being part of the Http2Protocol types
• Make upgrade/downgrade work with Http2 in HttpClientProtocol
• Allow HttpServerProtocol to work with Http2.

I have stumbled across an issue when trying to implement the IMessageReader interface in F#.
This is not so much an issue with the BedrockFramework as it is the FSharp compiler but I thought I might make it known here as well.

The FSharp compiler seems to struggle with the tupled arguments required for the TryParseMessage method.

I have added to an existing issue in the FSharp repository here

I have also created a minimal reproduction of the issue:
https://github.com/auslavs/errorFS2014

is there a common pattern to express HttpApplication, MqttApplication and others?

for me it is about transforming IDuplexPipe to IProtocol. What I think of is:

public interface IProtocol<TRequest,TResponse>
  {
    ChannelReader<TRequest> Input { get; }
    ChannelWriter<TResponse> Output { get; }
  }

public interface IProtocol<TMsg> : IProtocol<TMsg,TMsg>
  {
  }

Is there any reason NRTs aren't enabled here?

If not, I'm happy to add them.

I'll start of with BedrockFramework, then move on to Experimental.

Hello, I'm using bedrock in a multithreaded (worker) context. I have the write machinery guarded with a semaphor and have no issues writing regular traffic, but I consistently get a stack trace indicating that there is a write collision when a connection is closed. Is there a way to detect that this object is being disposed, or to synchronize with the CompleteAsync() call somehow? Is there something I'm not understanding about how to interact with the PipeWriter perhaps?

https://github.com/davidfowl/BedrockFramework/blob/master/src/Bedrock.Framework/Infrastructure/DuplexPipeStreamAdapter.cs#L40-L53

Is there an abstraction planned for different congestion control implementations for protocols? Or where does congestion control fit into the framework?

Hi, i've started to learn this framework and built simple rabbitmq-like api.
It would be great, if we can pass some payload to ConnectAsync method. Use case: add connection name, or some additional client provided properties for example application name etc.
How can i achive this functionality ?

I've tried to add middleware to add connectionName into ConnectionContext.Items, but my middleware executes after actual connection to the server.

services.AddTransient<IMqClient>(sp =>
{
    static Task Middleware(ConnectionContext connectionContext, Func<Task> next)
    {
        connectionContext.Items["connectionName"] = "test1";

        return next();
    }

    var cb = new ClientBuilder(sp);
    cb.Use(Middleware);

    var client = cb.UseSockets().Build();

    return new MqClient.MqClient(client);
});

Thanks.

Add BenchmarkDotNet microbenchmarks for ProtocolReader

There is a default implementation of connection pooling, a common concern for client side applications. Eventually, we would like this to be part of the main project, but there are a few key gaps we need to fill before:

• Figure out how to provide a custom implementation for connection pooling. Current plans are to follow something similar to ObjectPool
• Implement a few other clients that would utilize custom connection pooling logic (SqlClient, HttpClient, etc.)
• Lots of tests!

Similar to #83, any plans to add helper support for a transport using SerialPort?

Memory ownership is tied to a Pipe which means you need to copy the data if you take it out of the pipe. And similarly, you need to copy data to get it into the pipe.

It seems by design. But maybe we could eliminate these copies by having more advanced Memory ownership? Though there may be a usability cost we don't want to pay.

We should have a well tested blazing fast implementation of the various HTTP/1.x parsers:

Server

• - Decide on the representation for an HTTP request and body on the server side
• - Implement HttpRequestMessageReader - This should parse the start line and headers
• - Implement HttpResponseMessageWriter - This should write the response line and headers

Client

• - Decide on the representation for an HTTP request and body on the client side.
• - Implement HttpRequestMessageWriter, this should write an HTTP/1.1 request and headers
• - Implement HttpResponseMessageReader, this should write an HTTP/1.1 request and headers

Shared

• - ChunkedBodyMessageReader - Reads an HTTP/1.1 chunked request or chunked response body.
• - ChunkedBodyMessageWriter - Writes a HTTP/1.1 chunked request or chunked response body.
• - FormUrlEncodedMessageReader - Reads a HTTP/1.1 form url encoded request body.
• - FormUrlEncodedMessageWriter - Writes a HTTP/1.1 form url encoded request body.
• - MultipartMessageReader - Reads a HTTP/1.1 multipart encoded request body.
• - MultipartMessageWriter - Writes a HTTP/1.1 multipart encoded request or response body.

Each of these can be implemented and tested separately (for the most part) and some of them have already been prototyped in the experimental project.

If so it would be good to elaborate a bit in the Readme. I assume this works at Layer 4 (Transport) and theoretically could support UDP as well.

There is a race condition in current implementation of BindAsync.

public ValueTask<IConnectionListener> BindAsync(EndPoint endpoint, CancellationToken cancellationToken = default)
{
    endpoint ??= MemoryEndPoint.Default;

    if (_listeners.TryGetValue(endpoint, out _))
    {
        throw new AddressInUseException($"{endpoint} listener already bound");
    }

    MemoryConnectionListener listener;
    // here we can have multiple threads adding a listener to the collection, but only one of them will win
    _listeners[endpoint] = listener = new MemoryConnectionListener() { EndPoint = endpoint };
    return new ValueTask<IConnectionListener>(listener);
}

Once we have an HTTP implementation that we like, we should enter the benchmarks.

When using bedrock for client side connections the connection is aborted with ThrowLatchedException.
Reproducer below has a unit test run with both Pipelines.Sockets.Unofficial & Bedrock. Boolean determines which.

https://gist.github.com/Ninds/9e4dfe6c8ba974ec128fbbce089f3ff1

High performance, low level networking APIs for building custom severs and clients.
It may be : servers
@davidfowl
I'd like to also contribute to code.

There's a prototype of the NamedPipeTransport in the experimental project. We'd like to move this over to the main project but it needs to be cleaned up and tested to make sure it works well.

• - Properly implement the NamedPipeConnectionContext. This needs to implement a bunch of features (like connection aborted)
• - Add extension method on both the server and client (ListenNamedPipe)
• - Add tests, lots and lots of tests

Protocols

• - HTTP/1.1
• - HTTP/2
• - WebSocket
• - TLS

Middleware

• - Pooling
• - TLS

Transports

• - NamedPipes
• - UnixDomainSockets
• - Sockets
• - InMemory

Core

• - ProtocolReader
• - ProtocolWriter
• - IMessageReader
• - IMessageWriter
• - Server/ClientBuilder

Other

• - API Docs
• - Tutorial Docs
• - Samples/ best practices
• -Decide on target TFMs (ns2.0, ns2.1?)

First off, this is a super rad API. I was able to parse raw messages to and from Kafka in an hour or two from scratch, without any of the normal socket/buffer manipulation that'd normally have to be done.

I've not attempted such performance-oriented protocol parsing code like this before, and I'm wondering if there are any good open-source examples, code, blogs, whitepapers, etc of low/amortized allocation protocol parsing code that deals with a large amount of payload types?

I've done quite a bit of googling on the subject, but I haven't come across something that deals with handling hundreds of distinct payloads...

The full Kafka API surface area is ~370 versioned payloads (some redundancy, but a lot of them are versioned) and ~20 primitives - and I know that my current approach of new'ing up typed request/response pairs isn't an ideal approach for various reasons. Especially, when some Kafka libraries can hit ~1 million msg/sec, and the requests often need to be kept around until the responses are returned.

I can think of multiple ways to try and solve this, but I can also think of multiple downsides to most of these approaches, and any help addressing maintainability alongside performance would be awesome.

As a followup, would Bedrock.Framework.Experimental be a good place to try novel approaches to common issues in this space?

Today IMessageReader<T> looks like this:

public interface IMessageReader<TMessage>
{
    bool TryParseMessage(in ReadOnlySequence<byte> input, ref SequencePosition consumed, ref SequencePosition examined, out TMessage message);
}

This works when the caller has a ReadOnlySequence<byte> (usually from a PipeReader) and is parsing messages at that level. Most of our IMessageReader<T> implementations end up creating a SequenceReader<byte> internally over the ReadOnlySequence<byte> to do further parser.

When trying to compose these parsers internally, it usually results is translating from ReadOnlySequence<byte> -> SequenceReader<byte> which is inefficient and inconvenient. Ideally you should be able to keep the SequenceReader<byte> rather than switching back to ReadOnlySequence<byte> and vice versa.

As an example, see https://github.com/YairHalberstadt/BedrockFramework/pull/1/files

Options:

• Use SequenceReader<byte> instead of ReadOnlySequence<byte>
• Add TryParseMessage overload for SequenceReader

public interface IMessageReader<TMessage>
{
    bool TryParseMessage(ref SequenceReader<byte> input, out TMessage message);
    bool TryParseMessage(in ReadOnlySequence<byte> input, ref SequencePosition consumed, ref SequencePosition examined, out TMessage message);
}

cc @YairHalberstadt

Details here need to be fleshed out and I'm going to lean on @Drawaes to help flesh this out and break it down into work items. Ideally we'd have several parsers for various parts of the pipeline (handshake writer and reader etc).

• - ClientHelloMessageReader - This will parse the client hello message
• - ClientHelloMessageWriter - This will write the client hello message

Now that there is the ability to read and write raw WebSocket frames, the actual protocol can be implemented. There are probably some API questions worth getting opinions on.

The corefx WebSocket API leans very heavily in a Stream paradigm. Is that the direction we want to go, or would a more message based API be better? The WebSocket protocol is actually not hugely friendly here, living in no-man's-land between a streaming protocol and a message based protocol. Frames can be exabytes long, and messages can have an infinite number of frames, the number of which isn't in any header information.

A Pipe type of API is actually almost perfect, except with Pipe there isn't a way to indicate end-of-message out of ReadAsync.

Hi!

Is this a good place to ask questions?

I'm looking into the custom protocol example. MyCustomProtocol.

Is there a way to obtain the number of bytes being sent/written?

The goal would be to print typical progress showing data transfer and so on.

Thanks!

Hi, David. Thanks for your wonderful framework for network transport!

I found an unexpected situation, using the demo client application, if I input English alphabets, it works very well, but if I input some Chinese text, such as "我" then press Enter, the connection will disconnect.

Debugged into the code, found that if input Chinese, after the text transfer to server, in the server side, the connection.Transport.Input immediately become completed, so CopyToAsync break out.

Still investigating why.

There are a few things that can be done to improve the performance of the ProtocolReader:

• - Store the last ReadResult from the PipeReader and synchronously parse the protocol while there's still a buffer around. This reduces call to ReadAsync/Advance.
• - ProtocolReader currently uses an async state machine in ReadAsync per call, it could implement IValueTaskSource to amortize this cost and reuse the ProtocolReader itself to avoid the per operation allocation (this is tricky and should be done last)

I would like to:

• set NoDelay for client sockets
• set the tcp connect() timeout (From my experiments it looks like ConnectAsync() timeout can be as high as 75 seconds, but it varies randomly).

I believe a torn read/write may happen here:
https://github.com/davidfowl/BedrockTransports/blob/e02b10a83d18bdd79e18f8bbc52f40cc17a707bb/Bedrock.Framework/Server/ServerApplication.cs#L48

and here:
https://github.com/davidfowl/BedrockTransports/blob/e02b10a83d18bdd79e18f8bbc52f40cc17a707bb/Bedrock.Framework/Server/ServerApplication.cs#L73

I've updated my code in order to follow you re-design, previously I just implemented the server side, but now I'm digging into the client.
My process actually spawns a single server that uses socket connection with custom protocol, and listen to different endpoints read from config (but still a single Server instance that just have multiple listeners). Whenever I ctrl+C or i close the other end of the communciation, server react in a proper way and disconnect/dispose (even if I see some memory problem, everytime I create connections and close them, memory keep increasing, but it may be caused by other stuff I inject in DI that may survive, will look at that)

About client, I haven't a clear view on how to spawn multiple connections and how to manage properly their life cycle (e.g. if I call connection.abort() the connection doesn't die like I was expecting)

My approach currently is to create a single Client object and use it to call this.client.ConnectAsync in a task whenever I need to spawn a new connection to an endpoint
https://github.com/MithrilMan/MithrilShards/blob/41316d2e13fd287a352c402f0aa03e73aecac6aa/src/MithrilShards.Network.Bedrock/BedrockForgeConnectivity.cs#L115-L145

in the code above I used even a .Use(this.SetupClientConnection) during Client build I'm actually doing nothing there (is it useful in this context? probably not)
as a test, I spawn a connection right there calling my Attemptconnection and I use a custom class to handle the connection lifetime.

As I said, internally in my class I was expecting to be able to call connection.abort to interrupt that connection but it wasn't working, so in order to make it works like I want, I added a CancellationTokenSource property in one of my internal classes (PeerContext) that I can use to trigger the disconnection.

I assigned then the token of such property to ConnectionClosed property of my connection object
connection.ConnectionClosed = peerContext.ConnectionCancellationTokenSource.Token;

this link take to the code part that implement this logic
https://github.com/MithrilMan/MithrilShards/blob/41316d2e13fd287a352c402f0aa03e73aecac6aa/src/MithrilShards.Network.Bedrock/MithrilForgeClientConnectionHandler.cs#L71-L109

and as you can see there I pass connection.ConnectionClosed as the cancellation token for reader.ReadAsync
ReadResult<INetworkMessage> result = await reader.ReadAsync(protocol, connection.ConnectionClosed).ConfigureAwait(false);

TLTR, actually server and client (at least for what I tested: socket custom protocol implementation) have a different level of easy of use and it's not clear which step to take in order to implement the client connection lifetime in a proper way, can you point me(us?) on the right track?

P.S.
wouldn't be appropriate to have ConfigureAwait(false) calls on async calls in your examples?
thanks and sorry for the wall of text

It seems that there is no such thing as a pure.net microservices wheel in.net core, and it is desirable to have such a high-performance.net core microservices architecture in the direction of the future

The WebSocketMessageReader and WebSocketMessageWriter should support reading and writing raw websocket frames, we need to define the frame type of course and that's part of this design. I started porting the corefx implementation here (https://github.com/davidfowl/BedrockFramework/compare/davidfowl/websockets) which is still very incomplete.

• - Define message types for reading and writing
• - Implement WebSocketMessageReader
• - Implement WebSocketMessageWriter
• - Add tests, lots and lots of tests

The AdvanceTo is only invoked when read fails; most protocols will involve multiple messages, but the second call to ReadAsync reliably fails with:

Unhandled exception. System.InvalidOperationException: Advance must be called before calling ReadAsync

Repro is here

When reading requests in a loop I end up with the exception below, I do not have an issue when reading from the server in a tight loop with http client (ensuring a single connection). There is a gist with both the client code I am using and the server code (both bedrock)

https://gist.github.com/Drawaes/699b42621f31d83f8bf68b1acdcf0185

Unhandled exception. System.InvalidOperationException: The examined position must be greater than or equal to the consumed position.
   at System.IO.Pipelines.ThrowHelper.ThrowInvalidOperationException_InvalidExaminedOrConsumedPosition()
   at System.IO.Pipelines.Pipe.AdvanceReader(BufferSegment consumedSegment, Int32 consumedIndex, BufferSegment examinedSegment, Int32 examinedIndex)
   at System.IO.Pipelines.Pipe.AdvanceReader(SequencePosition& consumed, SequencePosition& examined)
   at System.IO.Pipelines.Pipe.DefaultPipeReader.AdvanceTo(SequencePosition consumed, SequencePosition examined)
   at Bedrock.Framework.Protocols.ProtocolReader.DoAsyncRead[TReadMessage](Nullable`1 maximumMessageSize, IMessageReader`1 reader, CancellationToken cancellationToken) in C:\code\BedrockFramework\src\Bedrock.Framework\Protocols\ProtocolReader.cs:line 113