Server cannot accept multiple clients in multithreading about python-mbedtls

[Synss]

Hi @P2Uming, I will look into this shortly.

[Synss]

While trying some more, I got some reproducible segmentation faults. There is definitely something wrong here.

[mingssun]

Hi @Synss, Thanks for your effort, Looking forward to the fix!

[Krolken]

I am not sure the solution with a wrapped socket is the best one of a server implementation.

A server need to accept multiple "connection" from many clients. It seems like the solution with a wrapped socket only will support 1 client as it says in the documentation that you need to make a new bind() for new clients. But you can only bind on one port at a time.

A better and maybe simpler solution would maybe be to have a DTLSSessionManager that sits between the socket and application.
The manager would store DTLS handshake state for each client that is connecting to it and return the correct responses.
As it keeps a record of clients it could also support sleeping clients and clients behind NATs if you would use the CID identifier but resuming the DTLS session. In an IoT environment it is important to keep the number of handshakes to a minimum.

So my proposed solution would be to not wrap the socket. Keep the socket as standard and handle the DTLS state in another class. Since you would use a standard socket it should be simple to integrate into different I/O paradigms. Just use a simple socketserver or use asyncio.

some psuedo-code

sock = make_socket()
appliation_reader, application_writer = get_app_reader_writer()

def on_datagram(data, addr):
   """Our server socket receives a datagram"""
   # If there is a DTLS connection the decrypted datagram will be passed to application_reader.
   # If not the next handshake package will be sent back
   sock.send(DTLSSessionManager.recv(data, addr))

I am just not sure on what parts of the mbedtls I need to interact with to support this setup.

[Krolken]

I tired setting up a small sample using "socketserver"

from socketserver import ThreadingUDPServer, BaseRequestHandler
import logging

from mbedtls import tls

logger = logging.getLogger(__name__)


class SimpleUdpHandler(BaseRequestHandler):
    """
    Simple request handler for UDP that will log incoming data.
    """

    def handle(self):
        srv_conf = tls.DTLSConfiguration(
            ciphers=(  # PSK Requires the selection PSK ciphers.
                "TLS-ECDHE-PSK-WITH-CHACHA20-POLY1305-SHA256",
                "TLS-RSA-PSK-WITH-CHACHA20-POLY1305-SHA256",
                "TLS-PSK-WITH-CHACHA20-POLY1305-SHA256",
            ),
            pre_shared_key_store={
                "client0": b"a secret",
                "client1": b"other secret",
                "client42": b"the secret",
                "client100": b"yet another one",
            },
        )
        dtls_context = tls.ServerContext(srv_conf)
        dtls_buffer = tls.TLSWrappedBuffer(dtls_context)

        data = self.request[0]
        socket = self.request[1]
        # dtls_socket = dtls_context.wrap_socket(socket)
        print(dtls_context._state)
        dtls_buffer._setcookieparam(str((self.client_address[0])).encode())
        dtls_buffer.receive_from_network(data)

        # dtls_socket.do_handshake()
        dtls_buffer._as_bio()
        dtls_buffer.do_handshake()

        print(dtls_context._state)

        print(data)
        print(socket)
        logger.debug(
            f"Received UDP Message",
            extra={
                "data": f"{data!r}",
                "host": self.client_address[0],
                "port": self.client_address[1],
            },
        )


if __name__ == "__main__":
    request_handler = SimpleUdpHandler

    with ThreadingUDPServer(("localhost", 2883), request_handler) as server:
        click.secho(
            f"Staring {server.__class__.__name__} ",
            fg="bright_yellow",
            blink=True,
        )
        server.serve_forever()

If I try to use a wrapped socket I get the error mbedtls.exceptions.TLSError: TLSError([0x004E] 'NET - Sending information through the socket failed')

But I wanted to try and only use the buffer to be able to handle the sockets and threads myself. But the _as_bio() is not availabe on the python object. If I just call dtls_buffer.do_handshake() i get mbedtls.exceptions.TLSError: TLSError([0x7100] 'SSL - Bad input parameters to function')

[Synss]

Indeed,

It seems like the solution with a wrapped socket only will support 1 client as it says in the documentation that you need to make a new bind() for new clients. But you can only bind on one port at a time.

is entirely correct. The wrapped socket comes from https://www.python.org/dev/peps/pep-0543/#toc-entry-10 so it is not something I will change. However,

the _as_bio() is not availabe on the python object.

the _as_bio() functions should not be necessary, at all. It was a useful shortcut I took in the beginning but it is on my to-do list to remove it and perform the handshake explicitly. That would result in further simplification such as making TLSWrappedSocket pure Python.

If that helps here I might as well do it sooner than later. It should be possible to perform the handshake using the buffer and a regular socket.socket(). However, it is not easy to explain and removing _as_bio() would be the perfect example for this.

[Krolken]

I made a public method calling _as_bio() in my fork and then I could get futher.
What I can't find in the mbedtls code is what generates the payloads.
I assume I should add the ClientHello in the buffer, then try to move the state. Since there is no cookie we get the HelloVerifyRequeset exception.
But where do I get the payload for the HelloVerify? If i use buffer.consume_output I get None.

[Synss]

The handshake does not use the buffer. Be sure to call _as_bio() on TLSWrappedBuffer. Then, you will probably need to jungle between the two buffers. You should probably publicise them as well, or at least some part of them (like their current size) if you want to keep on debugging. I am working on it as well.

[Synss]

Sorry, I realise I misunderstood your question and you are that far already.

But where do I get the payload for the HelloVerify? If i use buffer.consume_output I get None.

The best for the details of the handshake with a raw socket is probably to check upstream:

• https://github.com/ARMmbed/mbedtls/blob/master/library/ssl_srv.c#L4256
• https://github.com/ARMmbed/mbedtls/blob/master/library/ssl_cli.c#L4208

[Synss]

@Krolken : As it seems you wanted to have a go at it as well I just pushed my dev branch

https://github.com/Synss/python-mbedtls/tree/dev

where the last commit performs a TLS handshake in the TLSWrappedBuffer instead of bypassing it. That means, handshake does not require the TLSWrappedSocket anymore (and actually does not use it).

I only tested it on TLS, DTLS will not work. But the idea is here. I will keep working on it and rebase/rewrite the history of the dev branch, too, so do not build on it: it will keep moving! 😉

[Krolken]

Thanks. Pulled it last night and tried some things. I am not able to get it to raise a HelloVerifyRequest Exception.
Also tried rewriting the handshake with sendto(), which did work on returning the response but since it did not raise a HelloVerifyRequest it just sent b'' and the client interpreted it as EOF.
Then of course it wouldn't work on the second try with my current setup as it is a threaded server and it would create a new thread for the response.

My current plan is to keep a dict of source_address and port mapping to the context as shared state. Just need to figure out how to initialize the handshake properly.

Now I am doing something like this.

dtls_context = tls.ServerContext(srv_conf)
dtls_buffer = tls.TLSWrappedBuffer(dtls_context)
data = get_udp_data()
dtls_buffer.receive_from_network(data)
dtls_buffer._setcookieparam(self.client_address[0].encode('ascii'))
# Assumed this would raise HelloVerify
dtls_buffer.context._do_handshake_step()
# and assumed that the data to send back would be in the buffer.
print(dtls_buffer.peek_outgoing(1024))

[Synss]

I have not forgotten about this issue but my personal computer is broken.

[Krolken]

No problem. I have been bogged down with other stuff.

[Synss]

Current master should have most of the necessary changes we discussed here.

• TLSWrappedSocket and TLSWrappedBuffer are pure Python: less magic happens here and they may be swapped for other implementations.
• The handshake now happens entirely in the TLS context instead of the wrapped socket and buffer.

I also have example client/server under programs.

[Synss]

On master, ClientContext and ServerContext are picklable (and therefore threadable or multiprocessing-friendly) and the contexts can wrap several sockets or buffers.

Now, some changes are still in progress for the handshake but, unless I am overlooking something important, the points above should fix this issue.

[Synss]

Now, I/O and crypto are independent and you can select any strategy you want on the I/O layer. The latest patches exemplify the crypto part without any actual I/O (test_tls).

Note that session tickets are currently not available on master. This is known and I am working on it. I may still release the current code without session handling shortly and add the session handling back later.

There have been small API changes since 1.6-1.7.