ermiry / cerver

Cerver is a development framework, created by Ermiry, designed to provide you with the necessary tools for creating fast and reliable multipurpose servers written in C.

Hello there! 👋, I am Erick Salas

Never stop creating

• 🔭 I’m currently working on Cerver Framework

• 🌱 I’m currently learning Custom implementations of ML and AI algorithms in C/C++ using cuda

• 💬 Ask me about Linux, C/C++, Cloud Services, ML, Flutter, React, Dev-Ops

• 📫 How to reach me [email protected]

cerver's Issues

Ability to create encrypted connections using SSL

Add the ability to create connection that can support an extra layer of security in the packets / data they can transfer.

Check that all allocated structures & buffers are deleted in cerver receive methods

In cerver receive methods (handler) we allocate at least two different structures (ReceiveHandle & CerverReceive), after huge refactor in logic due to the use of thpool, we need to check that we are not leaving any memory floating around.

Enhance client example

• Better code structure
• Showcase the creation of multiple client using different methods

Refactor thpool with custom thread structures

Refactor original thpool code to use latest cerver thread custom structure and methods

Connection gets stuck when handling a request as a web cerver using thpool

We can generate the response correctly, but when calling the following methods to end the connection, we get stuck somewhere

// after the performed action, close the client socket
Client *client = client_get_by_sock_fd (receive->cerver, receive->socket->sock_fd);
// client_drop (receive->cerver, client);
client_remove_connection_by_sock_fd (receive->cerver, 
	client, receive->socket->sock_fd);

Check client_unregister_from_cerver ()

Specially when called from

client_remove_connection_by_sock_fd ()
client_drop ()
client_unregister_from_cerver ()

• Room for optimization - check connections size before performing operations
• When are we removing client from htab

Idea

Maybe when removing client from htba, or not removing it is causing segfault
At the time of this writing, htba is not thread safe

Create a large file transmission example to test handlers reliability

Possible double lock in cerver_poll_register_connection ()

Possible double lock of cerver->poll_lock when calling cerver_poll_register_connection () and we need to realloc the poll array as it might be full!

Possible solution

Split in two methods, one is the public method which will manage the flow (and will lock), and the second will perform the actual insertion in the array (and will NO lock).

Refactor htab collection to be thread safe

As we are accessing htab from different threads in latest handler, we need to make it thread safe.

Maybe this will solve issue #27

Create option to select how many threads the cerver can use for receive

Latest cerver (1.2.3 - 1.3) can only handle incoming messages in one thread, however, we have commented logic for handling incoming messages using the thpool, bu it its just lying there. Add option to select to use the pool thread or not for incoming messages.

Add application wide tests and unit testing to critical components

As our code become more complex with a lot more features on the way, we need a better way to test current elements of the framework whenever we are adding new features.

Add option to handle connections in dedicated threads

Right now we only use poll () to select which socket to call recv () on, and then we handle the buffer using the thpool or in the same thread. We need to offer an option to create a dedicated thread for each connection.

Also create benchmarks to test which is the best option in each case.

Refactor examples to use latest handler methods

• test
• handlers
• multi
• game

Check why Sockets are not being pushed to pool in cerver_receive_handle_failed ()

In cerver_receive_handle_failed () if we find a rogue connection (socket) that does not belong to any Connection we have cerver_sockets_pool_push () placed in comments.

Create a cerver auth example

Create an example to showcase cerver built in authentication methods

Web socket implementation

Implement a fully fetch web socket module

• Set route modifier to enable web sockets connections
• Correctly handle http request headers to enable web sockets
• Disable connection closing after the first http request has finished
• Correctly handle data frames (receive & send)
• Implement callback methods to be called on various web socket events (open, close, message, etc)

Refactor admin cerver flow and methods

Refactor how admins work in cerver to increase their performance and their usage

• Remove dedicated socket to listen for new admin connections (accept & receive)
• Remove dedicated receive methods, as they are duplicates of the ones found the main handler
• Remove existing credentials logic and replace it with the ability to take a user defined method instead
• Remove dedicated auth methods, as they are copies of the main auth methods and use them instead
• Refactor admin handlers to use latest Handler structure

Ability to display & interact with a menu to check cerver stats

Add the ability to interact with a menu (inspired by Flutter) to directly check on cerver status & display information about current cerver structures & clients. This will be useful to help in the debugging process & will complement cerver-connect which requires to setup dedicated admins.

Add support for udp connections

We have some methods already in place to handle sending and receiving packets using the udp protocol, however, it has never been implemented.

Update client methods with latest cerver-client source code

• update cerver connect methods
• update cerver request methods
• update client handler methods

Correctly destroy handlers

Wait for all handlers to be done and correctly dispose threads

Refactor client handler methods

Refactor client handler methods to correct use latest code from main cerver handler. Also replace handler Action to the new Handler structure.

• Refactor client structure
• Refactor client handler methods
• Correctly init handler threads when client starts, if needed
• End client handlers in client teardown

Add option for authentication using mongo directly

Use ermiry's custom mongo driver for a new option to handle authentication for normal and admin clients.

Create a new example to showcase app, app error and custom handlers implementations

Create handlers example to showcase how cerver handler's should work now and rename previous handlers example to multi

Segfault on client disconnect

When a client sends use a CLIENT_CLOSE_CONNECTION packet, we get a segfault some times.

Tested with 16 clients, all connect and send test messages as expected, however, almost when always, when 14/16 client have disconnected, when disconnecting 15th we get a segfault in the server.

Server configuration:
cerver_set_receive_buffer_size (my_cerver, 16384);
cerver_set_thpool_n_threads (my_cerver, 4);

Ideas

Relate to use #26 for a possible solution

Refactor how client & connection stats are updating in cerver handler methods

• Correctly update client & connection stats in cerver_receive (). This means that we need to get the client & connection related to the socket in this method, rather than in the actual handling methods.
• Getting the client & connection will require to tweak existing handler methods

Subroutine to check for inactive clients

• Add the ability to set cerver's option to check for inactive clients
• Refactor client structure to keep track of the last time it had activity
• Create a dedicated thread for client inactivity checking
• Check for when a client was last active (made a request) and if it was some time ago, drop the client.
• Every time the cerver receives a packet from a client (connection) or a packet is sent to him, the inactivity time must be updated
• We need a mutex inside client structure to correctly handle dropping clients from multiple threads

Segfault in client_connection_request_to_cerver ()

Sock receive is NULL when receiving packets in client_connection_request_to_cerver () in v 1.3.1

Refactor / Re-implement avl tree

After testing, we found that at least two segfaults were caused directly by the avl data structure:

• The first one related to issue #40 was solved by refactoring avl_remove_node_r (). This was found to be in every version at least from 1.3.1 but probably even before, not real way to tell.

• The second one related to issue #27 solved by adding additional checks in methods for NULL values.

This is specially concerning as we need to do more testing to check if we are indeed getting the correct behavior of an avl after the modifications. More testing will be done, but a good idea will be to completely re-implement the avl collection.

Refactor client start methods

• Create a new client_start () that should be used after client_connect ()
• Refactor existing start method into client_connect_and_start ()

Received NULL ptr when attempting to remove a client from cerver's client tree

This error only happens only when removing the first client that connected to the cerver when there ate least two clients connected. If only one client is connected or we remove the second or any other client before the first one, it does not happens.

Tested in cerver versions as low as 1.3.1 and we get the same result.

Upgrade cerver http capabilities

Upgrade the existing http capabilities of the framework by implementing the following:

• A fully fetch http parser based on https://github.com/nodejs/http-parser
• Multi-part parser based on https://github.com/iafonov/multipart-parser-c
• Add native support for jwt tokens
  • This will remove the need of having a js middle-ware as in current implementations as the most request future is to directly handle private routes the same way pasport-jwt is configured to do so.

Create an array of sockets that the cerver can bind and listen to

Add the ability for the same cerver to handle connections to multiple sockets

Create a dedicated requests example

Create a new dedicated requests example to handle and test requests form cerver-client

Add client events

Add the events from cerver-client into cerver's client. This events are mostly when connecting one cerver to another or to better manage client connections / requests to another type of server.

Problems in client_remove_connection_by_sock_fd ()

• In the default case of client_remove_connection_by_sock_fd () we are not correctly removing the connection from the client
• Connections should be closed & removed from client in one method, and in another one, they should be unregistered from the cerver
• Refactor connection_remove () with a similar logic to the one implemented in admin cerver

Update client & connection stats in client_receive ()

Check that we are indeed correctly updating cerver client connection stats in client_receive () & client_packet_handler () & related packet methods.

Use latest handler structure in single handler server

Single handler server (original) is not using the latest handler structure. This will allow use to also create dedicated handler data when the main handler starts. We need to change app_packet, app_error_packet and custom_packet handler from being Action to Handler

Create an admin exmaple

Use this example to show case how admins can connect to the cerver and how to enable them in the cerver. This will also be useful for testing admin methods.

Enable or disable packet checking

We have the methods in place to correctly check if a packet is valid or not but have been disabled. Add the ability to select to perform packet checking (version, protocol id, etc) or not.

Remove compress clients (poll fds array) option

Remove the option to compress clients (poll fds array) as it is no longer used

• Remove cerver_poll_compress_clients ()
• Remove cerver_on_hold_poll_compress ()
• Remove compress option from cerver structure
• Remove compress checks from poll methods

Refactor how we access connection->sock_receive

We are using a global map to get connection's sock_receive reference, which is unnecessary, as we have sock_receive inside the connection structure itself

Create a basic client example

Create a new example to showcase how to use cerver clients to connect to another server and perform requests.

Sockets are not being created using the cerver's socket pool

Every time we create a new connection, a new socket is allocated right way without checking the cerver's sokcet pool first.

Inconsistent receives when using thpool in cerver_poll ()

When using only a single thread for handling messages from already connected clients, we get the expected result for one or more clients connected at once.

However, when we choose to handle using the thpool, we get inconsistent received and handler methods. And it appears that the socket or therad just hang up and stop sending and / or receiving messages.

Client sending test packets to cerver

We also get a segfault when exiting the cerver.

Possible Cause

As we don't have the array of fds in cerver as a thread safe structure, when calling cerver_receive_handle_failed () when a client disconnects, may cause the segfault as we are still trying to access it from the cerver_poll () at the same time. This, however, will not explain the inconsistent handles.

Finish game cerver methods

We still have game methods that need to be finished!

Check game_packet_handler () in game.c

Add dedicated module to get stats, processes and general info from host machine

Reference https://github.com/hyperic/sigar

Better cerver to cerver connections

Correctly handle:

• Cerver requires authentication - cerver_check_info ()
• Cerver teardown - disconnect event - client_cerver_packet_handler ()

Check on_hold_connection_get_by_sock () is unused

Compiler marks it as an unused method in cerver 1.4

Added #pragma GCC diagnostic ignored "-Wunused-function" to suppress warning until issue is reolved.

Critical error in cerver_poll_handle () when clients disconnects without sending any packet

When a cerver client connects & immediately disconnects due to an error on his part, cerver_poll_handle () gets stuck trying to handle an apparent signal 25 (value taken from cerver->fds[i].revents)