Can f-stack network stack run as 1 process on multiple cores with multiple threads? about f-stack

[whl739]

lcore_mask in config.ini means hexadecimal bitmask of cores to run on, it can be configured to multi cores, the default 3 means we will start two processes on cpu0 and cpu1. But if you have only one core, it will start one process.

If you start f-stack without start.sh, you should do it like this:
1 process:
sudo ./demo ./config.ini -c 0x1 --proc-type=primary --num-procs=1 --proc-id=0
2 processes:

sudo ./demo ./config.ini -c 0x1 --proc-type=primary --num-procs=2 --proc-id=0
sudo ./demo ./config.ini -c 0x2 --proc-type=secondary--num-procs=2 --proc-id=1

[whl739]

We don't support 1 process on multiple cores with multiple threads for now.
In order to run FreeBSD TCP/IP stack in parallel without lock, we just run one process with one thread on each core.

[taoswords]

Yelp. 2 processes will be OK if I start the 1st process as primary and the 2nd process as secondary.

My case is to run multiple workers to deal with the tcp sessions on 1 NIC port with 1 IP. The original idea is to run multiple threads to deal with different sessions based on different tcp ports. The beneifit is that the different tcp session will involve different time-consumption phases, and it is not suitable to put all in the single f-stack network stack.

If f-stack do not support 1 process on multiple cores with multiple threads so far, is it possible for me to use f-stack to satisfy my case?How could the primary process to share incoming network packets with the secondary process?

-- Where should I initialize the IP? On the primary process? Or both on the primary and secondary?
-- How could I make the different tcp streams to different process even if the IP initialization is resolved?

[whl739]

If f-stack do not support 1 process on multiple cores with multiple threads so far, is it possible for me to use f-stack to satisfy my case?How could the primary process to share incoming network packets with the secondary process?

F-Stack can not fully adapt your case, you must make some changes.In lib/ff_dpdk_if.c:process_packets:

enum FilterReturn filter = protocol_filter(data, len);

this code will check the incoming packets, if they are ARP packets or KNI packets, will be transmitted to other processes using rte_ring.You can refer to this, peek the packet, determine that whether this packet is needed for this process, if not, transmit to other process.

-- Where should I initialize the IP? On the primary process? Or both on the primary and secondary?

Both on the primary and secondary.

-- How could I make the different tcp streams to different process even if the IP initialization is resolved?

Refer to the ARP processing.

By the way, we'll add some hook functions for custom packet handling, for example, bypass, forward, copy,etc..

[taoswords]

Thanks. I will look inside and try to filter the packets.

[taoswords]

one more question:

Given the config.ini is configured with 1 port with 1 IP. I run 2 processes with 1st as primary and 2nd as secondary based on current f-stack.
I wonder which process will receive the packets. Will the primary process receives all the packets? Or the traffic is randomly received among these 2 processes? Since these 2 processes all share the same config.ini and will see the same configuration on the same port, the primary process and secondary process will randomly pick up the packets?

[whl739]

According to the result of RSS hash, packets will be received to different processes.
Currently the primary and secondary processess will receive packets from all nics.

[gpstrive]

@oceanspark
actually, you can achieved the purpose by run 2 process (or many), and each process has its own rx ring.

[jlijian3]

Is it support nginx proxy on multiple processes? How to make the packets from backend server received by the process correctly?
For example，nginx process0 send syn to the backend server，the syn-ack will received by process0 but not process1.

[taoswords]

@gpstrive Yeah, different rx ring can do if the transactions are stateless. I want to control all the streams/datagrams for every transaction. Just as @whl739 said, we need to do some custom packet handling. I am trying to add some hook points for purpose. Certainly, if f-stack supports hooks, maybe it is easier ^-^

[taoswords]

@jlijian3 I do not quite understand your mentioned proxy-pass. Maybe it is like the flow director. I just know f-stack uses RSS hash to distribute the flows to different process, and it is up to the users to take actions on the received packets, like dropping, dealing with some extra procedures, or just forwarding, and so on.

[jlijian3]

Sorry，I mean nginx proxy.

[whl739]

Proxy is supported by nginx-fstack.
When F-Stack connects to a server, we will choose a port(use toeplitz_hash to calculate the rss result) that packets will be received on the same process.

[gpstrive]

@oceanspark
My case is to run multiple workers to deal with the tcp sessions on 1 NIC port with 1 IP. The original idea is to run multiple threads to deal with different sessions based on different tcp ports.

If you start 2 processes, different tcp sessions distribute to different process and the same session always go to the same process, does it satisfy your case ?

[taoswords]

@gpstrive I think your approach to start 2 processes to handle tcp sessions is ok in your case. However, my case is different and is required to handle mixed udp and tcp packets, and some time-consuming work to do. So I consider to hook and re-distribute the packets to different processes even if I benefit the high capability of F-stack to receive packets.