wuklab / lite Goto Github PK

I want to write a RDMA kernel module with Linux 5.1.1 & OFED 4.7 . But RDMA kernel APIs have changed a lot from Linux 3.10 to Linux 5.1.1 .And it's hard for me to find some guide about this.
I'm wondering if it is possible that you could recommend some books or PDF files about RDMA kernel module programming.
Thank you for your time.

There are only three example programs under lite-userspace. So I am confused about how to use these functions to write my own program.Do I need to read the source code and learn about these functions' usage?

Faild to add cluster when client node equal or greater than 2 node.

error:
[root@localhost cluster-manager]# ./mgmt-server
Initialize Server
Hostname: localhost.localdomain
IB-port: 1
Eth-port: 18500
Option: 0
max qp 163776
UD qpn 96
local address: LID 0x0000, QPN 0x000061, PSN 0x1d3b53, GID fe80::6eb3:11ff:fe4d:ca8
local address: LID 0x0000, QPN 0x000062, PSN 0xb455a1, GID fe80::6eb3:11ff:fe4d:ca8
loopback create successfully
Do a post-receive with 2048
IB Preparation for the incoming 1 connection from 172.16.3.105
send NODE_ID 1
Get Connection from 172.16.3.105: 0001:0000:00004b:50f524:fe800000000000006eb311fffe4d0b88
server_keep_server_alive: UD message from 1 with qpn 74 and lid 0: 0x7f32990888c0
IB Preparation for the incoming 2 connection from 172.16.3.104
send NODE_ID 2
Get Connection from 172.16.3.104: 0002:0000:000069:4bd5c0:00000000000000000000000000000000
server_keep_server_alive: UD message from 2 with qpn 104 and lid 0: (nil)
Segmentation fault (core dumped)

master-system：
[root@localhost cluster-manager]# uname -r
3.10.108-lite-kernel

the kernel 3.10.108 and apply lite-patch. if I not apply patch，I can‘t install official IB ofed libraries.

client-system:
same with master-system. The libraries through yum install.

{lite/core/lite_core.c line 5764~5767
addr.sin_addr.s_addr = htonl((((((ip_a << 8) | ip_b) << 8) | ip_c) << 8) | ip_d);
printk(KERN_ALERT "establish connection to %x to port %d\n",addr.sin_addr.s_addr, port);
sockfd = sock_create(PF_INET, SOCK_STREAM, IPPROTO_TCP, &excsocket);
ret = excsocket->ops->connect(excsocket, (struct sockaddr *)&addr, sizeof(addr), 0);
}
where is listen function?

Today I tried to compile the userspace program and I get problem like this:
lite-lib.h:153:12: warning: inline function ‘userspace_liteapi_memset’ declared but never defined
inline int userspace_liteapi_memset(unsigned lite_handler, int offset, int size);
I searched the code and did not find the definition of 'userspace_liteapi_memset'
Many other functions also have the problem.
Make error:
lite_join.c:(.text+0x26): undefined reference to userspace_liteapi_join' lite_join.c:(.text+0x2b): undefined reference to userspace_liteapi_get_node_id'
collect2: error: ld returned 1 exit status
make: *** [lite_join.o] Error 1

when I look into the source code , I am confused about the parameter of hash_for_each_possible_rcu
in function client_add_askmr_table, there is one line code like this:
hash_for_each_possible_rcu ( ADD_ASKMR_TABLE_HASHTABLE , current_hash_ptr, hlist , bucket)
there are 4 parameters
the definition of hash_for_each_possible_rcu is:
#define hash_for_each_possible_rcu(name, obj, node, member, key)
hlist_for_each_entry_rcu(obj, node, &name[hash_min(key, HASH_BITS(name))], member)
there are 5 parameters
and the definition of hlist_for_each_entry_rcu is:
#define hlist_for_each_entry_rcu(pos, head, member)
for (pos = hlist_entry_safe (rcu_dereference_raw(hlist_first_rcu(head)),
typeof((pos)), member);
pos;
pos = hlist_entry_safe(rcu_dereference_raw(hlist_next_rcu(
&(pos)->member)), typeof((pos)), member))
there are only 3 parameters