lvsgate / lvs-dpdk Goto Github PK

按照readme进行了安装，过程没有问题，第六步执行测试用例时大量报错
EAL: Detected 2 lcore(s) EAL: Probing VFIO support... EAL: PCI device 0000:02:01.0 on NUMA socket -1 EAL: probe driver: 8086:100f net_e1000_em EAL: PCI device 0000:02:01.0 on NUMA socket -1 EAL: probe driver: 8086:100f net_e1000_em EAL: PCI device 0000:02:05.0 on NUMA socket -1 EAL: probe driver: 8086:100f net_e1000_em PKTIO: initialized loop interface. No crypto devices available odp_cpu_count : 2 Num worker threads: 1 first CPU: 1 cpu mask: 0x2 I 9 1:1164054272 ofp_uma.c:45] Creating pool 'udp_inpcb', nitems=1024 size=904 total=925696 I 9 1:1164054272 ofp_uma.c:45] Creating pool 'tcp_inpcb', nitems=2048 size=904 total=1851392 odp_pool.c:479:odp_pool_create():Cannot init DPDK mbuf pool: Cannot allocate memory E 9 1:1164054272 ofp_uma.c:54] odp_pool_create failed I 9 1:1164054272 ofp_uma.c:45] Creating pool 'tcpcb', nitems=2048 size=784 total=1605632 odp_pool.c:479:odp_pool_create():Cannot init DPDK mbuf pool: Cannot allocate memory E 9 1:1164054272 ofp_uma.c:54] odp_pool_create failed I 9 1:1164054272 ofp_uma.c:45] Creating pool 'tcptw', nitems=409 size=80 total=32720 I 9 1:1164054272 ofp_uma.c:45] Creating pool 'syncache', nitems=30720 size=168 total=5160960 odp_pool.c:479:odp_pool_create():Cannot init DPDK mbuf pool: Cannot allocate memory E 9 1:1164054272 ofp_uma.c:54] odp_pool_create failed I 9 1:1164054272 ofp_uma.c:45] Creating pool 'tcpreass', nitems=320 size=48 total=15360 I 9 1:1164054272 ofp_uma.c:45] Creating pool 'sackhole', nitems=65536 size=40 total=2621440 odp_pool.c:479:odp_pool_create():Cannot init DPDK mbuf pool: Cannot allocate memory E 9 1:1164054272 ofp_uma.c:54] odp_pool_create failed I 9 1:1164054272 ofp_init.c:245] Slow path threads on core 0 I 9 1:1164054272 ofp_ifnet.c:143] Device '0' addr 00:0c:29:7d:2e:ba I 9 1:1164054272 ofp_ifnet.c:152] Device '0' MTU=1500 odp_thread.c:166:odp_thread_init_local():There is a thread already running on core 0 odp_thread.c:166:odp_thread_init_local():There is a thread already running on core 0 I 14 1:1164054272 main.c:369] if 0 rx_q 0 bind to cpu 1 odp_thread.c:166:odp_thread_init_local():There is a thread already running on core 0 ../linux-generic/odp_packet_io.c:224:setup_pktio_entry():Unable to init any I/O type. E 14 1:1164054272 ofp_ifnet.c:23] odp_pktio_open failed E 14 1:1164054272 main.c:337] Failed to init interface 1 E 14 1:1164054272 main.c:477] create_ifnet_and_bind_queues failed

3. Fetch and compile ofp 配置configure时最后的cxxflags配置不进去，手动修改了configure的17981行
   17981 AM_CPPFLAGS="$AM_CPPFLAGS $ODP_CFLAGS $ODPHELPER_CFLAGS $XYZ"
   在最后加上变量XYZ,然后改成
   ./configure --with-odp-lib=odp-dpdk --with-odp=<ODP-DPDK INSTALLATION DIR> --enable-shared=no --enable-sp=yes --disable-mtrie XYZ=-I<ODP-DPDK INSTALLATION DIR>/include/odp/arch/x86_64-linux/
   才成功配置
   第三步最后还差了个make install

classifier_main.c:(.text+0x45a): undefined reference to odph_linux_pthread_create' classifier_main.c:(.text+0x476): undefined reference to odph_linux_pthread_join'
../../lib/.libs/libofp.a(ofp_cli.o): In function ofp_start_cli_thread': ofp_cli.c:(.text+0x3894): undefined reference to odph_linux_pthread_create'
../../lib/.libs/libofp.a(ofp_cli.o): In function ofp_stop_cli_thread': ofp_cli.c:(.text+0x3a18): undefined reference to odph_linux_pthread_join'
../../lib/.libs/libofp.a(ofp_init.o): In function ofp_init_global': ofp_init.c:(.text+0xaf2): undefined reference to odph_linux_pthread_create'
../../lib/.libs/libofp.a(ofp_init.o): In function ofp_term_global': ofp_init.c:(.text+0xd6e): undefined reference to odph_linux_pthread_join'
ofp_init.c:(.text+0xf0c): undefined reference to odph_linux_pthread_join' ofp_init.c:(.text+0xf26): undefined reference to odph_linux_pthread_join'
../../lib/.libs/libofp.a(ofp_ifnet.o): In function ofp_ifnet_create2': ofp_ifnet.c:(.text+0x10f3): undefined reference to odph_linux_pthread_create'
ofp_ifnet.c:(.text+0x1143): undefined reference to `odph_linux_pthread_create'

测试环境：vmware RHEL7.2

follow below command:
export ODP_PLATFORM_PARAMS="-n 4"
./ofp_vs -i 0,1 -c 2 -o 0 -p 1 -f ofp.conf

I 1 0:693169024 ofp_uma.c:44] Creating pool 'tcptw', nitems=409 size=80 total=32720
I 1 0:693169024 ofp_uma.c:44] Creating pool 'syncache', nitems=30720 size=168 total=5160960
I 2 0:693169024 ofp_uma.c:44] Creating pool 'tcpreass', nitems=320 size=48 total=15360
I 2 0:693169024 ofp_uma.c:44] Creating pool 'sackhole', nitems=65536 size=40 total=2621440
I 3 0:693169024 ofp_init.c:195] Slow path threads on core 0
I 3 0:693169024 ofp_ifnet.c:143] Device '0' addr 00:0c:29:1a:76:13
I 3 0:693169024 ofp_ifnet.c:152] Device '0' MTU=1500
odp_thread.c:166:odp_thread_init_local():There is a thread already running on core 0
../linux-generic/odp_timer.c:683:timer_notify():
1 ticks overrun on timer pool "TmrPool", timer resolution too high
I 9 0:693169024 main.c:365] if 0 rx_q 0 bind to cpu 0
I 9 0:693169024 ofp_ifnet.c:143] Device '1' addr 00:0c:29:1a:76:1d
I 9 0:693169024 ofp_ifnet.c:152] Device '1' MTU=1500
odp_thread.c:166:odp_thread_init_local():There is a thread already running on core 0
odp_thread.c:166:odp_thread_init_local():There is a thread already running on core 0
I 13 0:693169024 main.c:365] if 1 rx_q 0 bind to cpu 0
odp_thread.c:166:odp_thread_init_local():There is a thread already running on core 0
odp_thread.c:166:odp_thread_init_local():There is a thread already running on core 0
odp_thread.c:166:odp_thread_init_local():There is a thread already running on core 0
E 22 0:693169024 ofp_vs_ctl.c:3093] genl_osp_resolve return -12

E 22 0:693169024 ofp_vs_core.c:466] ofp_vs_ctl_init Unknown error -12 -12

E 22 0:693169024 main.c:495] ofp_vs_init() failed

End Worker
E 23 0:693169024 ofp_vs_sched.c:211] unregister_ip_vs_scheduler(): [rr] scheduler is not in the list. failed

E 23 0:693169024 ofp_vs_sched.c:211] unregister_ip_vs_scheduler(): [snat_sched] scheduler is not in the list. failed

Segmentation fault (core dumped)
[@linux /opt/ofp/example/ofp_vs]#

linux/types.h 和 sys/types.h 以及 stdint.h 类型定义冲突
错误如下（比较多，截了一段）：
/usr/src/linux/include/linux/types.h:112:17: error: conflicting types for ‘u_int64_t’
typedef __u64 u_int64_t;
^
In file included from /usr/include/sys/uio.h:23:0,
from /usr/include/sys/socket.h:26,
from /usr/include/netinet/in.h:23,
from /usr/include/netdb.h:27,
from ../include/layer4.h:30,
from layer4.c:24:
/usr/include/sys/types.h:203:1: note: previous declaration of ‘u_int64_t’ was here
__u_intN_t (64, DI);
^
In file included from ../include/../libipvs-2.6/ip_vs.h:12:0,
from ../include/check_data.h:38,
from ../include/check_api.h:27,
from layer4.c:25:
/usr/src/linux/include/linux/types.h:113:17: error: conflicting types for ‘int64_t’
typedef __s64 int64_t;
^
In file included from /usr/lib64/gcc/x86_64-suse-linux/4.8/include/stdint.h:9:0,
from /usr/include/netinet/in.h:22,
from /usr/include/netdb.h:27,
from ../include/layer4.h:30,
from layer4.c:24:
/usr/include/stdint.h:40:19: note: previous declaration of ‘int64_t’ was here
typedef long int int64_t;
^
In file included from ../include/../libipvs-2.6/ip_vs.h:12:0,
from ../include/check_data.h:38,
from ../include/check_api.h:27,
from layer4.c:25:

I build odp with option 'enable-shared=no' so that ofp_vs can build with static lib of odp (libodp-dpdk.a libodphelper.a ).
But this ofp_vs can not work with error:
odp_packet_dpdk.c:162:setup_pkt_dpdk():No driver found for interface: 0
../linux-generic/odp_packet_io.c:224:setup_pktio)entry():Unable to init an I/O type.

I found that dpdk driver lib: librte_pmd_*.a are not link in libodp-dpdk.a, so there is a error "No driver found for interface: 0"

How can I build ofp_vs with static lib of odp?

严格按照文档上的步骤执行的，在as7.2环境下，ofp编译不过去
make[2]: Entering directory /opt/work/ofp/example/ofp_vs' CC ofp_vs-main.o In file included from ./include/net/ip_vs.h:26:0, from ./include/ofp_vs.h:27, from main.c:15: ./include/linux/ip_vs.h:14:36: fatal error: libnl3/netlink/netlink.h: No such file or directory #include <libnl3/netlink/netlink.h> ^ compilation terminated. make[2]: *** [ofp_vs-main.o] Error 1 make[2]: Leaving directory /opt/work/ofp/example/ofp_vs'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/opt/work/ofp/example'
make: *** [all-recursive] Error 1

• step1:
  I visited https://dragon.linux-vs.org/~dragonfly/htm/lvs_freebsd.htm
  and want to lasted ipvs demo on freebsd。but did not visited it.
  

• step2:
  http://www.freebsdsoftware.org/net/ipvs.html

can you help me ,thanks!

make[2]: Entering directory /opt/work/ofp/example/ofp_vs' CCLD ofp_vs ofp_vs-ofp_vs_ctl.o: In function ip_vs_genl_fill_stats':
/opt/work/ofp/example/ofp_vs/ofp_vs_ctl.c:1792: undefined reference to nla_nest_cancel' ofp_vs-ofp_vs_ctl.o: In function ip_vs_genl_fill_service':
/opt/work/ofp/example/ofp_vs/ofp_vs_ctl.c:1847: undefined reference to nla_nest_cancel' ofp_vs-ofp_vs_ctl.o: In function ip_vs_genl_fill_laddr':
/opt/work/ofp/example/ofp_vs/ofp_vs_ctl.c:2004: undefined reference to nla_nest_cancel' ofp_vs-ofp_vs_ctl.o: In function ip_vs_genl_dump_dest':
/opt/work/ofp/example/ofp_vs/ofp_vs_ctl.c:2158: undefined reference to nla_nest_cancel' collect2: error: ld returned 1 exit status make[2]: *** [ofp_vs] Error 1 make[2]: Leaving directory /opt/work/ofp/example/ofp_vs'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/opt/work/ofp/example'
make: *** [all-recursive] Error 1

刚才把ofp_vs重新编译了下，跑了下，这是日志
`root@localhost ofp_vs]# ./ofp_vs -i 0,1 -c 2 -o 0 -p 1 -f ofp.conf
RLIMIT_CORE: 0/-1
Setting to max: 0

ODP system info

ODP API version: 1.14.0
CPU model:
CPU freq (hz): 3407872000
Cache line size: 0
Core count: 0
Outer port: 0
Inner port: 1

Running ODP appl: "ofp_vs"

IF-count: 2
Using IFs: 0 1

EAL: Detected 2 lcore(s)
EAL: No free hugepages reported in hugepages-1048576kB
EAL: Probing VFIO support...
EAL: PCI device 0000:02:01.0 on NUMA socket -1
EAL: probe driver: 8086:100f net_e1000_em
EAL: PCI device 0000:02:01.0 on NUMA socket -1
EAL: probe driver: 8086:100f net_e1000_em
EAL: PCI device 0000:02:05.0 on NUMA socket -1
EAL: probe driver: 8086:100f net_e1000_em
EAL: PCI device 0000:02:06.0 on NUMA socket -1
EAL: probe driver: 8086:100f net_e1000_em
../linux-generic/_ishmphy.c:150:_odp_ishmphy_map():mmap failed:Cannot allocate memory
../linux-generic/_ishm.c:866:_odp_ishm_reserve():No huge pages, fall back to normal pages. check: /proc/sys/vm/nr_hugepages.
../linux-generic/_ishmphy.c:150:_odp_ishmphy_map():mmap failed:Cannot allocate memory
../linux-generic/_ishmphy.c:150:_odp_ishmphy_map():mmap failed:Cannot allocate memory
../linux-generic/_ishmphy.c:150:_odp_ishmphy_map():mmap failed:Cannot allocate memory
PKTIO: initialized loop interface.
../linux-generic/_ishmphy.c:150:_odp_ishmphy_map():mmap failed:Cannot allocate memory
No crypto devices available
../linux-generic/_ishmphy.c:150:_odp_ishmphy_map():mmap failed:Cannot allocate memory
../linux-generic/_ishmphy.c:150:_odp_ishmphy_map():mmap failed:Cannot allocate memory
odp_cpu_count : 2
Num worker threads: 1
first CPU: 1
cpu mask: 0x2
../linux-generic/_ishmphy.c:150:_odp_ishmphy_map():mmap failed:Cannot allocate memory
../linux-generic/_ishmphy.c:150:_odp_ishmphy_map():mmap failed:Cannot allocate memory
../linux-generic/_ishmphy.c:150:_odp_ishmphy_map():mmap failed:Cannot allocate memory
I 5 1:1536392960 ofp_uma.c:45] Creating pool 'udp_inpcb', nitems=1024 size=904 total=925696
I 5 1:1536392960 ofp_uma.c:45] Creating pool 'tcp_inpcb', nitems=2048 size=904 total=1851392
I 5 1:1536392960 ofp_uma.c:45] Creating pool 'tcpcb', nitems=2048 size=784 total=1605632
I 5 1:1536392960 ofp_uma.c:45] Creating pool 'tcptw', nitems=409 size=80 total=32720
I 5 1:1536392960 ofp_uma.c:45] Creating pool 'syncache', nitems=30720 size=168 total=5160960
I 6 1:1536392960 ofp_uma.c:45] Creating pool 'tcpreass', nitems=320 size=48 total=15360
I 6 1:1536392960 ofp_uma.c:45] Creating pool 'sackhole', nitems=65536 size=40 total=2621440
I 7 1:1536392960 ofp_init.c:245] Slow path threads on core 0
I 7 1:1536392960 ofp_ifnet.c:143] Device '0' addr 00:0c:29:7d:2e:ba
I 7 1:1536392960 ofp_ifnet.c:152] Device '0' MTU=1500
odp_thread.c:166:odp_thread_init_local():There is a thread already running on core 0
I 12 1:1536392960 main.c:369] if 0 rx_q 0 bind to cpu 1
I 12 1:1536392960 ofp_ifnet.c:143] Device '1' addr 00:0c:29:7d:2e:c4
I 12 1:1536392960 ofp_ifnet.c:152] Device '1' MTU=1500
odp_thread.c:166:odp_thread_init_local():There is a thread already running on core 0
odp_thread.c:166:odp_thread_init_local():There is a thread already running on core 0
odp_thread.c:166:odp_thread_init_local():There is a thread already running on core 0
I 16 1:1536392960 main.c:369] if 1 rx_q 0 bind to cpu 1
odp_thread.c:166:odp_thread_init_local():There is a thread already running on core 0
odp_thread.c:166:odp_thread_init_local():There is a thread already running on core 1
../linux-generic/odp_timer.c:687:timer_notify():
1 ticks overrun on timer pool "TmrPool", timer resolution too high
I 20 1:1536392960 ofp_vs_ctl.c:3139] ofp_vs_ctl_init ok

I 20 1:1536392960 ofp_vs_proto.c:217] Registered protocols (TCP, UDP, ICMP)

Segmentation fault (core dumped)
`