haggaie / rdma-experiment

RDMA experiment VMs, instructions, and example.

RDMA experiment

See the experiment instructions (Hebrew).

Hebrew editing instructions

To edit hebrew Markdown on Windows, Notepad++ can be used, along with the markdown-plus-plus syntax highlighting plugin. To change text direction when editing use the Ctrl+Alt+L and Ctrl+Alt+R shortcut keys.

On Linux, gedit can be used.

VM installation instructions

You will need the following tools:

• VirtualBox - a desktop virtualization system.
• Vagrant - a scripted tool for managing virtual machines.

To create the virtual machines use:

vagrant up

This will create a virtual machine with the SoftRoCE RDMA driver enabled on its virtual network interface. To access the machine, use:

vagrant ssh

The repository files should be available inside the VM under the /vagrant directory.

RDMA functionality should be available on the VM's Ethernet interface, which vagrant normally configures with the IP address 10.0.2.15.

rdma-experiment's Issues

Move to Ubuntu Focal 20.04

Move the packer image to use focal and update the rest of the dependencies accordingly.

One benefit from using 20.04 would be that the issue that required logging the serial console to a file (https://bugs.launchpad.net/cloud-images/+bug/1874453) should be solved.

bash: line 4: /sbin/ifdown: No such file or directory

It is reported at cilium/cilium#1918. Seems like ifupdown need be installed at provision script.

Ubuntu: '20180210.0.0
Virtualbox: 5.2.2 r119230 (Qt5.6.3)
MacOS: 10.13.3 (17D47)

Vagrat script creates 2VM or Just One?

Is the Vagrant Script create 2 VM or Just one?
It would be nice if actually 2 VM are created for better understanding of IB verb programming

validate send failed

Thanks for making the great environment to ease development.

I'm trying to build and run client/server application, but got below failure. Do you have any idea on why it failed?

vagrant@ubuntu-bionic:/vagrant/src/build$ ./rdma_client -s 192.168.0.11
rdma_client: start
validate send failed
rdma_post_send: Unknown error -22
rdma_client: end -1

vagrant@ubuntu-bionic:/vagrant/src/build$ sudo rxe_cfg status
  Name    Link  Driver  Speed  NMTU  IPv4_addr  RDEV  RMTU
  enp0s3  yes   e1000
  enp0s8  yes   e1000                           rxe0  1024  (3)

ibv_rc_pingpong works fine btw client and server

Eliminate uart modifyvm line in Virtualbox

Need to modify the resulting box that packer generates so that it doesn't hard-code /dev/null as the uart filename. See https://bugs.launchpad.net/cloud-images/+bug/1874453/comments/53

Client is connects to server when client puts its own IP address?with Server IP it refuses to connect