Create an end-to-end testing harness to facilitate automated regression testing of VFd feature combinations, e.g. strip/no-strip, VLAN filter/MAC filter, and BUM. The harness will produce synthetic packets, pass them through a NIC, and compare what is received on the receiver side under all combinations of the above settings.

VFD take away the capability of running TCPDUMP , which i think is a MUST have function while trying to troubleshoot.

After the deletion of heat stack, the json files were no longer present in the vfd/config directory.
But the 'iplex show all' still showed the VF's in the output in 'DOWN' state.

And below is the vfd output that states it failed due to the below check:

So changing that condition to vfc->vfid < 0 should solve this?

Vfd.log

1487015211 2017/02/13 19:46:51Z [1] deleting vf from file: /var/lib/vfd/config/94d61f92-2c4b-436d-baaf-6a416557ae8d.json
1487015211 2017/02/13 19:46:51Z [2] del: config data: name: Instance_admin-vmx-contrail-1_fpc0/94d61f92-2c4b-436d-baaf-6a416557ae8d
1487015211 2017/02/13 19:46:51Z [2] del: config data: pciid: 0000:02:00.0
1487015211 2017/02/13 19:46:51Z [2] del: config data: vfid: 0
1487015211 2017/02/13 19:46:51Z [1] vfd_del_vf failed: unable to read config file: /var/lib/vfd/config/94d61f92-2c4b-436d-baaf-6a416557ae8d.json
1487015211 2017/02/13 19:46:51Z [2] sending response: /tmp/IPLEX_32497(23) [1] 173 bytes
1487015211 2017/02/13 19:46:51Z [2] response written to pipe
1487015213 2017/02/13 19:46:53Z [2] raw message: ({"action": "delete", "params": {"loglevel": 0, "resource": null, "r_fifo": "/tmp/IPLEX_32500", "filename": "/var/lib/vfd/config/d5e5cac4-14e9-463c-8639-f854488a360a.json"}}
)
1487015213 2017/02/13 19:46:53Z [1] deleting vf from file: /var/lib/vfd/config/d5e5cac4-14e9-463c-8639-f854488a360a.json
1487015213 2017/02/13 19:46:53Z [2] del: config data: name: Instance_admin-vmx-contrail-1_fpc0/d5e5cac4-14e9-463c-8639-f854488a360a
1487015213 2017/02/13 19:46:53Z [2] del: config data: pciid: 0000:02:00.1
1487015213 2017/02/13 19:46:53Z [2] del: config data: vfid: 0
1487015213 2017/02/13 19:46:53Z [1] vfd_del_vf failed: unable to read config file: /var/lib/vfd/config/d5e5cac4-14e9-463c-8639-f854488a360a.json

... should the consequent of this if be bracketized?

It is possible that the poll for a VF's queues to go ready may continue after the configuration file has been deleted via an iplex delete.

VFd user interface needs to ensure that there is no pending queue polling for the VF as it is cleaned up.

the old MAC's are still persistent and in control of VFd even though after updating the MAC's list in the config.

Currently the version is generated manually and cannot be associated with a commit. The version number needs to be generated automatically and capable of being directly related to a specific commit.

If iplex sends a request which is not recognised by VFd, there is no response returned to iplex and the command just hangs.

When a VF config has a PCIID address that does not exist, or is not available to VFd, VFd core dumps as it tries to recover.

Some terminals (Putty?) is sending a signal on window resize which is reaching VFd, and causing VFd to terminate.

Firmware bug was suspect by Intel. However, a software fix was later discovered, and provided by Intel. This patch will be upstreamed in 1705?

In the case where the igb_uio driver is being used inside of a VM with a DPDK application, the status that is presented by VFd for the VF is 'DOWN' while the actual state is up and functioning normally. This is not happening in all environments.

A request has been made for VFd to capture the name given in the configuration json, and to ensure that the name matches before using the configuration to reset the PF/VF combination. This prevents the "user" from accidentally submitting the wrong configuration which has a PF/VF combination that duplicates another.

This in itself is not a foolproof mechanism to prevent the reset of a PF/VF as there exists the possibility for a race condition if two configuration files, both of which specify the same PF/VF combination, exist when VFd is started.

Add a feature to configure mirroring of traffic from one VF to another. It should be possible to specify a VF to mirror and a list of VLAN tags on that VF. An output VF and optionally VLAN tag must also be specified.

Iplex requires sudo privledges to submit a request because the fifo that VFd listens to is owned by root and is not writable by the world. It would be nice to have a way that a normal user could run the iplex show, ping, and possibly dump requests without having to grant them sudo authorisation.

The current limitation is that iplex is written in python which does not allow it to be executed as a setuid programme which would allow a normal user to run show/ping commands. A stand alone binary, which could be installed as suid owned by root, needs to be created in order to allow this.

Currently if an add request is received for a VF/PF combination that is already in use VFd will reject the request. This is an enhancement request which asks that the parameters be examined and the add accepted if they are all the same.

Justification: some virtualisation managers seem to attempt to add a configuration when a guest is rebooted without first deleting it; it is felt that this is not a failure and thus VFd should not reject the request.

Packet loss when using VFd through a switch has been reported. No packet loss is seen when NIC is connected back to back to another NIC.

Root cause is under investigation.

The PCIID listed isn't correct; should be all odd or all even, but is not:

PF/VF ID PCIID Link...
pf 0 0000:08:00.0 UP ...
vf 1 0000:08:10.1 DOWN...
vf 2 0000:08:10.2 DOWN...
vf 3 0000:08:10.3 UP ...
vf 4 0000:08:10.4 DOWN...
vf 5 0000:08:10.5 DOWN...
vf 6 0000:08:10.6 DOWN...
vf 7 0000:08:10.7 DOWN...
vf 8 0000:08:11.0 DOWN..
:

The actual number of vfs (port->nvfs_config) can be bigger than the configured number
(port->num_vfs) so target id can also be bigger.

For example: PF has 16 VFs, but only vfs 13 and 14 are configured with json files and vf 13 is has mirror settings targeting vf 14. This configuration will fail since port->num_vfs = 2 and the check is target_id > port->num_vfs instead of target_id > port->nvfs_config.

With Niantic nic, we are seeeing a .04% packet loss rate (bare metal) when the queue drop enable and split drop enable bits are set.

Eg, "vfid": "-13"

It would be nice to have a script at the top most directory which builds everything.

The VF configuration should be reset to default/initial state when VFD is terminating due to sigterm.

When the physical interface link goes down, and an 'iplex show all' is executed, the state for all VFs under the PF continues to show 'UP'. The state for the PF shows correctly.

If the VFs created on a PF is less than 32, the queue checking mechanism is getting it wrong and as a result the status for a VF is likely to be incorrect and we don't detect when the queue becomes ready which potentially prevents us from 'activating' the VF.

Hi,
I'm getting below error upon enabling mirroring , i keep getting this error at step 5 of" Changing Mirroring State "[iplex add] in (https://github.com/att/vfd/wiki/Debugging-Tricks):

iplex add 86d13cde-4a29-41e1-8956-4b62424431dd

{ "state": "ERROR", "msg": "unable to add vf: /var/lib/vfd/config/86d13cde-4a29-41e1-8956-4b62424431dd.json: mirror target is out of range or is the same as this VF (28): 29" }

• Also i tried to sudo iplex mirror ,but I can not see traffic in target VM [50.50.50.52_Mirroring_Guest]

problematic VF:

cat /var/lib/vfd/config/86d13cde-4a29-41e1-8956-4b62424431dd.json
{"strip_stag": true, "macs": [], "name": "MO_TEST_VM53/86d13cde-4a29-41e1-8956-4b62424431dd", "link_status": "auto", "pciid": "0000:83:00.0", "allow_bcast": true, "vfid": 28, "mac_anti_spoof": true, "vlan_anti_spoof": true, "allow_mcast": true, "insert_stag": false, "vlans": [1000], "allow_un_ucast": true,"mirror": { "target": 29, "direction": "all" }}

Target VF :

cat /var/lib/vfd/config/8e442a16-4e93-4162-8dac-83a8853a0cc5.json
{"strip_stag": true, "macs": [], "name": "MO_TEST_VM52_mirror/8e442a16-4e93-4162-8dac-83a8853a0cc5", "link_status": "auto", "pciid": "0000:83:00.0", "allow_bcast": true, "vfid": 29, "mac_anti_spoof": true, "vlan_anti_spoof": true, "allow_mcast": true, "insert_stag": false, "vlans": [1000], "allow_un_ucast": false}

===================Logs & Info========================
50.50.50.52_Mirroring_Guest:

iplex show all|grep 0000:83:17.2

vf 29 0000:83:17.2 UP 0 0 0 0 107 32846 0 0

#:~# nova interface-list c2fc40cd-d529-43aa-bebd-b8cc691c82f0
+------------+--------------------------------------+--------------------------------------+--------------+-------------------+
| Port State | Port ID | Net ID | IP addresses | MAC Addr |
+------------+--------------------------------------+--------------------------------------+--------------+-------------------+
| ACTIVE | 8e442a16-4e93-4162-8dac-83a8853a0cc5 | 4eb7e90b-b026-4857-b1f5-098ce144627e | 50.50.50.52 | 02:8e:44:2a:16:4e |
+------------+--------------------------------------+--------------------------------------+--------------+-------------------+

cat /var/lib/vfd/config/8e442a16-4e93-4162-8dac-83a8853a0cc5.json
{"strip_stag": true, "macs": [], "name": "MO_TEST_VM52_mirror/8e442a16-4e93-4162-8dac-83a8853a0cc5", "link_status": "auto", "pciid": "0000:83:00.0", "allow_bcast": true, "vfid": 29, "mac_anti_spoof": true, "vlan_anti_spoof": true, "allow_mcast": true, "insert_stag": false, "vlans": [1000], "allow_un_ucast": false}

50.50.50.52_Mirroring_Guest:

iplex show all|grep 0000:83:17.2

vf 29 0000:83:17.2 UP 0 0 0 0 107 32846 0 0

#:~# nova interface-list c2fc40cd-d529-43aa-bebd-b8cc691c82f0
+------------+--------------------------------------+--------------------------------------+--------------+-------------------+
| Port State | Port ID | Net ID | IP addresses | MAC Addr |
+------------+--------------------------------------+--------------------------------------+--------------+-------------------+
| ACTIVE | 8e442a16-4e93-4162-8dac-83a8853a0cc5 | 4eb7e90b-b026-4857-b1f5-098ce144627e | 50.50.50.52 | 02:8e:44:2a:16:4e |
+------------+--------------------------------------+--------------------------------------+--------------+-------------------+

cat /var/lib/vfd/config/8e442a16-4e93-4162-8dac-83a8853a0cc5.json
{"strip_stag": true, "macs": [], "name": "MO_TEST_VM52_mirror/8e442a16-4e93-4162-8dac-83a8853a0cc5", "link_status": "auto", "pciid": "0000:83:00.0", "allow_bcast": true, "vfid": 29, "mac_anti_spoof": true, "vlan_anti_spoof": true, "allow_mcast": true, "insert_stag": false, "vlans": [1000], "allow_un_ucast": false}

In the case where there are no MAC addresses listed in the config file, and a guest adds a MAC to the white list, VFd will incorrectly attempt to set one too many MAC addresses into the white list.

When there exist, at VFd start up time, two VF configuration files which reference the same PF/VF combination the guest's communication (Tx) is affected. Packets can be observed coming into the guest (via tcpdump), but responses do not pass through the NIC.

Support for multiple NICs: develop harness to dynamically detect which NIC is installed, and call appropriate PMD functions for that NIC.

Add support for the Intel FVL-25 Fortville NIC using within the multi-NIC framework.

During initialisation the number of ports reported by DPDK is checked with what is expected in the config file and errors/warnings are issued based on what is expected vs what is found. If less ports are available than expected a warning is issued and execution continues. If more ports are reported than expected a critical message, indicating abort is to be performed, is written, but the abort doesn't happen.

This section of main.c needs to be examined and cleaned up. My suggestion is to issue a critical error and abort if the number of reported devices does NOT match the number of devices in the config as we have specifically whitelisted all of the devices in the config and if one is missing that is not a state that I think we should continue with.

Code in main.c is about line 1598.

Looks like dpdk-16.11 missing ping_vfs function for Niantic (17.02 has it for i40e)
also parameter have to be passed from configuration file to control VF notification

VFd needs to allow users to provide a MAC address for a VF. This requirement is to support nova integration where MAC is supplied by nova. The VF must use the supplied MAC address whenever the VF is created or reinitialized. However, to support VF teaming, VFd should also allow the VM to override this MAC address and change it to something else.

If a guest pushes a default MAC address, then pushes one or more whitelist addresses, a hole is created at index [1]:

1528385274 2018/06/07 15:27:54Z [2] dump: pf/vf: 0/5 vlan[0] 21
1528385274 2018/06/07 15:27:54Z [2] dump: pf/vf: 0/5 mac[0] fa:ce:ed:09:00:05
1528385274 2018/06/07 15:27:54Z [2] dump: pf/vf: 0/5 mac[1]
1528385274 2018/06/07 15:27:54Z [2] dump: pf/vf: 0/5 mac[2] fa:ce:ed:09:9a:05
1528385274 2018/06/07 15:27:54Z [2] dump: pf/vf: 0/5 mac[3] fa:ce:ed:09:9b:05
1528385274 2018/06/07 15:27:54Z [2] dump: pf/vf: 0/5 mac[4] fa:ce:ed:09:9c:05

To support the use of the 'set mac-vlan' callback which allows a guest to add MAC addresses to the white list in real-time, the capability of setting a static list of MAC addresses in the VFs config file may break.

Specifically, when a guest virtual machine initialises the vNIC a 'reset' is received (MAC address of all zeros) which causes VFd to clear the list of MAC addresses added to the VF. This behavour was added to emulate the behavour of various drivers and is correct. However, the feature of allowing MAC addresses to be supplied via the VF configuration is broken with this as the MAC addresses are cleared and as they were no t initially inserted by the guest, they will not be replaced.

To return VFd to support the ability to supply 'white list' MAC addresses in the VF configuration file, the following change is being proposed:

If the VF config contains one or more MAC addresses in the array, the first will be used as the default MAC address (visible to the guest) and the remainder will be added to the white list (guest will receive packets with these MAC addresses as the dest, and will be allowed to send packets with these MAC addresses as the source. Any set mac-vlan callbacks will be ignored by VFd.

When the VF config has an empty array for MAC addresses, or the field is missing from the config, VFd will cause a random MAC address to be defined as the default, and set mac-vlan callbacks will be processed allowing the guest to add/delete MAC addresses from the white list.

Please comment.

Several API calls have moved from the eth_dev area to the ixgbe PMD portion of the code with 17.02 and thus to build with 17.02 the following changes will need to be made to VFd.

The following five APIs for VF management from the PF have been removed from the ethdev, renamed and added to the ixgbe PMD:
rte_eth_dev_set_vf_rate_limit
rte_eth_dev_set_vf_rx
rte_eth_dev_set_vf_rxmode
rte_eth_dev_set_vf_tx
rte_eth_dev_set_vf_vlan_filter

The API’s have been renamed to the following:
rte_pmd_ixgbe_set_vf_rate_limit
rte_pmd_ixgbe_set_vf_rx
rte_pmd_ixgbe_set_vf_rxmode
rte_pmd_ixgbe_set_vf_tx
rte_pmd_ixgbe_set_vf_vlan_filter

If a VF configuration file has a nil ("") string as an element in the MAC address array VFd segfaults and aborts.

If a "user" places multiple configuration files into the VFd config directory while VFd is not running, and those configuration files all reference the same PF/VF combination, VFd will select one during startup to use as the configuration for the PF/VF. There are several possible, unpleasant, outcomes of this that need to be addressed:

1. The 'unused' configuration files are left to rot, and could inadvertently be used to remove the configuration which would impact a guest/application currently using the VF. If the correct configuration file is selected, when it is removed/deleted, the rotten file could then be used to configure the VF (if VFd is restarted) allowing for a configured, but unused, VF (probably not harmful, but misleading from a stats perspective).

2. The method of selecting the configuration file for use is non-deterministic (to some degree) and as such the wrong/old configuration could be applied to the VF.

A mechanism must be put into place which prevents this.

sometimes "iplex show all" is showing PF speed and duplex fields as 0

gcc complains about implicit fallthrough here:

@ScottDaniels - Not clear whether the share percentage in the VF configuration is related to the PF TC min_bw configuration.

For example: TC0 in the PF QoS cfg has min_bw of 50% of link speed. Does that mean that if VF0 has qshare configuration of 10% in TC0 it will get 5% of the link speed?
From the code it seems that VF TC configuration doesn't care of the PF configuration at all (Unless I missed something.

in vfd_rif.c qshares array generation is wrong when no normalization is needed.
for( j = i; j < port->num_vfs; j++ ) {

should be

for( j = 0; j < port->num_vfs; j++ ) {

If a delete request is sent to VFd, and the configuration file isn't fully qualified, the delete will fail because it isn't looking in the correct place (live directory). This isn't a problem if iplex is used to send the delete request as it always builds a fully qualified filename.

Support of base names only is the intended direction for containerised VFd, and thus this needs to be fixed.

When an attempt to add a VF with multiple MAC addresses is made, it is rejected.

There is currently an issue when mcast is set to true, and un-ucast is set to false, and the reverse. The symptom is that setting one to false seems to cause both to be set to false. I've verified that after updating the NIC with the desired values, reading the values back from the NIC show the expected settings, so this might not be a VFd bug, but an oddity of the underlying hardware.

in sriov.h
line 152 should be .jumbo_frame = 1,

I have seen cases where the loopback setting is turned off for the port when a physical link goes down and then up. I was not able to force this to happen all of the time.

We need to add a patch directory which contains patches to DPDK which must be applied to the current base DPDK at vfd build time.

Feature Request:

At present it is not possible to clear the packet/byte counters reported by VFd. This would be a useful feature.

VFd is currently setting the hardware based stripping of the CRC to false. It needs to be true.