Implement `dry-run` test in the CI about robots-configuration

[pattacini]

I've started off with the development in https://github.com/robotology/robots-configuration/tree/466-implement-dry-run-test-in-the-ci.

In the CI, for the moment, I'm sticking to manually checking out the development branches. I'll have to manage the correct versioning among the dependencies (e.g., yarp, icub-main...) and robots-configuration smartly later on.

Only the ETH robots can be verified via a dry-run. We need to come back to this later as well.

To enable the dry-run on a ETH robot, it suffices to follow these instructions¹ (thanks @davidetome):

• #321 (comment)

This is an example of the dry-run executed on iCubGenova11 via

>> yarprobotinterface > log.txt 2>&1

whose output is log.txt.

The return code is:

>> echo $?
134

Therefore, it may be better to parse the content of the log, in search of keywords like "unknown plugin" and the like.

Finally, the idea is to create a suite of CMake tests to automatize this check for all the ETH robots installed.

Footnotes

1. Setting skipCalibration to true is not required, actually. ↩

[pattacini]

Aligned F2F with @traversaro too.
There are 2 dry-run:

• yarprobotinterface --dryrun.
• the dry-run of the embObj devices, which is the one I've been focusing so far.

[pattacini]

The failure upon closure is due to cartesian/right_arm-cartesian.xml, whereas cartesian/left_arm-cartesian.xml is just fine (weird).

However, even when yarprobotinterface shuts down gracefully, its return code is 1. Probably, there are other reasons for that. Need for inquiring about the log.

[pattacini]

Today, I've reached a status where I can handle CAN vs. ETH robots via the ctest infrastructure.

The output of the dry-run is not parsed yet.
In the meantime, I've realized that iCubErzelli03 just hangs up: investigation is needed here.

Opened a draft PR:

• #471

[traversaro]

The dry-run is meaningful only for the embObj devices. Thus, I'll disable the inertial.

Do you have any idea why? I tought the dry-run was not actually running the devices, so I am a bit surprised that a device is actually being run, but probably I actually misunderstood what dry-run is doing.

See robotology/yarp#2752 for the context from which I understood that "dryrun" was not actually opening the devices.

[pattacini]

R1SN000 removed as it is no longer maintained, and no one uses it (it still relies on controlboardwrapper2). This robot will be still accessible through the git history.

Latest status:

The following tests FAILED:
         25 - dry-run_iCubGenova04 (Failed)
         29 - dry-run_iCubGenova09 (Failed)
         34 - dry-run_iCubHeidelberg01 (Failed)
         42 - dry-run_iCubLisboa01 (Failed)
         47 - dry-run_iCubMoscow01 (Failed)
         62 - dry-run_iCubShanghai01 (Failed)
         73 - dry-run_iRonCub-Mk1 (Failed)

[pattacini]

Nice idea from @traversaro 💡
I'll take a look at it starting the next sprint.

cc @Nicogene

[traversaro]

The return code is:

>> echo $?
134

Therefore, it may be better to parse the content of the log, in search of keywords like "unknown plugin" and the like.

Note that at the moment the dryrun do not check the existing of the plugin mentioned in the xml, at least this is what I understood, but I may be wrong. However, it seems that the yarprobotinterface is crashing:

[ERROR] yarprobotinterface intercepted a segmentation fault caused by a faulty plugin:
[ERROR] /workspace/install/bin/../lib/libYARP_os.so.3(_ZN4yarp2os6BottleD2Ev+0x67) [0x7f416934a397]
Trace requested at /workspace/yarp/src/yarprobotinterface/Module.cpp:77 by code called from:
/workspace/install/bin/../lib/libYARP_os.so.3(_Z16yarp_print_traceP8_IO_FILEPKcj+0x3b) [0x7f416935733b]
yarprobotinterface(+0x9c84) [0x5625654d0c84]
/lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f4168e8b520]
/workspace/install/bin/../lib/libYARP_os.so.3(_ZN4yarp2os6BottleD2Ev+0x67) [0x7f416934a397]
/workspace/install/bin/../lib/libYARP_os.so.3(+0xc2c4a) [0x7f416938ac4a]
/workspace/install/bin/../lib/libYARP_os.so.3(+0xc2c10) [0x7f416938ac10]
/workspace/install/bin/../lib/libYARP_os.so.3(_ZN4yarp2os8PropertyD2Ev+0x75) [0x7f416938ad05]
/workspace/install/bin/../lib/libYARP_dev.so.3(+0xa98e6) [0x7f4168de98e6]
/workspace/install/bin/../lib/libYARP_dev.so.3(_ZN4yarp3dev10PolyDriver5closeEv+0x4c) [0x7f4168de74fc]
/workspace/install/bin/../lib/libYARP_robotinterface.so.3(_ZN4yarp14robotinterface6Device5closeEv+0x48) [0x7f41694cb898]
/workspace/install/bin/../lib/libYARP_robotinterface.so.3(_ZN4yarp14robotinterface5Robot7Private12closeDevicesEv+0x2fa) [0x7f41694d9f4a]
/workspace/install/bin/../lib/libYARP_robotinterface.so.3(_ZN4yarp14robotinterface5Robot10enterPhaseENS0_11ActionPhaseE+0x29a4) [0x7f41694e2c24]
yarprobotinterface(+0xbcbd) [0x5625654d2cbd]
/workspace/install/bin/../lib/libYARP_os.so.3(_ZN4yarp2os8RFModule9runModuleERNS0_14ResourceFinderE+0xf7) [0x7f41693b47c7]
yarprobotinterface(+0x8dbf) [0x5625654cfdbf]
/lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f4168e72d90]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0x80) [0x7f4168e72e40]
yarprobotinterface(+0x8e85) [0x5625654cfe85]

So probably if the yarprobotinterface did not crash the output would be 0?

[pattacini]

Fixed the cartesiancontrollerserver closure via robotology/icub-main@537df47.

[pattacini]

Things to do:

1. Create the ctest infrastructure.
2. Edit the conf files upon installation to enable dry-run.
3. Parse the output of yarprobotinterface.
4. Manage CAN vs. ETH robots.
5. Manage in the CI the correct versions of the deps wrt robots-configuration.

[pattacini]

I've realized that iCubErzelli03 just hangs up: investigation is needed here.

Actually, iCubErzelli03 was running just fine. The calibration was taking up some time, hence I had to disable it, eventually¹. Thus, yarprobotinterface was running happily without closing up, and I had to modify the test to manage the graceful shutdown.

After all, it turned out that for many ETH robots the dry-run doesn't execute flawlessly as it happens for iCubErzelli03 or iCubHamburg01², for instance.

Footnotes

1. Therefore, I stopped relying on sed and switched to xmlstarlet for editing XML files. ↩

2. These robots do have the head only. ↩

[pattacini]

5. Manage in the CI the correct versions of the deps wrt robots-configuration.

To keep things simple, I went for:

• Sticking to icub-main@devel and configuring ycm, yarp, icub-firmware-shared versions accordingly.
• Running the dry-run once per week (and upon manual trigger).

This way, we can test a specific line of development only, but it should be sufficient for our purposes.
Let's see how it goes to then refine the dry-run further if required.

I need to merge the PR to make the new CI available for testing purposes.
Also, the dry-run will publish the reports as a zipped artifact.

The last thing to address is the parsing of yarprobotinterface output.

[pattacini]

With the latest changes, I managed to further narrow down the failures to:

The following tests FAILED:
          1 - dry-run_R1SN000 (Failed)
          2 - dry-run_R1SN001 (Failed)
          3 - dry-run_R1SN002 (Failed)
          4 - dry-run_R1SN003 (Failed)
         26 - dry-run_iCubGenova04 (Failed)
         30 - dry-run_iCubGenova09 (Failed)
         35 - dry-run_iCubHeidelberg01 (Failed)
         43 - dry-run_iCubLisboa01 (Failed)
         48 - dry-run_iCubMoscow01 (Failed)
         63 - dry-run_iCubShanghai01 (Failed)
         74 - dry-run_iRonCub-Mk1 (Failed)

In detail, I had to:

• disable ros and ros2 devices.
• disable wholebodydynamics device.
• handle the specific case of cfw2 device.
• fix some XML files missing the xmlns:xi="http://www.w3.org/2001/XInclude" attribute.

⚠️ There are references to batteryWrapper in some robot XML files, which I suppose has been deprecated (and removed in yarp-3.8.0) in favor of battery_nws_yarp. Thereby, we need to patch some robot files.

[pattacini]

I've added also the option for the native yarprobointerface --dryrun, discovering several errors due to missing files.

Concerning the check related to the devices, there are still errors due to the use of batteryWrapper (see #477), some missing remapper/wrapper and finally some errors related to iRonCub-Mk1 because of the deprecated controlboardwrapper2.

We will need to address the whole bunch of these errors relying on the CI in the upcoming days¹.

In the future, we may consider refactoring the CI to rely on robotology-superbuild, although the latter doesn't include yet all the required deps and it's not easy to tweak the versioning.

Footnotes

1. The CI is performed once a week and upon a manual trigger, and will publish the test outcome as an artifact to download. ↩

[pattacini]

As agreed with @traversaro, iRonCub-Mk1 has been removed via b7c437a.

[pattacini]

I finally managed to make the CI run flawlessly.
Of course, we do still have errors to fix in the XML files: we'll do that incrementally starting the next sprint.

The state of play:

The following tests FAILED:
	  3 - dry-run_R1SN002_xml (Failed)
	 13 - dry-run_iCubAberystwyth01_xml (Failed)
	 15 - dry-run_iCubAbuDhabi01_xml (Failed)
	 17 - dry-run_iCubAnkara01_xml (Failed)
	 21 - dry-run_iCubBielefeld01_xml (Failed)
	 23 - dry-run_iCubBielefeld02_xml (Failed)
	 25 - dry-run_iCubBielefeld03_xml (Failed)
	 41 - dry-run_iCubFrankfurt01_xml (Failed)
	 49 - dry-run_iCubGenova04_xml (Failed)
	 50 - dry-run_iCubGenova04_dev (Failed)
	 58 - dry-run_iCubGenova09_dev (Failed)
	 63 - dry-run_iCubGrenoble01_xml (Failed)
	 67 - dry-run_iCubHeidelberg01_xml (Failed)
	 68 - dry-run_iCubHeidelberg01_dev (Failed)
	 69 - dry-run_iCubHertfordshire01_xml (Failed)
	 71 - dry-run_iCubHertfordshire02_xml (Failed)
	 75 - dry-run_iCubKarlsruhe01_xml (Failed)
	 77 - dry-run_iCubLausanne01_xml (Failed)
	 81 - dry-run_iCubLethbridge01_xml (Failed)
	 83 - dry-run_iCubLisboa01_xml (Failed)
	 84 - dry-run_iCubLisboa01_dev (Failed)
	 85 - dry-run_iCubLisboa02_xml (Failed)
	 94 - dry-run_iCubMoscow01_dev (Failed)
	 95 - dry-run_iCubMunich01_xml (Failed)
	101 - dry-run_iCubOffenbach01_xml (Failed)
	103 - dry-run_iCubOffenbach02_xml (Failed)
	105 - dry-run_iCubParis01_xml (Failed)
	107 - dry-run_iCubParis02_xml (Failed)
	109 - dry-run_iCubPisa01_xml (Failed)
	111 - dry-run_iCubPlymouth01_xml (Failed)
	113 - dry-run_iCubPlymouth02_xml (Failed)
	117 - dry-run_iCubPrototype01_xml (Failed)
	119 - dry-run_iCubRome02_xml (Failed)
	124 - dry-run_iCubShanghai01_dev (Failed)
	131 - dry-run_iCubTokyo01_xml (Failed)
	135 - dry-run_iCubUrbana01_xml (Failed)
	143 - dry-run_iCubZurich01_xml (Failed)
	149 - dry-run_vizzy_xml (Failed)

The dry-run logs are available as an artificial to download from the workspace.

In the upcoming days, I'll see how to improve the CI in order to:

• attach it to PR's;
• use the YARP_NO_DEPRECATED cmake flag;
• address the building of some yarp-device-* repos;

[pattacini]

Therefore, it may be better to parse the content of the log, in search of keywords like "unknown plugin" and the like.

I was more referring to any kind of generic error currently handled by the yarprobotinterface. A better example is plugin found but could not open.

For the failure you're right, although yarprobotinterface seg-faults upon closure are not rare in my experience. I'll have a look at it.

[pattacini]

I stumbled upon the following error during a dry-run:

[DEBUG] |yarp.dev.PolyDriver|head-inertial| Parameters are (comport "/dev/ttyUSB0") (device imuBosch_BNO055) (id head-inertial) (robotName icub)
[ERROR] |yarp.device.imuBosch_BNO055| Can't open /dev/ttyUSB0, No such file or directory
[ERROR] |yarp.dev.PolyDriver|head-inertial| Driver <imuBosch_BNO055> was found but could not open

I'm using a gitpod-based sandbox to simulate the CI environment; therefore, no real device plugged in.

Does anyone have experience with running the Bosch unit in dry-run?
cc @davidetome @traversaro @Nicogene @marcoaccame

[pattacini]

Just aligned F2F w/ @davidetome.
The dry-run is meaningful only for the embObj devices. Thus, I'll disable the inertial.

[pattacini]

Once fixed the inertial (see #476), I managed to narrow down the failures:

Test project /workspace/build
      Start  1: dry-run_R1SN000
 1/77 Test  #1: dry-run_R1SN000 ..................***Failed    0.07 sec
      Start  2: dry-run_R1SN001
 2/77 Test  #2: dry-run_R1SN001 ..................***Failed    4.24 sec
      Start  3: dry-run_R1SN002
 3/77 Test  #3: dry-run_R1SN002 ..................***Failed    0.16 sec
      Start  4: dry-run_R1SN003
 4/77 Test  #4: dry-run_R1SN003 ..................***Failed    4.23 sec
      Start  5: dry-run_SkinFingertipDemo
 5/77 Test  #5: dry-run_SkinFingertipDemo ........   Passed    0.02 sec
      Start  6: dry-run_SkinTableTop
 6/77 Test  #6: dry-run_SkinTableTop .............   Passed    0.02 sec
      Start  7: dry-run_ergoCubSN000
 7/77 Test  #7: dry-run_ergoCubSN000 .............***Failed    3.20 sec
      Start  8: dry-run_iCubAberystwyth01
 8/77 Test  #8: dry-run_iCubAberystwyth01 ........   Passed    0.02 sec
      Start  9: dry-run_iCubAbuDhabi01
 9/77 Test  #9: dry-run_iCubAbuDhabi01 ...........   Passed    0.01 sec
      Start 10: dry-run_iCubAnkara01
10/77 Test #10: dry-run_iCubAnkara01 .............   Passed    0.02 sec
      Start 11: dry-run_iCubBarcelona01
11/77 Test #11: dry-run_iCubBarcelona01 ..........   Passed    0.02 sec
      Start 12: dry-run_iCubBielefeld01
12/77 Test #12: dry-run_iCubBielefeld01 ..........   Passed    0.21 sec
      Start 13: dry-run_iCubBielefeld02
13/77 Test #13: dry-run_iCubBielefeld02 ..........   Passed    0.21 sec
      Start 14: dry-run_iCubBielefeld03
14/77 Test #14: dry-run_iCubBielefeld03 ..........   Passed    0.21 sec
      Start 15: dry-run_iCubChemnitz01
15/77 Test #15: dry-run_iCubChemnitz01 ...........***Failed    4.23 sec
      Start 16: dry-run_iCubDarmstadt01
16/77 Test #16: dry-run_iCubDarmstadt01 ..........***Failed   11.32 sec
      Start 17: dry-run_iCubDijon01
17/77 Test #17: dry-run_iCubDijon01 ..............   Passed    0.02 sec
      Start 18: dry-run_iCubEdinburgh01
18/77 Test #18: dry-run_iCubEdinburgh01 ..........   Passed   11.35 sec
      Start 19: dry-run_iCubErzelli01
19/77 Test #19: dry-run_iCubErzelli01 ............   Passed   11.35 sec
      Start 20: dry-run_iCubErzelli02
20/77 Test #20: dry-run_iCubErzelli02 ............   Passed   11.38 sec
      Start 21: dry-run_iCubErzelli03
21/77 Test #21: dry-run_iCubErzelli03 ............   Passed    1.22 sec
      Start 22: dry-run_iCubFrankfurt01
22/77 Test #22: dry-run_iCubFrankfurt01 ..........   Passed    0.31 sec
      Start 23: dry-run_iCubGenova01
23/77 Test #23: dry-run_iCubGenova01 .............   Passed    0.03 sec
      Start 24: dry-run_iCubGenova02
24/77 Test #24: dry-run_iCubGenova02 .............   Passed   11.43 sec
      Start 25: dry-run_iCubGenova03
25/77 Test #25: dry-run_iCubGenova03 .............   Passed    0.03 sec
      Start 26: dry-run_iCubGenova04
26/77 Test #26: dry-run_iCubGenova04 .............***Failed   11.42 sec
      Start 27: dry-run_iCubGenova06
27/77 Test #27: dry-run_iCubGenova06 .............   Passed    0.03 sec
      Start 28: dry-run_iCubGenova07
28/77 Test #28: dry-run_iCubGenova07 .............   Passed   11.47 sec
      Start 29: dry-run_iCubGenova08
29/77 Test #29: dry-run_iCubGenova08 .............   Passed    5.40 sec
      Start 30: dry-run_iCubGenova09
30/77 Test #30: dry-run_iCubGenova09 .............***Failed    5.34 sec
      Start 31: dry-run_iCubGenova10
31/77 Test #31: dry-run_iCubGenova10 .............***Failed   12.43 sec
      Start 32: dry-run_iCubGenova11
32/77 Test #32: dry-run_iCubGenova11 .............***Failed    7.34 sec
      Start 33: dry-run_iCubGrenoble01
33/77 Test #33: dry-run_iCubGrenoble01 ...........   Passed    0.03 sec
      Start 34: dry-run_iCubHamburg01
34/77 Test #34: dry-run_iCubHamburg01 ............   Passed    1.13 sec
      Start 35: dry-run_iCubHeidelberg01
35/77 Test #35: dry-run_iCubHeidelberg01 .........***Failed    6.30 sec
      Start 36: dry-run_iCubHertfordshire01
36/77 Test #36: dry-run_iCubHertfordshire01 ......   Passed    0.03 sec
      Start 37: dry-run_iCubHertfordshire02
37/77 Test #37: dry-run_iCubHertfordshire02 ......   Passed    0.02 sec
      Start 38: dry-run_iCubHongKong01
38/77 Test #38: dry-run_iCubHongKong01 ...........   Passed   11.41 sec
      Start 39: dry-run_iCubKarlsruhe01
39/77 Test #39: dry-run_iCubKarlsruhe01 ..........   Passed    0.27 sec
      Start 40: dry-run_iCubLausanne01
40/77 Test #40: dry-run_iCubLausanne01 ...........   Passed    0.02 sec
      Start 41: dry-run_iCubLausanne02
41/77 Test #41: dry-run_iCubLausanne02 ...........   Passed   11.38 sec
      Start 42: dry-run_iCubLethbridge01
42/77 Test #42: dry-run_iCubLethbridge01 .........   Passed    0.03 sec
      Start 43: dry-run_iCubLisboa01
43/77 Test #43: dry-run_iCubLisboa01 .............***Failed    7.30 sec
      Start 44: dry-run_iCubLisboa02
44/77 Test #44: dry-run_iCubLisboa02 .............   Passed    0.25 sec
      Start 45: dry-run_iCubLondon01
45/77 Test #45: dry-run_iCubLondon01 .............   Passed    0.03 sec
      Start 46: dry-run_iCubLugano01
46/77 Test #46: dry-run_iCubLugano01 .............   Passed    0.03 sec
      Start 47: dry-run_iCubManchester01
47/77 Test #47: dry-run_iCubManchester01 .........   Passed    0.03 sec
      Start 48: dry-run_iCubMoscow01
48/77 Test #48: dry-run_iCubMoscow01 .............***Failed   12.41 sec
      Start 49: dry-run_iCubMunich01
49/77 Test #49: dry-run_iCubMunich01 .............   Passed    0.03 sec
      Start 50: dry-run_iCubNancy01
50/77 Test #50: dry-run_iCubNancy01 ..............***Failed   11.39 sec
      Start 51: dry-run_iCubNottingham01
51/77 Test #51: dry-run_iCubNottingham01 .........***Failed   11.39 sec
      Start 52: dry-run_iCubOffenbach01
52/77 Test #52: dry-run_iCubOffenbach01 ..........   Passed    0.24 sec
      Start 53: dry-run_iCubOffenbach02
53/77 Test #53: dry-run_iCubOffenbach02 ..........   Passed    0.23 sec
      Start 54: dry-run_iCubParis01
54/77 Test #54: dry-run_iCubParis01 ..............   Passed    0.03 sec
      Start 55: dry-run_iCubParis02
55/77 Test #55: dry-run_iCubParis02 ..............   Passed    0.02 sec
      Start 56: dry-run_iCubPisa01
56/77 Test #56: dry-run_iCubPisa01 ...............   Passed    0.02 sec
      Start 57: dry-run_iCubPlymouth01
57/77 Test #57: dry-run_iCubPlymouth01 ...........   Passed    0.02 sec
      Start 58: dry-run_iCubPlymouth02
58/77 Test #58: dry-run_iCubPlymouth02 ...........   Passed    0.03 sec
      Start 59: dry-run_iCubPrague01
59/77 Test #59: dry-run_iCubPrague01 .............***Failed   12.44 sec
      Start 60: dry-run_iCubPrototype01
60/77 Test #60: dry-run_iCubPrototype01 ..........   Passed    0.24 sec
      Start 61: dry-run_iCubRome02
61/77 Test #61: dry-run_iCubRome02 ...............   Passed    0.03 sec
      Start 62: dry-run_iCubSeoul01
62/77 Test #62: dry-run_iCubSeoul01 ..............   Passed    0.03 sec
      Start 63: dry-run_iCubShanghai01
63/77 Test #63: dry-run_iCubShanghai01 ...........***Failed   12.39 sec
      Start 64: dry-run_iCubSheffield01
64/77 Test #64: dry-run_iCubSheffield01 ..........***Failed    6.33 sec
      Start 65: dry-run_iCubShenzhen01
65/77 Test #65: dry-run_iCubShenzhen01 ...........***Failed   12.49 sec
      Start 66: dry-run_iCubSingapore01
66/77 Test #66: dry-run_iCubSingapore01 ..........   Passed   11.28 sec
      Start 67: dry-run_iCubTokyo01
67/77 Test #67: dry-run_iCubTokyo01 ..............   Passed    0.02 sec
      Start 68: dry-run_iCubTwente01
68/77 Test #68: dry-run_iCubTwente01 .............***Failed   11.35 sec
      Start 69: dry-run_iCubUrbana01
69/77 Test #69: dry-run_iCubUrbana01 .............   Passed    0.24 sec
      Start 70: dry-run_iCubValparaiso01
70/77 Test #70: dry-run_iCubValparaiso01 .........***Failed   12.40 sec
      Start 71: dry-run_iCubWaterloo01
71/77 Test #71: dry-run_iCubWaterloo01 ...........***Failed   12.29 sec
      Start 72: dry-run_iCubZagreb01
72/77 Test #72: dry-run_iCubZagreb01 .............***Failed   12.43 sec
      Start 73: dry-run_iCubZurich01
73/77 Test #73: dry-run_iCubZurich01 .............   Passed    0.24 sec
      Start 74: dry-run_iRonCub-Mk1
74/77 Test #74: dry-run_iRonCub-Mk1 ..............***Failed    8.31 sec
      Start 75: dry-run_iiwaBoston01
75/77 Test #75: dry-run_iiwaBoston01 .............   Passed    0.02 sec
      Start 76: dry-run_naoPrague01
76/77 Test #76: dry-run_naoPrague01 ..............   Passed    0.02 sec
      Start 77: dry-run_vizzy
77/77 Test #77: dry-run_vizzy ....................   Passed    0.03 sec

68% tests passed, 25 tests failed out of 77

Total Test time (real) = 315.42 sec

The following tests FAILED:
          1 - dry-run_R1SN000 (Failed)
          2 - dry-run_R1SN001 (Failed)
          3 - dry-run_R1SN002 (Failed)
          4 - dry-run_R1SN003 (Failed)
          7 - dry-run_ergoCubSN000 (Failed)
         15 - dry-run_iCubChemnitz01 (Failed)
         16 - dry-run_iCubDarmstadt01 (Failed)
         26 - dry-run_iCubGenova04 (Failed)
         30 - dry-run_iCubGenova09 (Failed)
         31 - dry-run_iCubGenova10 (Failed)
         32 - dry-run_iCubGenova11 (Failed)
         35 - dry-run_iCubHeidelberg01 (Failed)
         43 - dry-run_iCubLisboa01 (Failed)
         48 - dry-run_iCubMoscow01 (Failed)
         50 - dry-run_iCubNancy01 (Failed)
         51 - dry-run_iCubNottingham01 (Failed)
         59 - dry-run_iCubPrague01 (Failed)
         63 - dry-run_iCubShanghai01 (Failed)
         64 - dry-run_iCubSheffield01 (Failed)
         65 - dry-run_iCubShenzhen01 (Failed)
         68 - dry-run_iCubTwente01 (Failed)
         70 - dry-run_iCubValparaiso01 (Failed)
         71 - dry-run_iCubWaterloo01 (Failed)
         72 - dry-run_iCubZagreb01 (Failed)
         74 - dry-run_iRonCub-Mk1 (Failed)

Still need to look at those remaining failures and the implementation of yarprobotinterface --dryrun.