Error for adding demo_nodes_py or demo_nodes_cpp to IMAGE_INSTALL about meta-ros

[robwoolley]

Thanks @Jpgig, meta-ros sets PYTHON_SITEPACKAGES_DIR in this class (

meta-ros/meta-ros-common/classes/ros_opt_prefix.bbclass

Line 28 in e45cd71

PYTHON_SITEPACKAGES_DIR = "${ros_libdir}/${PYTHON_DIR}/site-packages"

) to use ros_libdir = "${ros_prefix}/${baselib}" which should be /usr/lib64 on your aarch64 target.

I took a look at the CMakeLists.txt for example_interfaces ( https://github.com/ros2/example_interfaces/blob/humble/CMakeLists.txt#L52 ). I don't see a reason why it would use /usr/lib. The install macro is relative and the recipe seems to inherit all the build system classes that would ensure that CMAKE_INSTALL_DIR or CMAKE_PREFIX_DIR is set appropriately.

I will have to do some test builds for mickledore+humble to see if I can reproduce the failure.

[Jpgig]

Hi Rob, did you get some time to build this?

[robwoolley]

Hi Jiten,

I noticed that the "mickledore" branch was missing some commits from "mickledore-next". One of which specifically related to using the correct arch-specific libdir: 689d33b

There is a high-chance that this will solve you issue. Please update your meta-ros layer to include the latest commits and do a rebuild. Please let me know if this helps solve your issue.

Regards,
Rob

[Jpgig]

Thanks Rob, I took the latest commit still getting same error.

[robwoolley]

Hi Jiten,

I did fresh test builds for both recipes (demo-nodes-py and demo-nodes-cpp) for Humble and Mickledore on both

qemux86_64:

Build Configuration:
BB_VERSION = "2.4.0"
BUILD_SYS = "x86_64-linux"
NATIVELSBSTRING = "debian-11"
TARGET_SYS = "aarch64-oe-linux"
MACHINE = "qemuarm64"
DISTRO = "nodistro"
DISTRO_VERSION = "nodistro.0"
TUNE_FEATURES = "aarch64 armv8a crc cortexa57"
TARGET_FPU = ""
DISTRO_NAME = "OpenEmbedded"
ROS_DISTRO = "humble"
ROS_VERSION = "2"
ROS_PYTHON_VERSION = "3"
meta-raspberrypi = "mickledore:5c2d60b3f50d9b89978b58db0bf8068acf7e7877"
meta-ros2-humble
meta-ros2
meta-ros-common = "mickledore:2b64ab2a209f689689af3774f2aca9d3f0d6db17"
meta-python
meta-oe = "mickledore:39968837196cb48209b71e8852dd04a2f8ccdca8"
meta = "mickledore:7e3489c0c5970389c8a239dc7b367bcadf554eb5"

and qemuarm64:

Build Configuration:
BB_VERSION = "2.4.0"
BUILD_SYS = "x86_64-linux"
NATIVELSBSTRING = "debian-11"
TARGET_SYS = "aarch64-oe-linux"
MACHINE = "qemuarm64"
DISTRO = "nodistro"
DISTRO_VERSION = "nodistro.0"
TUNE_FEATURES = "aarch64 armv8a crc cortexa57"
TARGET_FPU = ""
DISTRO_NAME = "OpenEmbedded"
ROS_DISTRO = "humble"
ROS_VERSION = "2"
ROS_PYTHON_VERSION = "3"
meta-raspberrypi = "mickledore:5c2d60b3f50d9b89978b58db0bf8068acf7e7877"
meta-ros2-humble
meta-ros2
meta-ros-common = "mickledore:2b64ab2a209f689689af3774f2aca9d3f0d6db17"
meta-python
meta-oe = "mickledore:39968837196cb48209b71e8852dd04a2f8ccdca8"
meta = "mickledore:7e3489c0c5970389c8a239dc7b367bcadf554eb5"

This was done using the mcf tool with the ros2-humble-mickledore.mcf configuration. Both builds were successful.

It is possible that there is something specific to your setup that I'm not seeing. Can you share step-by-step instructions for creating your build environment and running the build?

Can you also share the Build Configuration information from the beginning of your build? (eg as shown above)

Regards,
Rob

[Jpgig]

I'm following below steps:

1. Added the ROS layer as per the link below:
   https://gist.github.com/robwoolley/2feef3cf763d371d9a0c534cae1b38b6
2. I also added below to local.conf:
   IMAGE_INSTALL:append = " sros2 demo-nodes-py"
   EXTRA_OECMAKE:pn- += " -DSECURITY=ON"
3. Then building using below command:
   bitbake wrlinux-image-small

I see below for ros layer:
meta-ros-common
meta-ros2
meta-ros2-humble = "mickledore-next:2b64ab2a209f689689af3774f2aca9d3f0d6db17"

[robwoolley]

I managed to reproduce the problem with example_interfaces Wind River Linux LTS 23. The ament_cmake_python package in ROS automatically tries to detect the PYTHON_INSTALL_DIR directory.

The code can be seen here: https://github.com/ament/ament_cmake/blob/9743c9cd3e5aef9566acefea6e6e3724e0e5657e/ament_cmake_python/ament_cmake_python-extras.cmake#L43

When running the python code under devshell, I can see it returns the following:
\>\>\> print(os.path.relpath(sysconfig.get_path('purelib', **kwargs), start='${CMAKE_INSTALL_PREFIX}').replace(os.sep, '/'))

lib/python3.11/site-packages

I compared this to the mickledore release for the Poky reference distro:

bitbake-getvar -r example-interfaces 'FILES:example-interfaces'

#
# $FILES:example-interfaces [5 operations]
# set /builds/oe-ros-ci/build-ci/openembedded-core/meta/conf/bitbake.conf:340
# "${bindir}/* ${sbindir}/* ${libexecdir}/* ${libdir}/lib*${SOLIBS} ${sysconfdir} ${sharedstatedir} ${localstatedir} ${base_bindir}/* ${base_sbindir}/* ${base_libdir}/${SOLIBS} ${base_prefix}/lib/udev ${prefix}/lib/udev ${base_libdir}/udev ${libdir}/udev ${datadir}/${BPN} ${libdir}/${BPN}/ ${datadir}/pixmaps ${datadir}/applications ${datadir}/idl ${datadir}/omf ${datadir}/sounds ${libdir}/bonobo/servers"
# append /builds/oe-ros-ci/build-ci/meta-ros/meta-ros-common/classes/ros_faulty_solibs.bbclass:5
# "${libdir}/lib*${SOLIBSDEV}"
# :prepend /builds/oe-ros-ci/build-ci/meta-ros/meta-ros-common/classes/ros_component.bbclass:23
# " ${datadir}/${ROS_BPN} ${libdir}/${PYTHON_DIR} ${libdir}/${ROS_BPN} "
# :prepend /builds/oe-ros-ci/build-ci/meta-ros/meta-ros2/classes/ros_ament_cmake.bbclass:28
# " ${datadir}/ament_index "
# rename from FILES:${PN} data.py:97 [expandKeys]
# "${bindir}/* ${sbindir}/* ${libexecdir}/* ${libdir}/lib*${SOLIBS} ${sysconfdir} ${sharedstatedir} ${localstatedir} ${base_bindir}/* ${base_sbindir}/* ${base_libdir}/${SOLIBS} ${base_prefix}/lib/udev ${prefix}/lib/udev ${base_libdir}/udev ${libdir}/udev ${datadir}/${BPN} ${libdir}/${BPN}/ ${datadir}/pixmaps ${datadir}/applications ${datadir}/idl ${datadir}/omf ${datadir}/sounds ${libdir}/bonobo/servers ${libdir}/lib*${SOLIBSDEV}"
# pre-expansion value:
# " ${datadir}/ament_index ${datadir}/${ROS_BPN} ${libdir}/${PYTHON_DIR} ${libdir}/${ROS_BPN} ${bindir}/* ${sbindir}/* ${libexecdir}/* ${libdir}/lib*${SOLIBS} ${sysconfdir} ${sharedstatedir} ${localstatedir} ${base_bindir}/* ${base_sbindir}/* ${base_libdir}/${SOLIBS} ${base_prefix}/lib/udev ${prefix}/lib/udev ${base_libdir}/udev ${libdir}/udev ${datadir}/${BPN} ${libdir}/${BPN}/ ${datadir}/pixmaps ${datadir}/applications ${datadir}/idl ${datadir}/omf ${datadir}/sounds ${libdir}/bonobo/servers ${libdir}/lib*${SOLIBSDEV}"
FILES:example-interfaces=" /usr/share/ament_index /usr/share/example_interfaces /usr/lib/python3.11 /usr/lib/example_interfaces /usr/bin/* /usr/sbin/* /usr/libexec/* /usr/lib/lib*.so.* /etc /com /var /bin/* /sbin/* /lib/.so. /lib/udev /usr/lib/udev /lib/udev /usr/lib/udev /usr/share/example-interfaces /usr/lib/example-interfaces/* /usr/share/pixmaps /usr/share/applications /usr/share/idl /usr/share/omf /usr/share/sounds /usr/lib/bonobo/servers /usr/lib/lib*.so"

You can see from above that /usr/lib/python3.11 is derived from ${libdir} so I check libdir to see where it is defined. It is as expected:

bitbake-getvar -r example-interfaces libdir

#
# $libdir [2 operations]
# exported /builds/oe-ros-ci/build-ci/openembedded-core/meta/conf/bitbake.conf:46
# [export] "1"
# set /builds/oe-ros-ci/build-ci/openembedded-core/meta/conf/bitbake.conf:46
# "${exec_prefix}/${baselib}"
# pre-expansion value:
# "${exec_prefix}/${baselib}"
export libdir="/usr/lib"

bitbake-getvar -r example-interfaces baselib

#
# $baselib [2 operations]
# set /builds/oe-ros-ci/build-ci/openembedded-core/meta/conf/bitbake.conf:10
# "${BASELIB}"
# set /builds/oe-ros-ci/build-ci/openembedded-core/meta/conf/bitbake.conf:11
# [vardepvalue] "${baselib}"
# pre-expansion value:
# "${BASELIB}"
baselib="lib"

So, then I check Wind River Linux LTS 23 which is based on mickledore as well:

bitbake-getvar -r example-interfaces 'FILES:example-interfaces'

WARNING: You have included the meta-security layer, but 'security' has not been enabled in your DISTRO_FEATURES. Some bbappend files and preferred version setting may not take effect. See the meta-security README for details on enabling security support.
WARNING: Host distribution "debian-11" has not been validated with this version of the build system; you may possibly experience unexpected failures. It is recommended that you use a tested distribution.
#
# $FILES:example-interfaces [5 operations]
# set /builds/oe-ros-ci/wrlinuxlts23-ros/work/layers/oe-core/meta/conf/bitbake.conf:342
# "${bindir}/* ${sbindir}/* ${libexecdir}/* ${libdir}/lib*${SOLIBS} ${sysconfdir} ${sharedstatedir} ${localstatedir} ${base_bindir}/* ${base_sbindir}/* ${base_libdir}/${SOLIBS} ${base_prefix}/lib/udev ${prefix}/lib/udev ${base_libdir}/udev ${libdir}/udev ${datadir}/${BPN} ${libdir}/${BPN}/ ${datadir}/pixmaps ${datadir}/applications ${datadir}/idl ${datadir}/omf ${datadir}/sounds ${libdir}/bonobo/servers"
# append /builds/oe-ros-ci/wrlinuxlts23-ros/work/layers/meta-ros/meta-ros-common/classes/ros_faulty_solibs.bbclass:5
# "${libdir}/lib*${SOLIBSDEV}"
# :prepend /builds/oe-ros-ci/wrlinuxlts23-ros/work/layers/meta-ros/meta-ros-common/classes/ros_component.bbclass:23
# " ${datadir}/${ROS_BPN} ${libdir}/${PYTHON_DIR} ${libdir}/${ROS_BPN} "
# :prepend /builds/oe-ros-ci/wrlinuxlts23-ros/work/layers/meta-ros/meta-ros2/classes/ros_ament_cmake.bbclass:28
# " ${datadir}/ament_index "
# rename from FILES:${PN} data.py:97 [expandKeys]
# "${bindir}/* ${sbindir}/* ${libexecdir}/* ${libdir}/lib*${SOLIBS} ${sysconfdir} ${sharedstatedir} ${localstatedir} ${base_bindir}/* ${base_sbindir}/* ${base_libdir}/${SOLIBS} ${base_prefix}/lib/udev ${prefix}/lib/udev ${base_libdir}/udev ${libdir}/udev ${datadir}/${BPN} ${libdir}/${BPN}/ ${datadir}/pixmaps ${datadir}/applications ${datadir}/idl ${datadir}/omf ${datadir}/sounds ${libdir}/bonobo/servers ${libdir}/lib*${SOLIBSDEV}"
# pre-expansion value:
# " ${datadir}/ament_index ${datadir}/${ROS_BPN} ${libdir}/${PYTHON_DIR} ${libdir}/${ROS_BPN} ${bindir}/* ${sbindir}/* ${libexecdir}/* ${libdir}/lib*${SOLIBS} ${sysconfdir} ${sharedstatedir} ${localstatedir} ${base_bindir}/* ${base_sbindir}/* ${base_libdir}/${SOLIBS} ${base_prefix}/lib/udev ${prefix}/lib/udev ${base_libdir}/udev ${libdir}/udev ${datadir}/${BPN} ${libdir}/${BPN}/ ${datadir}/pixmaps ${datadir}/applications ${datadir}/idl ${datadir}/omf ${datadir}/sounds ${libdir}/bonobo/servers ${libdir}/lib*${SOLIBSDEV}"
FILES:example-interfaces=" /usr/share/ament_index /usr/share/example_interfaces /usr/lib64/python3.11 /usr/lib64/example_interfaces /usr/bin/* /usr/sbin/* /usr/libexec/* /usr/lib64/lib*.so.* /etc /com /var /bin/* /sbin/* /lib64/.so. /lib/udev /usr/lib/udev /lib64/udev /usr/lib64/udev /usr/share/example-interfaces /usr/lib64/example-interfaces/* /usr/share/pixmaps /usr/share/applications /usr/share/idl /usr/share/omf /usr/share/sounds /usr/lib64/bonobo/servers /usr/lib64/lib*.so"

Notice that it is /usr/lib64/python3.11 here, so let's check libdir and baselib:

bitbake-getvar -r example-interfaces libdir

`WARNING: You have included the meta-security layer, but 'security' has not been enabled in your DISTRO_FEATURES. Some bbappend files and preferred version setting may not take effect. See the meta-security README for details on enabling security support.
WARNING: Host distribution "debian-11" has not been validated with this version of the build system; you may possibly experience unexpected failures. It is recommended that you use a tested distribution.
#
# $libdir [2 operations]
# exported /builds/oe-ros-ci/wrlinuxlts23-ros/work/layers/oe-core/meta/conf/bitbake.conf:46
# [export] "1"
# set /builds/oe-ros-ci/wrlinuxlts23-ros/work/layers/oe-core/meta/conf/bitbake.conf:46
# "${exec_prefix}/${baselib}"
# pre-expansion value:
# "${exec_prefix}/${baselib}"
export libdir="/usr/lib64"

bitbake-getvar -r example-interfaces baselib

`WARNING: You have included the meta-security layer, but 'security' has not been enabled in your DISTRO_FEATURES. Some bbappend files and preferred version setting may not take effect. See the meta-security README for details on enabling security support.
WARNING: Host distribution "debian-11" has not been validated with this version of the build system; you may possibly experience unexpected failures. It is recommended that you use a tested distribution.
#
# $baselib [3 operations]
# set /builds/oe-ros-ci/wrlinuxlts23-ros/work/layers/oe-core/meta/conf/bitbake.conf:10
# "${BASELIB}"
# set /builds/oe-ros-ci/wrlinuxlts23-ros/work/layers/oe-core/meta/conf/bitbake.conf:11
# [vardepvalue] "${baselib}"
# set /builds/oe-ros-ci/wrlinuxlts23-ros/work/layers/oe-core/meta/conf/multilib.conf:2
# "${@d.getVar('BASE_LIB:tune-' + (d.getVar('DEFAULTTUNE') or 'INVALID')) or d.getVar('BASELIB')}"
# pre-expansion value:
# "${@d.getVar('BASE_LIB:tune-' + (d.getVar('DEFAULTTUNE') or 'INVALID')) or d.getVar('BASELIB')}"
baselib="lib64"

Note that there is a multilib.conf being used in the definition of baselib that is resulting in the value of lib64.

This is because the Wind River Linux LTS 23 BSPs support the multilib feature that lets one combine 32-bit and 64-bit libraries.

However, the Python sysconfig for "purelib" is reporting back "/usr/lib". It is unclear to me what the expected behaviour should be in this scenario. It might be necessary to ask the Yocto developers that maintain Python upstream.

However, a simple workaround may be to just add the following line in any affected packages (eg example-interfaces and demo-nodes-*).

EXTRA_OECMAKE += " -DPYTHON_INSTALL_DIR=${baselib}/${PYTHON_DIR}/site-packages"

This is something that I am happy to add to meta-ros as a temporary measure for the mickledore branch. It may take me some time to get to in the queue. If you could try it out and send me a PR with the changes, that would be excellent. You can refer to the commit I pointed to before to see examples of how this was done for other packages: 689d33b

That said, it will likely require a more permanent fix for multilib in the later releases. You may wish to file a bug report against Wind River Linux LTS 23 explaining that the Python sysconfig may be reporting the wrong information on multilib systems. Then they can work to get it fixed upstream if it is needed.

[robwoolley]

I took another look at the information above. I noticed that the python directory is being added to FILES in the ros_component.bbclass file:

# :prepend /builds/oe-ros-ci/wrlinuxlts23-ros/work/layers/meta-ros/meta-ros-common/classes/ros_component.bbclass:23
# " ${datadir}/${ROS_BPN} ${libdir}/${PYTHON_DIR} ${libdir}/${ROS_BPN} "

When I look at that file I notice that it is using ${libdir}${PYTHON_DIR}:
https://github.com/ros/meta-ros/blob/master/meta-ros-common/classes/ros_component.bbclass#L21

It may be more correct to use ${nonarch_libdir} instead.

[Jpgig]

So your suggestion to add below:
EXTRA_OECMAKE += " -DPYTHON_INSTALL_DIR=${nonarch_libdir}/${PYTHON_DIR}/site-packages"
instead of this:
EXTRA_OECMAKE += " -DPYTHON_INSTALL_DIR=${baselib}/${PYTHON_DIR}/site-packages"
Below is the screen of my file which is under meta-ros/meta-ros2-humble/generated-recipes/demos folder:

[Jpgig]

I'm able to compile now after adding below line to example-interfaces_0.9.3-2.bbappend recipe file:
EXTRA_OECMAKE += " -DPYTHON_INSTALL_DIR=${baselib}/${PYTHON_DIR}/site-packages"
but still getting below error when trying to run : ros2 run demo_nodes_py talker

[jiaxshi]

Hi @Jpgig
You can refer to my PR for fixing this issue:
9ac73da