awesomebytes / ros_overlay_on_gentoo_prefix

ros-overlay built on Gentoo Prefix ready to use

ros_overlay_on_gentoo_prefix

ROS Kinetic: ROS Melodic:

This project answers the following questions:

• Do you have a system (computer or robot) where you can't update the software (ROS or its dependencies) anymore?
• Do you have a system where you lack permissions to install or update software?
• Do you want a system where you don't depend on any software of the platform itself but you can still use the platform software if you want?

Gentoo Prefix provides an Operating System in userspace (you can place it in any folder that you have write permissions) which requires no root privileges to install or run software.a

ros-overlay provides recipes to build different ROS versions on Gentoo systems.

This project builds upon both by building nightly binary releases of a full Gentoo Prefix + ROS (Kinetic & Melodic).

Building ROS over Gentoo Prefix (on /tmp/gentoo for amd64 over a Docker image of Ubuntu 16.04) thanks to ros-overlay.

This is a project closely related to Gentoo Prefix CI.

Builds page on Azure Pipelines: https://dev.azure.com/12719821/12719821/_build?definitionId=5

Ready-to-use releases: https://github.com/awesomebytes/ros_overlay_on_gentoo_prefix/releases

Ready-to-use built Dockers:

• ros-kinetic/ros_base: docker pull awesomebytes/roogp_ros_kinetic_ros_base
• ros-kinetic/desktop: docker pull awesomebytes/roogp_ros_kinetic_desktop_3
• ros-melodic/ros_base: docker pull awesomebytes/roogp_ros_melodic_ros_base
• ros-melodic/desktop: docker pull awesomebytes/roogp_ros_melodic_desktop_3

You can get into the dockers by doing: docker run -it awesomebytes/IMAGE_NAME /bin/bash

You'll find

Currently building

• ros-kinetic/ros_base
• ros-kinetic/desktop
• ros-melodic/ros_base
• ros-melodic/desktop

Try it

Go to https://github.com/awesomebytes/ros_overlay_on_gentoo_prefix/releases and download the latest release of your choice (ros-kinetic/ros_base is 2GB~), it's divided in 1GB parts (GitHub Releases accepts files up to 2GB only) that are named like gentoo_on_tmp_with_ros-kinetic_ros_base-amd64_2020-03-08T21at41plus00at00.tar.gz.part-00.

Put the parts together and extract (4.4GB~):

# Put the .part-XX files together
cat gentoo_on_tmp* > gentoo_on_tmp.tar.lzma
# Extract the 'gentoo' folder that contains the OS
tar xvf gentoo_on_tmp.tar.lzma
# Probably delete the intermediate files
rm gentoo_on_tmp*
# Just enter the Gentoo Prefix environment
./gentoo/startprefix
# To use ROS you'll need to source ROS and setup a couple of environment variables
source /tmp/gentoo/opt/ros/<ROS_VERSION_HERE>/setup.bash
export CATKIN_PREFIX_PATH=/tmp/gentoo/opt/ros/<ROS_VERSION_HERE>
export ROS_IP=<YOUR_IP_HERE>

ros-kinetic/desktop is ~4GB, extracted ~6GB.

The startprefix script is used to drop the user in a shell in the Gentoo Prefix environment. It creates a softlink to /tmp/gentoo if it does not exist in order to work.

Work in progress & Future work

Contributions more than welcome!

• ros-kinetic/desktop_full: it worked at some point, needs some love to bring it back to life.
• ros2: should work easily, haven't put the time.
• Support for MacOS: Azure Pipelines has MacOS VMs to use, but I have never used MacOS so I've never put the time.
• Support for ARM: Need an emulator or a Raspberry Pi or similar setup somewhere to use as buildfarm.
• Support for Android: for ARM needs work to setup an emulator, but for x86 Gentoo Prefix needs to be bootstrapped in /data/local/tmp and that's pretty much it.

ros_overlay_on_gentoo_prefix's Issues

Fix ros_base image docker build

Hi !

When I built the ros_base docker image, I had to remove the "bluetooth" word from these lines :

RUN echo "dev-lang/python sqlite bluetooth readline" >> $EPREFIX/etc/portage/package.use
RUN EXTRA_ECONF="--enable-optimizations --with-lto" $PREFIXED emerge python:2.7[sqlite,bluetooth,readline] python:3.6[sqlite,bluetooth,readline]

I also had to add --autounmask-keep-masks option to the following command (deduced from what the emerge build system was telling me) :

RUN $PREFIXED emerge dev-util/cmake

And finally had to change the cmake version in this command to 3.13.5 :

RUN echo ">dev-util/cmake-3.12.3" >> /tmp/gentoo/etc/portage/package.mask

Then I docker-cp-ed the gentoo folder on my host machine, went into it, and launched : ./startprefix

Got this message :

bash: eval: line 31: syntax error near unexpected token `newline'
bash: eval: line 31: `Usage: lesspipe <file>'

Which did not prevent me from entering the prefix so I guess everything is fine then ?

When looking a bit into it, it seems the bash command on my host .bashrc :

[ -x /usr/bin/lesspipe ] && eval "$(SHELL=/bin/sh lesspipe)"

is causing this error message, but since its just a comfort command, I guess its not a problem.

Maybe create GitHub issues when builds break?

The issues could contain the last bit of the log, a link to the build page, a link to the log and usual things to check to fix it (checking ros-overlay, checking gentoo bug tracker...).

I did a quick search and there are either CLI or Python libs to interact with issues.

Unsafe load()

Hi! Thank you for your great job)
Can I ask you to add patch to prefix, which fix this bug:

Unsafe load() call disabled by Gentoo. See bug #659348

It appeared when I tried to run rosdep command (

Add job that tests the produced images

It should just try to extract the released version and execute something meaningful.
Another one that would use 'the latest release' would also be good.

This will avoid faulty releases as sometimes they don't work cause:

• We ran out of disk space at some point and we are missing some split file (Hopefully I fixed this issue already)
• The upload to GitHub releases returned some error. For some reason this doesn't crash the job.
• Some other unknown error that I haven't yet encountered.

docker build --squash

Investigate the --squash parameter of docker build. It may help to have less layers to not run into the 127 layers limit of Docker that we ran before. Specially useful for https://github.com/awesomebytes/pepper_os

Catkin issue Failed Patch: 0001-Workaround-error-issue-ros-overlay-711.patch

In order to fix "Failed to emerge ros-kinetic/catkin-0.7.20-r1" because of the patch 0001-Workaround-error-issue-ros-overlay-711.patch, you need to remove it from ros-overlay and ebuild the manifest. Add these commands can be a quick workaround for now :
RUN rm /tmp/gentoo/var/repo/ros-overlay/ros-kinetic/catkin/files/0001-Workaround-error-issue-ros-overlay-711.patch
RUN $PREFIXED ebuild /tmp/gentoo/var/repo/ros-overlay/ros-kinetic/catkin/*.ebuild manifest

Automate using latest release of gentoo_prefix_ci

Here: https://github.com/awesomebytes/ros_overlay_on_gentoo_prefix/blob/master/ros_base/Dockerfile#L40

Add sqlite USE flag to python

ros2 support

Reposting this as a public issue so others may help or be interested:

Hello! I am @alex-kozinov
We've met before, when I asked you about some issues with ROS overlays over gentoo prefix. And you've been very helpful! Only with your support and your knowledge our team got an opportunity to work under Nao robots, thank you a lot (:
But recently we found out about ros2 and its advantages. But we don't know how to install it on Nao robot. Can you help us to do it? Maybe we will together implement it in repo https://github.com/awesomebytes/ros_overlay_on_gentoo_prefix

You can certainly use my bootstrapping method via ros_overlay_on_gentoo_prefix to get ros2 running. You'll need to fix up some issues as they come, most probably, but itshould be okay.
ROS2 offers a lot of cool things, but, also note that it may be missing packages to be ported that you may want to use. This has been the case for some friends last year, things may have improved meanwhile though (they didn't use Nao/Pepper). Most probably, the naoqi_driver will not be ready for ROS2. However, you may also run ROS1 and ROS2 software at the same time thanks to a bridge, I think.
Some work was done in a different project by a friend, @esteve, to build ROS1 and ROS2 for Pepper: https://github.com/esteve/ros2_pepper Note that it compiles 'by hand' all dependencies and ROS itself instead of leveraging the OS package manager to do that for you. I think my approach with Gentoo Prefix is better because of that. But it may be useful to have as a reference.
You may want to start your journey by trying to follow the instructions in: https://index.ros.org/doc/ros2/Installation/Foxy/Linux-Install-Debians/ But installing the ros2 packages as:

emerge ros-foxy/ros_base

I would start by using docker containers to make testing easier:

# Get the latest successful docker image
docker pull awesomebytes/roogp_ros_melodic_ros_base
# Get a shell inside of it
docker run -it --name=ros2 awesomebytes/roogp_ros_melodic_ros_base /bin/bash
./gentoo/startprefix
export EPREFIX=/tmp/gentoo
# Update our portage package manager (like doing apt-get update)
cd $EPREFIX/usr/local/portage && git clean -f && git reset --hard
emaint sync -a
# Try to emerge ros-foxy
emerge ros-foxy/ros_base

Note that you can get additional shells in that docker container by doing docker exec -it ros2 /bin/bash.

You can check the names of packages from the ros-overlay repo: https://github.com/ros/ros-overlay/tree/main/ros-foxy

Let me know how you go.