robojackets / urc-software

Primary codebase for the RoboNav team's URC rover.

urc-software

Welcome to the RoboJackets/RoboNav software repo for the University Rover Challenge (URC)! This document will give you a brief description of the repo's layout and an overview of the repo.

Directory Structure

• .github CI pipeline and PR/issue templates
• cmake CMake files to aid with building
• documents Research, design, and documentation
• external Where all our submodules are located
• urc_analysis Nodes used for scientific analysis
• urc_arm_moveit_config Moveit config folder for rover arm
• urc_bringup Location of the universal launch file + heartbeat node
• urc_control deprecated Controller manager node and controllers bring up
• urc_controllers ros2-control controllers
• urc_gazebo Helper nodes used for simulation purposes
• urc_hw ros2-control hardware interface
• urc_hw_description URDF description for the rover
• urc_manipulation Collection of nodes used for the robotic arm
• urc_nanopb nanopb related files and settings
• urc_msgs Custom ROS messages used in various packages
• urc_platform Manages our nanopb protocol buffers
• urc_nav2 Launch scripts and configs for Nav2 packages
• urc_navigation Collection of nodes that form our navigation stack
• urc_perception Collection of nodes that form our perception stack
• urc_platform Nodes that are platform specific and used to communicate with the hardware, ie. IMU, joystick and motor controller
• urc_util A collection of utility nodes and classes

Installation Instructions

Essential
You will need to be using Ubuntu 22.04 to run ROS2. This can be accomplished with any of the following methods:

• Ubuntu 22.04: Native Installation or WSL (Windows/Linux) Strongly recommended!
• NoVNC Docker Installation Instructions (Mac/Windows/Linux) Less viable, use for Apple Silicon

Specific Features

• XBox Controller Setup
• Depth Camera Setup
• ROS2 Control Gazebo Setup
• Radio Communication Between Rover and Ground Station

Helpful Resources

• Useful Commands: ROS2 Commands, Git Commands
• Design Presentation Requirements
• Drone Repository
• Firmware Repository

Team-Related Links

• Slack
• Google Drive
• Software Training

External Documentation and Background Reading

• ROS2 Humble Documentation
• MoveIt2 Documentation
• Nav2 Documentation
• ROS2 Control Documentation

Common Issues

NanoPB Not Building

Fix (will only build after the last time):

colcon build --symlink-install ; chmod +x build/urc_nanopb/nanopb/generator/protoc-gen-nanopb
colcon build --symlink-install ; chmod +x build/urc_nanopb/nanopb/generator/nanopb_generator.py
colcon build --symlink-install

urc-software's Issues

Migrate joystick_driver node from Jessiii to ROS2

Description

Joystick controls are no longer being used just for testing like in IGVC. They will be a key part of any manual control portions of the URC competition, and are currently needed to test prototypes.

Jessi code to migrate can be found here.

Must include unit tests (may need to resolve #21 first).

Requirements

• Node is ported over and works successfully
• Unit tests pass

Localization using `robot_localization`

Description

Currently, we do not have any nodes that performs localization. It would be a good start to utilize the robot_localization package to set up a localization node.

Helpful Resources:

• Configuring robot_localization
• Implementation inigvc_software
  • config file
  • launch file

AC: Create an ekf localization node using the robot_localization package.

Implement Elevation Mapping

Description

For our rover to navigate autonomously, it needs to be able to categorize the traversibility of its surroundings. grid_map and elevation_mapping allow for costmaps based on height differentials, with assigned cost values based on the steepness of areas.

Requirements

• Implement the elevation_mapping library
• Test and bug-check the configuration

Create model of robot arm in gazebo.

To test out manipulation code, we need to have an arm in simulation we can move around.

AC: Robot arm model in gazebo that meets specs of real life arm.

---

Join #robot-arm-model on the Slack to discuss!

Custom Messages

Migrate igvc_msgs so that we can send custom messages using ROS2. Systems for creating messages have changed since ROS1 to being based on native ROS interfaces.

Old code can be seen here:
https://github.com/RoboJackets/igvc-software/tree/master/igvc_msgs

Gazebo Testing in CI Pipeline

Description

Currently there is no way of accurately testing if the Gazebo testing environment will launch on every machine. Running some launch files to verify Gazebo's functionality would help out the pipeline a lot.

Requirements

• Finish #13 before starting
• Integrate some basic Gazebo tests into the CI pipeline

Migrate igvc_gazebo scan_to_pointcloud node to ROS2

AC: scan_to_pointcloud node is rewritten to be in ROS2.

New Robot Model

Description

Currently, our robot model is a box with wheels attached to it that doesn't really do anything.

However, in the igvc-software repository, we have a robot model of Jessiii in gazebo that simulates and publishes sensor messages to their respective topics. This includes camera images, IMU messages, LiDAR pointclouds, GPS coordinates, and more.

AC: Create a robot model that publishes messages from the same sensors that were Jessiii.

Update Rover Model

Description

Having the arm mounted on our rover would make testing in Gazebo far more robust. To do this, we need to have the current rover model and the arm model mounted on top of it.

Requirements

• Import rover URDF
• Create new rover model
• Mount arm model on the rover model

Improve Nanopb Building

Description

The current way we are building Nanopb is pretty scuffed. The way the Nanopb files are auto generated causes a lot of issues for our colcon build system, and the only solution I could find was isolating nanopb and urc_nanopb during an initial first build and sourcing . install/setup.bash before building everything.

Ideally we should be able to colcon build during a first build without having to worry about Nanopb.

Requirements

1. Figure out a solution that enables a much cleaner and less error prone build of nanopb
2. Update the CI pipeline and all instructions with the new solution

Create a temporary model for Walli in simulation.

We don't need a full complete model of Walli right now, just something to test that basic code works.

AC: Box on wheels model of robot that can move.

Migrate motor_controller node from Jessie to ROS2

Description

Motor controller operation is critical to the basic functionality and testing of any URC prototypes.

Jessie code to migrate can be found here.

Must include unit tests (may need to resolve #21 first).

Requirements

• Node is ported over and works successfully
• Unit tests pass

Migrate igvc_gazebo magnetometer node to ROS2

AC: magnetometer node is rewritten to be in ROS2

Implement Live Camera Feed

Description

Finish #42 first!

To operate remotely during the manual control segments of the URC competition, we need to be able to see a live feed of what's around the rover.

Requirements

• Camera is capable of the tasks outlined here
• Camera can be viewed from the web interface
• Minimal latency if possible

ARUCOTagDetected nav tree node

Description

Custom nav2 tree node that returns SUCCESS if there is an ARUCO tag detected, and FAILURE otherwise.

Requirements

• Implements a nav2 tree node
• meets the description

Migrate igvc_gazebo control node to ROS2

AC: control node is rewritten to be in ROS2

GPS interface map

We need an interface for tracking the robot on a live map (with the ability to add points, clear points from the map, rotate the map, and rotate the map to align with the robot's coordinate frame).

Consider maybe augmenting this map's waypoints with the ability to send them to the rover as an input for urc_navigation's GPS navigation (see necessary input for #81). This would require the ability to add points sequentially, then convert them into GPS coordinates and feed them to the rover in a message formatted as a list of GPS coordinates.

Wheel Odometry Node

Create a wheel odometry node that takes in wheel encoder messages and produces an odometry estimate.

AC: Wheel Odometry Node

Mac VM Instructions

Description

Onboarding M1 Mac users was a significant obstacle last semester. Having clear virtualization instructions should make the process of easing Mac users into the codebase far easier.

Please only tackle this issue if you have an M1 Mac! We need to verify this works on the hardware in question

Potential VM options:

• UTM
• QEMU
• Parallels

Requirements

• Decide on the best VM option
• Write the installation instructions
• Verify your installation instructions work

CalculateGoalForGate nav tree node

Description

Node that calculates the goal pose for a gate target (between the two ARUCO tag poles)

Requirements

• implements nav2 tree node
• generates a pose that's a valid goal

LED Node

urc_platform needs to have a node that sends LED commands to an LED strip in order to signify teleop, autonomously navigating to goal, or goal reached.

Should take in a msg of type urc_msgs/msg/NavigationState containing a teleop true/false and a goal_reached true/false. Blue on teleop, red on navigating to goal point, green on goal reached.

Comprehensive CI Pipeline

Currently, the pipeline only checks if everything builds correctly. Ideally we would like to perform all of the following:

• Build the project
  • colcon build
• Run unit tests
  • colcon test
• Run C++ linter
  • clang-tidy
• Run Python linter
  • pylint
• Run C++ style checker
  • clang-format
• Run Python style checker
  • flake8
• Include GitHub problem matchers to make things look cool
  • 
• Utilize a Docker container

RoboCup has a pretty sweet CI setup with GitHub actions here which is definitely worth looking at.

AC: Improved CI pipeline using GitHub Actions with features listed above

Remove implicit rosbridge float32 -> int cast workaround

Description

In #44, we discovered that rosbridge would convert floats of 0.0 to ints of 0 implicitly. This caused a lot of problems, and we implemented some scuffed workarounds to avoid it. The issue we posted in the rosbridge_suite repo has been resolved with the PR here, so it should be safe to remove the workarounds now.

Requirements

• Remove the implicit casting workarounds
• Test that the improved new method works

Remote Communications Framework

Description

To pilot the rover, we will need to be able to remotely publish and receive messages at distances up to 1 km. A web interface created using some of the libraries and tools here would make this possible.

Requirements

• Remote web interface
• Able to publish/receive messages from the rover

---

Join #urc-remote-communication on Slack to discuss!

Automated install script

Description

Bash/zsh script to automate the Ubuntu/Linux install instructions

Requirements

• .sh script

Swap to ament_uncrustify for formatting

Description

Colcon and our current framework do not play well with clang-format. Using an ament_code_style.cfg file with ament_uncrustify should resolve most of the issues we are experiencing right now.

Should resolve the lining issues mentioned in #21.

Example of what the new formatting should look like

Requirements

• No more linting errors when running colcon test-result --verbose
• Code format is more or less identical to what would be delivered using clang-format
• Get rid of any remaining Makefiles

Custom decision node for nav tree

Description

A decision node that keeps track of the goals in the decision tree. The first 3 times it will travel left in the tree, which is to go to the waypoint, and the other 4 times it will travel right to deal with either stopping at poles or going between them.

Requirements

• Traverses tree left 3 times then right 4 times, counting up if child node returns success (1)

CalculateGoalForPole nav tree node

Description

Node that calculates the goal pose for a pole target (within 2 meters of the ARUCO tag pole)

Requirements

• implements nav2 tree node
• generates a pose that's a valid goal

SendSignal nav tree node

Description

Node that sends the signals required to indicate that the rover has reached the end goal.

Requirements

• Implements a nav2 tree node
• Publishes a topic / calls a service to send the message and light up the red LED.

Useful Commands File

Description

There are a lot of commands that members should know, but might forget or never get a chance to learn.

Requirements

• Add a file in /documents with useful commands
• Ask current groups for command suggestions

Google Test Errors

Description

Mentioned in #19, the current code does not test properly.

Steps To Reproduce

1. Run colcon test-result --verbose

Result:

1. Google Test result files missing

build/urc_platform/test_results/urc_platform/test_scan_to_pointcloud.gtest.xml: 1 test, 1 error, 0 failures, 0 skipped
- urc_platform test_scan_to_pointcloud.gtest.missing_result
  <<< error message
    The test did not generate a result file.
  >>>

Expected behavior

Running colcon test-result --verbose should successfully run the Google Tests (create a separate issue or update this one with more information if they don't pass).

Additional information

Python ROS node testing interface to model our unit tests after

Migrate igvc_gazebo ground_truth node to ROS2

AC: ground_truth node is rewritten to be in ROS2.

Make Arm Controller Functional

Description

As of now, most of our arm controller code in the joy_to_servo_pub node is nonfunctional. It is held together by placeholders, and needs to be changed to reflect the actual situation once we have more information on the arm and the body of the rover.

Requirements

• Update the rosparams
• Make a README file with the controller mappings
• Update the rover body dimensions
• Properly remap topics in the launch file
• Verify that the node works correctly in Gazebo
  • You should not be able to collide the arm with the body!

ARUCO Tag Detection

Description

A CV node to detect ARUCO tags

Requirements

• Detects ARUCO tags with accuracy

Output Requirement

• number of unique ArUco tags seen (in [0, 2]) and locations of the tags seen

Gather Elevation Map Data

Get good quality elevation map for URC area.

Set up gazebo launch files.

Migrate the gazebo launch files to this codebase, removing extraneous things which aren't necessary yet.

AC: Able to launch a gazebo simulation with a single flat plane world with a launch file titled "test_world.launch".

GPS Navigation

Pull coordinates from our competition area, and navigate sequentially through those coordinates.

This issue comprises of a node/set of nodes that are able to add a set of GPS waypoints to a list (based on a message recieved), have the robot navigate to those GPS waypoints sequentially (and end at those waypoints within a +/- 1.5 m error), and return when the robot has completed the path.

After path completion, rover should stop listen for new waypoints to add to the list and a "continue" command, such that it begins executing the next set of waypoints.

Sets of waypoints will be published on a topic in order to be transferred to the robot. Message could either be generated as a json, or sent directly from the web interface (see #80).

Since legs 1 and 2 of the autonomous portion are largely flat and do not require significant obstacle avoidance, a simple movement algorithm such as a PID or LQR to get a robot from one position to another should suffice.

Create docker development environment for URC

Add magnetometer plugin to robot model

Description

The robot is missing a magnetometer plugin in the robot model. On swervi, this was done with the hector gazebo plugin, which is now available in ros2. However, there is no ros humble branch for the plugin.

This can probably be solved by including the foxy devel branch of the plugin as an external git submodule in the urc repo, and then adding the magnetometer plugin to the robot model from there.
Link to plugin

Implement Robotic Arm Controls

Description

Write nodes implementing manual robotic arm controls on the rover.

I'd recommend tackling this as a team and writing design documentation along the way. You may need to collaborate with urc-firmware, electrical, and science to get this done.

Requirements

• Robotic arm is capable of the tasks outlined here
• Design decisions are properly documented

Implement Remote Joystick Functionality

Description

Finish #42 first!

To operate remotely during the manual control segments of the URC competition, we need to be able to pass joystick messages from our web interface to the rover.

Requirements

• Joystick input messages able to be sent to the rover from the web interface
• Minimal latency if possible

Nav2 Behavior Trees

Description

In IGVC, a navigation state machine using move_base_flex was used to help decide what paths should be taken and to offer recovery options during the planning and execution of paths. Making a similar version of this system using Nav2 behavior trees will make our autonomous stack far more functional.

Fleshing out the mapping/path planning stack first might be preferable, but this issue should be able to be resolved by setting up a basic framework that can be expanded upon later.

Requirements

• Implement Nav2 behavior trees
  • Must have way to get a path
  • Must have way to execute a path
  • Must have way to recover after failed get/execute

Nav2 Path Planning

Description

Finish #56 first!

Path planning based on our costmaps enables the autonomous movement necessary for part 3 of the URC Competition.

Requirements

• Implement local and global path planning algorithms based on the competition
• Integrate into the Nav2 behavior tree stack
• Test and bug-check the configuration

Better Web Interface Design

Description

When I wrote the web interface, I only aimed to get it up and running. The code could definitely use some cleaning up and beautification from someone with a good amount of design and frontend experience.

Code is here on the repo

Requirements

• Add CSS to the web interface
• Improve the overall look of the interface
• Clean up existing code if necessary

Nanopb breaks build when CMake files are removed

Description

Manually removing the CMake build files with rm -rf build install log is a good way to verify things build on a clean system and troubleshoot CMake errors. However, it breaks nanopb in ways that I cannot figure out how to easily resolve without a fresh reinstall.

Steps To Reproduce

1. colcon build
2. rm -rf build install log
3. colcon build

Result: CMake build error hell

• This bug has been reproduced by someone else

Expected behavior

CMake build files can be cleared without major issues

DeterminePoleOrGate nav tree node

Description

nav2 tree node that splits based on the number of ARUCO tags detected

Requirements

• implements a nav2 tree node
• ticks left child if 2 ARUCO tags detected
• ticks right child if 1 ARUCO tag detected

Migrate Multiclass Semantic Segmentation

Description

Currently, the URC repository does not utilize machine learning of any kind for obstacle detection. A good way to kickstart our ML efforts would be to migrate the vision based perception strategy from the IGVC repository.

The vision based perception strategy in the IGVC repository relies on a UNET with an EfficientNet encoder to perform semantic segmentation on input images. The neural net for the IGVC competition was trained to classify pixels as line, barrel, or ground. For the URC, the network will probably have to be retrained to classify obstacles for the URC competition. Retraining will involve some effort to label images.

Our encoder is pretrained on the ImageNet dataset which allows us to utilize transfer learning to train our model. Networks trained from scratch need a massive amounts of data and compute. Even with the appropriate dataset and compute you would need to wait days or weeks to train such a model. Using this strategy we can get away with using 500ish images (versus millions of images needed if training from scratch) to fine tune the weights of the pretrained encoder and achieve highly accurate results.

Requirements

• Migrate multiclass_segmentation.py and the corresponding multiclass segmentation folder
• Migrate multiclass_segmentation.launch
• Successfully launches

CI PR Tests Error

Description

Ever since the CI pipeline rewrite, there has always been one test case that never manages to pass thanks to an error in GitHub actions. We have always just used admin overwrite access to ignore it, but we should probably figure out how to avoid needing this step during the merge process.

Steps To Reproduce

1. Open a PR
2. Pass all CI tests

Result:

Expected behavior

CI pipeline for PRs allows merging once all tests pass

Implement UDP Communication

Description

Update the UDP code in motor_controller in urc_platform so that ROS can get information from the Teensy

Requirements

• Teensy and ROS can talk to one another

Migrate and Improve Nanopb Build Pipeline

Description

Protobuf (short for "protocol buffers") is a data formatting scheme used to transmit data, usually across a network. We are using an implementation of Protobuf called Nanopb on Jessiii and Swervi, and we plan on using it for URC. In particular, we use nanopb to pass data from the computer running ROS to the microcontrollers connected to the same network.

To get a quick introduction to Nanopb, look here.

Basically, we have a .proto file which specifies what our message will look like. Then, we feed the .proto file into a python script called nanopb_generator.py, which produces a .pb.c file and a .pb.h file. Finally, when code that uses the nanopb messages defined in the .proto file, the compiler uses the .pb.c and the .pb.h files.

However, the way the igvc-software handles nanopb is clunky. Basically, software copies the .proto file from the igvc-firmware repository and runs the nanopb_generator.py script manually to produce the igvc.pb.c and the igvc.pb.h files (which you can see right here). The firmware repository handles this much more gracefully. The firmware repository has a igvc.proto file and a clone of the nanopb Github repository for the nanopb_generator.py script. Then, in the CMakeLists.txt, the .proto file is automatically converted into the .pb.c and the .pb.h files.

This automatic transformation of the .proto files will be a much needed upgrade for the urc-software stack. Doing this requires several steps:

1. Add the Nanopb repository as a git submodule.
2. Create a test .proto file and a test .cpp file in the urc_platform node. Look here for an easy example.
3. Edit the CMakeLists.txt to automatically transform the .proto file. Follow the example here (Look for the part that says "Nanopb Config". You basically want to copy up to the part that prints "Creating NanoPB target...Done!")

Requirements

• Ensure that the test .cpp script can use the messages in the .proto file. If it compiles, it should work.