watonomous / wato_monorepo Goto Github PK

We do this with semantic segmentation and lane detection.

Run inference with BEVFusion. https://github.com/WATonomous/Perception-Research/tree/main/src/bevfusion

Obstacle.msg: https://git.uwaterloo.ca/WATonomous/wato_monorepo/-/blob/develop/src/ros_msgs/common_msgs/msg/Obstacle.msg

• Read in a lidar + camera scan from the /lidar + /camera topic
• Run inference with bevfusion
• output 3D bounding box predictions to /detections_3d

Another person will also need to help out with creating visualization tools, annotating the image directly and visualizing that.

Instead of our three nodes for camera detection, we want one unified node that outputs one message with detections for resource efficiency, and temporal synchronization, or we could have a post processing node that applies a time synchronizer using http://wiki.ros.org/message_filters

June 13 2024

• Running in foxglove with post-processing synchronization (?)
• Still testing post-process temporal synchronization
• Still working on getting SLURM + foxglove working on all machines

June 13 2024

• Done!

Port over the camera object detection node for https://git.uwaterloo.ca/WATonomous/wato_monorepo/-/tree/develop/src/camera_detection into our camera_object_detection node.

Dockerfile: https://git.uwaterloo.ca/WATonomous/wato_monorepo/-/blob/develop/docker/camera_detection/Dockerfile

They make the distinct between left and right camera, you can ignore that for now. We will use the same format for Obstacle for now.

• Read in an image from the /camera topic
• Run inference with YOLO
• output bounding box predictions to /detections_2d, Obstacle.msg is defined here

Another person will also need to help out with creating visualization tools, annotating the image directly and visualizing that.

Refer to https://git.uwaterloo.ca/WATonomous/wato_monorepo/-/blob/develop/src/camera_detection/yolov5_detection.py

Just for consistency with traffic_signs

June 13 2024

• Addressed most PR comments
• Need to create labels server still
• WHY AREN'T BUILDS BEING SAVED TO THE REGISTRY!!! I DON'T WANT TO WAIT 321312HRS FOR MY CONTAINER TO BUILD TO DO WORK!!!

To be consistent across monorepo, and good practice.

For example:
https://github.com/WATonomous/wato_monorepo/blob/main/src/perception/camera_object_detection/config/eve_config.yaml

Port over the lidar object detection node for https://git.uwaterloo.ca/WATonomous/wato_monorepo/-/tree/perp/add-pp/src/lidar_cuda into our lidar_object_detection node. Original repo is here: https://github.com/open-mmlab/mmdetection3d

Obstacle.msg: https://git.uwaterloo.ca/WATonomous/wato_monorepo/-/blob/develop/src/ros_msgs/common_msgs/msg/Obstacle.msg

• Read in a lidar scan from the /lidar topic
• Run inference with pointpillars
• output 3D bounding box predictions to /detections_3d

Another person will also need to help out with creating visualization tools, annotating the image directly and visualizing that.

LiDAR velocity estimation which is redundancy for radar velocity detection

June 13 2024

• Can't build due to docker filesystem error

because too many messages, I think ZED also does it this way

Bags can be found in /mnt/wato-drive/rosbags

You can also try this repo to start up a server and see rosbags: https://git.uwaterloo.ca/WATonomous/rosbag-server

Steps:

1. Use https://gitlab.com/ternaris/rosbags to convert the rosbags. On the server, you can use the VM container from Perception Research to run the pip install rosbags command. Make sure everything is mounted properly in the docker-compose.yaml so the rosbags generated on the container can be accessed outside the container.
2. Store the ros2 bags inside /mnt/wato-drive2/ros2bags
3. Write a bag_bridge profile for monorepo_v2, essentially a port from https://git.uwaterloo.ca/WATonomous/wato_monorepo/-/blob/develop/profiles/docker-compose.bag_bridge.yaml, but we are using ROS2 now, and the mount paths are different (should be like this /mnt/wato-drive2/ros2bags:/bags)

In the end, the following command should work:

watod2 bag_bridge ros2 bag play -r 1.0 /path/to/bag

On line 34, I need to add a apt update in front , https://github.com/WATonomous/wato_monorepo/blob/8a82913a92717fb9b77ccc211830eb999179edc2/docker/interfacing/sensor_interfacing/sensor_interfacing.Dockerfile

else I get the following error

The reference on line 24 doesn't include an apt update.

The image_size parameter name is misleading because it doesn't actually represent the resolution of the camera, but rather the resolution of the image that is fed to the model.

We need 3D object tracks to send to world modelling
Potential Implementation:
https://github.com/ZHOUYI1023/awesome-radar-perception?tab=readme-ov-file#Velocity-Estimation

June 13 2024

• Have tracked objects working in Nuscenes!
• Working on integrating actual detections from the perceptions pipeline (blocked by 2D to 3D(

1. Investigate SOTA dnn-based monocular depth estimation models and test performance on our GPUs
2. Integrate model into ROS 2 node
3. (separate task) Generating point clouds from depth estimation outputs

Old ros2 node for reference: WATonomous/deepracer_ws#2

June 13 2024
- Moving Forward:

• Fix sourcing issue
• Take in rosbag of nuscenes data and get real time predictions
• Integrate into WATO infrastructure

We don't care about object clustering here, we care about velocity detections.
Near scan and far scan filtering from radar pc

June 13 2024
Using Polymath driver

• Working with reading from a file
• Not set up or tested with radar data