Comments (27)
from isaac_ros_visual_slam.
Yes, Elbrus SVIO has been used successfully with ZED2.
Thank you for your reply!
so deployingf to jetson 4.6 with jetpack4.5 argument to try on jetson with zed2?
It would require to buils the sample app somehow?
how to build it with analogy to
bazel build //apps/samples/zed_camera
there is no elbrus sample app at the Host PC
./../engine/engine/build/deploy.sh --remote_user user -p //apps/samples/elbrus_visual_slam_zed:elbrus_visual_slam_zed-pkg -d jetpack45 -h 192.168.x.x
jetpack 4.6 argument doesn't seem supported for jetson deployment?
also can not find as for now deployment line for the elbrus zed svo
Ffrom the instruction:
The “elbrus_visual_slam_zed” sample application uses the ZED camera, which performs software undistortion inside the StereoLabs SDK.
but there is no such app in the sample app folder
It only has
april_tags follow_me imu_calibration manipulation pwm stereo_rectification vicon_mocap
argus_camera hgmm_matching imu_sim navigation_rosbridge python_api subprocess zed_camera
BUILD image_undistortion __init__.py pick_and_place realsense_camera teleport_sim
camera imu interactive_markers proto_to_json stereo_dummy v4l2_camera
sdk/packages_jetpack45/visual_slam$ ls
BUILD libvisual_slam_module.so
/sdk/packages_x86_64_cuda_11_1/visual_slam$ ls
BUILD libvisual_slam_module.so
from isaac_ros_visual_slam.
where to get these files on jetson? packages/visual_slam/apps/elbrus_visual_slam_zed_python.py
trying to build on the Host PC with
bazel build packages/visual_slam/apps/elbrus_visual_slam_zed
@hemalshahNV but how to deploy it to jetson once built?
from isaac_ros_visual_slam.
from isaac_ros_visual_slam.
@hemalshahNV Thank you for your response: "Are you using Elbrus in Isaac SDK?"
No, I used the ROS node with realsense without the IsaaC.
However as you confirmed that zed2 works wit hElbrus, coudl you extend if it is supported in a standalone mode [ like with realsense] or with Isaac + Elbrus mode? Both? neither of the two?
Moreover, the linked reference thread is at the first message in the github issues post
https://forums.developer.nvidia.com/t/will-it-work-with-the-zed2-the-elbrus/196452
from isaac_ros_visual_slam.
anyway posted at both places, could you extend on steps to run zed2 with elbrus anyhow please e.g. in the setup you confirmed previously, please?
from isaac_ros_visual_slam.
I'm afraid I'm not following what you're trying to do exactly, but let me see if I can help on the ROS-side of the conversation here and address the Isaac SDK part of it in the Dev Forum post you made (my bad, did not notice the link in your first post). You would first have to go prepare the Zed2 ROS2 packages here. Then, there is an Isaac ROS launch file that should bring up the VO nodes with topics remapped for Zed. All you would have to do is launch the ZED camera ROS2 node then launch this Isaac ROS VO launch file and you should be up and running. We had configured the Realsense version of this launch file to also bring up the Realsense camera node as well but didn't have a chance to do something similar for ZED though, so we had to leave launching the ZED camera node up to the user.
from isaac_ros_visual_slam.
@hemalshahNV
Thank you for your reply.
I will be trying to setup the zed2ros2 package to run it on jetson in foxy2 container docker implementation provided by you.
But I will have to separately copy the file https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_visual_odometry/blob/main/isaac_ros_visual_odometry/launch/isaac_ros_visual_odometry_zed2.launch.py manually to Jetson, right? or it won't work with just copying the single file but would need to deploy also the environment with Isaac[bazel] from Host PC to Jetson?
from isaac_ros_visual_slam.
@hemalshahNV
after installing zed2 srapper copying pyhton script getting this on ros2 foxy jetson
after running zed2 node.:
ros2 launch zed_wrapper zed2.launch.py
how to run the python file
ros2 launch isaac_ros_visual_odometry isaac_ros_visual_odometry_zed2.launch.py
like that?
ros2 topic echo /visual_odometry/tf_stamped
However the output is empty of the latter.
Any ideas?
from isaac_ros_visual_slam.
ros2 launch isaac_ros_visual_odometry isaac_ros_visual_odometry_zed2.launch.py
[INFO] [launch]: All log files can be found below /home/user/.ros/log/-------
[INFO] [launch]: Default logging verbosity is set to INFO
[INFO] [component_container-1]: process started with pid [198027]
[component_container-1] [INFO] [1638199221.682004070] [visual_odometry_launch_container]: Load Library: /home/user/your_ws/src/install/isaac_ros_visual_odometry/lib/libvisual_odometry_node.so
[component_container-1] [INFO] [1638199221.959088912] [visual_odometry_launch_container]: Found class: rclcpp_components::NodeFactoryTemplate<isaac_ros::visual_odometry::VisualOdometryNode>
[component_container-1] [INFO] [1638199221.959244050] [visual_odometry_launch_container]: Instantiate class: rclcpp_components::NodeFactoryTemplate<isaac_ros::visual_odometry::VisualOdometryNode>
[INFO] [launch_ros.actions.load_composable_nodes]: Loaded node '/visual_odometry_node' in container '/visual_odometry_launch_container'
from isaac_ros_visual_slam.
I will be trying to setup the zed2ros2 package to run it on jetson in foxy2 container docker implementation provided by you.
But I will have to separately copy the file https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_visual_odometry/blob/main/isaac_ros_visual_odometry/launch/isaac_ros_visual_odometry_zed2.launch.py manually to Jetson, right? or it won't work with just copying the single file but would need to deploy also the environment with Isaac[bazel] from Host PC to Jetson?
You will need to checkout the Isaac ROS VO repository to your Jetson and build the packages within using the ROS2 standard colcon. There is no Bazel involved. Your steps in the subsequent post look right. You may want to check that the topics are all mapped correctly with ros2 topic list
and ros2 topic info
to make sure everything is wired appropriately.
from isaac_ros_visual_slam.
@hemalshahNV Thank you for your reply!
It seems I built all components on jetson. right?
colcon build --symlink-install && source install/setup.bash
Starting >>> isaac_ros_common
Starting >>> isaac_ros_test
Starting >>> zed_interfaces
Starting >>> isaac_ros_nvengine_interfaces
Starting >>> isaac_ros_visual_odometry_interfaces
Finished <<< isaac_ros_common [2.72s]
Finished <<< isaac_ros_visual_odometry_interfaces [4.44s]
Finished <<< isaac_ros_test [4.89s]
Starting >>> isaac_ros_image_proc
Starting >>> isaac_ros_stereo_image_proc
Finished <<< isaac_ros_nvengine_interfaces [4.98s]
Starting >>> isaac_ros_nvengine
Finished <<< isaac_ros_image_proc [1.90s]
Starting >>> isaac_ros_visual_odometry
Finished <<< isaac_ros_nvengine [1.82s]
Finished <<< zed_interfaces [7.26s]
Starting >>> zed_components
Finished <<< isaac_ros_visual_odometry [1.24s]
Finished <<< zed_components [1.09s]
Starting >>> zed_wrapper
Finished <<< zed_wrapper [0.85s]
Starting >>> zed_ros2
Finished <<< zed_ros2 [0.65s]
Finished <<< isaac_ros_stereo_image_proc [14.6s]
Starting >>> isaac_ros_image_pipeline
Finished <<< isaac_ros_image_pipeline [0.63s]
Summary: 13 packages finished [21.1s]
The topic list shows the below.
~/your_ws$ ros2 topic list
/diagnostics
/parameter_events
/rosout
/tf
/tf_static
/visual_odometry/tf_stamped
/zed2/joint_states
/zed2/robot_description
/zed2/zed_node/atm_press
/zed2/zed_node/confidence/confidence_map
/zed2/zed_node/depth/camera_info
/zed2/zed_node/depth/depth_registered
/zed2/zed_node/disparity/disparity_image
/zed2/zed_node/imu/data
/zed2/zed_node/imu/data_raw
/zed2/zed_node/imu/mag
/zed2/zed_node/left/camera_info
/zed2/zed_node/left/image_rect_color
/zed2/zed_node/left/image_rect_gray
/zed2/zed_node/left_cam_imu_transform
/zed2/zed_node/left_raw/camera_info
/zed2/zed_node/left_raw/image_raw_color
/zed2/zed_node/left_raw/image_raw_gray
/zed2/zed_node/odom
/zed2/zed_node/path_map
/zed2/zed_node/path_odom
/zed2/zed_node/point_cloud/cloud_registered
/zed2/zed_node/pose
/zed2/zed_node/pose_with_covariance
/zed2/zed_node/rgb/camera_info
/zed2/zed_node/rgb/image_rect_color
/zed2/zed_node/rgb/image_rect_gray
/zed2/zed_node/rgb_raw/camera_info
/zed2/zed_node/rgb_raw/image_raw_color
/zed2/zed_node/rgb_raw/image_raw_gray
/zed2/zed_node/right/camera_info
/zed2/zed_node/right/image_rect_color
/zed2/zed_node/right/image_rect_gray
/zed2/zed_node/right_raw/camera_info
/zed2/zed_node/right_raw/image_raw_color
/zed2/zed_node/right_raw/image_raw_gray
/zed2/zed_node/stereo/image_rect_color
/zed2/zed_node/stereo_raw/image_raw_color
/zed2/zed_node/temperature/imu
/zed2/zed_node/temperature/left
/zed2/zed_node/temperature/right
the topic info shows
ros2 topic info /visual_odometry/tf_stamped
Type: isaac_ros_visual_odometry_interfaces/msg/VisualOdometryTransformStamped
Publisher count: 1
Subscription count: 1
zed topics have the output, but the visual odometry topic is silent
from isaac_ros_visual_slam.
~/your_ws/src$ ls
isaac_ros_common isaac_ros_image_pipeline isaac_ros_visual_odometry zed-ros2-wrapper
****
from isaac_ros_visual_slam.
does zed2 need certain firmware? sdk version? I just installed the latest available sdk from stereolabs website. zed2 node seems working with it as ther is e..g pose topic poblished
does the phrase below reffer to docker container?
Loaded node '/visual_odometry_node' in container '/visual_odometry_launch_container
The setup is systemwide, no container is used
from isaac_ros_visual_slam.
after removing src folder then just cloning zed wrapper/ odometry
then building just two packages
cat log.txt
In file included from /home/user/your_ws/install/zed_components/include/zed_components/sl_tools.h:50,
from /home/user/your_ws/install/zed_components/include/zed_components/zed_camera_component.hpp:29,
from /home/user/your_ws/zed-ros2-wrapper/zed_wrapper/src/zed_wrapper.cpp:26:
/usr/local/zed/include/sl/Camera.hpp:784:13: warning: ISO C++ prohibits anonymous structs [-Wpedantic]
};
^
/usr/local/zed/include/sl/Camera.hpp:937:13: warning: ISO C++ prohibits anonymous structs [-Wpedantic]
};
^
/usr/local/zed/include/sl/Camera.hpp:1114:13: warning: ISO C++ prohibits anonymous structs [-Wpedantic]
};
^
/usr/local/zed/include/sl/Camera.hpp:1278:13: warning: ISO C++ prohibits anonymous structs [-Wpedantic]
};
^
/usr/local/zed/include/sl/Camera.hpp:1282:13: warning: ISO C++ prohibits anonymous structs [-Wpedantic]
};
^
/usr/local/zed/include/sl/Camera.hpp:1286:13: warning: ISO C++ prohibits anonymous structs [-Wpedantic]
};
^
/usr/local/zed/include/sl/Camera.hpp:1526:13: warning: ISO C++ prohibits anonymous structs [-Wpedantic]
};
^
/usr/local/zed/include/sl/Camera.hpp:1530:13: warning: ISO C++ prohibits anonymous structs [-Wpedantic]
};
^
/usr/local/zed/include/sl/Camera.hpp:1534:13: warning: ISO C++ prohibits anonymous structs [-Wpedantic]
};
^
/usr/local/zed/include/sl/Camera.hpp: In function ‘cudaError sl::__cudaSafeCall(cudaError, const char*, const char*, int)’:
/usr/local/zed/include/sl/Camera.hpp:2048:89: warning: unused parameter ‘file’ [-Wunused-parameter]
daError __cudaSafeCall(cudaError err, const char *func, const char *file, const int line) {
~~~~~~~~~~~~^~~~
/usr/local/zed/include/sl/Camera.hpp:2048:105: warning: unused parameter ‘line’ [-Wunused-parameter]
daSafeCall(cudaError err, const char *func, const char *file, const int line) {
~~~~~~~~~~^~~~
In file included from /home/user/your_ws/install/zed_components/include/zed_components/sl_tools.h:50,
from /home/user/your_ws/install/zed_components/include/zed_components/zed_camera_component.hpp:29,
from /home/user/your_ws/zed-ros2-wrapper/zed_wrapper/src/zed_wrapper.cpp:26:
/usr/local/zed/include/sl/Camera.hpp: At global scope:
/usr/local/zed/include/sl/Camera.hpp:5335:12: warning: type qualifiers ignored on function return type [-Wignored-qualifiers]
inline const void /*@cond SHOWHIDDEN*/SL_SDK_EXPORT/*@endcond*/ getZEDSDKBuildVersion(int &major, int& minor, int& patch) {
^~~~~
In file included from /home/user/your_ws/install/zed_components/include/zed_components/sl_tools.h:50,
from /home/user/your_ws/install/zed_components/include/zed_components/zed_camera_component.hpp:29,
from /home/user/your_ws/zed-ros2-wrapper/zed_wrapper/src/zed_wrapper.cpp:26:
/usr/local/zed/include/sl/Camera.hpp:5724:2: warning: extra ‘;’ [-Wpedantic]
};
^
from isaac_ros_visual_slam.
Are you seeing any traffic on topic /zed2/zed_node/left/image_rect_gray
? (ros2 topic hz /zed2/zed_node/left/image_rect_gray
) The isaac_ros_visual_odometry_zed2.launch.py launch file should work remap to the topics coming from the Zed camera in your ros2 topic list, so everything should be wired together. We need to confirm that Zed camera is actually outputting anything.
from isaac_ros_visual_slam.
@hemalshahNV the p;ose works
ros2 topic hz /zed2/zed_node/pose
average rate: 15.156
the rect grey seems empty
ros2 topic hz /zed2/zed_node/left/image_rect_gray
from isaac_ros_visual_slam.
@hemalshahNV
Thanks
seems done by installing the package below
https://github.com/ros-perception/image_common/archive/refs/tags/3.0.0.tar.gz
ros2 topic hz /visual_odometry/tf_stamped
average rate: 15.116
min: 0.059s max: 0.073s std dev: 0.00334s window: 17
average rate: 15.146
min: 0.059s max: 0.073s std dev: 0.00301s window: 33
average rate: 15.141
min: 0.059s max: 0.073s std dev: 0.00284s window: 49
from isaac_ros_visual_slam.
so for zed2 there is no launch script for vizualization anyhow with issac sim? else?
the most simple way to vizualise will be which from the text outputs for zed2? Thanks
from isaac_ros_visual_slam.
@hemalshahNV
Could you confirm there is still no way to run elbrus on Jetson with basel? only with ros2?
Thanks
AV
from isaac_ros_visual_slam.
odometry package won't build anymore in container
I need to test ros2 elbrus with zed somehow
/workspaces/isaac_ros-dev/src/isaac_ros_visual_slam-0.9.1-ea2/isaac_ros_visual_odometry/src/visual_odometry_node.cpp:598:25: error: request for member ‘num_parameters’ in ‘elbrus_camera’, which is of non-class type ‘int’
elbrus_camera.num_parameters = 4;
^~~~~~~~~~~~~~
/workspaces/isaac_ros-dev/src/isaac_ros_visual_slam-0.9.1-ea2/isaac_ros_visual_odometry/src/visual_odometry_node.cpp:605:25: error: request for member ‘distortion_model’ in ‘elbrus_camera’, which is of non-class type ‘int’
elbrus_camera.distortion_model = kPinhole;
^~~~~~~~~~~~~~~~
/workspaces/isaac_ros-dev/src/isaac_ros_visual_slam-0.9.1-ea2/isaac_ros_visual_odometry/src/visual_odometry_node.cpp:606:25: error: request for member ‘num_parameters’ in ‘elbrus_camera’, which is of non-class type ‘int’
elbrus_camera.num_parameters = 4;
^~~~~~~~~~~~~~
/workspaces/isaac_ros-dev/src/isaac_ros_visual_slam-0.9.1-ea2/isaac_ros_visual_odometry/src/visual_odometry_node.cpp: At global scope:
/workspaces/isaac_ros-dev/src/isaac_ros_visual_slam-0.9.1-ea2/isaac_ros_visual_odometry/src/visual_odometry_node.cpp:618:1: error: ‘ELBRUS_Image’ does not name a type
ELBRUS_Image VisualOdometryNode::VisualOdometryImpl::ToElbrusImage(
^~~~~~~~~~~~
/workspaces/isaac_ros-dev/src/isaac_ros_visual_slam-0.9.1-ea2/isaac_ros_visual_odometry/src/visual_odometry_node.cpp:633:1: error: ‘ELBRUS_ImuMeasurement’ does not name a type
ELBRUS_ImuMeasurement VisualOdometryNode::VisualOdometryImpl::ToElbrusImuMeasurement(
^~~~~~~~~~~~~~~~~~~~~
/workspaces/isaac_ros-dev/src/isaac_ros_visual_slam-0.9.1-ea2/isaac_ros_visual_odometry/src/visual_odometry_node.cpp: In destructor ‘isaac_ros::visual_odometry::VisualOdometryNode::VisualOdometryImpl::~VisualOdometryImpl()’:
/workspaces/isaac_ros-dev/src/isaac_ros_visual_slam-0.9.1-ea2/isaac_ros_visual_odometry/src/visual_odometry_node.cpp:742:7: error: ‘elbrus_handle’ was not declared in this scope
if (elbrus_handle != nullptr) {ELBRUS_DestroyTracker(elbrus_handle);}
^~~~~~~~~~~~~
/workspaces/isaac_ros-dev/src/isaac_ros_visual_slam-0.9.1-ea2/isaac_ros_visual_odometry/src/visual_odometry_node.cpp:742:7: note: suggested alternative: ‘elbrus_cameras’
if (elbrus_handle != nullptr) {ELBRUS_DestroyTracker(elbrus_handle);}
^~~~~~~~~~~~~
elbrus_cameras
/workspaces/isaac_ros-dev/src/isaac_ros_visual_slam-0.9.1-ea2/isaac_ros_visual_odometry/src/visual_odometry_node.cpp:742:34: error: ‘ELBRUS_DestroyTracker’ was not declared in this scope
if (elbrus_handle != nullptr) {ELBRUS_DestroyTracker(elbrus_handle);}
^~~~~~~~~~~~~~~~~~~~~
make[2]: *** [CMakeFiles/visual_odometry_node.dir/src/visual_odometry_node.cpp.o] Error 1
make[1]: *** [CMakeFiles/visual_odometry_node.dir/all] Error 2
make: *** [all] Error 2
from isaac_ros_visual_slam.
@hemalshahNV
We managed to run zedm with prior version of the odometry
However, it only seem to generate one topic as resulting output
/visual_odometry/tf_stamped
is that correct?
from isaac_ros_visual_slam.
@hemalshahNV does it have zed support by now? with the most recent release?
Thanks
from isaac_ros_visual_slam.
We have not developed any ZED support specifically but there is no reason that it should not work that we know of if you setup the ZED SDK and everything properly.
from isaac_ros_visual_slam.
@hemalshahNV Thank you for following up!
However, while installing the SDK from stereolabs is not an issue at all, but a trivial task,
the problem is to get zed stereolabs humble wrapper node installed in humble container supplied from you.
Neither it seems to build even if to update the os to 22.04 to get apt ros humble support
If you will have any luck with testing the zed coherence with slam or nvblox, let me know please
from isaac_ros_visual_slam.
@hemalshahNV hello,
Was isaac_ros_visual_odometry_zed2.launch.py deleted from the repository? I wonder what were the launch parameters for ZED and visual slam nodes.
I tried making ZED camera work with Isaac Visual SLAM, but unfortunately I'm getting messages that the tracker got lost.
I've remapped all the frames, tried disabling ZED's tf2, nothing helped..
from isaac_ros_visual_slam.
@shtern94 , yes, we removed the ZED launch file because it was not being tested regularly at the time as changes to the Stereolabs SDK were coming through. In an upcoming release, we're working on adding more complete ZED support and currently debugging ZED and Isaac ROS Visual SLAM in fact. When we find anything, I'll report back.
from isaac_ros_visual_slam.
Related Issues (20)
- /visual_slam/status is incorrect ? HOT 1
- IMU fusion status HOT 6
- Cuda issue when running roslaunch command. HOT 2
- Feature request : localization only mode. HOT 2
- isaac_ros_visual_slam_zed.launch.py If an error occurs when running HOT 2
- No messages being published HOT 2
- Problem with the quickstart HOT 1
- no vio with zedm camera HOT 1
- LoadMapAdnLocalize crashes HOT 3
- cuvslam HOT 1
- EuRoc dataset
- Is it possible to increase odometry/slam rate past camera rate?
- Realsense camera not shown When I run the launch file
- problem with colcon build
- isaac sim freeze when ros visual slam is being run
- cuVSLAM not publishing any points or point cloud
- Support for Zed cameras under Isaac ROS version 3 with JP 6 HOT 10
- Low frame rate HOT 2
- estabilishing communcation of ROS2 to another PC
- Why wait one second for a transform?
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from isaac_ros_visual_slam.