Table of Contents |
---|
1. Overview
This application note explains how to use the ZED camera with the Emcraft software distribution for the Renesas RZ/V2H SoC.
2. High-level Overview
The ZED camera support may be seen as a part of the ROS2 support layer. Information on the ROS2 support layer is available from Using ROS2 Layer and APIs .
The ZED camera support is adapted from the TI’s Robotic SDK https://software-dl.ti.com/jacinto7/esd/robotics-sdk/latest/docs/source/README.html, which is based on the OpenCV
library.
3. ZED Camera Support Packages
The Emcraft software distribution installs to the target image the following ZED camera support packages:
...
NOTE: More ROS2 demo nodes from the TI’s SDK are going to be adapted soon.
3.1. Understanding ZED Packages Build Process
Emcraft makes use of a separate build environment to build the ZED camera support packages integrated into the Emcraft software distribution for the RZ/V2H SoC.
The build environment is based on the GitHub - ros/meta-ros: OpenEmbedded Layers for ROS 1 and ROS 2 Yocto meta-layer and is configured to build packages for the ARM64 and AMD64 architectures. The built binary packages are then published via a private Debian package archive. The Emcraft software distribution for the RZ/V2H SoC has the PPA included to the APT source list.
4. Verifying ZED Camera Support
4.1. Running ZED Capture ROS2 Node
4.2. Using ROS2 Interface to the ZED Camera
Step through the following procedure to demonstrate use of the ROS2 interface to the ZED camera:
Open a terminal window and login onto the RZ/V2H EVK board using
ssh
:Code Block <terminal 1>$ ssh user@<evk IP>
Connect the ZED camera to the EVK USB 3.x interface.
Check that ZED camera is recognized by USB:
Code Block user@rzv2hevkalphaemc:~$ v4l2-ctl --list-devices ... ZED 2i: ZED 2i (usb-15860000.usb-1): /dev/video0 /dev/video1 /dev/media5 ...
Run the ROS2 camera node wrapper:
Code Block <terminal 1>$ ros2 launch zed_capture zed_capture_launch.py zed_sn_str:=SN10026720
Note: If the camera is attached to a different
/dev/videoX
node, then add thecam_id:=<node num>
argument toros2 launch
(egcam_id:=2
for/dev/video2
).
The node will start with providing logs and information on the detected ZED camera:Code Block [INFO] [launch]: All log files can be found below /home/user/.ros/log/2023-12-18-11-24-52-897616-rzv2hevkalpha-2902 [INFO] [launch]: Default logging verbosity is set to INFO [INFO] [zed_capture-1]: process started with pid [2945] [zed_capture-1] [INFO] [1702898698.084998416] [zed_capture]: Initialize the ZED camera [zed_capture-1] [ WARN:0] global ./modules/videoio/src/cap_gstreamer.cpp (2075) handleMessage OpenCV | GStreamer warning: Embedded video playback halted; module source reported: Could not read from resource. [zed_capture-1] [ WARN:0] global ./modules/videoio/src/cap_gstreamer.cpp (1053) open OpenCV | GStreamer warning: unable to start pipeline [zed_capture-1] [ WARN:0] global ./modules/videoio/src/cap_gstreamer.cpp (616) isPipelinePlaying OpenCV | GStreamer warning: GStreamer: pipeline have not been created [zed_capture-1] [INFO] [1702898699.271579125] [StereoCamera]: YUV422 [zed_capture-1] [INFO] [1702898699.271702458] [StereoCamera]: Stereo Camera Mode HD, width 1280, height 720 [zed_capture-1] [INFO] [1702898699.432589500] [zed_capture]: Loading camera info from yaml files [zed_capture-1] [INFO] [1702898699.435826125] [zed_capture]: camera calibration URL: file:///opt/ros/humble/share/zed_capture/config/SN10026720_HD_camera_info_left.yaml [zed_capture-1] [INFO] [1702898699.439227625] [zed_capture]: camera calibration URL: file:///opt/ros/humble/share/zed_capture/config/SN10026720_HD_camera_info_right.yaml [zed_capture-1] [INFO] [1702898705.495579435] [zed_capture]: Successfully found the ZED camera ...
Open another terminal window and login onto the EVK:
Code Block <terminal 2>$ ssh user@<evk IP>
Verify that the camera related topics are present:
Code Block <terminal 2>$ ros2 topic list /camera/left/camera_info /camera/left/image_raw /camera/right/camera_info /camera/right/image_raw /parameter_events /rosout
Capture the camera information via the
camera/left/camera_info
topic:Code Block <terminal 2>$ ros2 topic echo /camera/left/camera_info
The application will start receiving the camera info message:
Code Block header: stamp: sec: 1695145271 nanosec: 503948931 frame_id: left_frame height: 720 width: 1280 distortion_model: plumb_bob d: - -0.174814 - 0.0277134 - 0.0 - 0.0 - 0.0 k: - 701.695 - 0.0 - 645.375 - 0.0 - 701.695 - 351.9755 - 0.0 - 0.0 - 1.0 r: - 0.9999991771672719 - 0.00021264049636184653 - -0.0012650884548001328 - -0.00021565273732970637 - 0.9999971412291467 - -0.0023813919103813186 - 0.001264578457844275 - 0.0023816627706823587 - 0.999996364255276 p: - 686.7164431988998 - 0.0 - 643.5791931152344 - 0.0 - 0.0 - 686.7164431988998 - 342.6826591491699 - 0.0 - 0.0 - 0.0 - 1.0 - 0.0 binning_x: 0 binning_y: 0 roi: x_offset: 0 y_offset: 0 height: 0 width: 0 do_rectify: false --- ...
Capture the camera information via the
camera/right/camera_info
topic:Code Block <terminal 2>$ ros2 topic echo /camera/right/camera_info
The application will start receiving the camera info message:
Code Block header: stamp: sec: 1695145324 nanosec: 367362789 frame_id: right_frame height: 720 width: 1280 distortion_model: plumb_bob d: - -0.174312 - 0.0273345 - 0.0 - 0.0 - 0.0 k: - 700.655 - 0.0 - 636.98 - 0.0 - 700.655 - 343.9995 - 0.0 - 0.0 - 1.0 r: - 1.0 - 2.1923516186951437e-14 - -1.2767118909117024e-13 - -2.1624428776507077e-14 - 0.9999971641585754 - 0.002381527830432345 - 1.2772216711642694e-13 - -0.002381527830432344 - 0.9999971641585754 p: - 686.7164431988998 - 0.0 - 643.5791931152344 - -30357.52616045848 - 0.0 - 686.7164431988998 - 342.6826591491699 - 0.0 - 0.0 - 0.0 - 1.0 - 0.0 binning_x: 0 binning_y: 0 roi: x_offset: 0 y_offset: 0 height: 0 width: 0 do_rectify: false --- ...
Capture the camera information via the
camera/left/camera_info
topic:Code Block <terminal 2>$ ros2 topic echo /camera/left/image_raw
The application will start receiving the camera info message:
Code Block header: stamp: sec: 1695145357 nanosec: 770892888 frame_id: left_frame height: 720 width: 1280 encoding: yuv422 is_bigendian: 0 step: 2560 data: - 132 - 11 - 134 - 12 - 132 - 11 - 134 - 11 - 132 - 11 - 134 - 11 - 132 - 11 - 134 - 11 - 132 - 12 - 134 - 11 - 132 - 12 - 134 - 11 - 132 - 12 - 134 - 12 - 132 - 12 - 134 - 12 - 132 - 11 - 134 - 11 - 132 - 12 - 134 - 12 - 132 - 12 - 134 - 12 - 132 - 12 - 134 - 12 - 132 - 12 - 134 - 12 - 132 - 12 - 134 - 12 - 132 - 12 - 134 - 13 - 132 - 12 - 134 - 12 - 132 - 12 - 135 - 12 - 132 - 13 - 135 - 12 - 133 - 13 - 135 - 14 - 133 - 19 - 136 - 18 - 133 - 23 - 137 - 24 - 133 - 28 - 137 - 25 - 133 - 33 - 138 - 29 - 134 - 46 - 137 - 90 - 120 - 162 - 135 - 170 - 106 - 153 - 135 - 105 - 121 - 57 - 133 - 29 - 131 - 37 - 133 - 40 - 130 - 54 - 133 - 49 - 130 - 36 - 132 - 26 - 131 - 27 - 133 - 29 - 131 - 32 - 134 - 25 - '...' --- ...
Capture the camera information via the
camera/right/camera_info
topic:Code Block <terminal 2>$ ros2 topic echo /camera/right/image_raw
The application will start receiving the camera info message:
Code Block header: stamp: sec: 1695145404 nanosec: 252655285 frame_id: right_frame height: 720 width: 1280 encoding: yuv422 is_bigendian: 0 step: 2560 data: - 131 - 29 - 134 - 25 - 131 - 25 - 134 - 25 - 132 - 29 - 134 - 27 - 132 - 22 - 134 - 14 - 132 - 13 - 134 - 13 - 132 - 14 - 134 - 13 - 132 - 13 - 134 - 14 - 132 - 13 - 134 - 13 - 132 - 13 - 134 - 13 - 132 - 13 - 134 - 14 - 132 - 13 - 134 - 12 - 132 - 13 - 134 - 14 - 132 - 13 - 134 - 13 - 132 - 13 - 134 - 13 - 132 - 13 - 134 - 12 - 132 - 13 - 134 - 13 - 132 - 13 - 134 - 13 - 132 - 13 - 134 - 13 - 133 - 13 - 134 - 13 - 133 - 13 - 134 - 13 - 133 - 13 - 135 - 13 - 133 - 13 - 135 - 13 - 133 - 13 - 135 - 13 - 133 - 13 - 135 - 13 - 132 - 13 - 135 - 13 - 132 - 13 - 135 - 13 - 132 - 14 - 135 - 13 - 132 - 13 - 134 - 13 - 132 - 12 - 134 - 12 - 132 - 13 - 134 - 12 - 132 - 12 - 134 - 12 - 132 - 12 - 134 - 12 - '...' --- ...
5. Recording Capture and Playing Back the Captured Data in RVIZ
The ZED camera video capture can be played back on a host with installed rviz2
. The capture is streamed by the ROS2 layer using the RTPS protocol as being captured, hence it can be played back online. Also, in the absence of an Ethernet connection between the RZ/V2H EVK and the host running rviz2
, the ROS2 data can be captured into a bag and then played back by the host.
...