...
| Table of Contents | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
1. Overview
This application note describes how to run the DRP-AI sample for the YOLOv2 model with the Renesas RZ/V2H EVK (RZ/V2H SoC-based embedded computer), using a USB Full HD resolution (1920x1080) camera.
This functionality is available with the Emcraft software distribution for the RZ/V2H, Release 1.0.5-42 or higher.
1.1. DRP-AI
DRP-AI is an AI accelerator developed by Renesas, designed to provide high-performance and low-power AI processing for embedded applications. It addresses the challenges of implementing AI on embedded devices, which may have limited resources or power constraints. The DRP-AI accelerator integrates pre- and post-processing with AI inference, resulting in superior AI processing performance compared to many other accelerators. The DRP-AI Translator tool maximises DRP-AI performance by enabling it to handle all operations of an AI model. The DRP-AI TVM, based on the Apache TVM ML compiler framework, supports multiple AI frameworks and allows operation in conjunction with the CPU, expanding the range of AI models that can be run. The DRP-AI Support Package provides the necessary driver and guide for operating the DRP-AI. The DRP-AI accelerator is embedded in the Renesas RZ/V2H series MPUs, delivering high-speed AI processing with high power efficiency.
Pleaser refer to the DRP-AI page on the Renesas web site for more information.
1.2. YOLO v2
YOLOv2 (You Only Look Once version 2) is a real-time object detection algorithm that aims to efficiently detect and classify objects in images or video frames. It is an improvement over the original YOLO algorithm, offering better accuracy and performance.
...
In this example, YOLOv2 identifies and localises a bicycle, car, and dog within the input image. The confidence scores indicate the algorithm's level of confidence in the predictions, while the bounding boxes specify the location and size of each detected object.
1.3. Supported Classes
Here is a list of objects that the YOLOv2 sample is capable of detecting:
...
Detection example provided with YOLO v2 sample:
...
1.4. Logitech BRIO 4K USB Camera
The While the demo supports any USB Full HD resolution (1920x1080) , Emcraft has explicitly validated with the Logitech BRIO 4K USB Camera is an Ultra HD camera manufactured by Logitech. It is designed to capture high-resolution video and provide exceptional image quality.
The Logitech BRIO 4K USB Camera can be purchased from various online retailers, including Amazon.
2. Running the DRP-AI Demo
2.1. Connecting Logitech BRIO 4K USB Camera
To connect the Logitech BRIO 4K USB Camera to the Renesas RZ/V2H kit, plug the camera into one of the USB 3.0 ports of the RZ/V2H kit.
2.2. Accessing Camera from Linux
The Logitech BRIO 4K USB Camera conforms to the USB Video Class (UVC) specification. Devices of this class are handled by the uvcvideo kernel driver, which implements the standard V4L2 API. Thus, a broad range of V4L2 compatible tools can be used for capturing video from the camera.
...
| Code Block |
|---|
user@rzv2hevkalpha:~$ v4l2-ctl --list-devices| grep -A3 Logi Logi 4K Stream Edition (usb-15860000.usb-1): /dev/video2 /dev/video3 /dev/video4 |
Install dependencies
...
2.3. Installing Dependencies
To install the required build dependencies, run the following command:
| Code Block |
|---|
user@rzv2hevkalpha:~$ sudo apt install libgtkd-3-dev |
...
2.4. Modifying Source Code
Unpack the source code from the rzv2h_drpai-sample-application_ver0.90.tar.gz archive:
| Code Block |
|---|
user@rzv2hevkalpha:~$ tar zxvf rzv2h_drpai-sample-application_ver0.90.tar.gz user@rzv2hevkalpha:~$ cd rzv2h_drpai-sample-application_ver0.90/app_yolov2_cam/src |
and open the define.h file in a text editor:
| Code Block |
|---|
user@rzv2hevkalpha:~$ vim define.h |
Change INPUT_CAM_TYPE from 1 to 0 for the Input Camera value:
| Code Block |
|---|
/* Input Camera support */ /* n = 0: USB Camera, n = 1: eCAM22 */ #define INPUT_CAM_TYPE 1 |
...
and remove the following modules mmngr, gstreamer-1.0 gstbase-1.0 gstapp-1.0 mmngrbuf. These modules are not required for building this sample.:
| Code Block |
|---|
target_link_libraries(${PROJECT_NAME}
pthread glib-2.0 mmngr gobject-2.0 gstreamer-1.0 gstbase-1.0 gstapp-1.0
opencv_imgcodecs opencv_imgproc opencv_core opencv_highgui
jpeg webp tiff z tbb gtk-3 png16 gdk-3 cairo
lzma rt cairo-gobject
xkbcommon wayland-cursor wayland-egl wayland-client epoxy
fribidi harfbuzz fontconfig
glib-2.0 gobject-2.0 gdk_pixbuf-2.0 gmodule-2.0 pangocairo-1.0
atk-1.0 gio-2.0 pango-1.0 freetype pixman-1 uuid pcre
mount resolv expat pangoft2-1.0 blkid
EGL GLESv2 mmngrbuf) |
After editing the CMakeList.txt, target_link_libraries will look like this:
| Code Block |
|---|
target_link_libraries(${PROJECT_NAME}
pthread glib-2.0 gobject-2.0
opencv_imgcodecs opencv_imgproc opencv_core opencv_highgui
jpeg webp tiff z tbb gtk-3 png16 gdk-3 cairo
lzma rt cairo-gobject
xkbcommon wayland-cursor wayland-egl wayland-client epoxy
fribidi harfbuzz fontconfig
glib-2.0 gobject-2.0 gdk_pixbuf-2.0 gmodule-2.0 pangocairo-1.0
atk-1.0 gio-2.0 pango-1.0 freetype pixman-1 uuid pcre
mount resolv expat pangoft2-1.0 blkid
EGL GLESv2) |
...
2.5. Building Source Code
To build the source code, create a build directory in src folder and run cmake in it:
| Code Block |
|---|
user@rzv2hevkalpha:~$ mkdir build user@rzv2hevkalpha:~$ cd build user@rzv2hevkalpha:~$ cmake .. user@rzv2hevkalpha:~$ make |
2.6. Running the
...
Sample
To run the sample, follow these steps:
...
Switch user to root to access the Weston graphics:
| Code Block |
|---|
user@rzv2hevkalpha:~$ sudo su |
Set the appropriate environment variables so the application can connect to Weston:
| Code Block |
|---|
root@rzv2hevkalpha:~# export XDG_RUNTIME_DIR=/run/user/0 root@rzv2hevkalpha:~# export QT_QPA_PLATFORM=wayland root@rzv2hevkalpha:~# export WAYLAND_DISPLAY=wayland-0 |
Run the sample, setting the DRP-AI frequencies of as follows: DRP0 to 315MHz and AI-MAC to 315MHz:
...