1. Overview

This application note describes how to run the DRP-AI sample for the YOLOv2 model with the Renesas RZ/V2H EVK (RZ/V2H SoC-based embedded computer), using a USB Full HD resolution (1920x1080) camera.

This functionality is available with the Emcraft software distribution for the RZ/V2H, Release 1.0.5-42 or higher.

1.1. DRP-AI

DRP-AI is an AI accelerator developed by Renesas, designed to provide high-performance and low-power AI processing for embedded applications. It addresses the challenges of implementing AI on embedded devices, which may have limited resources or power constraints. The DRP-AI accelerator integrates pre- and post-processing with AI inference, resulting in superior AI processing performance compared to many other accelerators. The DRP-AI Translator tool maximises DRP-AI performance by enabling it to handle all operations of an AI model. The DRP-AI TVM, based on the Apache TVM ML compiler framework, supports multiple AI frameworks and allows operation in conjunction with the CPU, expanding the range of AI models that can be run. The DRP-AI Support Package provides the necessary driver and guide for operating the DRP-AI. The DRP-AI accelerator is embedded in the Renesas RZ/V2H series MPUs, delivering high-speed AI processing with high power efficiency.

Pleaser refer to the DRP-AI page on the Renesas web site for more information.

1.2. YOLO v2

YOLOv2 (You Only Look Once version 2) is a real-time object detection algorithm that aims to efficiently detect and classify objects in images or video frames. It is an improvement over the original YOLO algorithm, offering better accuracy and performance.

Please refer to original the YOLO v2 web site for more information.

Here is an example of YOLOv2 detecting objects:

In this example, YOLOv2 identifies and localises a bicycle, car, and dog within the input image. The confidence scores indicate the algorithm's level of confidence in the predictions, while the bounding boxes specify the location and size of each detected object.

1.3. Supported Classes

Here is a list of objects that the YOLOv2 sample is capable of detecting:

• aeroplane

• bicycle

• bird

• boat

• bottle

• bus

• car

• cat

• chair

• cow

• diningtable

• dog

• horse

• motorbike

• person

• pottedplant

• sheep

• sofa

• train

• tvmonitor

Detection example provided with YOLO v2 sample:

1.4. Logitech BRIO 4K USB Camera

While the demo supports any USB Full HD resolution (1920x1080) , Emcraft has explicitly validated with the Logitech BRIO 4K USB Camera is an Ultra HD camera manufactured by Logitech. It is designed to capture high-resolution video and provide exceptional image quality.

The Logitech BRIO 4K USB Camera can be purchased from various online retailers, including Amazon.

2. Running the DRP-AI Demo

2.1. Connecting Logitech BRIO 4K USB Camera

To connect the Logitech BRIO 4K USB Camera to the Renesas RZ/V2H kit, plug the camera into one of the USB 3.0 ports of the RZ/V2H kit.

2.2. Accessing Camera from Linux

The Logitech BRIO 4K USB Camera conforms to the USB Video Class (UVC) specification. Devices of this class are handled by the uvcvideo kernel driver, which implements the standard V4L2 API. Thus, a broad range of V4L2 compatible tools can be used for capturing video from the camera.

Depending on additional hardware connected to the Renesas board, the Logitech camera may not be the only V4L2 device present in the system. In order to access the camera the user needs to determine the correct video device file node. The below command lists all device node files registered for the Logitech camera:

user@rzv2hevkalpha:~$ v4l2-ctl --list-devices| grep -A3 Logi
Logi 4K Stream Edition (usb-15860000.usb-1):
	/dev/video2
	/dev/video3
	/dev/video4

2.3. Installing Dependencies

To install the required build dependencies, run the following command:

user@rzv2hevkalpha:~$ sudo apt install libgtkd-3-dev

2.4. Modifying Source Code

Unpack the source code from the rzv2h_drpai-sample-application_ver0.90.tar.gz archive:

user@rzv2hevkalpha:~$ tar zxvf rzv2h_drpai-sample-application_ver0.90.tar.gz
user@rzv2hevkalpha:~$ cd rzv2h_drpai-sample-application_ver0.90/app_yolov2_cam/src

and open the define.h file in a text editor:

user@rzv2hevkalpha:~$ vim define.h

Change INPUT_CAM_TYPE from 1 to 0 for the Input Camera value:

/* Input Camera support */
/* n = 0: USB Camera, n = 1: eCAM22 */
#define INPUT_CAM_TYPE 1

To exclude the installation of unnecessary libraries, open the CmakeList.txt file in a text editor:

user@rzv2hevkalpha:~$ vim CMakeList.txt

and remove the following modules mmngr, gstreamer-1.0 gstbase-1.0 gstapp-1.0 mmngrbuf. These modules are not required for building this sample:

target_link_libraries(${PROJECT_NAME} 
	pthread glib-2.0 mmngr gobject-2.0 gstreamer-1.0  gstbase-1.0 gstapp-1.0 
	opencv_imgcodecs opencv_imgproc opencv_core opencv_highgui
	jpeg webp tiff z tbb gtk-3 png16 gdk-3 cairo
	lzma rt cairo-gobject
	xkbcommon wayland-cursor wayland-egl wayland-client epoxy
	fribidi harfbuzz fontconfig
	glib-2.0 gobject-2.0 gdk_pixbuf-2.0 gmodule-2.0 pangocairo-1.0
	atk-1.0 gio-2.0 pango-1.0 freetype pixman-1 uuid pcre
	mount resolv expat pangoft2-1.0 blkid
	EGL GLESv2 mmngrbuf)

After editing the CMakeList.txt, target_link_libraries will look like this:

target_link_libraries(${PROJECT_NAME} 
	pthread glib-2.0 gobject-2.0 
	opencv_imgcodecs opencv_imgproc opencv_core opencv_highgui
	jpeg webp tiff z tbb gtk-3 png16 gdk-3 cairo
	lzma rt cairo-gobject
	xkbcommon wayland-cursor wayland-egl wayland-client epoxy
	fribidi harfbuzz fontconfig
	glib-2.0 gobject-2.0 gdk_pixbuf-2.0 gmodule-2.0 pangocairo-1.0
	atk-1.0 gio-2.0 pango-1.0 freetype pixman-1 uuid pcre
	mount resolv expat pangoft2-1.0 blkid
	EGL GLESv2)

2.5. Building Source Code

To build the source code, create a build directory in src folder and run cmake in it:

user@rzv2hevkalpha:~$ mkdir build
user@rzv2hevkalpha:~$ cd build
user@rzv2hevkalpha:~$ cmake ..
user@rzv2hevkalpha:~$ make

2.6. Running the Sample

To run the sample, follow these steps:

1. Go to rzv2h_drpai-sample-application_ver0.90/app_yolov2_cam/exe directory.

2. Locate the app_yolov2_cam binary file.

3. Replace the existing app_yolov2_cambinary file with the one that you have built in the previous step:

user@rzv2hevkalpha:~$ rzv2h_drpai-sample-application_ver0.90/app_yolov2_cam/exe
user@rzv2hevkalpha:~$ rm app_yolov2_cam
user@rzv2hevkalpha:~$ cp ../src/build/app_yolov2_cam .

Switch user to root to access the Weston graphics:

user@rzv2hevkalpha:~$ sudo su

Set the appropriate environment variables so the application can connect to Weston:

root@rzv2hevkalpha:~# export XDG_RUNTIME_DIR=/run/user/0  
root@rzv2hevkalpha:~# export QT_QPA_PLATFORM=wayland
root@rzv2hevkalpha:~# export WAYLAND_DISPLAY=wayland-0

Run the sample, setting the DRP-AI frequencies as follows: DRP0 to 315MHz and AI-MAC to 315MHz:

root@rzv2hevkalpha:~# ./app_yolov2_cam 3 5