Using PMD flexx2 3D Camera in TCP Streaming Mode on Renesas EVK

Owned by Natalia Sofronova
Last updated: Mar 22, 2024Version comment
3 min read

1. Overview

This application note describes how to connect the PMD flexx2 3D camera to the Renesas EVK board, and set up a TCP server streaming 3D video stream over network. A remote host running the Royaleviewer UI will be used for visualizing the stream received over the network.

The PMD flexx2 3D camera is built on the Infineon’s IRS2381C REAL3™ Time-of-Flight Image Sensor and implements a PMD patented 3D Time-of-Flight Technology. The camera characteristics include up to 2.4M 3D points per second and 7 m measurement range with incredible data quality. The flexx2 camera comes with a special software development kit (SDK) “Royale”, that supports popular programming extensions including Matlab, OpenCV, and ROS 1+2. Refer to the PMD flexx2 product page for additional details:

https://3d.pmdtec.com/en/3d-cameras/flexx2/

2. Connecting the Camera

To connect the PMD flexx2 camera to the Renesas EVK kit, use a USB 3.1 Gen 1 compatible USB cable and connect the camera to the USB1 port of the carrier board.

The following picture illustrates connection of the camera to the kit:

3. Starting Camera Network Streaming Server

Open a serial or ssh connection to the Renesas EVK, and start the flexx2 camera network streaming server using the below command:

user@rzv2hevkalpha:~$ tcpserver TCP Server for Royale ------------------------------------------------ For more options call --help Searching for cameras ... Camera(s) found: [0] - Name: Flexx2, ID: 8630-9407-1AC7-3C7C Opening the first camera. Successfully connected to the camera device. *********** Server started *********** Using level: 2 Current port: 5000 Press Return to stop the server

After start the server will listen for connections on the 5000 port (default).

4. Running Royaleviewer on a Remote Host

Install an x86 version of the Royale package to a remote host that will be used for visualizing the flexx2 3D stream coming from the Renesas EVK. The host should be located in the same network as Renesas EVK, or has other options (special route or port forwarding) for accessing TCP port 5000 on the Renesas EVK.

Navigate to the folder where the Royale package is installed on the host, and start the royaleviewer application. When the application opens click the “Tools” icon in the left top corner, then select the “Open Ethernet Camera” item and specify the Renesas EVK IP address:

Click “Connect” after entering all the details. The Royaleviewer will initialize the connection and start streaming a 3D visualization of the area surrounding the Renesas EVK.

The following picture illustrates Royaleviewer running on a host and displaying 3D stream from a remote flexx2 camera: