1. Overview

This application note describes how to connect the PMD flexx2 3D camera to the NavQPlus board, and set up a TCP server streaming 3D video stream over network. A remote host running the Royaleviewer UI will be used for visualizing the stream received over the network.

The PMD flexx2 3D camera is built on the Infineon’s IRS2381C REAL3™ Time-of-Flight Image Sensor and implements a PMD patented 3D Time-of-Flight Technology. The camera characteristics include up to 2.4M 3D points per second and 7 m measurement range with incredible data quality. The flexx2 camera comes with a special software development kit (SDK) “Royale”, that supports popular programming extensions including Matlab, OpenCV, and ROS 1+2. Refer to the PMD flexx2 product page for additional details:

https://3d.pmdtec.com/en/3d-cameras/flexx2/

2. Connecting the Camera

To connect the PMD flexx2 camera to the NAVQPlus kit, use a USB 3.1 Gen 1 compatible USB cable and connect the camera to the USB1 port of the carrier board. Alternatively, the flexx2 camera can be connected to a USB hub that is plugged into the USB1 port of the NAVQ+.

The following picture illustrates connection of the camera to the kit:

Open

3. Software Setup

With some USB hubs the NavQPlus USB1 port may not switch into the host mode automatically, leading to a situation when the flexx2 camera cannot be detected by the NavQPlus USB subsystem. In such case the USB1 port should be switched to host mode manually using the below command:

The default password for the sudo command in the above command example is “user”.

4. Starting Camera Network Streaming Server

Open a serial or ssh connection to the NavQPlus, and start the flexx2 camera network streaming server using the below command:

After start the server will listen for connections on the 5000 port (default).

5. Running Royaleviewer on a Remote Host

Install an x86 version of the Royale package to a remote host that will be used for visualizing the flexx2 3D stream coming from the NavQPlus. The host should be located in the same network as NavQPlus, or has other options (special route or port forwarding) for accessing TCP port 5000 on the NavQPlus.

Navigate to the folder where the Royale package is installed on the host, and start the royaleviewer application. When the application opens click the “Tools” icon in the left top corner, then select the “Open Ethernet Camera” item and specify the NavQPlus IP address:

Open

Click “Connect” after entering all the details. The Royaleviewer will initialize the connection and start streaming a 3D visualization of the area surrounding the NavQPlus.

The following picture illustrates Royaleviewer running on a host and displaying 3D stream from a remote flexx2 camera:

6. Known Problems And Limitations

NAVQP-174: the WIFI connection may freeze when the flexx2 camera is connected to the USB1 port of the NavQPlus. The Ethernet connection should be used for working around this problem.

NAVQP-119: under some circumstances automatic mode switching between device and host mode is not working for the USB1 port. In this case USB1 port should be manually switched into the host mode for the flexx2 camera to appear in the system.