Camera Application for Qualcomm® Robotics RB3 Development Kit

Skill LevelArea of FocusOperating SystemPlatform/Hardware
BeginnerBluetooth, Computer Vision, Embedded, SensorsLinuxQualcomm Robotics RB3 Robotics Dev Kit
The Qualcomm Robotics RB3 Development Kit, which is based on the Qualcomm® SDA845 processor, has 4 cameras which are used in combinations with different concurrency modes, such as “2-Camera concurrency mode” and “3-Camera concurrency modes”. These modes are designed to take measurements such as ‘Depth Calculation’, etc. to capture good images and help minimize drawbacks such as ‘Light Scattering’
This demo application will walk-through setting up the concurrency mode to capture images.
  • Qualcomm Robotics RB3 Development Kit
  • Power Adapter - 12 V with 2500 mA required by the 96Boards specification
  • USB to Micro USB cable - For serial console interface and ADB, Fast boot commands
  • USB to USB Type C cable - For connecting the USB3.0 Type C port and flashing images
  • Host PC OS for Qualcomm Robotics RB3 SDK build process

camera_test.tar

(found on the Thundercomm Qualcomm Robotics RB3 Development Kit web page, under the Technical Documents tab in the “Sample-apps-codes” section)

Prerequisites:

Complete the build instructions in the "Hello RB3" project, and make sure that the following steps are done before proceeding with running the application on device.

1. Setup the development environment

  • 1.1) Install Android SDK tools (ADB, Fastboot)
  • 1.2) Install and configure the USB driver
  • 1.3) Flash the firmware image on the board
  • 1.4) Setup serial port for debugging purpose
  • 1.5) Install the App Toolchain SDK

2. Camera Application:

  • 2.1. From the Thundercomm Qualcomm Robotics RB3 Development Kit web page, under the Technical Documents tab in the “Sample-apps-codes” section, download source code for camera application: “camera_test.tar

    Camera-app sample-code location
  • 2.2. After decompressing the .tar file, the directory is as follows:

    $ cd “Path to downloaded source code”
    $ tree -L 1
    Directory structure of camera app

    Look for Makefile in the downloaded folder

3. Build and transfer the application.

  • 3.1. Run the Make file to compile and build the application.

    $ cat Makefile Command to view contents of Make file-1Command to view contents of make file-2
  • 3.2. Build the application:

    $ source /usr/local/oecore-x86_64/environment-setup-armv7a-neon-oemllib32-linux-gnueabi
    $ make
  • 3.3. Connect the board to the host PC via Type C cable and ensure that the ADB port can be used (check using ‘adb devices’ and connect using ‘adb shell’)

    Transfer the application on to the RB3 board:

    $ adb push hal3_test /bin/ Push Camera app binary to RB3

4. Let’s explore how to use the four cameras concurrency that help in developing the robotics solutions for tracking and evaluation of objects present in the captured images.

The Qualcomm Robotics RB3 Development Kit supports four camera interfaces namely Stereo, ToF, Tracking and Main camera (OV8856)

Camera devices can be attached in various combinations and Sensor ID reflects these combinations, as shown in the matrix below:

Sensor ID Matrix

In the example of “Sr. No” 1.4, here we are connecting only one of the supported sensors. In this case, “cameraID = 0” will select “Tracking”.

In the example of “Sr. No” 2.3, here we are connecting stereo camera along with tracking (2 camera sensor scenario). In this case, “cameraID=1” will select “Tracking” while “cameraID=0” will select “Stereo”

4.1. The combinations of cameras which result in different concurrency modes, such as 2-camera concurrency mode and 3-camera concurrency mode, help in minimizing drawbacks in capturing perfect images such as ‘Light scattering issue’ and also helps in measurements such as ‘Depth Calculation’.

4.2. A simple demonstrate on how to capture an image using Main camera in jpeg format is given below. The sensor ID specified in the command will determine concurrency of cameras used.

4.3. The hal3_test application is available on the Thundercomm website to capture image streams provided by the cameras. This application is written using the HAL3 API.

4.4. To run the hal3_test application, log in to the Qualcomm Robotics RB3 device as a root (using serial or adb shell).

Start preview with snapshot (size:3264x2448,format:jpeg)

$ adb shell
# hal3_test
CAM0>>A:id=2,psize=1920x1080,pformat=yuv420,ssize=3264x2448,sformat=jpeg
(A: Add a camera device for test)
CAM0>> P:1 // P : trigger dump preview
CAM0>> s:1 // s : Trigger snapshot
CAM0>> D // Delete current camera device
CAM0>> Q // Quit
# exit

The following command will pull the image from the Qualcomm Robotics RB3 device board to the host PC, which can be viewed using image viewer:

$ adb pull /data/misc/camera/”filename”
NameTitle/Company
Rakesh SankarSr. System Architect
Global Edge Soft Ltd
Akshay KulkarniTechnical Lead
Global Edge Soft Ltd
Vihaari SreeramSenior Software Engineer
Global Edge Soft Ltd
Patcha VengamambaSoftware Engineer
Global Edge Soft Ltd

By submitting your content (“Submission”), you are granting Qualcomm a royalty-free, perpetual, non-exclusive, unrestricted, worldwide license to: (a) post, use, copy, sublicense, adapt, transmit, publicly perform or display any such Submission, (b) use, reproduce, modify, adapt, publish, translate, create derivative works from, distribute, perform, play, host, communicate, make available and publish your Submission without restriction and (c) sublicense to third parties the unrestricted right to exercise any of the foregoing rights granted with respect to the Submission. The foregoing grants shall include the right to exploit any ideas, concepts, intellectual property, or proprietary rights in such Submission, including but not limited to rights under copyright, trademark, service mark or patent laws under any relevant jurisdiction without Qualcomm owing any monies to you whatsoever. You represent and warrant that you own all right, title and interest in and to the Submission, or you have been granted sufficient rights in and to the Submission allowing the foregoing use of such Submission.