Software Architecture

The Qualcomm Robotics RB5 development kit includes support for multiple SDKs and tools, including the Qualcomm Neural Processing SDK for AI, Qualcomm Robotics Vision SDK, Qualcomm Computer Vision SDK (FastCV), Hexagon DSP SDK, and Robotics Operating System (RoS).

These SDK's enable developers to implement common robotics features such as:

  • Object tracking, detection, and classification
  • Object depth estimation and obstacle avoidance
  • Visual localization and mapping (vSLAM)
  • Visual inertial odometry (VIO)
  • Path planning and 3D map formation
  • Face detection and recognition
  • Scene classification and understanding
  • Gesture and hand tracking
  • Voice activation and audio classification
  • Deep learning

Qualcomm Robotics RB5 software is based on Ubuntu, which allows developers to install 3rd party app and libraries to reduce the time-to-market .

The RB5 platform includes the following features:

  • Multiple multimedia subsystems (display, graphics, audio, video, and camera) that can be accessed from user space
    • Qualcomm Technologies GStreamer plugins provide access to video and camera subsystems using the Qualcomm Multimedia Framework (QMMF)
    • Qualcomm-customized Pulse GStreamer plugins can be be used for audio functionality
    • Display can be Wayland/XOrg. Access is direct through the Wayland protocol for Weston applications.
  • Support for ROS2.0. The robotics vision modules interact with ROS2.0 per the interface specification of ROS2.0.
  • Sensor data available through Sensor ROS nodes
  • Qualcomm Technologies specific AI/ML components
    • Qualcomm Neural Processing SDK accessible via a GStreamer plugin
    • Support for TFLite and its associated GStreamer plugin. Standard TF Lite applications can be executed in the shell.
    • Qualcomm Computer Vision SDK (FastCV) support
    • Hexagon DSP support

Since the platform has Ubuntu, the standard Ubuntu application packages can be installed via apt-get.

Additional details are presented in subsequent sections of this document. Qualcomm Multimedia Framework provides access to low level API's using GStreamer elements. It is recommended to use GStreamer to access the multimedia subsystem. Refer to GStreamer documentation for more information.