Camera capture (encoding) of streams has the following highlights:
- GStreamer SRC plugin (qmmfsrc)
- Can be used to capture camera frames via the Qualcomm MMF service
- Is capable of providing multiple encoded (AVC/HEVC) bitstreams and YUV streams.
- Is a client to the Qualcomm MMF server. The Qualcomm MMF server runs as a daemon in the system and provides easy RPC APIs to implement camera/recorder use cases.
- Up to 7 cameras are supported concurrently (excluding the UVC camera).
- The encoded bit stream provided by GST plugin can be used by different types of sink elements (such as filesink) to save the bit stream into the file system. It can also be used by streaming plugins (RTP over TCP or UDP) to send the stream to a network.
- The raw YUV stream can be used for multiple use cases:
- The waylandsink element uses this YUV stream to render the camera frames to a physical display to achieve a live camera preview use case
- Postprocessing elements can use this YUV stream to improve image quality (for example, image quality, downscaling, padding, color conversion, etc.)
- The ML inferencing engine (SNPE, TF-LITE) can use this YUV stream to do inferencing on live camera
- Qualcomm MMF interacts with HAL3 and further HAL3 interacts with the camera back end (Camx) and camera driver to configure the sensor and get a stream from the camera sensor.
- Buffers for each camera stream are allocated by Qualcomm MMF using libGBM and submitted to HAL
QRB5165 uses the Qualcomm Spectra 480 camera architecture, which has six CSI interfaces for external cameras
- Six 4-lane interfaces with CSI D-PHY 1.2 or C-PHY 1.2
- Each D-PHY 1.2 interface has a data rate of 2.5 Gbps/lane.
- Each C-PHY 1.2 interface has a data rate of 10.26 Gbps per T (4.5 Gsps/T)
The following table provides detailed specifications of the camera ISP.
|RB3 Spectra 280||RB5 Spectra 480|
Maximum real-time sensor input resolution
2x full IFE processing: 20 MP (4:3), 16 MP (16:9 aspect ratio)
1x IFE_Lite processing: Mono/YUV interface for miscellaneous CV use cases
2x full IFE processing: 25 MP (4:3), 18 MP (16:9 aspect ratio)
5x IFE_Lite processing: Mono/YUV interface for miscellaneous CV use cases
DPHY 1.2: 4/4/4/2 (2.5 Gbps/lane)
CPHY 1.0: 5.71 Gbps/T
Connect up to 7x cameras, 3x concurrent
DPHY 1.2: 4/4/4/4/4/4 (2.5 Gbps/lane)
CPHY 1.2: 10.28 Gbps/T
Connect up to 12x camera, 7x concurrent
2x 16M @ 30 fps – Dual camera
1x 32M @ 30 fps – Single camera
2x 25M @ 30 fps – Dual camera
1x 50M @ 30 fps – Single camera
8-bit/10-bit, 4K/60 support with in-line hardware EIS (no MCTF)
10-bit 4K/120, 8K/30 support with in-line hardware EIS
IQ and imaging
BPS tap-out points
The following table provides a list of camera sensors and modules supported by RB5.
|IMX577||Main camera sensor|
|OV9282||Tracking camera sensor|
|Intel Realsense D435i||Depth camera|
Depth: 1280x720/90 fps
RGB: 1920x1080/30 fps
|YUV/RAW Bayer||Intel Realsense D435i||USB|
|MN34906||ToF camera sensor|
Panasonic TOF V4T
|ON SEMI AR0231 Sensor||GMSL camera sensor|