Video Decode

The Qualcomm RB5 platform has a dedicated hardware block called the video processing unit (VPU) for hardware decoding and encoding. The VPU supports up to 4K240/8K60 decoding and 4K120/8K30 encoding. Native decode support is is also provided for the H.265 Main 10, H.265 Main, H.264 High, and MPEG-2 codecs.

  • The video decode (qtivdec) GStreamer plugin uses V4L2 IOCTLs to decode the H264/H265 bit stream. This plugin is mainly used in playback and transcode use cases.
  • For the playback use case, the output of decoder (GBM buffers) are relayed to the display framework (wayland/weston) for rendering. GBM buffers are shared between three components: video decode, waylandsink, and weston compositor are zero-copy.
  • For transcode use cases, the output of the video decoder is processed by the encoder element (either JPEG or OMX encode)

The following example shows simultaneous 1080p capture.

gst-launch-1.0 -e qtiqmmfsrc name=qmmf ! video/x-h264,format=NV12,width=1920,height=1080,framerate=30/1 ! h264parse ! mp4mux ! queue ! filesink location="/data/mux1.mp4" qmmf. ! video/x-h264,format=NV12,width=1920,height=1080,framerate=30/1 ! h264parse ! mp4mux ! queue ! filesink location="/data/mux2.mp4" qmmf. ! video/x-h264,format=NV12,width=1920,height=1080,framerate=30/1 ! h264parse ! mp4mux ! queue ! filesink location="/data/mux3.mp4"

In the following example, the qtiqmmfsrc element is used to generate two encoded video streams (4K and 480p resolution) and one 1080p YUV stream. The first encoded video stream is then processed by the h264parse and mp4mux elements and the filesink element stores it in a file on the device. Next, the second encoded stream is processed by the h264parse and mpegtsmux elements and streamed over TCP by the tcpserversink element. Finally, a raw YUV stream is rendered on the display panel by the waylandsink element. The raw streams only exercise the ISP pipeline, It will convert the Bayer signal from camera hardware to the YUV format.

gst-launch-1.0 -e qtiqmmfsrc name=qmmf qmmf.video_0 ! video/x-h264,format=NV12,width=3840,height=2160,framerate=30/1 ! h264parse ! mp4mux ! queue ! filesink location="/data/mux1.mp4" qmmf.video_1 ! video/x-h264,format=NV12,width=640,height=480,framerate=30/1 ! h264parse config-interval=1 ! mpegtsmux name=muxer ! queue ! tcpserversink port=8900 host=127.0.0.1 qmmf.video_2 ! "video/x-raw(memory:GBM),format=NV12,width=1920,height=1080,framerate=30/1" ! waylandsink sync=false fullscreen=true enable-last-sample=false

In the following example, the qtiqmmfsrc element is used to decode a video stream (4K) video stream and send it to waylandsink.

gst-launch-1.0 filesrc location=/data/mux_4K.mp4 ! qtdemux ! queue ! h264parse ! qtivdec ! waylandsink x=960 y=0 width=3840 height=2160