qtiqmmfsrc

QTI's qtiqmmfsrc plugin can be used to capture video frames via the Qualcomm MMF service.

The plugin consists of the main class called GstQmmfSrc which acts as a wrapper on top of the Qualcomm MMF Recorder Client with separate pads for video and image streams. The pads store the creation time parameters (passed as GstCaps during pipeline creation) for the particular stream while the GstQmmfSrc takes that information, translates it to the Qualcomm MMF Recorder Client parameters and calls the necessary APIs on each state transition of the element. For video and image pads the camera device ID which will be used for this instance of the plugin can be set via the "camera" property (by default this ID is 0).

During transitioning between NULL and READY state the plugin will open and initialize the camera device with the given ID. Next, when transitioning to PAUSED state from READY, the plugin translates the set pad parameters and makes calls to the Qualcomm MMF service in order to create the source streams for each pad. And finally, the session transitions to the PLAYING state.

When a frame is received in the main class it will create a GstBuffer out of it and send it to the relevant pad buffer queue. From there the pad will push the buffer to its linked sink pad from the next plugin. Buffer allocation is happening inside the Qualcomm MMF service while the plugin only ensures that buffers are returned back to the service when they are no longer in use.

For video, buffers are sent by the Qualcomm MMF service when the plugin state is PLAYING, but for image pad a "capture-image" signal must be sent. For each "capture-image" a single buffer will be sent from the Qualcomm MMF service.

Pad templates

video_%u (output)

video/x-h264

        profile: { (string)baseline, (string)main, (string)high }
         level: { (string)1, (string)1.3, (string)2, (string)2.1, (string)2.2, (string)3, (string)3.1, (string)3.2, (string)4, (string)4.1, (string)4.2, (string)5, (string)5.1, (string)5.2 }
         width: [ 16, 3840 ]
        height: [ 16, 2160 ]
     framerate: [ 0/1, 30/1 ]

video/x-h264(memory:GBM)
profile: { (string)baseline, (string)main, (string)high }
         level: { (string)1, (string)1.3, (string)2, (string)2.1, (string)2.2, (string)3, (string)3.1, (string)3.2, (string)4, (string)4.1, (string)4.2, (string)5, (string)5.1, (string)5.2 }
         width: [ 16, 3840 ]
        height: [ 16, 2160 ]
     framerate: [ 0/1, 30/1 ]
video/x-h265
     profile: { (string)main }
        level: { (string)3, (string)4, (string)5, (string)5.1, (string)5.2 }
        width: [ 16, 3840 ]
     height: [ 16, 2160 ]
    framerate: [ 0/1, 30/1 ]
video/x-h265(memory:GBM)
        profile: { (string)main }
         level: { (string)3, (string)4, (string)5, (string)5.1, (string)5.2 }
         width: [ 16, 3840 ]
        height: [ 16, 2160 ]
     framerate: [ 0/1, 30/1 ]
video/x-raw
        format: { (string)NV12 }
         width: [ 16, 3840 ]
        height: [ 16, 2160 ]
     framerate: [ 0/1, 30/1 ]
video/x-raw(memory:GBM)
        format: { (string)NV12 }
         width: [ 16, 3840 ]
        height: [ 16, 2160 ]
     framerate: [ 0/1, 30/1 ]

Presence – on request

Direction – src

image_%u (output)

image/jpeg
         width: [ 16, 4000 ]
        height: [ 16, 3000 ]
     framerate: [ 0/1, 30/1 ]

image/jpeg(memory:GBM)
         width: [ 16, 4000 ]
        height: [ 16, 3000 ]
     framerate: [ 0/1, 30/1 ]
video/x-bayer
     format: { (string)RAW8 (string)RAW10, (string)RAW12, (string)RAW16 }
        width: [ 16, 4000 ]
     height: [ 16, 3000 ]
    framerate: [ 0/1, 30/1 ]
video/x-bayer(memory:GBM)
         level: { (string)RAW8 (string)RAW10, (string)RAW12, (string)RAW16 }
         width: [ 16, 4000 ]
        height: [ 16, 3000 ]
     framerate: [ 0/1, 30/1 ]
video/x-raw
        format: { (string)NV12 }
         width: [ 16, 4000 ]
        height: [ 16, 3000 ]
     framerate: [ 0/1, 30/1 ]
video/x-raw(memory:GBM)
        format: { (string)NV12 }
         width: [ 16, 4000 ]
        height: [ 16, 3000 ]
     framerate: [ 0/1, 30/1 ]

Presence – on request

Direction – src

Pad Properties

video_%u (output)

source-index

Index of the source video pad to which this pad will be linked.

type: Integer
access: read/write
default: -1

framerate

Target framerate in frames per second for displaying

type: Double
access: read/write
default: 30.0

range: 0 - 30

bitrate

Target bitrate in bits per second for compressed streams

type: Unsigned Integer
access: read/write
default: 6000000

bitrate-control

Bitrate control method for compressed streams

type: Enum (GstVideoControlRate)
access: read/write
default: 3, "maxbitrate"

Available Values:

(0): disable - Disable
(1): variable - Variable
(2): constant - Constant
(3): maxbitrate - Max Bitrate
(4): constant-skip-frames - Variable Skip Frames
(5): variable-skip-frames - Constant Skip Frames
(6): maxbitrate-skip-frames - Max Bitrate Skip Frames

quant-i-frames

Quantization parameter on I-frames for compressed streams

type: Unsigned Integer
access: read/write
default: 27

quant-p-frames

Quantization parameter on P-frames for compressed streams

type: Unsigned Integer
access: read/write
default: 28

quant-b-frames

Quantization parameter on B-frames for compressed streams

type: Unsigned Integer
access: read/write
default: 28

min-qp

Minimum QP value allowed during rate control for compressed streams

type: Unsigned Integer
access: read/write
default: 10

range : 0 - 51

max-qp

Max QP value allowed during rate control for compressed streams

type: Unsigned Integer
access: read/write
default: 51

range : 0 - 51

min-qp-i-frames

Minimum QP value allowed on I-Frames during rate control for compressed streams

type: Unsigned Integer
access: read/write
default: 10

range : 0 - 51

max-qp-i-frames

Max QP value allowed on I-Frames during rate control for compressed streams

type: Unsigned Integer
access: read/write
default: 51

range : 0 - 51

min-qp-p-frames

Minimum QP value allowed on for P-Frames during rate control for compressed streams

type: Unsigned Integer
access: read/write
default: 10

range : 0 - 51

max-qp-p-frames

MaxQP value allowed on for P-Frames during rate control for compressed streams

type: Unsigned Integer
access: read/write
default: 51

range : 0 - 51

min-qp-b-frames

Minimum QP value allowed on B-Frames during rate control for compressed streams

type: Unsigned Integer
access: read/write
default: 10

range : 0 - 51

max-qp-b-frames

Maximum QP value allowed on B-Frames during rate control for compressed streams

type: Unsigned Integer
access: read/write
default: 51

range : 0 - 51

Main Properties

name

The name of the object.

type: String
access: read/write
default: "qmmfsrc0"

camera

Camera device ID to be used by video/image pads.

type: Unsigned Integer
access: read/write
default: 0

slave

Set camera as a slave device. This is used for multi-stream use cases.

type: Boolean
access: read/write
default: false

shdr

Super High Dynamic Range Imaging.

type: Boolean
access: read/write
default: false

eis

Image Stabilization technology to reduce the effects of camera shake.

type: Boolean
access: read/write
default: false
Note: This feature is not supported currently

effect

Effect applied to the camera frames.

type: Enum "GstCameraEffectMode"
access: read/write
(0): off             - No color effect will be applied.
(1): mono             - A 'monocolor' effect where the image is mapped into a single color.
(2): negative         - A 'photo-negative' effect where the image's colors are inverted.
(3): solarize         - A 'solarisation' effect (Sabattier effect) where the image is wholly or partially reversed in tone.
(4): sepia            - A 'sepia' effect where the image is mapped into warm gray, red, and brown tones.
(5): posterize        - A 'posterization' effect where the image uses discrete regions of tone rather than a continuous gradient of tones.
(6): whiteboard     - A 'whiteboard' effect where the image is typically displayed as regions of white, with black or grey details.
(7): blackboard     - A 'blackboard' effect where the image is typically displayed as regions of black, with white or grey details.
(8): aqua             - An 'aqua' effect where a blue hue is added to the image.
default: 0, "off"

antibanding

Camera antibanding routine for the current illumination condition.

type: Enum "GstAntibandingMode"
access: read/write
(0): off             - The camera device will not adjust exposure duration to avoid banding problems.
(1): 50hz             - The camera device will adjust exposure duration to avoid banding problems with 50Hz illumination sources.
(2): 60hz             - The camera device will adjust exposure duration to avoid banding problems with 60Hz illumination sources.
(3): auto             - The camera device will automatically adapt its antibanding routine to the current illumination condition.
default: 3, "auto"

ae-compensation

Auto Exposure Compensation.

type: Integer
access: read/write
default: 0

ae-lock

Auto Exposure lock

type: Boolean
access: read/write
default: false

awb-mode

Auto White Balance mode.

type: Enum "GstCameraAWBMode"
access: read/write
(0): off             - The camera device's auto-white balance routine is disabled.
(1): auto             - The camera device's auto-white balance routine is active.
(2): shade            - The camera device uses shade light as the assumed scene illumination for white balance.
(3): incandescent     - The camera device uses incandescent light as the assumed scene illumination for white balance.
(4): fluorescent     - The camera device uses fluorescent light as the assumed scene illumination for white balance.
(5): warm-fluorescent - The camera device uses warm fluorescent light as the assumed scene illumination for white balance.
(6): daylight         - The camera device uses daylight light as the assumed scene illumination for white balance.
(7): cloudy-daylight - The camera device uses cloudy daylight light as the assumed scene illumination for white balance.
(8): twilight         - The camera device uses twilight light as the assumed scene illumination for white balance.
default: 1, "auto"

awb-lock

Auto White Balance lock.

type: Boolean
access: read/write
default: false

ae-mode

Auto White Balance mode.

type: Enum "GstCameraAEMode"
access: read/write
(0): off             - The auto exposure routine is disabled. Manual exposure time will be used.
(1): on             - The auto exposure routine is active.
default: 1, "on"

exposure-time

Manual exposure time in nanoseconds. Used when the AE mode is set to 'off'

type: 64 bit Integer
access: read/write
default: 3333333

ae-metering-mode

Auro exposure metering mode.

type: Enum ""GstCameraAEMeteringMode"
access: read/write
(0): average         - The camera device's exposure metering is calculated as average from the whole frame.
(1): center-weighted - The camera device's exposure metering is calculated from the center region of the frame.
(2): spot             - The camera device's exposure metering is calculated from a chosen spot.
(3): custom         - The camera device's exposure metering is calculated from a custom metering table.

default: 0, "average"

af-mode

Auto Focus mode.

type: Enum ""GstCameraAFMode"
access: read/write
(0): off             - The auto focus routine is disabled.
(1): auto             - The auto focus routine is active.
(2): macro            - In this mode, the auto focus algorithm is optimized for focusing on objects very close to the camera.
(3): continuous     - In this mode, the AF algorithm modifies the lens position continually to attempt to provide a constantly-in-focus image stream.
(4): edof             - The camera device will produce images with an extended depth of field automatically; no special focusing operations need to be done before taking a picture.
default: 1, "auto"

iso-mode

ISO exposure mode.

type: Enum "GstCameraISOMode"
access: read/write
(0): auto             - The ISO exposure mode will be chosen depending on the scene.
(1): deblur         - The ISO exposure sensitivity set to prioritize motion deblur.
(2): 100             - The ISO exposure sensitivity set to prioritize level 100.
(3): 200             - The ISO exposure sensitivity set to prioritize level 200.
(4): 400             - The ISO exposure sensitivity set to prioritize level 400.
(5): 800             - The ISO exposure sensitivity set to prioritize level 800.
(6): 1600             - The ISO exposure sensitivity set to prioritize level 1600.
(7): 3200             - The ISO exposure sensitivity set to prioritize level 3200.
default: 0, "auto"

infrared-mode

Infrared Mode.

type: Enum ""GstCameraIRMode"
access: read/write
(0): off             - The infrared is disabled.
(1): on             - The infrared is active until canceled.
(2): auto             - The infrared is turned OFF or ON depending on the conditions.
default: 0, "off"

noise-reduction

Noise reduction filter mode.

type: Enum ""GstCameraNoiseReduction"
access: read/write
(0): off             - No noise reduction filter is applied.
(1): hq             - TNR (Temoral Noise Reduction) mode.
default: 0, "off"

zoom

Camera zoom rectangle ('<X, Y, WIDTH, HEIGHT >') in sensor active pixel array coordinates.

type: GstValueArray of GValues of type Integer
access: read/write
default: NONE

exposure-table

A GstStructure describing exposure table.

type: String (Either the exposure table in string form or path for file with exposure table values)
access: read/write
default: NONE

defog-table

A GstStructure describing defog table.

type: String (Either the exposure table in string form or path for file with defog table values)
access: read/write
default: NONE

Note: To view the effect of some of the above-mentioned properties the camera sensor must be tuned.