Forums - qtivdec Gstreamer plugin

22 posts / 0 new
Last post
qtivdec Gstreamer plugin
andris.kuciks
Join Date: 22 Jul 20
Posts: 16
Posted: Fri, 2021-01-29 05:56

Hello!
I tried this example and everything works okay. Afterwards I tried to stream webcam data other way arround - from my PC to RB5. When I run gst with this command -

gst-launch-1.0 rtspsrc location=rtsp://192.168.50.150:8554/test ! rtph264depay ! queue ! h264parse ! avdec_h264 ! videoconvert ! waylandsink 

- everything works fine, I'm able to see webcam stream on RB5 weston  client. The problem is that I want to use qtivdec plugin to enable hardware decoding but whenever I launch it with the same command as above but with different decoder  (avdec_h264 -> qtivdec) I just green square, no video, no errors, just green screen. I tried to change some properties of the qtivdec plugin no success. Tried to change the pipeline a bit, most of the time result was the same and rarely it ended with errors.

Could you please give some advice  or example would be perfect on how to setup this plugin? In github examples there is only a example of streaming from RB5 to network but no example on how to do it other way arround.

Andris

  • Up0
  • Down0
andris.kuciks
Join Date: 22 Jul 20
Posts: 16
Posted: Wed, 2021-02-03 02:01

There isn't anyone who could help with this issue

After setting GST_DEBUG=4 getting these errors when gstreamer tries to open the webcam stream:
0:00:03.556641690 25754   0x55b82c8ed0 ERROR                qtivdec gstqtivdec.c:1992:gst_qtivdec_set_format:<qtivdec0> Unable to get width/height value

 

  • Up0
  • Down0
kevin.dai
Join Date: 21 Oct 20
Posts: 137
Posted: Wed, 2021-02-03 10:30

I will check it and give you update

  • Up0
  • Down0
kevin.dai
Join Date: 21 Oct 20
Posts: 137
Posted: Thu, 2021-02-04 15:29

Can you please share the video?

  • Up0
  • Down0
andris.kuciks
Join Date: 22 Jul 20
Posts: 16
Posted: Thu, 2021-02-04 23:26

Hi Kevin,
There isn't a video to share. As I mentioned before its just a webcam stream over RTSP server. The stream source is "v4l2src device=/dev/video0" but the whole host (PC) pipeline is built like this:
( v4l2src device=/dev/video0 ! videoconvert ! x264enc tune="zerolatency" byte-stream=true bitrate=10000 ! rtph264pay name=pay0 pt=96 )
There is no video output only when I try to use qtivdec Gstreamer plugin as mentioned in first post. I checked Gstreamer log messages and no significant error where generated. I tried video decode example from here and everything worked fine. I checked log messages from the example mentioned and pipeline I try to build, everything looked fine so I'm not really sure whats causing this.

About the error mentioned previously. RB5 generates twice the error about width/height but afterwards the pipeline sets it to 640x480 resolution and that could not be the source of this problem.

Regards,

Andris

  • Up0
  • Down0
kevin.dai
Join Date: 21 Oct 20
Posts: 137
Posted: Tue, 2021-02-09 13:41


Could you please share the logs with below setprop:

export GST_DEBUG=5
export GST_DEBUG_FILE=/data/gst_log.log

Kernel logs setprop:

adb shell "echo 0x103F > /d/msm_vidc/debug_level"
adb shell "echo 0x3F > /d/msm_vidc/fw_level"
adb shell "echo 0 > /proc/sys/kernel/kptr_restrict"

OR:

adb shell "echo 0x103f > /sys/kernel/debug/msm_vidc/debug_level"
adb shell "echo 0x3F > /sys/kernel/debug/msm_vidc/fw_level"

Kernel logs :-
adb logcat -c -b kernel
adb logcat -b kernel > kmsg.txt

adb logs :-
adb logcat -c
adb logcat -v threadtime > adb.txt

Pull the gst logs and share the all logs.

Do you get a chance to check qtivdec working with local video?

 

  • Up0
  • Down0
andris.kuciks
Join Date: 22 Jul 20
Posts: 16
Posted: Mon, 2021-02-15 00:21

Hi Kevin!
So tried to configure kernel setprops but it seems that `/sys/kernel/debug/msm_vidc` doesn't contain fw_level file. When I try creating and setting it, device returns "Perrmission denied". Tried to change all the permissions but still no success. Is this fw_level configuration crucial to you? Or I can just log the output without this setting?

Regards,

Andris

 

 

  • Up0
  • Down0
andris.kuciks
Join Date: 22 Jul 20
Posts: 16
Posted: Mon, 2021-02-15 01:56

Dear Kevin,
So I recorded the logs that you asked for. They're available here!

Regards,

Andris

  • Up0
  • Down0
kevin.dai
Join Date: 21 Oct 20
Posts: 137
Posted: Mon, 2021-02-15 10:28

Yes, there are persmmision issue to set fw_level

sh: /sys/kernel/debug/msm_vidc/fw_level: Permission denied

I will check it and also thank you log files

  • Up0
  • Down0
kevin.dai
Join Date: 21 Oct 20
Posts: 137
Posted: Thu, 2021-02-18 10:42

Can you add this in pipeline?

"qtivdec ! video/x-raw\(memory:GBM\),compression=ubwc ! "

gst-launch-1.0 rtspsrc location=rtsp://192.168.50.150:8554/test ! rtph264depay ! queue ! h264parse !                     qtivdec ! video/x-raw\(memory:GBM\),compression=ubwc  ! videoconvert ! waylandsink

Let me know result

 

  • Up0
  • Down0
kevin.dai
Join Date: 21 Oct 20
Posts: 137
Posted: Thu, 2021-02-18 12:06

I tried these commands to play local media file:

1. create a script file weston.sh

#! /bin/bash
mkdir -p /usr/bin/weston_socket
export XDG_RUNTIME_DIR=/usr/bin/weston_socket
export LD_LIBRARY_PATH=/usr/lib:/usr/lib/aarch64-linux-gnu/
weston --tty=1 --connector=29
 
2. push this file to device and run it
 
$ adb push weston.sh /data/.
$ adb shell
sh-4.4# source /data/weston.sh &

 

3. run playback

sh-4.4# export XDG_RUNTIME_DIR=/usr/bin/weston_socket &&
sh-4.4# gst-launch-1.0 filesrc location=/data/file_example_MP4_480_1_5MG.mp4 ! qtdemux name=demux demux. ! queue ! h264parse ! qtivdec ! video/x-raw\(memory:GBM\),compression=ubwc ! waylandsink fullscreen=true
 
It works fine for me
 
Please try it. 
 
 
 
 
 
 
  • Up0
  • Down0
andris.kuciks
Join Date: 22 Jul 20
Posts: 16
Posted: Tue, 2021-02-23 02:17

So I tried to add this:
"qtivdec ! video/x-raw\(memory:GBM\),compression=ubwc ! "

to the pipeline and the end result is still the same -> no image on RB5 only green cube in corner of weston display.


I tried this example and everything worked fine using "qtivdec"  plugin.


Is it possible to you to test the rstp server streaming a webcam video from a host PC to a RB5 connected in the same local network? Then maybe you would be able to replicate this issue and find a possible solution to fix this issue.

Regards,

Andris
 

  • Up0
  • Down0
kevin.dai
Join Date: 21 Oct 20
Posts: 137
Posted: Tue, 2021-02-23 10:59

The qtivdec has no problem, since it works on local vedio playback . There might be other issue in your test

  • Up0
  • Down0
andris.kuciks
Join Date: 22 Jul 20
Posts: 16
Posted: Tue, 2021-02-23 22:54

Hi!
But why then the pipeline works when I try using Gstreamer avdec_h264 plugin which should do the same thing as qtivdec but only using CPU instead of hardware H264 decoder? Or I misunderstood something?

It would be really great if you could try it yourself and report back the result you got!

  • Up0
  • Down0
kevin.dai
Join Date: 21 Oct 20
Posts: 137
Posted: Wed, 2021-02-24 09:33

We are on rtsp streaming test, when I have result, I will let you know.

Thanks

Kevin

  • Up0
  • Down0
kevin.dai
Join Date: 21 Oct 20
Posts: 137
Posted: Thu, 2021-02-25 14:05

Hi Andris

I did test  your command on my rtsp streaming, it works fine.

Here are steps I did:

1. Configure network in my windows PC 

In network setting, for Guset or public ,

turn on network discovery

turn on file and printer sharing

2 In change network properity

Select public network

3 . Create rtsp server on my Windows PC. 

get/download the rtsp simple server from here

https://reposhub.com/go/network/aler9-rtsp-simple-server.html

run rtsp server 

4. Publish streaming

download ffmpeg from internet and run commad to publish streaming 

C:\Users\kevin\Tools\ffmpeg\ffmpeg-N-101260-gd7f10b2497-win64-gpl-shared-vulkan\bin>ffmpeg -re -stream_loop -1 -i file.ts -c copy -f rtsp rtsp://192.168.0.22:8554/mystream

The file.ts is media video file to stream.

5. run command on your RB5

$source /data/weston.sh &

$export XDG_RUNTIME_DIR=/usr/bin/weston_socket

$gst-launch-1.0 rtspsrc location=rtsp://192.168.0.22:8554/mystream ! rtph264depay ! queue ! h264parse ! qtivdec ! videoconvert ! waylandsink

My rtsp server on my windows PC is 192.168.0.22

The qtivdec work fine for rtsp streaming

You can create rtsp sever and test  streaming on your RB5 device.

 

  • Up0
  • Down0
kevin.dai
Join Date: 21 Oct 20
Posts: 137
Posted: Thu, 2021-02-25 20:14

Test log:

 

sh-4.4# 
sh-4.4# 
264depay ! queue ! h264parse ! qtivdec ! videoconvert ! waylandsinkstream ! rtph2
Setting pipeline to PAUSED ...
gbm_create_device(156): Info: backend name is: msm_drm
Pipeline is live and does not need PREROLL ...
Progress: (open) Opening Stream
Progress: (connect) Connecting to rtsp://192.168.0.22:8554/mystream
Progress: (open) Retrieving server options
Progress: (open) Retrieving media info
Progress: (request) SETUP stream 0
Progress: (request) SETUP stream 1
Progress: (open) Opened Stream
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Progress: (request) Sending PLAY request
Progress: (request) Sending PLAY request
Progress: (request) Sent PLAY request
gbm_create_device(156): Info: backend name is: msm_drm
gbm_create_device(156): Info: backend name is: msm_drm
GET YUV Info success
GET YUV Info success
GET YUV Info success
GET YUV Info success
GET YUV Info success
GET YUV Info success
GET YUV Info success
GET YUV Info success
GET YUV Info success
GET YUV Info success
GET YUV Info success
GET YUV Info success
GET YUV Info success
GET YUV Info success
GET YUV Info success
GET YUV Info success
GET YUV Info success
GET YUV Info success
GET YUV Info success
GET YUV Info success
GET YUV Info success
GET YUV Info success
GET YUV Info success
GET YUV Info success
GET YUV Info success
GET YUV Info success
GET YUV Info success
GET YUV Info success
GET YUV Info success
GET YUV Info success
GET YUV Info success
GET YUV Info success
GET YUV Info success
GET YUV Info success
GET YUV Info success
GET YUV Info success
 
  • Up0
  • Down0
andris.kuciks
Join Date: 22 Jul 20
Posts: 16
Posted: Fri, 2021-02-26 05:44

Hi Kevin,

So I tried the steps you listed only when running rtsp stream from host I use this command:

ffmpeg -re -i /dev/video0 -c libx264 -f rtsp rtsp://192.168.88.237:8554/teststream

as I do not have a encoded video stream from webcam.
I tried multiple pipeline setups but still no luck with output. Still the best I got was a green cube...

I run the Gstramer on RB5 with exactly the same command as you supported :

GST_DEBUG=3 gst-launch-1.0 rtspsrc location=rtsp://192.168.88.237:8554/teststream ! rtph264depay ! queue ! h264parse ! qtivdec ! videoconvert ! waylandsink

When the pipeline is running I do not get any error messages only a  FIXME and WARN message looking like this:

572       0x7f8c0466f0 WARN    videodecoder gstvideodecoder.c:2780:gst_video_decoder_prepare_finish_frame:<qtivdec0> decreasing timestamp (0:00:00.200951361 < 0:00:00.268661757)

5272     0x55a7aa45a0 FIXME  videodecoder.c:933:gst_video_decoder_drain_out:<qtivdec0> Sub-class should implementdrain()

I wasn't able to find any solutions while googling. But this issue isn't relevant anymore as we decided to switch to other hardware.

Regards,

Andris

 

  • Up0
  • Down0
kevin.dai
Join Date: 21 Oct 20
Posts: 137
Posted: Fri, 2021-02-26 11:13

There are some things about your test.

1. Can you ping your device from youe device:

#ping 192.168.88.237

2. Have you started your rstp server

c:> rtsp-simple-server

3. Can you use media file to replace your  /dev/video0 -c libx264

4. Can you play your stream in other machine?

You can use VLC player or Media player to play your rtsp stream

 rtsp://192.168.88.237:8554/teststream

and verify if your streaming is good or not

 

 

 

  • Up0
  • Down0
andris.kuciks
Join Date: 22 Jul 20
Posts: 16
Posted: Mon, 2021-03-01 00:16

Hi Kevin!
1. Yes I could ping to RB5 from my PC as both devices were in the same local network.

2. I followed all the step you mentioned but I used webcam stream instead of local video file

3. Will test this soon and report back

4. Yes, I tested the stream on a laptop using VLC player

Regards,

Andris

  • Up0
  • Down0
andris.kuciks
Join Date: 22 Jul 20
Posts: 16
Posted: Mon, 2021-03-01 00:16

Hi Kevin!

So I tried to stream a video file from my pc over rtsp server. I followed all the steps you listed but instead of Windows I'm using Linux machine.

1. So I started the rtsp-simple-server

2. Running stream with: ffmpeg -re -stream_loop -1 -i /home/andris/Downlos/sample_640x360.ts -c copy -f rtsp rtsp://192.168.88.237:8554/stream

3.  Then I run these commands on RB5

$source /data/weston.sh &

$export XDG_RUNTIME_DIR=/usr/bin/weston_socket

$gst-launch-1.0 rtspsrc location=rtsp://192.168.88.237:8554/stream ! rtph264depay ! queue ! h264parse ! qtivdec ! videoconvert ! waylandsink

And I get these following errors:

FIXME videodecoder gstvideodecoder.c:933:gst_video_decoder_grain_out<qtivdec0? Sub-class should implement drain()
FIXME basesink gstbasesink.c:3145:gst_base_sink_default_event:<wailandsink-0> stream-start event without group-id. Consider implementing group-id handling in the upstream elements

WARN basesrc gstbasesrc.c:3055:gst_base_src_loop:<udpsrc1> error: Internal data stream error.

WARN gstbasesrc.c:3055:gst_base_src_loop:<udpsrc1> error: streaming stopped, reason not-negotiated (-4)

ERROR: from element /GstPipeline:pieline0/GstRTSPSrc:rtspsrc0/GstUDPSrc:udpsrc1: Internal data stram error

Additional debug info:

gstbasesrc.c (3055): gst_base_src_loop (): /GstPipeline:pipeline0/GstRTSPSrc:rtspsrc0/GstUDPSrc:udpsrc1: streaming stopped, reason not-negotiated (-4)

I tested the stream on different device (laptop) and everything worked fine.

Regards,

Andris

  • Up0
  • Down0
kevin.dai
Join Date: 21 Oct 20
Posts: 137
Posted: Mon, 2021-03-08 14:14

Andris comfirmed that this issue is resolved 

Close this ticket

  • Up0
  • Down0
or Register

Opinions expressed in the content posted here are the personal opinions of the original authors, and do not necessarily reflect those of Qualcomm Incorporated or its subsidiaries (“Qualcomm”). The content is provided for informational purposes only and is not meant to be an endorsement or representation by Qualcomm or any other party. This site may also provide links or references to non-Qualcomm sites and resources. Qualcomm makes no representations, warranties, or other commitments whatsoever about any non-Qualcomm sites or third-party resources that may be referenced, accessible from, or linked to this site.