Forums - TFLite models not working

3 posts / 0 new
Last post
TFLite models not working
zmonteci
Join Date: 22 May 18
Posts: 14
Posted: Tue, 2022-10-11 09:47

Is there any specific things we need to pay attention to when loading a TFLite model? In the sample app gst_tflite, there is a model included with the app and that works flawlessly. I downloaded different TFLite models (Resnet50, YoloV3, InceptionV3) and I tried to adapt them to the sample app.  The result of that is a plain video feed. Now, I am not sure how to figure out if the model is actually working or if the issue is in the Overlay plugin.  I would really like to understand how to run different TFLite models on the RB5.  Out of the all the sample tflite apps, I have only being able to run Pose estimation and Detection. I would also like some guidance on classification and segmentation.  I really appreciate some help with this.

Thank you,

Zilmara 

  • Up0
  • Down0
hs.chaya
Join Date: 20 Mar 20
Posts: 23
Posted: Thu, 2022-11-24 03:00

Hi,
 

Steps to load and execute AI model in RB5 board using gstreamer plugins

Step-1 : Create directory named rb5 in the home directory of RB5 board

adb shell

mkdir /home/rb5

exit

Step-2 : Push the AI model and labels to the RB5 board from host system

adb push model_name.tflite /home/rb5

adb push labelmap.txt home/rb5

Step-3 : Create configuration file by gstreamer property. File extension should be .config

 

Pipeline creation in command line to load and execute AI model

Running model with qtimletflite :

gst-launch-1.0 v4l2src ! jpegdec ! videoconvert ! qtimletflite config=/home/rb5/mle_tflite.config model=/home/rb5/mobilenet_v1.tflite labels=/home/rb5/labelmap.txt postprocessing=segmentation ! videoconvert ! jpegenc ! filesink location=image.jpeg

Above pipeline takes the inference frames from the camera source and are delivered to the GStreamer TF Lite plugin along with a model tflite. The TF Lite runtime can be running on the DSP, GPU or CPU. Inference results are gathered back in the GStreamer TF sink for postprocessing and that metadata is stored in the file.

*/
  • Up0
  • Down0
mbauroth
Join Date: 23 Mar 23
Posts: 4
Posted: Fri, 2023-04-21 03:55

The exact same thing happens to me. At least I am able to check some more details with logcat command, which gives very much indepth information whiel gstreamer pipeline is running.
But yeah: very few tflite models work, a lot more models do not work with qtimletflite plugin. Most obvious mistake: a different number of classes found in model and in labelmap.txt (some models want 1001 or 1024 or much less classes which don't crash the pipeline, but logcat shows the problem in the back). Then I thought the structure of the output layers is different (again logcat is showing extra data about the structure loaded from the model). But seems not to be the case too.
So I would like to repeat the initial question: does there exist any additional hints or thoughts why more than half of the tflite models do not work? Is there a documentation available about incompatibilities for example?

  • Up0
  • Down0
or Register

Opinions expressed in the content posted here are the personal opinions of the original authors, and do not necessarily reflect those of Qualcomm Incorporated or its subsidiaries (“Qualcomm”). The content is provided for informational purposes only and is not meant to be an endorsement or representation by Qualcomm or any other party. This site may also provide links or references to non-Qualcomm sites and resources. Qualcomm makes no representations, warranties, or other commitments whatsoever about any non-Qualcomm sites or third-party resources that may be referenced, accessible from, or linked to this site.