Hi,
I'm attempting to apply a TF Lite Object Detection model on a camera feed using GStreamer Qualcomm plugins. Here is my command line and error message:
gst-launch-1.0 qtiqmmfsrc ! video/x-raw, format=NV12, width=1280, height=720, framerate=30/1 ! qtimletflite config=/data/misc/camera/mle_tflite.config model=/data/misc/camera/detect.tflite labels=/data/misc/camera/labelmap.txt postprocessing=detection ! queue ! qtioverlay ! qtivtransform rotate=1 ! waylandsink fullscreen=true async=true sync=false enable-last-sample=false
0:00:03.445772956 5545 0x559d74de80 DEBUG qtimletflite mle_tflite.cc:431:gst_mle_create_engine:<mletflite0> Parsing from file is successful!
0:00:03.445832904 5545 0x559d74de80 DEBUG qtimletflite mle_tflite.cc:378:gst_mle_print_config:<mletflite0> ==== Configuration Begin ====0:00:03.445889154 5545 0x559d74de80 DEBUG qtimletflite mle_tflite.cc:379:gst_mle_print_config:<mletflite0> Model /data/misc/camera/detect.tflite0:00:03.445944050 5545 0x559d74de80 DEBUG qtimletflite mle_tflite.cc:380:gst_mle_print_config:<mletflite0> Labels /data/misc/camera/labelmap.txt0:00:03.445998061 5545 0x559d74de80 DEBUG qtimletflite mle_tflite.cc:382:gst_mle_print_config:<mletflite0> Pre-processing 10:00:03.446052071 5545 0x559d74de80 DEBUG qtimletflite mle_tflite.cc:384:gst_mle_print_config:<mletflite0> Pre-processing accelerator 20:00:03.787940831 5545 0x559d74de80 DEBUG qtimletflite mle_tflite.cc:387:gst_mle_print_config:<mletflite0> Mean(B,G,R): 0.000000, 0.000000, 0.0000000:00:03.788020154 5545 0x559d74de80 DEBUG qtimletflite mle_tflite.cc:390:gst_mle_print_config:<mletflite0> Sigma(B,G,R): 0.000000, 0.000000, 0.0000000:00:03.788081300 5545 0x559d74de80 DEBUG qtimletflite mle_tflite.cc:392:gst_mle_print_config:<mletflite0> Confidence threshold 0.6000000:00:03.788137758 5545 0x559d74de80 DEBUG qtimletflite mle_tflite.cc:393:gst_mle_print_config:<mletflite0> Input format 00:00:03.788381249 5545 0x559d74de80 DEBUG qtimletflite mle_tflite.cc:394:gst_mle_print_config:<mletflite0> Delegate 00:00:03.788440103 5545 0x559d74de80 DEBUG qtimletflite mle_tflite.cc:396:gst_mle_print_config:<mletflite0> Number of threads 20:00:03.788495729 5545 0x559d74de80 DEBUG qtimletflite mle_tflite.cc:398:gst_mle_print_config:<mletflite0> Post-processing detection0:00:03.788548698 5545 0x559d74de80 DEBUG qtimletflite mle_tflite.cc:399:gst_mle_print_config:<mletflite0> ==== Configuration End ====0:00:03.792476942 5545 0x559d74de80 ERROR qtimletflite mle_tflite.cc:574:gst_mle_tflite_set_info:<mletflite0> MLE init failed.
This model has been trained using the Ultralytics YOLOv8 architecture: https://github.com/ultralytics/ultralytics/tree/main
For reference, the exact same command is working fine with the formal TF Lite demo from Tensor Flow's website: https://www.tensorflow.org/lite/examples/object_detection/overview
Could you provide guidance on what could be wrong with my model and how to troubleshoot this?
Update: by looking at adb logcat, I was able to find more info on this error:
02-28 22:19:54.721 5233 5237 E : MLE: ValidateModelInfo: No: of labels 1, DO NOT match no: of predictions 6300
02-28 22:19:54.721 5233 5237 E : MLE: InitFramework: Provided model is not supported
This error tells basically that the MLE is not understanding properly the output format of my model, which is YOLOv8 by Ultralytics. The ouput is [1,5,6300]. I see that some TF Lite models have 4 elements in their output: location, classes, score and number of objects, but this can change for each OD architecture out there.
Could you share what is the expected model output format for qtimletflite postprocessing=detection?