Hi,
Got the new SNPE version 1.12.0 to try out the MobileNet SSD support.
snpe-tensorflow-to-dlc --graph ssd_mobilenet_v1_android_export.pb -i image_tensor 300,300,3 --model_version 1_12_0 --in_type image --dlc
ssd_mobilenet_v1_android_export.dlc --verbose --out_node detection_boxes --out_node detection_scores --out_node detection_classes --out_n
ode num_detections
Seems like non of the layers are recognized :
2018-02-14 10:56:55,314 - 106 - DEBUG - DEBUG_TF_OP_NAME_TYPE_PRINT: Operation(Postprocessor/BatchMultiClassNonMaxSuppression/map/TensorArrayStack_2/range/start) [Const])
2018-02-14 10:56:55,314 - 106 - DEBUG - DEBUG_TF_OP_NAME_TYPE_PRINT: Operation(Postprocessor/BatchMultiClassNonMaxSuppression/map/TensorArrayStack_2/range/delta) [Const])
2018-02-14 10:56:55,314 - 104 - DEBUG - DEBUG_TF_SCOPE_PRINT: Scope(Postprocessor/BatchMultiClassNonMaxSuppression/map/TensorArrayStack_4/range)
2018-02-14 10:56:55,314 - 106 - DEBUG - DEBUG_TF_OP_NAME_TYPE_PRINT: Operation(Postprocessor/BatchMultiClassNonMaxSuppression/map/TensorArrayStack_4/range/start) [Const])
2018-02-14 10:56:55,314 - 106 - DEBUG - DEBUG_TF_OP_NAME_TYPE_PRINT: Operation(Postprocessor/BatchMultiClassNonMaxSuppression/map/TensorArrayStack_4/range/delta) [Const])
2018-02-14 10:56:55,314 - 104 - DEBUG - DEBUG_TF_SCOPE_PRINT: Scope(num_detections)
2018-02-14 10:56:55,314 - 106 - DEBUG - DEBUG_TF_OP_NAME_TYPE_PRINT: Operation(num_detections) [Identity])
2018-02-14 10:56:55,314 - 104 - DEBUG - DEBUG_TF_SCOPE_PRINT: Scope(add_6)
2018-02-14 10:56:55,314 - 106 - DEBUG - DEBUG_TF_OP_NAME_TYPE_PRINT: Operation(add_6/y) [Const])
2018-02-14 10:56:55,314 - 106 - DEBUG - DEBUG_TF_OP_NAME_TYPE_PRINT: Operation(add_6) [Add])
2018-02-14 10:56:55,314 - 104 - DEBUG - DEBUG_TF_SCOPE_PRINT: Scope(detection_classes)
2018-02-14 10:56:55,314 - 106 - DEBUG - DEBUG_TF_OP_NAME_TYPE_PRINT: Operation(detection_classes) [Identity])
2018-02-14 10:56:57.777012: W tensorflow/core/framework/op_kernel.cc:1198] Internal: Retval[6682] has already been set.
2018-02-14 10:56:57.777016: W tensorflow/core/framework/op_kernel.cc:1198] Internal: Retval[6664] has already been set.
2018-02-14 10:56:57.777107: W tensorflow/core/framework/op_kernel.cc:1198] Internal: Retval[6685] has already been set.
2018-02-14 10:56:57.777107: W tensorflow/core/framework/op_kernel.cc:1198] Internal: Retval[6668] has already been set.
2018-02-14 10:56:57.777139: W tensorflow/core/framework/op_kernel.cc:1198] Internal: Retval[6667] has already been set.
2018-02-14 10:56:57.777142: W tensorflow/core/framework/op_kernel.cc:1198] Internal: Retval[6665] has already been set.
2018-02-14 10:56:57.777031: W tensorflow/core/framework/op_kernel.cc:1198] Internal: Retval[6666] has already been set.
2018-02-14 10:56:57.777591: W tensorflow/core/framework/op_kernel.cc:1198] Internal: Retval[6683] has already been set.
2018-02-14 10:56:57.778083: W tensorflow/core/framework/op_kernel.cc:1198] Internal: Retval[6670] has already been set.
2018-02-14 10:56:57.778110: W tensorflow/core/framework/op_kernel.cc:1198] Internal: Retval[6669] has already been set.
2018-02-14 10:56:57.977223: W tensorflow/core/framework/op_kernel.cc:1198] Internal: Retval[658] has already been set.
2018-02-14 10:56:57.977367: W tensorflow/core/framework/op_kernel.cc:1198] Internal: Retval[613] has already been set.
2018-02-14 10:56:57.977407: W tensorflow/core/framework/op_kernel.cc:1198] Internal: Retval[6509] has already been set.
2018-02-14 10:56:57.977428: W tensorflow/core/framework/op_kernel.cc:1198] Internal: Retval[6513] has already been set.
2018-02-14 10:56:57.977545: W tensorflow/core/framework/op_kernel.cc:1198] Internal: Retval[6515] has already been set.
2018-02-14 10:56:57.977547: W tensorflow/core/framework/op_kernel.cc:1198] Internal: Retval[6511] has already been set.
2018-02-14 10:56:57.977603: W tensorflow/core/framework/op_kernel.cc:1198] Internal: Retval[6500] has already been set.
2018-02-14 10:56:57.977629: W tensorflow/core/framework/op_kernel.cc:1198] Internal: Retval[6503] has already been set.
2018-02-14 10:56:57.977650: W tensorflow/core/framework/op_kernel.cc:1198] Internal: Retval[6506] has already been set.
2018-02-14 10:56:57.977671: W tensorflow/core/framework/op_kernel.cc:1198] Internal: Retval[6501] has already been set.
2018-02-14 10:56:57.977693: W tensorflow/core/framework/op_kernel.cc:1198] Internal: Retval[6507] has already been set.
2018-02-14 10:56:57.977727: W tensorflow/core/framework/op_kernel.cc:1198] Internal: Retval[617] has already been set.
2018-02-14 10:56:57.977752: W tensorflow/core/framework/op_kernel.cc:1198] Internal: Retval[616] has already been set.
2018-02-14 10:56:57.977773: W tensorflow/core/framework/op_kernel.cc:1198] Internal: Retval[614] has already been set.
2018-02-14 10:56:57.977793: W tensorflow/core/framework/op_kernel.cc:1198] Internal: Retval[615] has already been set.
2018-02-14 10:56:57.977868: W tensorflow/core/framework/op_kernel.cc:1198] Internal: Retval[6504] has already been set.
2018-02-14 10:56:57.993109: W tensorflow/core/framework/op_kernel.cc:1198] Internal: Retval[618] has already been set.
2018-02-14 10:56:57.993159: W tensorflow/core/framework/op_kernel.cc:1198] Internal: Retval[619] has already been set.
2018-02-14 10:56:57.993224: W tensorflow/core/framework/op_kernel.cc:1198] Internal: Retval[624] has already been set.
2018-02-14 10:56:57.993240: W tensorflow/core/framework/op_kernel.cc:1198] Internal: Retval[625] has already been set.
2018-02-14 10:56:57.993324: W tensorflow/core/framework/op_kernel.cc:1198] Internal: Retval[620] has already been set.
2018-02-14 10:56:57.993340: W tensorflow/core/framework/op_kernel.cc:1198] Internal: Retval[621] has already been set.
2018-02-14 10:56:57.993444: W tensorflow/core/framework/op_kernel.cc:1198] Internal: Retval[622] has already been set.
2018-02-14 10:56:57.993460: W tensorflow/core/framework/op_kernel.cc:1198] Internal: Retval[623] has already been set.
2018-02-14 10:56:58,001 - 391 - WARNING - ERROR_TF_FALLBACK_TO_ONDEMAND_EVALUATION: Unable to resolve operation output shapes in single pass. Using on-demand evaluation!
2018-02-14 10:56:58,008 - 301 - INFO - INFO_ALL_BUILDING_NETWORK:
==============================================================
Building Network
==============================================================
2018-02-14 10:56:58,009 - 326 - INFO - INFO_TF_BUILDING_INPUT_LAYER: Building layer (INPUT) with node: image_tensor, shape [300, 300, 3]
2018-02-14 10:57:17,838 - 364 - WARNING - WARNING_TF_SCOPE_OP_NOT_CONSUMED: Operation (Postprocessor/BatchMultiClassNonMaxSuppression/map/while/MultiClassNonMaxSuppression/ClipToWindow_47/Maximum_2) not consumed by converter: Maximum.
2018-02-14 10:57:17,838 - 364 - WARNING - WARNING_TF_SCOPE_OP_NOT_CONSUMED: Operation (Postprocessor/BatchMultiClassNonMaxSuppression/map/while/MultiClassNonMaxSuppression/FilterGreaterThan_20/Gather/Gather_1) not consumed by converter: Gather.
2018-02-14 10:57:17,838 - 364 - WARNING - WARNING_TF_SCOPE_OP_NOT_CONSUMED: Operation (Postprocessor/BatchMultiClassNonMaxSuppression/map/while/MultiClassNonMaxSuppression/ClipToWindow_5/Area/sub_1) not consumed by converter: Sub.
2018-02-14 10:57:17,838 - 364 - WARNING - WARNING_TF_SCOPE_OP_NOT_CONSUMED: Operation (Postprocessor/BatchMultiClassNonMaxSuppression/map/while/MultiClassNonMaxSuppression/ClipToWindow_68/Area/mul) not consumed by converter: Mul.
2018-02-14 10:57:17,838 - 364 - WARNING - WARNING_TF_SCOPE_OP_NOT_CONSUMED: Operation (Postprocessor/BatchMultiClassNonMaxSuppression/map/while/MultiClassNonMaxSuppression/Gather_9/Gather_1) not consumed by converter: Gather.
2018-02-14 10:57:17,838 - 364 - WARNING - WARNING_TF_SCOPE_OP_NOT_CONSUMED: Operation (Postprocessor/BatchMultiClassNonMaxSuppression/map/while/MultiClassNonMaxSuppression/ClipToWindow_68/Area/Squeeze) not consumed by converter: Squeeze.
2018-02-14 10:57:17,838 - 364 - WARNING - WARNING_TF_SCOPE_OP_NOT_CONSUMED: Operation (Postprocessor/BatchMultiClassNonMaxSuppression/map/while/MultiClassNonMaxSuppression/ClipToWindow_67/Maximum_2) not consumed by converter: Maximum.
2018-02-14 10:57:17,838 - 364 - WARNING - WARNING_TF_SCOPE_OP_NOT_CONSUMED: Operation (Postprocessor/BatchMultiClassNonMaxSuppression/map/while/MultiClassNonMaxSuppression/Reshape_52) not consumed by converter: Reshape.
2018-02-14 10:57:17,838 - 364 - WARNING - WARNING_TF_SCOPE_OP_NOT_CONSUMED: Operation (Postprocessor/BatchMultiClassNonMaxSuppression/map/while/MultiClassNonMaxSuppression/ClipToWindow_68/Greater) not consumed by converter: Greater.
2018-02-14 10:57:17,838 - 364 - WARNING - WARNING_TF_SCOPE_OP_NOT_CONSUMED: Operation (Postprocessor/BatchMultiClassNonMaxSuppression/map/while/TensorArrayReadV3_3) not consumed by converter: TensorArrayReadV3.
2018-02-14 10:57:17,838 - 364 - WARNING - WARNING_TF_SCOPE_OP_NOT_CONSUMED: Operation (Postprocessor/BatchMultiClassNonMaxSuppression/map/while/MultiClassNonMaxSuppression/ClipToWindow_59/Maximum_2) not consumed by converter: Maximum.
2018-02-14 10:57:17,838 - 364 - WARNING - WARNING_TF_SCOPE_OP_NOT_CONSUMED: Operation (Postprocessor/BatchMultiClassNonMaxSuppression/map/while/MultiClassNonMaxSuppression/ClipToWindow_68/Where) not consumed by converter: Where.
2018-02-14 10:57:17,838 - 364 - WARNING - WARNING_TF_SCOPE_OP_NOT_CONSUMED: Operation (Postprocessor/BatchMultiClassNonMaxSuppression/map/while/MultiClassNonMaxSuppression/ClipToWindow_44/Where) not consumed by converter: Where.
2018-02-14 10:57:17,838 - 364 - WARNING - WARNING_TF_SCOPE_OP_NOT_CONSUMED: Operation (Postprocessor/BatchMultiClassNonMaxSuppression/map/while/MultiClassNonMaxSuppression/stack_86) not consumed by converter: Pack.
2018-02-14 10:57:17,838 - 364 - WARNING - WARNING_TF_SCOPE_OP_NOT_CONSUMED: Operation (Postprocessor/BatchMultiClassNonMaxSuppression/map/while/MultiClassNonMaxSuppression/ClipToWindow_5/Reshape) not consumed by converter: Reshape.
2018-02-14 10:57:17,838 - 364 - WARNING - WARNING_TF_SCOPE_OP_NOT_CONSUMED: Operation (Postprocessor/BatchMultiClassNonMaxSuppression/map/while/MultiClassNonMaxSuppression/ClipToWindow_80/Gather/Gather) not consumed by converter: Gather.
2018-02-14 10:57:17,838 - 364 - WARNING - WARNING_TF_SCOPE_OP_NOT_CONSUMED: Operation (Postprocessor/BatchMultiClassNonMaxSuppression/map/while/MultiClassNonMaxSuppression/ClipToWindow_5/Cast) not consumed by converter: Cast.
2018-02-14 10:57:17,839 - 364 - WARNING - WARNING_TF_SCOPE_OP_NOT_CONSUMED: Operation (Postprocessor/BatchMultiClassNonMaxSuppression/map/while/MultiClassNonMaxSuppression/ClipToWindow_5/Gather/Gather) not consumed by converter: Gather.
2018-02-14 10:57:17,839 - 364 - WARNING - WARNING_TF_SCOPE_OP_NOT_CONSUMED: Operation (Postprocessor/BatchMultiClassNonMaxSuppression/map/while/MultiClassNonMaxSuppression/Shape_7) not consumed by converter: Shape.
... CUT THOUSANDS MORE LINES ...
Thanks in advance for any replies.
Rex
i meet this issue too , if you solved this problem ,please hlep me!
Sorry for the delay. There are instructions for MobilenetSSD in the users guide, in the "MobilenetSSD" chapter under "Model Conversion". Are you following those instructions exactly? They tell you which exact version of the model SNPE supports, and how to convert it. Please verify.
Which version of Tensorflow are you using?
If you get this to convert, then you need to also enable CPU Fallback if you try to run it on the GPU. Some of the layers are not supported on the GPU.
Hi jesliger,
Thanks for the reply. I tried the instructions you noted which are under the Examples Tutorials -> Model Conversion -> MobilenetSSD and they work as you stated. I will see how we can adapt our network to be compatible with this one.
Thanks again.
Rex
Hi,
Using the documented conversion and coco_ssd model - we only get the detection_classes and no boxes or scores using the documented conversion parameters.
conversion command -
application output -
dlc-info output -
| 157 | Postprocessor/Tile:0 | constant | Postprocessor/Tile:0 | Postprocessor/Tile:0 | 1x1917x4
You have to specify the output layers at runtime as well. Append them to the list that you provide the builder in the call to setOutputLayers.
SNPE 1.13.0 has a bit of improved documentation on the output layers and output buffer names.
Hi,
Thanks for the reply. As you suggested, adding all the required outputs in setOutputLayers(..) makes the other outputs available.
One side-effect here is that we have two "detection_classes" in the output buffers -
detection_classes seems more accurate than the other PostProcessor/...._classes - can you explain the differences between these outputs?
Thanks.
Rex
The tensorflow model shifts the classification by 1 as the index 0 is background.
For the classes you should use detection_classes only since it is the correct value shifted in order to match the labels file.
Hi dmarques,
Thanks for the insight. We had taken these shift factors into account.
So we ran a testing set and below I have linked an example of the problem.
Reference Image -
https://drive.google.com/file/d/1PM38uim6uIClnvqf7c5T7Z89lFDCEjQF/view?u...
Tensorflow CPU on Android using SNPE COCO v1 model -
https://drive.google.com/file/d/1vPnc0_9BJzjtT-4ij28Gp9yTgCQ1LibT/view?u...
SNPE GPU with CPU fallback on Android using SNPE COCO v1 model -
https://drive.google.com/file/d/1a3Zb8KrSdhDjZwYex0vwK6bkFfO-E-8S/view?u...
It is clear that SNPE is not giving the same results as Tensorflow so there must be a discrepancy in the way the model is handled or the way we handle the output. Any ideas are welcome.
Thanks in advance for any replies.
Rex
Hello Rex,
I use the Conversion example of SSD model in the documentation. When I run the command to convert the pb model into dlc model I get that:
Do you succeed generate the model into dlc file?
For me it seems the models has been generated but when I use it, it generated detection_classes:0.raw files for each image but all the files are the sames.
Hi Rex,
when run mobilenetssd, how to get “FloatBuffer meanImage”
Thanks
"Postprocessor/BatchMultiClassNonMaxSuppression_boxes" Is the output Ymin, Xmin, Ymax, Ymin? I use this order, but the results are not right. I followed the tutorial and used tensorflow 1.5 to transform the model.
hello jesliger!
I followed the instructions for MobilenetSSD in the users guide, in the "MobilenetSSD" chapter under "Model Conversion", to train and convert my model, but I always got the errors as bellow. and I try to open --allow unconsumed nodes, it converted success, but seems none nodes consumed at all, my tensorflow version:1.5.0, SNPE version:1.13.0.
...
...
2018-04-02 16:42:00,289 - 123 - ERROR - Conversion failed: ERROR_TF_OPERATION_NOT_MAPPED_TO_LAYER: Some operations in the Tensorflow graph were not resolved to a layer. You can use --allow_unconsumed_nodes for partial graph resolution!
Hi Rex,
May i know how you visualize the detected boxes on images using .dlc file. Any script you wrote??
#Keras_Retinanet
Brief of the problem: While trying to convert the model to DLC I am getting issues with respect to output nodes, I am able to convert keras model (.H5 file) to tensorflow model (.pb) but failing to get out the DLC file
For I am getting following listed error in the final step of conversion:
2019-11-04 14:36:18,000 - 175 - WARNING - Option: '--dlc' is DEPRECATED and will be removed in upcoming release. Please use '--output_path', '-o'
2019-11-04 14:36:18,000 - 175 - WARNING - Option: '--graph' is DEPRECATED and will be removed in upcoming release. Please use '--input_path', '-i'
2019-11-04 14:36:18.001569: I tensorflow/core/platform/cpu_feature_guard.cc:140] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2
2019-11-04 14:36:25.114553: W tensorflow/core/framework/op_kernel.cc:1202] OP_REQUIRES failed at function_ops.cc:80 : Internal: Retval[1522] has already been set.
2019-11-04 14:36:25.114674: W tensorflow/core/framework/op_kernel.cc:1202] OP_REQUIRES failed at function_ops.cc:80 : Internal: Retval[1528] has already been set.
2019-11-04 14:36:25.114742: W tensorflow/core/framework/op_kernel.cc:1202] OP_REQUIRES failed at function_ops.cc:80 : Internal: Retval[1527] has already been set.
2019-11-04 14:36:25.114823: W tensorflow/core/framework/op_kernel.cc:1202] OP_REQUIRES failed at function_ops.cc:80 : Internal: Retval[1523] has already been set.
2019-11-04 14:36:25.114886: W tensorflow/core/framework/op_kernel.cc:1202] OP_REQUIRES failed at function_ops.cc:80 : Internal: Retval[1524] has already been set.
2019-11-04 14:36:25.114990: W tensorflow/core/framework/op_kernel.cc:1202] OP_REQUIRES failed at function_ops.cc:80 : Internal: Retval[1553] has already been set.
2019-11-04 14:36:25.115067: W tensorflow/core/framework/op_kernel.cc:1202] OP_REQUIRES failed at function_ops.cc:80 : Internal: Retval[1550] has already been set.
2019-11-04 14:36:25.115145: W tensorflow/core/framework/op_kernel.cc:1202] OP_REQUIRES failed at function_ops.cc:80 : Internal: Retval[1556] has already been set.
2019-11-04 14:36:25.179434: W tensorflow/core/framework/op_kernel.cc:1202] OP_REQUIRES failed at function_ops.cc:80 : Internal: Retval[1551] has already been set.
2019-11-04 14:36:25.675460: W tensorflow/core/framework/op_kernel.cc:1202] OP_REQUIRES failed at function_ops.cc:80 : Internal: Retval[1554] has already been set.
2019-11-04 14:36:25.676226: W tensorflow/core/framework/op_kernel.cc:1202] OP_REQUIRES failed at function_ops.cc:80 : Internal: Retval[1530] has already been set.
2019-11-04 14:36:25.676299: W tensorflow/core/framework/op_kernel.cc:1202] OP_REQUIRES failed at function_ops.cc:80 : Internal: Retval[1529] has already been set.
2019-11-04 14:36:25,681 - 411 - WARNING - ERROR_TF_FALLBACK_TO_ONDEMAND_EVALUATION: Unable to resolve operation output shapes in single pass. Using on-demand evaluation!
2019-11-04 14:36:25,799 - 170 - INFO - INFO_ALL_BUILDING_NETWORK:
==============================================================
Building Network
==============================================================
2019-11-04 14:36:25,799 - 170 - INFO - INFO_TF_BUILDING_INPUT_LAYER: Building layer (INPUT) with node: input_1_2, shape [1, 224, 224, 3]
2019-11-04 14:37:23,803 - 165 - ERROR - Conversion failed: Cannot find bboxes or scores
Configuration for training model:
------------------------------------------------------
Keras: 2.2.4
Tensorflow 1.19.0
Python 3.6
:::output format : .H5
Configuration for .h5 to .pb (keras to tensorflow) :
----------------------------------------------------------------------
Keras: 2.2.4
Tensorflow: 1.16.0
Python: 2.7
<H5 to PB was Successfully converted>
Configuration for .pb to .dlc :
-------------------------------------------
Command for conversion:
snpe-tensorflow-to-dlc --graph /home/chandan/Documents/snpe/models/out_model1.pb --input_dim input_1 1,224,224,3 --out_node filtered_detections/map/TensorArrayStack/TensorArrayGatherV3 --out_node filtered_detections/map/TensorArrayStack_1/TensorArrayGatherV3 --out_node filtered_detections/map/TensorArrayStack_2/TensorArrayGatherV3 --dlc /home/chandan/Documents/snpe/models/out_model1.dlc --allow_unconsumed_nodes
Using SNPE: Snapdragon Neural Processing Engine SDK
SNPE. snpe-1.31.0.522
Python: 2.7
Tensorflow: 1.16.0
Keras: 2.2.4
<We get the error as listed above > - > ERROR - Conversion failed: Cannot find bboxes or scores
Problem 2: Can you please help me to get input shape, i am using default (1,224,224,3),
I can share you the architecture and weight file also,
please help!