Hi,All
I try to convert my ssd-mobilenet models to .dlc ,but meet below issue.Anybody can help me what happen?
I used the original pre-trained ssd-mobilenet models can convert to .dlc successfully,but when I Used my re-trained ssd-mobilenet models meet below error. how to fix it?
"ERROR - Conversion failed: ERROR_TF_SSD_NMS_CAN_NOT_RESOLVE_SCORE_THRESHOLD: Unable to resolve score threshold for Multi Class Non Max Suppression layer."
~/snpe-sdk$ snpe-tensorflow-to-dlc --graph $SNPE_ROOT/models/models_dlc/mobilenet_num_1.pb \
> -i Preprocessor/sub 1,300,300,3\
> --out_node detection_classes\
> --out_node detection_boxes\
> --out_node detection_scores\
> --dlc $SNPE_ROOT/models/models_dlc/mobilenet_num_1.dlc\
> --allow_unconsumed_nodes
2018-11-26 11:09:41.540407: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2018-11-26 11:09:43.320767: W tensorflow/core/framework/op_kernel.cc:1273] OP_REQUIRES failed at function_ops.cc:68 : Internal: Retval[2714] has already been set.
2018-11-26 11:09:43.320767: W tensorflow/core/framework/op_kernel.cc:1273] OP_REQUIRES failed at function_ops.cc:68 : Internal: Retval[2705] has already been set.
2018-11-26 11:09:43.320833: W tensorflow/core/framework/op_kernel.cc:1273] OP_REQUIRES failed at function_ops.cc:68 : Internal: Retval[2715] has already been set.
2018-11-26 11:09:43.320840: W tensorflow/core/framework/op_kernel.cc:1273] OP_REQUIRES failed at function_ops.cc:68 : Internal: Retval[2726] has already been set.
2018-11-26 11:09:43.320873: W tensorflow/core/framework/op_kernel.cc:1273] OP_REQUIRES failed at function_ops.cc:68 : Internal: Retval[2710] has already been set.
2018-11-26 11:09:43.320877: W tensorflow/core/framework/op_kernel.cc:1273] OP_REQUIRES failed at function_ops.cc:68 : Internal: Retval[2712] has already been set.
2018-11-26 11:09:43.320891: W tensorflow/core/framework/op_kernel.cc:1273] OP_REQUIRES failed at function_ops.cc:68 : Internal: Retval[2711] has already been set.
2018-11-26 11:09:43.320905: W tensorflow/core/framework/op_kernel.cc:1273] OP_REQUIRES failed at function_ops.cc:68 : Internal: Retval[2713] has already been set.
2018-11-26 11:09:43.320912: W tensorflow/core/framework/op_kernel.cc:1273] OP_REQUIRES failed at function_ops.cc:68 : Internal: Retval[2706] has already been set.
2018-11-26 11:09:43.320931: W tensorflow/core/framework/op_kernel.cc:1273] OP_REQUIRES failed at function_ops.cc:68 : Internal: Retval[2707] has already been set.
2018-11-26 11:09:43.320935: W tensorflow/core/framework/op_kernel.cc:1273] OP_REQUIRES failed at function_ops.cc:68 : Internal: Retval[2708] has already been set.
2018-11-26 11:09:43.320957: W tensorflow/core/framework/op_kernel.cc:1273] OP_REQUIRES failed at function_ops.cc:68 : Internal: Retval[2709] has already been set.
2018-11-26 11:09:43.406686: W tensorflow/core/framework/op_kernel.cc:1273] OP_REQUIRES failed at function_ops.cc:68 : Internal: Retval[1793] has already been set.
2018-11-26 11:09:43.406848: W tensorflow/core/framework/op_kernel.cc:1273] OP_REQUIRES failed at function_ops.cc:68 : Internal: Retval[2530] has already been set.
2018-11-26 11:09:43.406922: W tensorflow/core/framework/op_kernel.cc:1273] OP_REQUIRES failed at function_ops.cc:68 : Internal: Retval[2533] has already been set.
2018-11-26 11:09:43.406975: W tensorflow/core/framework/op_kernel.cc:1273] OP_REQUIRES failed at function_ops.cc:68 : Internal: Retval[1800] has already been set.
2018-11-26 11:09:43.406997: W tensorflow/core/framework/op_kernel.cc:1273] OP_REQUIRES failed at function_ops.cc:68 : Internal: Retval[1801] has already been set.
2018-11-26 11:09:43.407018: W tensorflow/core/framework/op_kernel.cc:1273] OP_REQUIRES failed at function_ops.cc:68 : Internal: Retval[1795] has already been set.
2018-11-26 11:09:43.407051: W tensorflow/core/framework/op_kernel.cc:1273] OP_REQUIRES failed at function_ops.cc:68 : Internal: Retval[2536] has already been set.
2018-11-26 11:09:43.407118: W tensorflow/core/framework/op_kernel.cc:1273] OP_REQUIRES failed at function_ops.cc:68 : Internal: Retval[2539] has already been set.
2018-11-26 11:09:43.407180: W tensorflow/core/framework/op_kernel.cc:1273] OP_REQUIRES failed at function_ops.cc:68 : Internal: Retval[2537] has already been set.
2018-11-26 11:09:43.407240: W tensorflow/core/framework/op_kernel.cc:1273] OP_REQUIRES failed at function_ops.cc:68 : Internal: Retval[2542] has already been set.
2018-11-26 11:09:43.407299: W tensorflow/core/framework/op_kernel.cc:1273] OP_REQUIRES failed at function_ops.cc:68 : Internal: Retval[2544] has already been set.
2018-11-26 11:09:43.407374: W tensorflow/core/framework/op_kernel.cc:1273] OP_REQUIRES failed at function_ops.cc:68 : Internal: Retval[2546] has already been set.
2018-11-26 11:09:43.407445: W tensorflow/core/framework/op_kernel.cc:1273] OP_REQUIRES failed at function_ops.cc:68 : Internal: Retval[2534] has already been set.
2018-11-26 11:09:43.407507: W tensorflow/core/framework/op_kernel.cc:1273] OP_REQUIRES failed at function_ops.cc:68 : Internal: Retval[2531] has already been set.
2018-11-26 11:09:43.407567: W tensorflow/core/framework/op_kernel.cc:1273] OP_REQUIRES failed at function_ops.cc:68 : Internal: Retval[2540] has already been set.
2018-11-26 11:09:43.407630: W tensorflow/core/framework/op_kernel.cc:1273] OP_REQUIRES failed at function_ops.cc:68 : Internal: Retval[1798] has already been set.
2018-11-26 11:09:43.407692: W tensorflow/core/framework/op_kernel.cc:1273] OP_REQUIRES failed at function_ops.cc:68 : Internal: Retval[1799] has already been set.
2018-11-26 11:09:43.407766: W tensorflow/core/framework/op_kernel.cc:1273] OP_REQUIRES failed at function_ops.cc:68 : Internal: Retval[1794] has already been set.
2018-11-26 11:09:43.407836: W tensorflow/core/framework/op_kernel.cc:1273] OP_REQUIRES failed at function_ops.cc:68 : Internal: Retval[1796] has already been set.
2018-11-26 11:09:43.407897: W tensorflow/core/framework/op_kernel.cc:1273] OP_REQUIRES failed at function_ops.cc:68 : Internal: Retval[1797] has already been set.
2018-11-26 11:09:43.408849: W tensorflow/core/framework/op_kernel.cc:1273] OP_REQUIRES failed at function_ops.cc:68 : Internal: Retval[1804] has already been set.
2018-11-26 11:09:43.408856: W tensorflow/core/framework/op_kernel.cc:1273] OP_REQUIRES failed at function_ops.cc:68 : Internal: Retval[1806] has already been set.
2018-11-26 11:09:43.408874: W tensorflow/core/framework/op_kernel.cc:1273] OP_REQUIRES failed at function_ops.cc:68 : Internal: Retval[1805] has already been set.
2018-11-26 11:09:43.408880: W tensorflow/core/framework/op_kernel.cc:1273] OP_REQUIRES failed at function_ops.cc:68 : Internal: Retval[1802] has already been set.
2018-11-26 11:09:43.408889: W tensorflow/core/framework/op_kernel.cc:1273] OP_REQUIRES failed at function_ops.cc:68 : Internal: Retval[1807] has already been set.
2018-11-26 11:09:43.408907: W tensorflow/core/framework/op_kernel.cc:1273] OP_REQUIRES failed at function_ops.cc:68 : Internal: Retval[1803] has already been set.
2018-11-26 11:09:43,413 - 393 - WARNING - ERROR_TF_FALLBACK_TO_ONDEMAND_EVALUATION: Unable to resolve operation output shapes in single pass. Using on-demand evaluation!
2018-11-26 11:09:48,274 - 106 - ERROR - Conversion failed: ERROR_TF_SSD_NMS_CAN_NOT_RESOLVE_SCORE_THRESHOLD: Unable to resolve score threshold for Multi Class Non Max Suppression layer.
Hi,
What SNPE version did you tried?
I have tried it on v1.20.3 and there is no problem on "Non Max Suppression" layer.
Thanks.
Hi,
Sorry that I just saw your issue happen on re-trained model. What the tensorflow version did you use? It looks it is tensorflow API too new are not supported by current SNPE SDK.
By the way, TF 1.6 I have tried to do re-trained and can work well.
Thanks.
@hi chenfeng what version TF you use,TF-gpu or cpu you use
Hi yingying,
do you succeed in this conversion? i tried mobilenetv2_ssd to dlc, and i use SNPE 1.22, which tensorflow version should i use? because i tried 1.12, 1.11, 1.6, they both failed. If you have successful conversion experience, please share me, thanks.
Hi chenfeng,
do you succeed in this conversion? i tried mobilenetv2_ssd to dlc, and i use SNPE 1.22, which tensorflow version should i use? because i tried 1.12, 1.11, 1.6, they both failed. If you have successful conversion experience, please share me, thanks.
Hi all,
i was able to get around this error by modifying the SNPE scripts that resolve tensorflow models. You can find the python script from <snpe root>/lib/python/converters/tensorflow/layers/ssd.py.
If you look at the script, there are classmethods for resolving score and iou thresholds named _resolve_score_threshold and _resolve_iou_threshold. If you inspect those methods you can see that they try to get the thresholds from specific path and if the model has bit different implementation of the batchmulticlassnonmaxsuppression layer, like i did, it can't find those thresholds. So what i did was, i modified those methods so that they just return the right threshold values. For example:
score_threshold = 0.3
return score_threshold
If you know the threshold that is probably the easiest way. Or you could try to modify the paths also.
I also noticed that they have added more support non max suppression and some aother layers for tensorflow converters in the latest version of SNPE 1.23. So maybe you should try the new version first.