Hello,
We tried to convert TensorFlow and Caffe's trained dataset into DLC.
But, that occurs errors.
Could you tell us the method to resolve the phenomena ?
The procedures is indicated on bellow in detail.
Procedure
--------------
Enviroment:
Ubuntu 14.04 64bit
SNPE v1.2.2
tensorflow v1.0 (tensorflow-1.0.0-cp27-none-linux_x86_64.whl)
caffe (git revision d8f79537)
Android NDK R11 (android-ndk-r11-linux-x86_64.zip)
Android Studio 2.3.3.0 (android-studio-ide-162.4069837-linux.zip)
Conversion of AlexNet and Inception v3 to DLC succeeded.
1)tensorflow
https://github.com/tensorflow/models/blob/master/object_detection/g3doc/...
wget http://download.tensorflow.org/models/object_detection/ssd_mobilenet_v1_...
tar zxvf ssd_mobilenet_v1_coco_11_06_2017.tar.gz
python $TENSORFLOW_HOME//lib/python2.7/site-packages/tensorflow/python/tools/optimize_for_inference.py --input frozen_inference_graph.pb --output ssd_inference_graph_optimize.pb --input_names 'image_tensor:0' --output_names 'detection_boxes,detection_scores,detection_classes'
==> Success
snpe-tensorflow-to-dlc --graph ssd_inference_graph_optimize.pb --input_dim 'image_tensor:0' 300,300,3 --out_node 'detection_boxes' --out_node 'detection_scores' --out_node 'detection_classes' --dlc ssd_inference_graph.dlc
==> Error
<Error Message>
2017-07-28 13:45:36,443 - 125 - ERROR - Encountered Error: graph_def is invalid at node u'Preprocessor/map/while/ResizeImage/ExpandDims/dim': More inputs specified ('Preprocessor/map/while/Switch:1') than the op expects..
Traceback (most recent call last):
File "/home/user/work/snpe/snpe-1.2.2/bin/x86_64-linux-clang/snpe-tensorflow-to-dlc", line 115, in main
model = loader.load(args.graph, in_nodes, in_dims, args.in_type, args.out_node, session)
File "/home/user/work/snpe/snpe-1.2.2/lib/python/converters/tensorflow/loader.py", line 67, in load
graph_def = self.__import_graph(graph_pb_or_meta_path, session, out_node_names)
File "/home/user/work/snpe/snpe-1.2.2/lib/python/converters/tensorflow/loader.py", line 129, in __import_graph
tf.import_graph_def(graph_def, name="")
File "/home/user/work/tensorflow_v1.0/local/lib/python2.7/site-packages/tensorflow/python/framework/importer.py", line 342, in import_graph_def
% (input_name,)))
ValueError: graph_def is invalid at node u'Preprocessor/map/while/ResizeImage/ExpandDims/dim': More inputs specified ('Preprocessor/map/while/Switch:1') than the op expects..
I changed " --input_dim 'image_tensor:0'" to "--input_dim 'image_tensor'" and retried.
python $TENSORFLOW_HOME//lib/python2.7/site-packages/tensorflow/python/tools/optimize_for_inference.py --input frozen_inference_graph.pb --output ssd_inference_graph_optimize.pb --input_names 'image_tensor' --output_names 'detection_boxes,detection_scores,detection_classes'
==> Success
snpe-tensorflow-to-dlc --graph ssd_inference_graph_optimize.pb --input_dim 'image_tensor' 300,300,3 --out_node 'detection_boxes' --out_node 'detection_scores' --out_node 'detection_classes' --dlc ssd_inference_graph.dlc
==> Error
<Error Message>
2017-07-28 13:51:10,868 - 125 - ERROR - Encountered Error: graph_def is invalid at node u'ToFloat': Input tensor 'image_tensor:0' Cannot convert a tensor of type float32 to an input of type uint8.
Traceback (most recent call last):
File "/home/user/work/snpe/snpe-1.2.2/bin/x86_64-linux-clang/snpe-tensorflow-to-dlc", line 115, in main
model = loader.load(args.graph, in_nodes, in_dims, args.in_type, args.out_node, session)
File "/home/user/work/snpe/snpe-1.2.2/lib/python/converters/tensorflow/loader.py", line 67, in load
graph_def = self.__import_graph(graph_pb_or_meta_path, session, out_node_names)
File "/home/user/work/snpe/snpe-1.2.2/lib/python/converters/tensorflow/loader.py", line 129, in __import_graph
tf.import_graph_def(graph_def, name="")
File "/home/user/work/tensorflow_v1.0/local/lib/python2.7/site-packages/tensorflow/python/framework/importer.py", line 368, in import_graph_def
node, 'Input tensor %r %s' % (input_name, te)))
ValueError: graph_def is invalid at node u'ToFloat': Input tensor 'image_tensor:0' Cannot convert a tensor of type float32 to an input of type uint8.
2) caffe
https://github.com/weiliu89/caffe/tree/ssd
ILSVRC models: models_VGGNet_ILSVRC2016_SSD_300x300.tar.gz
tar zxvf models_VGGNet_ILSVRC2016_SSD_300x300.tar.gz
snpe-caffe-to-dlc --caffe_txt deploy.prototxt --caffe_bin VGG_ILSVRC2016_SSD_300x300_iter_440000.caffemodel --dlc ssd300_caffe.dlc
==> Error
<Error Message>
ERROR_CAFFE_CAFFE_PARSING_ERROR: Caffe could not parse deploy.prototxt: 814:3 : Message type "caffe.LayerParameter" has no field named "norm_param".
INFO_CAFFE_CAFFE_INSTALLATION_ERROR: Caffe installation in use: /home/user/work/caffe/build/install/python/caffe/__init__.pyc
models_VGGNet_VOC0712Plus_SSD_300x300_ft.tar.gz is also the same.
--------------
Best Regards
Yoshihiro Tanno
Thank you for detailed information!
You are trying to load SSD types of models. Currently SNPE SDK does not support all layers needed for SSD models. Support will be added in future SDK releases.
As noted on https://developer.qualcomm.com/software/snapdragon-neural-processing-engine, "Due to the fast evolution of networks and layers, acceleration support is partial and will be expanded through time". The SSD netowork support is in that category right now.
Hello moljaca-san,
Thank you for your support.
We understood that SSD model suport will be added in future SDK releases.
We have a question.
Currently, Does SNPE SDK support Fast RCNN and Faster RCNN except SSD model?
Thanks a lot
Yoshihiro Tanno
Hi Yoshihiro,
Snapdragon NPE v1.2.2 does not support all layers needed for Fast RCNN and Faster RCNN networks. Support may be added in future SDK releases, depending on demand. In context of object detection, Fast RCNN and Faster RCNN networks have significantly bigger complexity comparing to SSD networks which seem more suitable for on-device deployment.
Best regards.
Hi moljaca-san,
Thank you for your support.
Then, Could you please tell us the SDK release schedule for the SDK version to be supported SSD networks ?
Best Regards,
Yoshihiro Tanno
Can you tell me the concrete layer in Fast RCNN and Faster RCNN that are not supported and what about the yolo and tiny-yolo?
I met a error when converting a YOLO object detection model to dlc using snpe-tensorflow-to-dlc, it said :
2017-09-13 15:38:40,877 - 388 - WARNING - WARNING_TF_SCOPE_OP_NOT_CONSUMED: Scope (yolo) operation(s) not consumed by converter: [u'Pad', u'Pad', u'Transpose'].
2017-09-13 15:38:41,781 - 388 - WARNING - WARNING_TF_SCOPE_OP_NOT_CONSUMED: Scope (yolo/conv_2) operation(s) not consumed by converter: [u'Mul', u'Maximum'].
2017-09-13 15:38:41,799 - 388 - WARNING - WARNING_TF_SCOPE_OP_NOT_CONSUMED: Scope (yolo/conv_4) operation(s) not consumed by converter: [u'Mul', u'Maximum'].
2017-09-13 15:38:41,814 - 388 - WARNING - WARNING_TF_SCOPE_OP_NOT_CONSUMED: Scope (yolo/conv_6) operation(s) not consumed by converter: [u'Mul', u'Maximum'].
2017-09-13 15:38:41,836 - 388 - WARNING - WARNING_TF_SCOPE_OP_NOT_CONSUMED: Scope (yolo/conv_7) operation(s) not consumed by converter: [u'Mul', u'Maximum'].
2017-09-13 15:38:41,846 - 388 - WARNING - WARNING_TF_SCOPE_OP_NOT_CONSUMED: Scope (yolo/conv_8) operation(s) not consumed by converter: [u'Mul', u'Maximum'].
2017-09-13 15:38:41,882 - 388 - WARNING - WARNING_TF_SCOPE_OP_NOT_CONSUMED: Scope (yolo/conv_9) operation(s) not consumed by converter: [u'Mul', u'Maximum'].
2017-09-13 15:38:41,898 - 388 - WARNING - WARNING_TF_SCOPE_OP_NOT_CONSUMED: Scope (yolo/conv_11) operation(s) not consumed by converter: [u'Mul', u'Maximum'].
2017-09-13 15:38:41,932 - 388 - WARNING - WARNING_TF_SCOPE_OP_NOT_CONSUMED: Scope (yolo/conv_12) operation(s) not consumed by converter: [u'Mul', u'Maximum'].
2017-09-13 15:38:41,943 - 388 - WARNING - WARNING_TF_SCOPE_OP_NOT_CONSUMED: Scope (yolo/conv_13) operation(s) not consumed by converter: [u'Mul', u'Maximum'].
2017-09-13 15:38:41,978 - 388 - WARNING - WARNING_TF_SCOPE_OP_NOT_CONSUMED: Scope (yolo/conv_14) operation(s) not consumed by converter: [u'Mul', u'Maximum'].
2017-09-13 15:38:41,990 - 388 - WARNING - WARNING_TF_SCOPE_OP_NOT_CONSUMED: Scope (yolo/conv_15) operation(s) not consumed by converter: [u'Mul', u'Maximum'].
2017-09-13 15:38:42,028 - 388 - WARNING - WARNING_TF_SCOPE_OP_NOT_CONSUMED: Scope (yolo/conv_16) operation(s) not consumed by converter: [u'Mul', u'Maximum'].
2017-09-13 15:38:42,037 - 388 - WARNING - WARNING_TF_SCOPE_OP_NOT_CONSUMED: Scope (yolo/conv_17) operation(s) not consumed by converter: [u'Mul', u'Maximum'].
2017-09-13 15:38:42,071 - 388 - WARNING - WARNING_TF_SCOPE_OP_NOT_CONSUMED: Scope (yolo/conv_18) operation(s) not consumed by converter: [u'Mul', u'Maximum'].
2017-09-13 15:38:42,084 - 388 - WARNING - WARNING_TF_SCOPE_OP_NOT_CONSUMED: Scope (yolo/conv_19) operation(s) not consumed by converter: [u'Mul', u'Maximum'].
2017-09-13 15:38:42,160 - 388 - WARNING - WARNING_TF_SCOPE_OP_NOT_CONSUMED: Scope (yolo/conv_20) operation(s) not consumed by converter: [u'Mul', u'Maximum'].
2017-09-13 15:38:42,183 - 388 - WARNING - WARNING_TF_SCOPE_OP_NOT_CONSUMED: Scope (yolo/conv_22) operation(s) not consumed by converter: [u'Mul', u'Maximum'].
2017-09-13 15:38:42,263 - 388 - WARNING - WARNING_TF_SCOPE_OP_NOT_CONSUMED: Scope (yolo/conv_23) operation(s) not consumed by converter: [u'Mul', u'Maximum'].
2017-09-13 15:38:42,281 - 388 - WARNING - WARNING_TF_SCOPE_OP_NOT_CONSUMED: Scope (yolo/conv_24) operation(s) not consumed by converter: [u'Mul', u'Maximum'].
2017-09-13 15:38:42,362 - 388 - WARNING - WARNING_TF_SCOPE_OP_NOT_CONSUMED: Scope (yolo/conv_25) operation(s) not consumed by converter: [u'Mul', u'Maximum'].
2017-09-13 15:38:42,522 - 388 - WARNING - WARNING_TF_SCOPE_OP_NOT_CONSUMED: Scope (yolo/conv_26) operation(s) not consumed by converter: [u'Mul', u'Maximum'].
2017-09-13 15:38:42,681 - 388 - WARNING - WARNING_TF_SCOPE_OP_NOT_CONSUMED: Scope (yolo/conv_28) operation(s) not consumed by converter: [u'Mul', u'Maximum'].
2017-09-13 15:38:42,850 - 388 - WARNING - WARNING_TF_SCOPE_OP_NOT_CONSUMED: Scope (yolo/conv_29) operation(s) not consumed by converter: [u'Mul', u'Maximum'].
2017-09-13 15:38:43,013 - 388 - WARNING - WARNING_TF_SCOPE_OP_NOT_CONSUMED: Scope (yolo/conv_30) operation(s) not consumed by converter: [u'Mul', u'Maximum'].
2017-09-13 15:38:43,021 - 388 - WARNING - WARNING_TF_SCOPE_OP_NOT_CONSUMED: Scope (yolo/flat_32) operation(s) not consumed by converter: [u'Shape', u'Slice', u'Slice', u'Prod', u'ExpandDims'].
2017-09-13 15:38:43,454 - 388 - WARNING - WARNING_TF_SCOPE_OP_NOT_CONSUMED: Scope (yolo/fc_33) operation(s) not consumed by converter: [u'Mul', u'Maximum'].
2017-09-13 15:38:43,527 - 388 - WARNING - WARNING_TF_SCOPE_OP_NOT_CONSUMED: Scope (yolo/fc_34) operation(s) not consumed by converter: [u'Mul', u'Maximum'].
2017-09-13 15:39:25,927 - 122 - ERROR - Conversion failed: Concatenation layer requires at least two inputs.
Hi All,
I try new release SDK v1.10.1 version transfer dlc but it has errors too.
Case 1: Run snpe-tensorflow-to-dlc command transfer mobilenet -> Success
pr-train mobilenet pb by myself -> optimized_mobilenet.pb
snpe-tensorflow-to-dlc --graph optimized_mobilenet.pb --input_dim 'input' 128,128,3 --out_node 'final_result' --dlc dlc/inception_v3.dlc --allow_unconsumed_nodes
Generate dlc file -> inception_v3.dlc file
Case 2: Run snpe-tensorflow-to-dlc command transfer ssd_mobilenet -> Fail
Download ssd_mobilenet_v1_coco_2017_11_17.tar from
https://github.com/tensorflow/models/blob/master/research/object_detecti...
snpe-tensorflow-to-dlc --graph frozen_inference_graph_optimize.pb -i image_tensor 300,300,3 --out_node detection_boxes --out_node detection_scores --out_node detection_classes --out_node num_detections --dlc test2.dlc --allow_unconsumed_nodes
Error message:
2018-01-15 17:49:15.299664: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
Hi.
Tensorflow mobilenet_ssd is not supported by SNPE at this time.
From snpe 1.17 ssd_mobilenet to dlc conversion is supportted as far as I know. You can use the following command line to do the conversion:
snpe-tensorflow-to-dlc --graph ssd_inference_graph_optimize.pb --input_dim Preprosessor/sub 1, 300,300,3 --out_node detection_boxes --out_node detection_scores --out_node detection_classes --dlc ssd_inference_graph.dlc
You need use " Preprocessor/sub" instead of "image_sensor". And the right dimension should be 1, width, height, 3. This command line will work with the ssd model as used in the snpe tutorial.