Hi,
I am trying to do MobileNet SSD Benchmarking from SNPE SDK. i am using SDK v1.22.0.
I am able to do Alexnet banchmarking with CPU only(Failed for GPU) which is given in SDK. it is failed for GPU because i think caffe is not installed with GPU support. I installed caffe using cmake.
Now i want to perform MobilenetSSD banchmarking.
For that i required following files:
/tmp/mobilenetssd.dlc - the converted neural network model file
/tmp/mobilenetssd.json - benchmark configuration file
/tmp/imagelist.txt - list of raw image paths (one per line)
/tmp/images - directory containing images specified in imagelist.txt above
-> For mobilenetssd.dlc file
Using TensorFlow:
Downloaded the model from http://download.tensorflow.org/models/object_detection/ssd_mobilenet_v1_coco_2017_11_17.tar.gz and try to create dlc file but it gives me following error
snpe-tensorflow-to-dlc --graph ssd_mobilenet_v1_coco_2017_11_17/frozen_inference_graph.pb -i Preprocessor/sub 1,300,300,3 --out_node detection_classes --out_node detection_boxes --out_node detection_scores --dlc mobilenet_ssd.dlc --allow_unconsumed_nodes2019-01-02 16:43:08.970001: F tensorflow/compiler/jit/deadness_analysis.cc:639] Check failed: it != predicate_map_.end() _SINKAborted (core dumped)
Using Caffe:
I have downloaded pretrained weights from https://github.com/chuanqi305/MobileNet-SSD/ and try to create dlc file but it gives me following error
snpe-caffe-to-dlc --caffe_txt MobileNetSSD_deploy.prototxt --caffe_bin MobileNetSSD_deploy.caffemodel --dlc caffe_mobilenet_ssd.dlcERROR_CAFFE_CAFFE_PARSING_ERROR: Caffe could not parse MobileNetSSD_deploy.prototxt: 1177:3 : Message type "caffe.LayerParameter" has no field named "permute_param".INFO_CAFFE_CAFFE_INSTALLATION_ERROR: Caffe installation in use: /home/user/qualcomm/caffe/build/install/python/caffe/__init__.pyc
So i download and install other caffe from https://github.com/weiliu89/caffe/tree/ssd and it will create dlc file. (I have installed caffe using cmake)
snpe-caffe-to-dlc --caffe_txt deploy.prototxt --caffe_bin mobilenet_iter_73000.caffemodel --dlc caffe_mobilenet_ssd.dlc/home/user/qualcomm/sdk/snpe-1.22.0.212/lib/python/snpe/snpe_caffe_to_dlc.py:2741: RuntimeWarning: error_code=1000; error_message=Layer is not supported. Layer detection_out of type SsdDetectionOutput not supported by GPU runtime; error_component=Model Validation; line_no=311; thread_id=140571183445760confidence_threshold=params.confidence_threshold)
-> For mobilenetssd.json file
I have created json file as given in SNPE SDK.
-> For imagelist.txt and images
I am confused that is we can use any raw images or we have to use some specific raw images for MobileNetSSD?
I tried raw images converted from alexnet model and try with snpe-net-run and snpe_bench.py but it gives me below error. Also tried https://github.com/chuanqi305/MobileNet-SSD/tree/master/images but got the same issue.
snpe-net-run --enable_cpu_fallback --container dlc/caffe_MobileNet-SSD.dlc --input_list data/cropped/raw_list.txt-------------------------------------------------------------------------------Model String: N/ASNPE v1.22.0.212-------------------------------------------------------------------------------Error: invalid input size provided. Please check all raw sizes.
Benchmarking using python snpe_bench.py -c mobilenet_sample.json -a
python snpe_bench.py -c mobilenet_sample.json -a2019-01-02 18:39:27,635 - INFO - snpe_bench: Running snpe_bench with {'config_file': 'mobilenet_sample.json', 'userbuffer_mode': '', 'perfprofile': '', 'profilinglevel': '', 'sleep': 0, 'host_name': None, 'debug': False, 'device_os_type_override': 'android', 'output_base_dir_override': None, 'run_on_all_connected_devices_override': True, 'device_id_override': None}2019-01-02 18:39:27,643 - INFO - snpe_bench: Overriding device id to [u'efedad32'], instead of [u'454d40f3'] from config file2019-01-02 18:39:27,657 - INFO - snpe_bench:--CONFIG--Name:MobileNetHostRootPath:/home/user/qaulcomm/sdk/snpe-1.22.0.212/benchmarks/mobilenetHostResultsDir:/home/user/qaulcomm/sdk/snpe-1.22.0.212/benchmarks/mobilenet/resultsDevices:[u'efedad32']DevicePath:/data/local/tmp/snpebmHostName:localhostRuns:2Model:Name:MobilenetModel:Dlc:/home/user/qaulcomm/sdk/snpe-1.22.0.212/models/mobilenet/dlc/caffe_mobilenet_ssd.dlcModel:Data:[u'../models/mobilenet/data/cropped']Model:InputDimensions:['data:1,300,300,3']Runtimes:[u'GPU', u'GPU_ub_float', u'CPU', u'CPU_ub_float']Architectures:['arm']Compiler:gcc4.9C++ Standard Library:gnustl_shared.soMeasurements:[u'timing']PerfProfile:high_performanceProfilingLevel:basicCpuFallback:FalseBufferTypes:None--END CONFIG--2019-01-02 18:39:28,247 - INFO - snpe_bench: md5 command to be used: /system/bin/md5sum2019-01-02 18:39:28,247 - INFO - snpe_bench: Perform MD5 check on files on device2019-01-02 18:39:32,090 - INFO - snpe_bench: md5 does not match for /data/local/tmp/snpebm/Mobilenet/target_raw_list.txt, copy from host again2019-01-02 18:39:34,693 - INFO - snpe_bench: md5 does not match for /data/local/tmp/snpebm/Mobilenet/cropped/raw_list.txt, copy from host again2019-01-02 18:39:36,327 - INFO - snpe_bench: Running on GPU_timing2019-01-02 18:39:36,328 - INFO - snpe_bench: Run 12019-01-02 18:39:36,978 - ERROR - snpe_bench: adb -H localhost -s efedad32 shell "sh /data/local/tmp/snpebm/artifacts/arm-android-gcc4.9/bin/snpe-bench_cmds.sh && echo ADB_SHELL_CMD_SUCCESS" failed with stderr of: error_code=1000; error_message=Layer is not supported. Layer detection_out of type SsdDetectionOutput not supported by GPU runtime; error_component=Model Validation; line_no=311; thread_id=-2437968282019-01-02 18:39:36,979 - WARNING - snpe_bench: adb shell command failed to execute:AdbShellCmdFailedException(u'error_code=1000; error_message=Layer is not supported. Layer detection_out of type SsdDetectionOutput not supported by GPU runtime; error_component=Model Validation; line_no=311; thread_id=-243796828\n',)2019-01-02 18:39:36,979 - WARNING - snpe_bench: Failed to perform benchmark for GPU_timing.2019-01-02 18:39:36,979 - INFO - snpe_bench: Running on GPU_ub_float_timing2019-01-02 18:39:36,979 - INFO - snpe_bench: Run 12019-01-02 18:39:37,627 - ERROR - snpe_bench: adb -H localhost -s efedad32 shell "sh /data/local/tmp/snpebm/artifacts/arm-android-gcc4.9/bin/snpe-bench_cmds.sh && echo ADB_SHELL_CMD_SUCCESS" failed with stderr of: error_code=1000; error_message=Layer is not supported. Layer detection_out of type SsdDetectionOutput not supported by GPU runtime; error_component=Model Validation; line_no=311; thread_id=-2511860122019-01-02 18:39:37,628 - WARNING - snpe_bench: adb shell command failed to execute:AdbShellCmdFailedException(u'error_code=1000; error_message=Layer is not supported. Layer detection_out of type SsdDetectionOutput not supported by GPU runtime; error_component=Model Validation; line_no=311; thread_id=-251186012\n',)2019-01-02 18:39:37,628 - WARNING - snpe_bench: Failed to perform benchmark for GPU_ub_float_timing.2019-01-02 18:39:37,628 - INFO - snpe_bench: Running on CPU_timing2019-01-02 18:39:37,628 - INFO - snpe_bench: Run 12019-01-02 18:39:38,452 - ERROR - snpe_bench: adb -H localhost -s efedad32 shell "sh /data/local/tmp/snpebm/artifacts/arm-android-gcc4.9/bin/snpe-bench_cmds.sh && echo ADB_SHELL_CMD_SUCCESS" failed with stderr of: -------------------------------------------------------------------------------Model String: N/ASNPE v1.22.0.212-------------------------------------------------------------------------------Error: invalid input size provided. Please check all raw sizes.2019-01-02 18:39:38,453 - WARNING - snpe_bench: adb shell command failed to execute:AdbShellCmdFailedException(u'-------------------------------------------------------------------------------\nModel String: N/A\nSNPE v1.22.0.212\n-------------------------------------------------------------------------------\nError: invalid input size provided. Please check all raw sizes.\n',)2019-01-02 18:39:38,453 - WARNING - snpe_bench: Failed to perform benchmark for CPU_timing.2019-01-02 18:39:38,453 - INFO - snpe_bench: Running on CPU_ub_float_timing2019-01-02 18:39:38,453 - INFO - snpe_bench: Run 12019-01-02 18:39:39,372 - ERROR - snpe_bench: adb -H localhost -s efedad32 shell "sh /data/local/tmp/snpebm/artifacts/arm-android-gcc4.9/bin/snpe-bench_cmds.sh && echo ADB_SHELL_CMD_SUCCESS" failed with stderr of: -------------------------------------------------------------------------------Model String: N/ASNPE v1.22.0.212-------------------------------------------------------------------------------Error: invalid input size provided. Please check all raw sizes.2019-01-02 18:39:39,372 - WARNING - snpe_bench: adb shell command failed to execute:AdbShellCmdFailedException(u'-------------------------------------------------------------------------------\nModel String: N/A\nSNPE v1.22.0.212\n-------------------------------------------------------------------------------\nError: invalid input size provided. Please check all raw sizes.\n',)2019-01-02 18:39:39,372 - WARNING - snpe_bench: Failed to perform benchmark for CPU_ub_float_timing.2019-01-02 18:39:39,372 - ERROR - snpe_bench: None of the selected benchmarks ran, therefore no results reported
So please suggest me what i am missing here. I am not sure if dlc file created is correct or not?
In SDK, sample of imagelist.txt is given like
#a/detection_out b/detection_outtmp/0#.rawtensortmp/1#.rawtensor
But i don't know how to use this, i don't have any rawtensor file. if there is specific raw images or imagelist.txt file is require then please suggest some.
Thanks,
Hiren
Hi, same problem here !
Trying to convert
http://download.tensorflow.org/models/object_detection/ssd_mobilenet_v2_...
or
http://download.tensorflow.org/models/object_detection/ssd_inception_v2_...
With the same command as yours using tensoflow, and I get the same issue :
2019-01-03 11:58:34.397865: F tensorflow/compiler/jit/deadness_analysis.cc:639] Check failed: it != predicate_map_.end() _SINK
Hi,
It seems that the input dimensions of your raw files are incorrect. Your model input dimensions are 1x300x300x3 and alexnet i think are 1x227x227x3. I think you can use create_inceptionv3_raws.py script to create right size raw files, just give 300 as input size argument.
Also I can see that you didn't use cpufallback when benchmarking the model. You need to use cpufallback when running ssd nets, because some of the layers do not support gpu runtime. So use cpufallback: true argument as it is given in json example.
Hi,
I am now able to run Benchmarking for MobilenetSSD after creating raw image of size 300 using create_inceptionv3_raws.py script.
But this benchmarking is failed to run in GPU. I think this is the problem with my dlc creation, because i installed caffe using cmake as given in http://caffe.berkeleyvision.org/installation.html#compilation and i think caffe is not installed with GPU support. I am not able to build caffe using make command. As per above blog CUDA is required to install caffe with GPU support, but i don't have NVIDIA card in my machine. So can someone please suggest how can i install caffe with GPU, DSP support. Or if there is some reference where i can find prebuilt dlc file with GPU and CPU support. I am trying to run SNPE model with all CPU, GPU and DSP support.
Thanks,
Hiren
Hi Hiren,
i am trying to use MobilenetSSD in my test praogram, and i have converted MObilenetSSD caffe model to dlc, also got raw images, test program is the example of Nativeapp(do a little modification), and errors occured everytime, i know the API setCPUFallbackMode should be used, and i just create an instance of this class to call the function:
zdl::SNPE::SNPEBuilder* tmpBuilderPtr=new zdl::SNPE::SNPEBuilder(&*container);
tmpBuilderPtr->setCPUFallbackMode(true);
it seems nothing help
could you please help to tell how to use this API?
By the way, the error is :
terminating with uncaught exception of type std::out_of_range: unordered_map::at: key not found
Hi Hiren,
We have tried converting the MobileNetSSD using given documentation by Qualcomm in SNPE Documentation.
https://developer.qualcomm.com/docs/snpe/convert_mobilenetssd.html
The conversion is done successfully for MobileNetSSD using SNPE1.21 and Tensorflow 1.11 on Ubuntu 16.04. Please try to get proper setup running according to setup documentation.
The Error: invalid input size provided. Please check all raw sizes. Will causes because of the wrong input sized raw image passed to the model for prediction.
If you are using the AlexNet model the input size will be 227x227x3. And required input size for MobileNetSSD is the 300x300x3. So You are passing the input image with a size of 227x227x3, because of that only you are getting the error.
If you want to convert the image into a raw file, You have to write the simple python script for it, as SNPE SDK didn't provide the tools for any size raw image conversion.
Steps for the image to raw image conversion:
1. Read image using OpenCV or any other API. (Make sure the image size should be in size of input required by model, Else resize the image into the script using OpenCV Resize API)
2. Convert the image into a Numpy array using Numpy API.
3. Dump that array into the file using the Numpy API(numpy.tofile()).
4. Once you have stored it into the file that is nothing but your Raw Image.
Input_list.txt file.
The file should be having the Input raw images file paths on each line. Output layer name is not required as its option for MobileNetSSD
<path_to_raw>/raw_file1.raw
<path_to_raw>/raw_file2.raw
Hi,
The model is unable to execute on GPU because its layers are not supported by NPE SDK on GPU.
You can confirm the network layers supported by NPE SDK for Caffe and TensorFlow frameworks here.