Bringing up Caffe and TensorFlow frameworks on the Qualcomm Neural Processing SDK for AI
Using MobileNet SSD model for object detection
It is necessary to convert Caffe- and TensorFlow-based models supported by the Snapdragon® Mobile Platform .dlc (Deep Learning Container) format before running them on the Qualcomm® Neural Processing SDK for AI. To understand some of the issues around that conversion process, consider the MobileNet SSD architecture.
MobileNet SSD and its layers
MobileNet SSD (Single Shot MultiBox Detector) is a small, lightweight neural network architecture for object detection. It has already been implemented in both TensorFlow and Caffe. Understanding the layers and other features of each framework is useful when running them on the Qualcomm Neural Processing SDK.
The Qualcomm® Neural Processing Engine (NPE) in the SDK supports a number of network layer types on the CPU, Qualcomm® Adreno™ GPU and Qualcomm® Hexagon™ DSP. Layers that are part of the TensorFlow or Caffe frameworks but are not supported by the SNPE will cause problems during the attempt to convert them to the .dlc format.
Bringing up MobileNet SSD on TensorFlow using SNPE
To convert TensorFlow-trained models, SNPE requires three pieces of information:
- Input layer name
- Output layer name
- Input shape
This table provides an example:
Layers | Shape | Model Size |
---|---|---|
Input Layer: Preprocessor/sub | 1x300x300x3 | 29.1 MB |
Output Layer: 1. detection_classes 1. detection_boxes 3. detection_scores | 1. 1x100 2. 1x100 3. 1x100 |
Follow these steps to convert a TensorFlow MobileNet SSD pre-trained model to run on SNPE:
1. Run the following commands in the terminal to download and extract the model:
$ wget
$ tar xzvf ssd_mobilenet_v2_quantized_300x300_coco_2019_01_03.tar.gz
2. Convert the model to .dlc format, passing the input layer, output layer and input shapes:
$ snpe-tensorflow-to-dlc –graph
Bringing up MobileNet SSD on Caffe using Qualcomm NPE
For MobileNetSSD on Caffe, a custom model is available on GitHub for training on the VOC2007 and VOC2012 data sets, with images of 20 classes. Train the model in Caffe as follows:
1. See the README file to install and prepare the SSD-Caffe project.
2. When finished, continue the model training to implement the MobileNet SSD detection network on Caffe.
Converting the model from Caffe to .dlc format is easier than converting from TensorFlow. The Qualcomm Neural Processing SDK includes an snpe-caffe-to-dlc conversion tool. A prototxt file (see table below) contains detailed information about the model:
Layers | Shape | Model Size |
---|---|---|
Input Layer: data | 1x300x300x3 | 23.1 MB |
Output Layer: detection_out | < random number R >, 7 |
3. Once the model training is completed, run the following command to convert the Caffe model to .dlc format.
$ snpe-caffe-to-dlc –caffe_txt < path_to_prototxt file > --caffe_bin < path_to_caffemodel file > --dlc < output file name with .dlc extension >
Snapdragon Processing Engine, Snapdragon, Qualcomm Adreno, Qualcomm Hexagon and Qualcomm Neural Processing SDK for AI are products of Qualcomm Technologies, Inc. and/or its subsidiaries.