Bringing up Caffe and TensorFlow frameworks on the Qualcomm Neural Processing SDK for AI

Using MobileNet SSD model for object detection

It is necessary to convert Caffe- and TensorFlow-based models supported by the Snapdragon® Mobile Platform .dlc (Deep Learning Container) format before running them on the Qualcomm® Neural Processing SDK for AI. To understand some of the issues around that conversion process, consider the MobileNet SSD architecture.

MobileNet SSD and its layers

MobileNet SSD (Single Shot MultiBox Detector) is a small, lightweight neural network architecture for object detection. It has already been implemented in both TensorFlow and Caffe. Understanding the layers and other features of each framework is useful when running them on the Qualcomm Neural Processing SDK.

The Qualcomm® Neural Processing Engine (NPE) in the SDK supports a number of network layer types on the CPU, Qualcomm® Adreno™ GPU and Qualcomm® Hexagon™ DSP. Layers that are part of the TensorFlow or Caffe frameworks but are not supported by the SNPE will cause problems during the attempt to convert them to the .dlc format.

Bringing up MobileNet SSD on TensorFlow using SNPE

To convert TensorFlow-trained models, SNPE requires three pieces of information:

• Input layer name
• Output layer name
• Input shape

This table provides an example:

Follow these steps to convert a TensorFlow MobileNet SSD pre-trained model to run on SNPE:

   1. Run the following commands in the terminal to download and extract the model:

$ wget

http://download.tensorflow.org/models/object_detection/ssd_mobilenet_v2_quantized_300x300_coco_2019_01_03.tar.gz

$ tar xzvf ssd_mobilenet_v2_quantized_300x300_coco_2019_01_03.tar.gz

   2. Convert the model to .dlc format, passing the input layer, output layer and input shapes:

$ snpe-tensorflow-to-dlc –graph /exported/frozen_inference_graph.pb -i Preprocessor/sub 1,300,300,3 –out_node detection_classes –out_node detection_boxes –out_node detection_scores –dlc mobilenet_ssd.dlc –allow_unconsumed_nodes

Bringing up MobileNet SSD on Caffe using Qualcomm NPE

For MobileNetSSD on Caffe, a custom model is available on GitHub for training on the VOC2007 and VOC2012 data sets, with images of 20 classes. Train the model in Caffe as follows:

    1. See the README file to install and prepare the SSD-Caffe project.

    2. When finished, continue the model training to implement the MobileNet SSD detection network on Caffe.

Converting the model from Caffe to .dlc format is easier than converting from TensorFlow. The Qualcomm Neural Processing SDK includes an snpe-caffe-to-dlc conversion tool. A prototxt file (see table below) contains detailed information about the model:

   3. Once the model training is completed, run the following command to convert the Caffe model to .dlc format.

$ snpe-caffe-to-dlc –caffe_txt < path_to_prototxt file > --caffe_bin < path_to_caffemodel file > --dlc < output file name with .dlc extension >

Snapdragon Processing Engine, Snapdragon, Qualcomm Adreno, Qualcomm Hexagon and Qualcomm Neural Processing SDK for AI are products of Qualcomm Technologies, Inc. and/or its subsidiaries.