I plan to use this MobileNet-YOLO Caffe implementation: https://github.com/eric612/MobileNet-YOLO
I know that it's possible to customize SNPE to add support for UDLs. But I also came across this Supported Network Layers section of the SNPE docs (https://developer.qualcomm.com/docs/snpe/network_layers.html).
I was wondering if I correctly add my UDL codes, that SNPE will be able to support this special Caffe version.
Opinions expressed in the content posted here are the personal opinions of the original authors, and do not necessarily reflect those of Qualcomm Incorporated or its subsidiaries (“Qualcomm”). The content is provided for informational purposes only and is not meant to be an endorsement or representation by Qualcomm or any other party. This site may also provide links or references to non-Qualcomm sites and resources. Qualcomm makes no representations, warranties, or other commitments whatsoever about any non-Qualcomm sites or third-party resources that may be referenced, accessible from, or linked to this site.
Hi,
Yes, it will work as expected. But I see that the layers you mentioned in your model are supported by caffe framework using DSP runtime.
Confirm this before implementing UDL for the model you are working.
Kindly follow the instructions in section "Modifying snpe-caffe-to-dlc to support BV MyCustomLayer" from https://developer.qualcomm.com/docs/snpe/udl_tutorial.html to add your customed layer.
I got SNPE working with Caffe MobileNet-YOLO.
This is my setup:
1. Use SNPE 1.24. Using 1.25-1.27 does not work, the UDL APIs return incorrect blob size and data.
2. Port the Upsample Forward implementation from MobileNet-YOLO (https://github.com/eric612/MobileNet-YOLO/blob/d8bf840aa8bc64bb64b5bc9e1...) into the execute() function of your UDL. Make sure to use the correct dims, SNPE uses BxHxWxC while MobileNet-YOLO uses NxCxHxW.
3. DO NOT USE the weights_data in the execute() function of your UDL. From the SNPE tutorial (https://developer.qualcomm.com/docs/snpe/udl_tutorial.html), the example shows that the output is a function of the input and the weights data. But in our testing, if you factor in the weights, the SNPE output will be wrong. Yes, it sounds weird, so maybe it's specific only to our model.
4. For ELTWISE layers with SUM operation, you have to manually add the layer weights to get the result, because it is not available from the converter.weights_provider.weights_map. Make sure to implement the ELTWISE sum based on BLAS saxpy: y=a*x+b, where a=2.0 (Caffe default coefficient for ELTWISE sum, https://caffe.berkeleyvision.org/tutorial/layers/eltwise.html).