I plan to use this MobileNet-YOLO Caffe implementation: https://github.com/eric612/MobileNet-YOLO
I know that it's possible to customize SNPE to add support for UDLs. But I also came across this Supported Network Layers section of the SNPE docs (https://developer.qualcomm.com/docs/snpe/network_layers.html).
I was wondering if I correctly add my UDL codes, that SNPE will be able to support this special Caffe version.
Hi,
Yes, it will work as expected. But I see that the layers you mentioned in your model are supported by caffe framework using DSP runtime.
Confirm this before implementing UDL for the model you are working.
Kindly follow the instructions in section "Modifying snpe-caffe-to-dlc to support BV MyCustomLayer" from https://developer.qualcomm.com/docs/snpe/udl_tutorial.html to add your customed layer.
I got SNPE working with Caffe MobileNet-YOLO.
This is my setup:
1. Use SNPE 1.24. Using 1.25-1.27 does not work, the UDL APIs return incorrect blob size and data.
2. Port the Upsample Forward implementation from MobileNet-YOLO (https://github.com/eric612/MobileNet-YOLO/blob/d8bf840aa8bc64bb64b5bc9e1...) into the execute() function of your UDL. Make sure to use the correct dims, SNPE uses BxHxWxC while MobileNet-YOLO uses NxCxHxW.
3. DO NOT USE the weights_data in the execute() function of your UDL. From the SNPE tutorial (https://developer.qualcomm.com/docs/snpe/udl_tutorial.html), the example shows that the output is a function of the input and the weights data. But in our testing, if you factor in the weights, the SNPE output will be wrong. Yes, it sounds weird, so maybe it's specific only to our model.
4. For ELTWISE layers with SUM operation, you have to manually add the layer weights to get the result, because it is not available from the converter.weights_provider.weights_map. Make sure to implement the ELTWISE sum based on BLAS saxpy: y=a*x+b, where a=2.0 (Caffe default coefficient for ELTWISE sum, https://caffe.berkeleyvision.org/tutorial/layers/eltwise.html).