Forums - DSP runtime does not support multiple output?

10 posts / 0 new
Last post
DSP runtime does not support multiple output?
zf.africa
Join Date: 15 Jun 17
Posts: 51
Posted: Mon, 2018-01-15 19:31

Dear developers,

I've recently working on running ssd on GPU runtime, when loading model, I need to set multiple layers as output layers.

And it works now on GPU runtime, and detect objects correctly, but when I change to DSP runtime,

it would fail on loading model.

If setting only one layer as output player, the network would be loaded successfully on DSP runtime.

The documentation says that quantization does not support the layer with multiple outputs,

but in my network each layer has only one output, I just set multiple layers as the network's output layers.

Is it a bug in DSP runtime or I use the DSP runtime wrong?

  • Up0
  • Down0
jesliger
Join Date: 6 Aug 13
Posts: 75
Posted: Tue, 2018-01-16 04:32

DSP supports multiple outputs. 

The snpe-dlc-quantize tool cannot quantize a network with multiple outputs.  You can still run those on DSP, but don't use snpe-dlc-quantize to quantize offline.  Give the float model to SNPE (like you do with GPU) and SNPE will quantize the appropriate parameters before giving it to the DSP.

To further debug your issue, we'd need more specifics of the model you are using and the error you are getting.

 

  • Up0
  • Down0
zf.africa
Join Date: 15 Jun 17
Posts: 51
Posted: Tue, 2018-01-16 06:45

Hi jesliger,

I use the same code when running GPU runtime, after changing to DSP runtime, loading model would report error log:

01-15 12:23:41.532 645-2718/? E/ANDR-PERF-RESOURCEQS: Failed to reset optimization [3, 0]
01-15 12:23:41.769 6904-6959/? D/SYMPHONY: Detected Symphony running as an Android application, using logcat for all debugging output
01-15 12:23:41.769 6904-6959/? I/SYMPHONY: 36mA t7f49bf8450 /home/host/build/aarch64-android-gcc4.9/SecondParty/symphony/src/symphony/src/lib/internal/soc/qti_soc_freqs.h:28 Unable to read from /sys/class/kgsl/kgsl-3d0/gpu_available_frequencies  0m
01-15 12:23:41.762 6959-6959/? W/JavaBridge: type=1400 audit(0.0:758): avc: denied { read } for name="gpu_available_frequencies" dev="sysfs" ino=33532 scontext=u:r:untrusted_app:s0:c512,c768 tcontext=u:object_r:sysfs:s0 tclass=file permissive=0
01-15 12:23:41.827 6904-6959/? I/SYMPHONY: 36mA t7f49bf8450 /home/host/build/aarch64-android-gcc4.9/SecondParty/symphony/src/symphony/src/lib/internal/soc/topology.c:656 Topology type = 30m
01-15 12:23:41.827 6904-6959/? I/SYMPHONY: 36mA t7f49bf8450 /home/host/build/aarch64-android-gcc4.9/SecondParty/symphony/src/symphony/src/lib/internal/soc/topology.c:660 Topology cluster cpu_big/seq: indices (4, 7), # freqs: 310m
01-15 12:23:41.827 6904-6959/? I/SYMPHONY: 36mA t7f49bf8450 /home/host/build/aarch64-android-gcc4.9/SecondParty/symphony/src/symphony/src/lib/internal/soc/topology.c:664 Topology cluster cpu_little: indices (0, 3), # freqs: 220m
01-15 12:23:41.827 6904-6959/? I/SYMPHONY: 36mA t7f49bf8450 /home/host/build/aarch64-android-gcc4.9/SecondParty/symphony/src/symphony/src/lib/internal/soc/topology.c:666 Topology cluster gpu: # freqs: 70m
01-15 12:23:41.827 6904-6959/? I/SYMPHONY: 36mA t7f49bf8450 /home/host/build/aarch64-android-gcc4.9/SecondParty/symphony/src/symphony/src/lib/internal/soc/topology.c:670 Topology cluster dsp: indices (546687998656, 7904546466120570276), # freqs: 310m
01-15 12:23:41.868 6904-6959/? E/com.phonegap.helloworld: vendor/qcom/proprietary/adsprpc/src/log_config.c:503:Unable to add watcher for folder . : errno is Permission denied
01-15 12:23:41.862 6959-6959/? W/JavaBridge: type=1400 audit(0.0:759): avc: denied { read } for name="/" dev="rootfs" ino=2 scontext=u:r:untrusted_app:s0:c512,c768 tcontext=u:object_r:rootfs:s0 tclass=dir permissive=0
01-15 12:23:41.875 6904-6973/? E/com.phonegap.helloworld: vendor/qcom/proprietary/adsprpc/src/apps_std_imp.c:538:Error 45: fopen failed for oemconfig.so. (No such file or directory)
01-15 12:23:41.882 6904-6973/? E/com.phonegap.helloworld: vendor/qcom/proprietary/adsprpc/src/apps_std_imp.c:538:Error 45: fopen failed for testsig-0x877776b5.so. (No such file or directory)
01-15 12:23:41.883 6904-6973/? E/com.phonegap.helloworld: vendor/qcom/proprietary/adsprpc/src/apps_std_imp.c:538:Error 45: fopen failed for testsig.so. (No such file or directory)
01-15 12:23:41.926 6904-6959/? E/Detect: damon: DSP Supported
01-15 12:23:41.934 645-2718/? E/ANDR-PERF-OPTSHANDLER: Warning: Core control support not present for lock_max_cores on cluster 0 for this target
01-15 12:23:41.934 645-2718/? E/ANDR-PERF-RESOURCEQS: Failed to apply optimization [4, 1]
01-15 12:23:41.968 2073-2073/? D/wpa_supplicant: wlan0: Control interface command 'SIGNAL_POLL'
01-15 12:23:42.262 6904-6973/? D/com.phonegap.helloworld: vendor/qcom/proprietary/adsprpc/src/fastrpc_apps_user.c:743: Error ffffffff: remote munmap64 failed. vaddrout 0x982c00000, size 1300000
01-15 12:23:42.262 6904-6973/? E/com.phonegap.helloworld: vendor/qcom/proprietary/adsprpc/src/apps_mem_imp.c:126::error: -1: 0 == (nErr = remote_munmap64((uintptr_t)vadsp, len))
01-15 12:23:42.262 6904-6973/? E/com.phonegap.helloworld: vendor/qcom/proprietary/adsprpc/src/apps_mem_imp.c:146:Error ffffffff: apps_mem_request_unmap64 failed
01-15 12:23:42.273 6904-6973/? D/com.phonegap.helloworld: vendor/qcom/proprietary/adsprpc/src/fastrpc_apps_user.c:576: Error ffffffff: remote handle invoke failed. domain 0, handle 3, sc 4020200, pra 0x7f44528348
01-15 12:23:42.273 6904-6959/? D/com.phonegap.helloworld: vendor/qcom/proprietary/adsprpc/src/fastrpc_apps_user.c:576: Error ffffffff: remote handle invoke failed. domain 0, handle 38b0e0a0, sc 20100, pra 0x7f49bf4770
01-15 12:23:42.273 6904-6973/? E/com.phonegap.helloworld: vendor/qcom/proprietary/adsprpc/src/listener_android.c:246:listener protocol failure ffffffff
01-15 12:23:42.275 6904-6973/? D/com.phonegap.helloworld: vendor/qcom/proprietary/adsprpc/src/fastrpc_apps_user.c:576: Error 27: remote handle invoke failed. domain 0, handle 3, sc 4020200, pra 0x7f44528348
01-15 12:23:42.275 6904-6973/? E/com.phonegap.helloworld: vendor/qcom/proprietary/adsprpc/src/listener_android.c:253::error: 39: 0 == (nErr = __QAIC_HEADER(adsp_listener_next2)( ctx, nErr, 0, 0, &ctx, &handle, &sc, inBufs, inBufsLen, &inBufsLenReq))
01-15 12:23:42.275 6904-6973/? E/com.phonegap.helloworld: vendor/qcom/proprietary/adsprpc/src/listener_android.c:335:Error 27: listener thread exited
01-15 12:23:42.275 6904-6973/? D/com.phonegap.helloworld: vendor/qcom/proprietary/adsprpc/src/fastrpc_apps_user.c:1074: Error 4b: adsp current process handle failed. domain 0
01-15 12:23:42.276 6904-6959/? E/Detect: snpe is null
01-15 12:23:42.276 6904-6959/? W/PluginManager: THREAD WARNING: exec() call to VIADetect.loadDLCModel blocked the main thread for 442ms. Plugin should use CordovaInterface.getThreadPool().
01-15 12:23:42.276 6904-6973/? D/com.phonegap.helloworld: vendor/qcom/proprietary/adsprpc/src/fastrpc_apps_user.c:576: Error 27: remote handle invoke failed. domain 0, handle 0, sc 20200, pra 0x7f445280f8
01-15 12:23:42.276 6904-6959/? W/CordovaPlugin: Attempted to send a second callback for ID: Detect1951923537
                                                Result was: "Invalid action"
01-15 12:23:42.276 6904-6973/? D/com.phonegap.helloworld: vendor/qcom/proprietary/adsprpc/src/fastrpc_apps_user.c:576: Error 2c: remote handle invoke failed. domain 0, handle ffffffff, sc 1000000, pra 0x0
01-15 12:23:42.277 6904-6973/? D/com.phonegap.helloworld: vendor/qcom/proprietary/adsprpc/src/fastrpc_apps_user.c:1074: Error 4b: adsp current process handle failed. domain 0
01-15 12:23:42.279 6904-6973/? D/com.phonegap.helloworld: vendor/qcom/proprietary/adsprpc/src/fastrpc_apps_user.c:576: Error 27: remote handle invoke failed. domain 0, handle 0, sc 20200, pra 0x7f445280f8
01-15 12:23:42.279 6904-6973/? D/com.phonegap.helloworld: vendor/qcom/proprietary/adsprpc/src/fastrpc_apps_user.c:576: Error 2c: remote handle invoke failed. domain 0, handle ffffffff, sc 1000000, pra 0x0
01-15 12:23:42.279 6904-6973/? D/com.phonegap.helloworld: vendor/qcom/proprietary/adsprpc/src/fastrpc_apps_user.c:1074: Error 4b: adsp current process handle failed. domain 0
01-15 12:23:42.279 6904-6904/? D/SystemWebChromeClient: file:///android_asset/www/js/index.js: Line 206 : Load mobilenet_ssd failed !
01-15 12:23:42.279 6904-6904/? I/chromium: [INFO:CONSOLE(206)] "Load mobilenet_ssd failed !", source: file:///android_asset/www/js/index.js (206)
01-15 12:23:42.281 6904-6973/? D/com.phonegap.helloworld: vendor/qcom/proprietary/adsprpc/src/fastrpc_apps_user.c:576: Error 27: remote handle invoke failed. domain 0, handle 0, sc 20200, pra 0x7f445280f8
01-15 12:23:42.281 6904-6973/? D/com.phonegap.helloworld: vendor/qcom/proprietary/adsprpc/src/fastrpc_apps_user.c:576: Error 2c: remote handle invoke failed. domain 0, handle ffffffff, sc 1000000, pra 0x0
01-15 12:23:42.281 6904-6973/? D/com.phonegap.helloworld: vendor/qcom/proprietary/adsprpc/src/fastrpc_apps_user.c:1074: Error 4b: adsp current process handle failed. domain 0
01-15 12:23:42.282 6904-6973/? D/com.phonegap.helloworld: vendor/qcom/proprietary/adsprpc/src/fastrpc_apps_user.c:576: Error 27: remote handle invoke failed. domain 0, handle 0, sc 20200, pra 0x7f445280f8
01-15 12:23:42.282 6904-6973/? D/com.phonegap.helloworld: vendor/qcom/proprietary/adsprpc/src/fastrpc_apps_user.c:576: Error 2c: remote handle invoke failed. domain 0, handle ffffffff, sc 1000000, pra 0x0

I am working on mobilenet_ssd (This is not exactly the same as that in tensorflow, small modifications has been done for converting to dlc model) , when setting only one output layer, loading network would return a valid SNPE pointer instead of NULL pointer.

I also generate testsig.so and set ADSP_LIBRARY_PATH in native code, but it cannot find testsig.so either.

The API isRuntimeAvailable says that DSP is supported, but it reports munmap fail, and also "Failed to apply optimization", I don't know if it matters.

I  bold some text I thought might important, hope it helps.

Thank you!

 

  • Up0
  • Down0
jesliger
Join Date: 6 Aug 13
Posts: 75
Posted: Tue, 2018-01-16 07:31

testsig is not needed, as the DSP library that we ship is already signed.

To debug further, we'd need the original network URL, and the modifications made.

  • Up0
  • Down0
zf.africa
Join Date: 15 Jun 17
Posts: 51
Posted: Tue, 2018-01-16 08:23

Hi jesliger,

I use the ssd_mobilenet from tensorflow, and turn off fused_batch_norm, the specific model is unfortunately not allowed to provided to you, as it is company's property.

I get 4 layers to output positions (values are greater than 1) and 4 layers to output probabilities (values are between 0~1). At first only probability layers can be recognized as output layers, after applying sigmoid function to positions layers' output ( sigmoid would normalize position values to between 0 ~ 1 ), all the 8 layers can be recognized as output layers.

But when loading network, I got a NULL pointer returned from SNPEBuilder.build().

  • Up0
  • Down0
jesliger
Join Date: 6 Aug 13
Posts: 75
Posted: Tue, 2018-01-16 09:06

Hi Damon.  Do you have a Qualcomm Customer Engineer (support) that you are working with?  Can you tell me which company you are from.

One thing to keep in mind: mobilenet SSD will not work properly in a quantized runtime (like the DSP) at this time.  Are you able to get good  (accurate) results using Tensorflow with this model?  Quantize everything (weights, nodes, activations) and run it in tensorflow... are the results good?  (Our experiments=, and google's, show that with the publicly released mobilenet, the results are not accurate, as the model doesn't work well under quantization.)

 

  • Up0
  • Down0
zf.africa
Join Date: 15 Jun 17
Posts: 51
Posted: Tue, 2018-01-16 17:43

Hi jesliger,

I'm from VIA Technology Inc.

I haven't tried to quantize mobilenet in tensorflow, only run it with float-number model, and get good results.

You have mentioned the quantize issue in another question( thanks for that :P), so I would like to try to solve the issue about loading multiple outputs first.

 

 

  • Up0
  • Down0
zf.africa
Join Date: 15 Jun 17
Posts: 51
Posted: Thu, 2018-01-18 02:01

Hi jesliger,

I find that the model could be only loaded on DSP runtime if setting final layer as output layer.

Setting other layers would lead to load model fail on DSP runtime.

I tried to quantize model with tool snpe-dlc-quantize, it did not help either.

  • Up0
  • Down0
aliamir
Join Date: 6 Jun 18
Posts: 3
Posted: Fri, 2018-12-07 14:07

@I was wondering if you were able to resolve the issue? I'm facing the same problem when trying to run mobilenet ssd network in DSP runtime. Running on GPU works without any issues and running in DSP with default output (one output) works as well. However, when I add the bounding boxes/confidence/classId to the outputs SNPE fails to build the graph. I've tried this with both SNPE 1.19.2 and 1.21.0 in both Java and C++ and had no luck.

I used the provided tutorial page to obtain and convert the network: link

  • Up0
  • Down0
18811348016
Join Date: 9 May 22
Posts: 2
Posted: Tue, 2022-06-21 05:51

Does AIP runtime support multiple output?  I set multiple output,  and I have already quantized the dlc model by enable_hta, it works on others runtime(cpu,gpu,dsp), but it errors when I set AIP runtime. I change it to one output on AIP, it success. So AIP runtime don't support multiple output?

  • Up0
  • Down0
or Register

Opinions expressed in the content posted here are the personal opinions of the original authors, and do not necessarily reflect those of Qualcomm Incorporated or its subsidiaries (“Qualcomm”). The content is provided for informational purposes only and is not meant to be an endorsement or representation by Qualcomm or any other party. This site may also provide links or references to non-Qualcomm sites and resources. Qualcomm makes no representations, warranties, or other commitments whatsoever about any non-Qualcomm sites or third-party resources that may be referenced, accessible from, or linked to this site.