I'm a 3rd-party app. developer.
My concern is if my app. was built with SNPE SDK, then how to make it work well in various Android devices ?
Even narrow down to Qualcomm's mobile platforms, if my app. works with SDK v1.12 and leverage GPU's 16-bit-floating runtime on 845, can it also works well with older mobile platform and newer one(in the future) ? We don't expect to prepare apps. for different platforms nor bundle multiple versions of sdk in one app.
Opinions expressed in the content posted here are the personal opinions of the original authors, and do not necessarily reflect those of Qualcomm Incorporated or its subsidiaries (“Qualcomm”). The content is provided for informational purposes only and is not meant to be an endorsement or representation by Qualcomm or any other party. This site may also provide links or references to non-Qualcomm sites and resources. Qualcomm makes no representations, warranties, or other commitments whatsoever about any non-Qualcomm sites or third-party resources that may be referenced, accessible from, or linked to this site.
SNPE only runs on Qualcomm devices. It is tested across multiple different Snapdragon devices. I believe CPU runtime should be supported on all devices. GPU will run on most devices that have a reasonable GPU. DSP runtime is limited to devices that have HVX DSPs.
You can use the "CPU Fallback" feature to ensure maximum compatibility across devices. The GPU supports falling back to CPU for layers that won't run on the CPU. The DSP doesn't support this feature yet, but look for it in an upcoming release.
Does "CPU Fallback" make a whole graph be computed by CPU or just some layers/operations not supported by hardware ?
As App just loads DLC file to your runtime in SDK, I would expect the runtime(bundled wtih specific version of compiler) to compile it to op kernels or something for processes to execute.
If my understanding is correct, why NPE would need "CPU Fallback" ?
Regarding CPU fallback.
Certain layers won't run on all runtimes. This can be because:
These limitations are documented in the user's guide.
Right now, SNPE supports GPU fallback to CPU. DSP fallback should come in a future release.
When you use GPU runtime with fallback enabled, the GPU will run all the layers it can run. Those layers that cannot be run on GPU will be run on CPU.