I'm a 3rd-party app. developer.
My concern is if my app. was built with SNPE SDK, then how to make it work well in various Android devices ?
Even narrow down to Qualcomm's mobile platforms, if my app. works with SDK v1.12 and leverage GPU's 16-bit-floating runtime on 845, can it also works well with older mobile platform and newer one(in the future) ? We don't expect to prepare apps. for different platforms nor bundle multiple versions of sdk in one app.
SNPE only runs on Qualcomm devices. It is tested across multiple different Snapdragon devices. I believe CPU runtime should be supported on all devices. GPU will run on most devices that have a reasonable GPU. DSP runtime is limited to devices that have HVX DSPs.
You can use the "CPU Fallback" feature to ensure maximum compatibility across devices. The GPU supports falling back to CPU for layers that won't run on the CPU. The DSP doesn't support this feature yet, but look for it in an upcoming release.
Does "CPU Fallback" make a whole graph be computed by CPU or just some layers/operations not supported by hardware ?
As App just loads DLC file to your runtime in SDK, I would expect the runtime(bundled wtih specific version of compiler) to compile it to op kernels or something for processes to execute.
If my understanding is correct, why NPE would need "CPU Fallback" ?
Regarding CPU fallback.
Certain layers won't run on all runtimes. This can be because:
These limitations are documented in the user's guide.
Right now, SNPE supports GPU fallback to CPU. DSP fallback should come in a future release.
When you use GPU runtime with fallback enabled, the GPU will run all the layers it can run. Those layers that cannot be run on GPU will be run on CPU.