Hi my model is taking almost 10 x inference time on DSP and AIP as compared to GPU . What am I missing ? In order to run model on DSP are there any constraints or further optimization which is required. I have followed the entire setup guide provided on Qualcomm developers and all layers are compatible with DSP but still its taking too much time. Any idea or suggestions are welcome . Thanks!
Opinions expressed in the content posted here are the personal opinions of the original authors, and do not necessarily reflect those of Qualcomm Incorporated or its subsidiaries (“Qualcomm”). The content is provided for informational purposes only and is not meant to be an endorsement or representation by Qualcomm or any other party. This site may also provide links or references to non-Qualcomm sites and resources. Qualcomm makes no representations, warranties, or other commitments whatsoever about any non-Qualcomm sites or third-party resources that may be referenced, accessible from, or linked to this site.
Hi Amitshuklaacer,
The statement "The higher processing power of DSP will help my model perform better compared to GPU", is not always true.
We worked with Face Expression Recognition(FER) model built using Keras and converted to the DLC file,
Comparison of Total Inference Time for GPU and DSP draws to the conclusion that DSP performance is 60% to that of GPU. Before drawing that conclusion we also have an account of the time consumed for RPC Execute ( acts as a communicator between CPU/GPU and DSP), SNPE Accelerator and Accelerator. On considering these mentioned parameters, it looks GPU is performing better than DSP for a single/lesser number of predictions. We can choose DSP as a run time only if we required to make a higher number of predictions using the FER model.
Request you to check the beanchmarking application from SNPE with multiple itterations and compare the results.
Here you can find the instructions on the usage of Bencharking Tool.