Forums - GPU degrading AIP performance when run in paralel

2 posts / 0 new
or Register
Last post
GPU degrading AIP performance when run in paralel
vkakula
Join Date: 21 Apr 23
Posts: 7
Posted: Wed, 2023-05-17 00:39
Top

I have tried runnning GPU & AIP in two threads paralley, then I have seen GPU throughput of 27.5 inferences/second & AIP throughtput of 116.1  inferences/second. But when I ran model on AIP in 2 threads in parallel, I got throughtput of AIP thread1 -> 97.6 &  AIP thread2 -> 92.8 inferences/second respectively. From this observation I understood that, AIP is acually capable of delivering 190 inferences/second, but when AIP is ran parallel with GPU, AIP performance is getting degraded, why is this happening. Is there any limitaion, can you please explain me.

Thanks in advance

  • Up0
  • Down0
Re: GPU degrading AIP performance when run in paralel
weihuan
Join Date: 12 Apr 20
Posts: 270
Posted: Fri, 2023-05-19 06:07
Top

Dear developer,

The runtime AIP is latency platform and no long support it. You could run the DSP backend instead.

BR.

Wei

  • Up0
  • Down0
or Register

Opinions expressed in the content posted here are the personal opinions of the original authors, and do not necessarily reflect those of Qualcomm Incorporated or its subsidiaries (“Qualcomm”). The content is provided for informational purposes only and is not meant to be an endorsement or representation by Qualcomm or any other party. This site may also provide links or references to non-Qualcomm sites and resources. Qualcomm makes no representations, warranties, or other commitments whatsoever about any non-Qualcomm sites or third-party resources that may be referenced, accessible from, or linked to this site.