Forums - Run model on GPU and HTP in parallel

3 posts / 0 new
Last post
Run model on GPU and HTP in parallel
manuel.k
Join Date: 27 Feb 23
Posts: 15
Posted: Wed, 2023-03-08 04:38

Hi everyone!

We are looking to run a fairly large model on device in a quick "burst", meaning as fast as possible but only once in a while and not continuously.

Would it make sense to run it on the GPU and HTP in parallel? We have to run on patches anyhow due to memory constraints and could therefore execute half of them on GPU and the other half on HTP.

To the best of our understanding, both GPU and HTP are QNN backends and both support float models (at least on selected devices.)

Thank you in advance, any help is appreciated!

Manuel

  • Up0
  • Down0
weihuan
Join Date: 12 Apr 20
Posts: 270
Posted: Sat, 2023-03-11 23:57

Dear developer,

The GPU and HTP are two different backend in our platform. You shall be applied tow SNPE instance in you application. One is for GPU backend and another presents to HTP backend.

BR.

Wei

  • Up0
  • Down0
manuel.k
Join Date: 27 Feb 23
Posts: 15
Posted: Tue, 2023-03-14 08:14

Dear Wei,

Thank you for getting back to me!

So does that mean we can use the GPU and HTP at the same time if we create one SNPE instance for each of them at the same time?

And then we could process a given amount of data up to 2x as fast because we one half on the GPU and the other half on the HTP?

Thank you,

Manuel

  • Up0
  • Down0
or Register

Opinions expressed in the content posted here are the personal opinions of the original authors, and do not necessarily reflect those of Qualcomm Incorporated or its subsidiaries (“Qualcomm”). The content is provided for informational purposes only and is not meant to be an endorsement or representation by Qualcomm or any other party. This site may also provide links or references to non-Qualcomm sites and resources. Qualcomm makes no representations, warranties, or other commitments whatsoever about any non-Qualcomm sites or third-party resources that may be referenced, accessible from, or linked to this site.