Forums - Debugging Model Inference Runtimes

4 posts / 0 new
or Register
Last post
Debugging Model Inference Runtimes
a.selvakumarasingam
Join Date: 18 Jan 21
Posts: 4
Posted: Tue, 2024-02-06 07:23
Top

Hi All,

I've recently converted an ONNX model through the SNPE to a DLC and have run inference. I noticed the model running on the GPU or DSP runtime is much slower than CPU. I understand this can happen when certain layers are not supported, but I'm looking for more information to optimizing this.

(1) Is there an updated list of layers that are supported. I came across the table on the reference guide, but it had a note saying that the list is outdated.

(2) How can I characterize this performance gap. Looking for ways to view the graph and see the points at which runtimes are switched. 

(3) This might be related to (2), but I have the same question for quantization. Does the quantizer recreate the graph based on the layer support for quantized inputs (adding quantized/dequantizer layers)? 

I'm open to suggestions from the community on ways to dissect this further.

 

 

  • Up0
  • Down0
Re: Debugging Model Inference Runtimes
sanjjey.a.sanjjey
Join Date: 17 May 22
Posts: 55
Posted: Tue, 2024-02-06 23:00
Top

Hi,

Which SNPE version did you used? 

Can you check with the latest SNPE version.

Thanks

  • Up0
  • Down0
Re: Debugging Model Inference Runtimes
a.selvakumarasingam
Join Date: 18 Jan 21
Posts: 4
Posted: Wed, 2024-02-07 12:24
Top

I'm using 2.17.0.231124. Do you know where I can find the updated list for that version?

  • Up0
  • Down0
Re: Debugging Model Inference Runtimes
sanjjey.a.sanjjey
Join Date: 17 May 22
Posts: 55
Posted: Wed, 2024-02-07 21:31
Top

you can check in qualcomm package manager, check the below given link. You can able to download and view the list of version here.

https://qpm.qualcomm.com/#/main/tools/details/qualcomm_neural_processing...

  • Up0
  • Down0
or Register

Opinions expressed in the content posted here are the personal opinions of the original authors, and do not necessarily reflect those of Qualcomm Incorporated or its subsidiaries (“Qualcomm”). The content is provided for informational purposes only and is not meant to be an endorsement or representation by Qualcomm or any other party. This site may also provide links or references to non-Qualcomm sites and resources. Qualcomm makes no representations, warranties, or other commitments whatsoever about any non-Qualcomm sites or third-party resources that may be referenced, accessible from, or linked to this site.