Forums - How to let some layers which DSP supports run on CPU

2 posts / 0 new
Last post
How to let some layers which DSP supports run on CPU
kuanhungliu
Join Date: 10 Jan 22
Posts: 6
Posted: Mon, 2022-02-21 18:54

Hi,
I have a model which has attention mechanism and found the MatMul op in attention on cpu is 10x faster  than on dsp.
Is there any possible solution can achieve run the MatMul  op on cpu and others on dsp?
I have trried HTA partitions and cpu_fallbacks.
 

  • Up0
  • Down0
kuanhungliu
Join Date: 10 Jan 22
Posts: 6
Posted: Wed, 2022-02-23 19:54

Hi, 

I find UDO can satisfy my requirement and I write a MatMul UDO to override original MatMul.
The UDO works fine on CPU and GPU but not ok in DSP.
I've checked that MatMul on DSP runtime is work fine with CPU.
But I still get error meesages shows below
enviroment: snpe v1.59
platform: arm64-v8a

error_code=910; error_message=DSP runtime system error. Failed to execute network.  Error code: -6, layer id: 0, op type: 0, extra: 0, backendErrorCode: 0.; error_component=DSP Runtime; line_no=833; thread_id=509629052152
 
  • Up0
  • Down0
or Register

Opinions expressed in the content posted here are the personal opinions of the original authors, and do not necessarily reflect those of Qualcomm Incorporated or its subsidiaries (“Qualcomm”). The content is provided for informational purposes only and is not meant to be an endorsement or representation by Qualcomm or any other party. This site may also provide links or references to non-Qualcomm sites and resources. Qualcomm makes no representations, warranties, or other commitments whatsoever about any non-Qualcomm sites or third-party resources that may be referenced, accessible from, or linked to this site.