Forums - Question on 16bits fixed point running on DSP

16 posts / 0 new
or Register
Last post
Question on 16bits fixed point running on DSP
mike.hsu1
Join Date: 28 Nov 21
Posts: 4
Posted: Sun, 2021-11-28 17:28
Top

Hi,

Does the 16bits fixed point on DSP be supported?  Although it claimed a new feature on DSP for 16bits fixed ponit supported in the SNPE Archecture in docurment (80-nl315-18_e), we still found the Runtime_t enumeration only exists this definition DSP = DSP_FIXED8_TF in SNPE v1.55. My platform is using CDSP Q6 V66 and is able to be running the SNPE on DSP with 8bits fixed point successfully.

I'm not sure if anyting I missed and just wonder to confirm if supported for the 16bits fixed point on DSP.

Thanks.

 

Best Regards,

MIke

  • Up0
  • Down0
Re: Question on 16bits fixed point running on DSP
justabuices
Join Date: 25 Dec 17
Posts: 5
Posted: Sun, 2021-11-28 19:29
Top

same question here. Thank you Mike.

  • Up0
  • Down0
Re: Question on 16bits fixed point running on DSP
zhiguol
Join Date: 16 Dec 19
Posts: 25
Posted: Mon, 2021-11-29 00:28
Top

Dear customer, 

Thanks for your question. 

Yes, 16bit are supported on DSP. 

Please refer to Snapdragon Neural Processing Engine SDK: Tools (qualcomm.com)

[ --bias_bitwidth=<val> ]
                        Use the --bias_bitdwith option to select the bitwidth to use when quantizing the biases, either 8 (default) or 32. Using 32 bit biases may
                        sometimes provide a small improvement in accuracy. Can't mix with --bitwidth.
  • Up0
  • Down0
Re: Question on 16bits fixed point running on DSP
mike.hsu1
Join Date: 28 Nov 21
Posts: 4
Posted: Mon, 2021-11-29 02:15
Top

Hi zhiguol,

Thanks for your suggestion but we are not sure how to configure the -bias-bitwidth to meet the 16bits FP supported on DSP?

-bias_bitwidth

Also, I have one more question, in this case, to support 16bits fixed point on DSP, how should we designate the value of zdl::DlSystem::Runtime_t::DSP and then call setRuntimeProcessorOrder()?

Thanks.

  • Up0
  • Down0
Re: Question on 16bits fixed point running on DSP
zhiguol
Join Date: 16 Dec 19
Posts: 25
Posted: Wed, 2021-12-01 00:23
Top

bias could be 8bit or 32bit, it is not related with 16bit. 

16bit is used to do the activation.

Thanks.

  • Up0
  • Down0
Re: Question on 16bits fixed point running on DSP
mike.hsu1
Join Date: 28 Nov 21
Posts: 4
Posted: Thu, 2021-12-02 23:13
Top

Hi zhiguol,

Thanks to provide the suggestion to set activation with 16bits. We will try it and feedback to you soon.

Also, as for the SDK SNPEBuilder, how should I set which the runtime type for the case? AIP_FIXED8_TF? DSP_FIXED8_TF? or others?

Thanks.

  • Up0
  • Down0
Re: Question on 16bits fixed point running on DSP
xinlia
Join Date: 27 Dec 19
Posts: 1
Posted: Fri, 2021-12-03 01:03
Top

Hi, Mike,

Could you please try with "quntization with 16bit activation" and "set runtime as DSP (DSP_FIXED8_TF) when building SNPE instance."

Thanks

  • Up0
  • Down0
Re: Question on 16bits fixed point running on DSP
mike.hsu1
Join Date: 28 Nov 21
Posts: 4
Posted: Tue, 2021-12-07 16:31
Top

Hi xinlia,

Thanks for your suggestion. I will test it and feedback to you later.

Thanks.

  • Up0
  • Down0
Re: Question on 16bits fixed point running on DSP
1020258195
Join Date: 8 Jul 21
Posts: 8
Posted: Tue, 2021-12-07 23:13
Top

Hi, xinlia

I quantizatized my model with 16bit activation and set runtime as DSP. But I got errors like this: 

'''

build engine error : error_code=910; error_message=DSP runtime system error. error_code=910; error_message=DSP runtime system error. Failed to load network.  Error code: -6, Backend Error Code: 0.; error_component=DSP Runtime; line_no=554; thread_id=548245913824; error_component=DSP Runtime; line_no=262; thread_id=548269658128

'''

  • Up0
  • Down0
Re: Question on 16bits fixed point running on DSP
ruzhongl
Join Date: 7 Jun 21
Posts: 42
Posted: Wed, 2021-12-08 00:59
Top

V66 dsp do not enable 16 bits, only v68(888,778) and future(Gen1) support 16bits act. You can give up to try on v66 dsp for 16 bits act supports.

 

However, AIP should support 16 bits act. You can set runtime to aip

snpe-dlc-quantize --enable_aip --act_bw //enable aip and set act to 16

 

  • Up0
  • Down0
Re: Question on 16bits fixed point running on DSP
842108128
Join Date: 26 May 21
Posts: 5
Posted: Wed, 2022-06-22 19:13
Top

Hi, I used V68, but I still got the error.

Quote:
error_code=910; error_message=DSP runtime system error. error_code=910; error_message=DSP runtime system error. Failed to load network.  Error code: 66538, Backend Error Code: 66538.; error_component=DSP Runtime; line_no=813; thread_id=498818858240; error_component=DSP Runtime; line_no=267; thread_id=511548450048
 
  • Up0
  • Down0
Re: Question on 16bits fixed point running on DSP
ruzhongl
Join Date: 7 Jun 21
Posts: 42
Posted: Wed, 2022-06-22 20:32
Top

sorry for later responce,seems the sytem do not send mail in time.

From logs, it seems backend(runtime) error, it is not related with act 8/16. For some markting device, the cdsp may not allow untrust_app to access cdsp. To confirm that, you can run Antutu to test AI score, if AI score is normal, it means the device SW support untrust_app access cdsp. 

Or you can check logcat logs to confirm whether it is caused by permission related issues.

 

In all, for v68(888,778), 8a8w, 16a8w are supported. For v69(8 Gen1/Gen1 Plus), 8a8w, 16a8w, FP16 are supported. 

  • Up0
  • Down0
Re: Question on 16bits fixed point running on DSP
842108128
Join Date: 26 May 21
Posts: 5
Posted: Wed, 2022-06-22 22:10
Top

It's a pleasure to receive a response from you.

Now I can load dlc (16a8w) sucessfully with snpe-net-run (snpe-1.60).

But it failed with SampleCode.

Quote:
Error while building SNPE object.

Quote:
error_code=910; error_message=DSP runtime system error. error_code=910; error_message=DSP runtime system error. Failed to load network.  Error code: 328682, Backend Error Code: 328682.; error_component=DSP Runtime; line_no=813; thread_id=497483738304; error_component=DSP Runtime; line_no=267; thread_id=510495368448

Is it right as shown below?

        bufferType = USERBUFFER_TF16;
        bitWidth = 16;
        runtime = zdl::DlSystem::Runtime_t::DSP;

 

  • Up0
  • Down0
Re: Question on 16bits fixed point running on DSP
ruzhongl
Join Date: 7 Jun 21
Posts: 42
Posted: Thu, 2022-06-23 07:07
Top

Suggest to use ITensor as input/output first.  If you use userbuffer, you need consider the act bw when setting userbuffer, it may litle difficult.

For ITensor, you can check next code, it is easier than userbuffer.

    else if(bufferType == ITENSOR)
    {
        // A tensor map for SNPE execution outputs
        zdl::DlSystem::TensorMap outputTensorMap;
 
For commercial project,  std::copy could be used to copy the Itensor data to your own container, the performance may not good as userbuffer. If ITensor indeed cannot satify the performance, you can use userbuffer then,
 
1 when build snpe instance , setUseUserSuppliedBuffers to true.
 
snpeBuilder.setUseUserSuppliedBuffers(true).
.setRuntime  DSP// 
 
2 when create userbuffer
CreateUserBuffer.cpp
 
userBufferEncoding = std::unique_ptr<zdl::DlSystem::UserBufferEncodingTfN>(new zdl::DlSystem::UserBufferEncodingTfN(stepEquivalentTo0,quantizedStepSize, bitWidth)); //// set bitwidth 16

 

For performance optimization , it may be another topic.  I am not focus on qualcomm  ecosystem and may not response  in time. You can raise case to QC if possiable.

 

  • Up0
  • Down0
Re: Question on 16bits fixed point running on DSP
ruzhongl
Join Date: 7 Jun 21
Posts: 42
Posted: Thu, 2022-06-23 07:07
Top

Suggest to use ITensor as input/output first.  If you use userbuffer, you need consider the act bw when setting userbuffer, it may litle difficult.

For ITensor, you can check next code, it is easier than userbuffer.

    else if(bufferType == ITENSOR)
    {
        // A tensor map for SNPE execution outputs
        zdl::DlSystem::TensorMap outputTensorMap;
 
For commercial project,  std::copy could be used to copy the Itensor data to your own container, the performance may not good as userbuffer. If ITensor indeed cannot satify the performance, you can use userbuffer then,
 
1 when build snpe instance , setUseUserSuppliedBuffers to true.
 
snpeBuilder.setUseUserSuppliedBuffers(true).
.setRuntime  DSP// 
 
2 when create userbuffer
CreateUserBuffer.cpp
 
userBufferEncoding = std::unique_ptr<zdl::DlSystem::UserBufferEncodingTfN>(new zdl::DlSystem::UserBufferEncodingTfN(stepEquivalentTo0,quantizedStepSize, bitWidth)); //// set bitwidth 16

 

For performance optimization , it may be another topic.  I am not focus on qualcomm  ecosystem and may not response  in time. You can raise case to QC if possiable.

 

  • Up0
  • Down0
Re: Question on 16bits fixed point running on DSP
842108128
Join Date: 26 May 21
Posts: 5
Posted: Tue, 2022-06-28 20:24
Top

okay,  thank you very much for your patient. I will try this.

  • Up0
  • Down0
or Register

Opinions expressed in the content posted here are the personal opinions of the original authors, and do not necessarily reflect those of Qualcomm Incorporated or its subsidiaries (“Qualcomm”). The content is provided for informational purposes only and is not meant to be an endorsement or representation by Qualcomm or any other party. This site may also provide links or references to non-Qualcomm sites and resources. Qualcomm makes no representations, warranties, or other commitments whatsoever about any non-Qualcomm sites or third-party resources that may be referenced, accessible from, or linked to this site.