Hi,
Does the 16bits fixed point on DSP be supported? Although it claimed a new feature on DSP for 16bits fixed ponit supported in the SNPE Archecture in docurment (80-nl315-18_e), we still found the Runtime_t enumeration only exists this definition DSP = DSP_FIXED8_TF in SNPE v1.55. My platform is using CDSP Q6 V66 and is able to be running the SNPE on DSP with 8bits fixed point successfully.
I'm not sure if anyting I missed and just wonder to confirm if supported for the 16bits fixed point on DSP.
Thanks.
Best Regards,
MIke
same question here. Thank you Mike.
Dear customer,
Thanks for your question.
Yes, 16bit are supported on DSP.
Please refer to Snapdragon Neural Processing Engine SDK: Tools (qualcomm.com)
Hi zhiguol,
Thanks for your suggestion but we are not sure how to configure the -bias-bitwidth to meet the 16bits FP supported on DSP?
Also, I have one more question, in this case, to support 16bits fixed point on DSP, how should we designate the value of zdl::DlSystem::Runtime_t::DSP and then call setRuntimeProcessorOrder()?
Thanks.
bias could be 8bit or 32bit, it is not related with 16bit.
16bit is used to do the activation.
Thanks.
Hi zhiguol,
Thanks to provide the suggestion to set activation with 16bits. We will try it and feedback to you soon.
Also, as for the SDK SNPEBuilder, how should I set which the runtime type for the case? AIP_FIXED8_TF? DSP_FIXED8_TF? or others?
Thanks.
Hi, Mike,
Could you please try with "quntization with 16bit activation" and "set runtime as DSP (DSP_FIXED8_TF) when building SNPE instance."
Thanks
Hi xinlia,
Thanks for your suggestion. I will test it and feedback to you later.
Thanks.
Hi, xinlia
I quantizatized my model with 16bit activation and set runtime as DSP. But I got errors like this:
'''
build engine error : error_code=910; error_message=DSP runtime system error. error_code=910; error_message=DSP runtime system error. Failed to load network. Error code: -6, Backend Error Code: 0.; error_component=DSP Runtime; line_no=554; thread_id=548245913824; error_component=DSP Runtime; line_no=262; thread_id=548269658128
'''
V66 dsp do not enable 16 bits, only v68(888,778) and future(Gen1) support 16bits act. You can give up to try on v66 dsp for 16 bits act supports.
However, AIP should support 16 bits act. You can set runtime to aip
snpe-dlc-quantize --enable_aip --act_bw //enable aip and set act to 16
Hi, I used V68, but I still got the error.
sorry for later responce,seems the sytem do not send mail in time.
From logs, it seems backend(runtime) error, it is not related with act 8/16. For some markting device, the cdsp may not allow untrust_app to access cdsp. To confirm that, you can run Antutu to test AI score, if AI score is normal, it means the device SW support untrust_app access cdsp.
Or you can check logcat logs to confirm whether it is caused by permission related issues.
In all, for v68(888,778), 8a8w, 16a8w are supported. For v69(8 Gen1/Gen1 Plus), 8a8w, 16a8w, FP16 are supported.
It's a pleasure to receive a response from you.
Now I can load dlc (16a8w) sucessfully with snpe-net-run (snpe-1.60).
But it failed with SampleCode.
Is it right as shown below?
Suggest to use ITensor as input/output first. If you use userbuffer, you need consider the act bw when setting userbuffer, it may litle difficult.
For ITensor, you can check next code, it is easier than userbuffer.
For performance optimization , it may be another topic. I am not focus on qualcomm ecosystem and may not response in time. You can raise case to QC if possiable.
Suggest to use ITensor as input/output first. If you use userbuffer, you need consider the act bw when setting userbuffer, it may litle difficult.
For ITensor, you can check next code, it is easier than userbuffer.
For performance optimization , it may be another topic. I am not focus on qualcomm ecosystem and may not response in time. You can raise case to QC if possiable.
okay, thank you very much for your patient. I will try this.