Forums - Can DSP automatically quantize float model to int8?

3 posts / 0 new
or Register
Last post
Can DSP automatically quantize float model to int8?
ysh329
Join Date: 25 Jul 18
Posts: 31
Posted: Wed, 2019-03-20 04:58
Top

Hi, all,

I found if I ran a un-quantized model (float32) using DSP or GPU_FP16. It doesn't have any issue.

 

Thus, I wanna ask if it did fp16-conversion before input layer?

 

thanks in advance

  • Up0
  • Down0
Re: Can DSP automatically quantize float model to int8?
ysh329
Join Date: 25 Jul 18
Posts: 31
Posted: Wed, 2019-03-20 05:31
Top

Besides, I read docs and it said: SNPE doesn't support quantize UDL layer, but dsp runtime can automatically make quantization during SNPE model initialization. 

Again, I tested quantized alexnet and non-quant. using dsp runtime, they have same inference time cost.

  • Up0
  • Down0
Re: Can DSP automatically quantize float model to int8?
jihoonk
Profile picture
Join Date: 28 Jan 13
Location: Seoul
Posts: 55
Posted: Wed, 2019-03-20 19:21
Top
Hi ysh329,
 
SNPE quantizes user-provided models automatically in the initialization step so there is no issue when you provide unquantized model and choose GPU_FP16 or DSP.
 
Input data quantization is done internally as well so you don't need to do fp16-conversion.
 
Even if you provide non-quantized Alexnet, SNPE quantizes the model in the initialization step. So quantized model runs in both cases and they have same inference time.
 
Thanks,
jihoonk
 
 
 
  • Up0
  • Down0
or Register

Opinions expressed in the content posted here are the personal opinions of the original authors, and do not necessarily reflect those of Qualcomm Incorporated or its subsidiaries (“Qualcomm”). The content is provided for informational purposes only and is not meant to be an endorsement or representation by Qualcomm or any other party. This site may also provide links or references to non-Qualcomm sites and resources. Qualcomm makes no representations, warranties, or other commitments whatsoever about any non-Qualcomm sites or third-party resources that may be referenced, accessible from, or linked to this site.