Frequently used terms in AI Tools and Resources
Below are a few terms and concepts that you may come across while using Artificial Intelligence tools and resources.
Deep Learning Container (DLC) file
The latest Snapdragon® mobile platform supports various ML Frameworks like TensorFlow, TFLite, Pytorch, ONNX, Caffe, Caffe2, etc. In order to support all these frameworks on the hardware, the model files like .pb, .onnx, .pth, etc. should be converted to .dlc files using the conversion tools shared along with the SDKs. Many layer level optimizations are also done during the conversion process so that the models can run efficiently on our platforms.
Quantization
Quantization techniques are applied on the trained model to reduce the size of the model along with improving the performance of the model. The DLC Files are converted from FP32 precision to lower precisions like INT4, INT8 etc. User can also use reduced precision floating point representations, like FP16. This is designed to reduce the size of the model and is also faster to execute. Static quantization of weights, biases, and activations are done with support for asymmetric dynamic range and arbitrary step size. Quantization is necessary for running the model on the AI Accelerator. The tools for this are shared along with the SDKs.
User Defined Operations (UDO)
Developers can define their own Ops as well and compile them to run on CPU/GPU/AI Accelerator (HTP). Developers can use a language like C/C++ for developing an Op for CPU. For GPU, OpenCL can be used. For Qualcomm HTP, the Assembly instructions of Hexagon DSP are used. There could potentially be a context switch between Accelerator Core and CPU/GPU while executing the UDO, so developers should choose the compilation target of UDO as per their performance requirements. More details can be found here.
Operations (Ops)
Ops are nodes in the Graph of a ML Model. Examples of Ops are Argmax, Conv2d, etc.
Snapdragon and Qualcomm branded products are products of Qualcomm Technologies, Inc. and/or its subsidiaries.