Snapdragon Neural Processing Engine SDK
Reference Guide
Creating a UDO Package

This section describes the process of creating a UDO package from a simple text specification of a user-defined operation using the snpe-udo-package-generator. From the SNPE API standpoint, a UDO package consists of a registration library and one or more implementation libraries. As such, while a user can create a UDO package independent of this prescription, this section describes the process of creating a partially defined UDO package which can be easily implemented and compiled to produce the relevant libraries.

Generating UDO Skeleton Code

To generate a package using SNPE tools, it is necessary to create a UDO configuration describing the operation,and the package details. See Defining a UDO Package for more information. Once a configuration has been specified to adequately represent the desired UDO, it can be supplied as an argument to the SNPE UDO package generator tool described in snpe-udo-package-generator. The intention of the tool is to generate partial skeleton code to aid rapid prototyping. This section describes the usage of the package generator tool and the artifacts it generates.

In order to run the snpe-udo-package-generator, the user is expected to have followed the setup instructions at SNPE Setup. The tool also has a dependency on the Mako Template Library, which can be found here: https://www.makotemplates.org/download.html. Once setup is complete, the following command can be used to generate a package:

snpe-udo-package-generator -p <my_config.json> -o <my-dir>

The above command will create a UDO package which will be a directory composed of skeleton code and build files that can be used to compile the package contents into stand-alone shared libraries. The config file referenced in UDO Tutorial has been used to generate the udo package contents below:

|-- Makefile
|-- common.mk
|-- config
|   `-- softmax.json
|-- include
|   |-- SnpeUdo
|   |   |-- UdoBase.h
|   |   |-- UdoImpl.h
|   |   |-- UdoImplCpu.h
|   |   |-- UdoImplDsp.h
|   |   |-- UdoImplGpu.h
|   |   |-- UdoReg.h
|   |   `-- UdoShared.h
|   |-- SoftmaxImplLibCpu.hpp
|   |-- SoftmaxImplLibDsp.h
|   |-- SoftmaxImplLibGpu.hpp
|   |-- SoftmaxUdoPackageCpuImplValidationFunctions.hpp
|   |-- SoftmaxUdoPackageDspImplValidationFunctions.hpp
|   |-- SoftmaxUdoPackageGpuImplValidationFunctions.hpp
|   `-- utils
|       |-- GPU
|       |   `-- UdoAdrenoOperation.hpp
|       |-- IUdoOpDefinition.hpp
|       |-- UdoCpuOperation.hpp
|       |-- UdoDspShared.h
|       |-- UdoMacros.hpp
|       |-- UdoOperation.hpp
|       `-- UdoUtil.hpp
|-- jni
|   |-- Android.mk
|   |-- Application.mk
|   `-- src
|       |-- CPU
|       |   |-- Makefile
|       |   |-- SoftmaxImplLibCpu.cpp
|       |   `-- SoftmaxUdoPackageImplLibCpu.cpp
|       |-- DSP
|       |   |-- Makefile
|       |   |-- SoftmaxUdoPackageImplLibDsp.c
|       |   `-- SoftmaxImplLibDsp.c
|       |-- GPU
|       |   |-- Makefile
|       |   |-- SoftmaxImplLibGpu.cpp
|       |   `-- SoftmaxUdoPackageImplLibGpu.cpp
|       |-- reg
|       |   |-- Makefile
|       |   |-- SoftmaxUdoPackageCpuImplValidationFunctions.cpp
|       |   |-- SoftmaxUdoPackageDspImplValidationFunctions.cpp
|       |   |-- SoftmaxUdoPackageGpuImplValidationFunctions.cpp
|       |   `-- SoftmaxUdoPackageRegLib.cpp
|       `-- utils
|           |-- GPU
|           |   `-- UdoAdrenoOperation.cpp
|           |-- UdoCpuOperation.cpp
|           `-- UdoUtil.cpp

Contents of a UDO package

  • The package can be compiled using the make build system for a Linux host machine or the Android-NDK build system for an Android device. Briefly, the make system is configured using the top level Makefile, common.mk and the individual makefiles in each runtime directory. The android-build system is configured using jni/Android.mk and jni/Application .mk. See Compiling a UDO package for more compilation details.
  • The config directory contains the JSON configuration used to create the package.
  • The include directory contains three kinds of files: headers from the SNPE UDO API, header files specific to the UDO package and its operations, and a directory of C++ helper utils which wrap the SNPE UDO API calls. Users should note that the utils API is included simply for convenience in creating implementation source code. The use of the utils is not a prerequisite for constructing or executing a UDO package.
  • The relevant source files for the package are organized under the jni/src directory. There will be a sub-directory for each core-type specified in the config.The registration (reg) directory contains files necessary to create the registration library, which is generally the point of entry for the SNPE API. There is also source code from the previously mentioned C++ helper utils. In general, users are only expected to edit code contained in runtime-specific or registration directories.

Generated Source Code

This section and the following sub-sections cover the source code generated in a package using the package contents displayed in generating-udo-implementation -stubs. When finalized, a UDO package is expected to contain a registration library and one or more implementation libraries. To produce the registration library, the source code in jni/src/reg is compiled. The implementation library is compiled using source code from each core-type specific directory. Recall that the package created by the tool will still need to be implemented. The following subsections will address the files that need to be implemented. All generated source code will have the tag Auto-generated in the header. The source code is considered partially complete in the generation stage, and it is the user's responsibility to implement certain files as needed to ensure proper compatibility and functionality with the SNPE API. All code to be implemented will have the tag add code here in the body to indicate that it needs to be implemented. Note that all libraries link against the C++ utils source code.

Completing the Registration Skeleton Code

As mentioned previously, the registration library is created from source code in jni/src/reg. The directory contains a Makefile to compile the package, one or more validation files separated per runtime, and the package specific file: SoftMaxUdoPackageRegLib .cpp which contains the function symbols that are resolved by the SNPE UDO API when the library is opened. The registration library file contains API calls that provide the SNPE UDO API with information about the nature of the operations in the model, as well as the implementation libraries they belong to. The user is only expected to edit the validation files, which are designed to contain any optional validation that a user would like to implement for the operations contained in the library. Users should note that implementing validation is at their discretion. A sample snippet from the validation file used in the Softmax example above is shown below:

SnpeUdo_ErrorType_t
SoftmaxCpuValidationFunction::validateOperation(SnpeUdo_OpDefinition_t* def) {
    /**
     * add code here
     */
    return SNPE_UDO_NO_ERROR;
}

Completing the Implementation Skeleton Code

The implementation library is created per core-type, from source code that lives under the core-type specific directory within jni/src. Using the CPU runtime as an example, the jni/src/CPU directory contains a Makefile to build the CPU implementation library, a package-specific source file: SoftmaxUdoPackageImplLibCpu.cpp for all operations to be contained in the library, and a per operation source file: SoftmaxImplLibCpu.cpp that should contain the runtime implementation. As in the registration case, the package-specific source file should not be edited in the general case. Similarly this file contains methods that return information about the operations contained in the implementation library, and methods that act as a layer of indirection above the code that is ultimately executed in the per operation file. In the CPU case, the two methods in SoftmaxImplLibCpu.cpp namely:createOp and snpeUdoExecute are the user's responsibility to edit. Note these methods create the operation and execute its implementation respectively, and as such, are completely determined by the user. A sample generated version of the implementation library is included below:

std::unique_ptr<UdoUtil::UdoOperation>
SoftmaxOpDef::createOp(void *perOpInfrastructure,
                      uint32_t numOfInputs,
                      SnpeUdo_TensorParam_t *inputs,
                      uint32_t numOfOutputs,
                      SnpeUdo_TensorParam_t *outputs,
                      uint32_t numOfStaticParams,
                      SnpeUdo_Param_t* params)
{
    /**
      * add code here
      * This method should
      * 1.) Validate Static Params
      * 2.) Create Op
      * 3.) Convert arguments and use appropriate constructor in Softmax.hpp
      */
    return nullptr;
}

SnpeUdo_ErrorType_t
SoftmaxOp::snpeUdoExecute(bool blocking,
                         const uint32_t ID,
                         SnpeUdo_ExternalNotify_t notifyFunc)
{
    /**
      * add code here
      */
    return SNPE_UDO_NO_ERROR;
}

Notes

  • In the general case, the package should only require functional edits that enable proper execution. The initial un-implemented package is guaranteed to compile.
  • One subtle distinction is that the generated DSP implementation source code expects one operation per implementation library. While in the GPU and the CPU cases, there may be an arbitrary number of operations in a library.
  • There are differences between the implementation source files for each runtime. In the GPU case, the execute workflow is already implemented and the user is only expected to implement the createOp method. In the contrast to the CPU and GPU, DSP uses a C API which does not depend on C++ helper utils discussed in the Generated Source Code section. This means that certain helper methods and constructors may not be available in the DSP case.