Executing neural networks with UDO

This section describes how to provide the set of libraries representing a UDO to SNPE, to execute network models containing the UDO. Execution requires the model containing the UDO be converted into DLC format using SNPE converter tools as explained in Preparing a model with UDO. Execution additionally requires the creation of registration and implementation libraries compatible with UDO APIs as described in Overview of UDO.

UDOs can be registered with a process and be utilized by any instance of SNPE that employs them throughout the lifetime of the process. They are registered with zdl::SNPE::SNPEFactory using the following API:

bool
SNPEFactory::addOpPackage( const std::string& regLibraryPath );

Note that the absolute path to the registration library is explicitly provided to SNPE using the above API, whereas the implementation libraries can exist anywhere on the system that should be discovered by the library loader (using LD_LIBRARY_PATH on Unix systems).

In addition to the native C++ API, the SNPE SDK provides a Java API for use in Android applications. The API is a part of the SNPE class:

public static boolean
addOpPackage(final Application application, String regLibraryPath)

Running snpe-net-run command-line tool with UDO-based DLC

This section outlines the use of snpe-net-run command line tool with a UDO-based DLC. The use of snpe-net-run is largely unchanged from its typical usage. The snpe-net-run tool registers a UDO registration library through the command line option --udo_package_path which accepts the absolute path to the registration library.

Example usage is as follows:

snpe-net-run --container <path_to_dlc> --input_list <path_to_input_list> --udo_package_path <path_to_registration_lib>

Running a UDO-based DLC on an unsigned process domain

To run on an unsigned process domain, the same principles as the above section on utilizing the snpe-net-run command line tool apply with a few extensions. Firstly, unsigned process domains only apply to the DSP target. Second, for an unsigned process domain to be created, an argument must be passed to the command line to request it. This comes in the form of the platform_options key:value argument pairing. To enable model execution on an unsigned process domain, simply add the following to the command line:

--use_dsp --platform_options unsignedPD:ON

Example usage is as follows:

snpe-net-run --container <path_to_dlc> --input_list <path_to_input_list> --udo_package_path <path_to_registration_lib> --use_dsp --platform_options unsignedPD:ON

Note: snpe-net-run requires implementation libraries to be discoverable by the system library loader.