Hi,
I would like to know , if there is any linux sample app to use snpe with a camera input . I have found similar app for android from the link provided below
https://developer.qualcomm.com/comment/16880
I am using QCS610 Chip, which supports only linux, So I couldn't use the example provided on the link.
Also I could find sample apps for linux under snpe-1.50.0.2622/examples/NativeCpp/ . But those are for single frames and not for the live input from the camera.
Could someone point an linux application which uses snpe to run any linux sample app that takes input from the live camera and provides it to a snpe .
Regards,
Vikaash.K.B
Hello Vikaash,
Let me just walk through the process of getting inference for camera input in SNPE runtime on QCS610 based board using code snippets(I assume you already went through the tutorial for building C++ application given in the SNPE documentation) .
This can be done through GST plugins available on the board and opencv library.
cv::VideoCapture cap ("qtiqmmfsrc ldc=TRUE !video/x-raw, format=NV12, width=1280, height=720, framerate=30/1 ! videoconvert ! appsink");
cv::Mat frame;
bool bSuccess = cap.read(frame);
cv::Mat input_image = preprocess(frame); // Preprocess input image
size_t img_size = input_image.channels() * input_image.cols * input_image.rows;
Load the DLC model, set the runtime and then build the network. Here we will use CPU as sole runtime.
Check Runtime_t documentation for other supported runtimes.
zdl::DlSystem::RuntimeList runtime_list;
zdl::DlSystem::Runtime_t runtime_cpu = zdl::DlSystem::Runtime_t::CPU;
std::unique_ptr<zdl::DlContainer::IDlContainer> container =
zdl::DlContainer::IDlContainer::open(zdl::DlSystem::String(<path-to-dlc>));
zdl::SNPE::SNPEBuilder snpeBuilder(container.get());
runtime_list.add(runtime_cpu);
//If there are multiple output layers, pass that instead of empty array.
std::unique_ptr<zdl::SNPE::SNPE> model_handler = snpeBuilder.setOutputLayers({})
Create built-in SNPE buffers called iTensor (you can also use user-backed buffer, instructions for the same can be found here) for loading input.
For getting the predictions for an input we can use execute method of SNPE interface class. This method expects an iTensor and also an output TensorMap to store output.
zdl::DlSystem::TensorMap output_tensor_map;
bool exec_status = model_handler->execute(tensor_ptr, output_tensor_map);
Once the inferencing is successful, output_tensor_map will have the output values. Assuming multiple output tensors, code snippet for postprocessing is as follows.
zdl::DlSystem::StringList out_tensors =
output_tensor_map.getTensorNames(); //Get all output tensor name
//output tensor name and corresponding output values as key value pairs
Note: Add necessary header files while buiding complete application.
For more information please refer SNPE C++ API documentation and tutorial on building C++ sample application.
Feel free to contact me for any other query.
Hi
I have been trying to test SNPE with GStreamer in QCS610. But looks like "qtimlesnpe" pipeline is not supported.
Below is the output of "gst-inspect-1.0 | grep qti"
Hi prabukumar,
qtimlesnpe element in not part of GStreamer in QC610 of Release: r00130.2, Please check with latest Release i.e. r00151.1 if it enabled now, we can use that.
Regards,
Sahil Bandar