I have test the output of squeezeNet_v_1.0 for the same model and the same image, although the classified result is the same, but the probabilities are different. For example, use the same cropped image "handicap_sign.jpg", the output of caffe is "street sign: 0.14200519", but the output of snpe is "street sign: 0.1140092". For alexnet, the output of caffe is "street sign: 0.18137495" and output of SNPE is "0.18454461". I have also tested other images, for example, the cropped image "chairs", the alexnet on caffe output "studio couch, day bed: 0.4382869" and SNPE output "studio couch, day bed: 0.34171513".
The test code of caffe is refered to http://nbviewer.jupyter.org/github/BVLC/caffe/blob/master/examples/00-cl...
The different is a little big, maybe for other tasks, problems will occured for different output. So I was wondering the layers in SNPE are refered to caffe or other architecture such as tensorflow, pytorch. But as other users has posted problem the output size of deconvolution layer is different with tensorflow, so which architecture the SNPE will output the same result with it. How can SNPE ensure that I trained a model with caffe or tensorflow and I can use the model to get a expected result on snpe?
Did you get the result using GPU runtime? Have you tried CPU runtime?
Thanks
Gino
the results I posted were tested on CPU, for GPU caffe output almost the same result with CPU, and SNPE output a little different with CPU, but the output on GPU is different for caffe and snpe.
I also got the same issue. The outputs of caffe and SNPE are similar but different. I used GPU and CPU runtime. Both outputs of the runtime are different to the result generated through Caffe running on a desktop. The results from GPU and CPU runtime of SNPE are also different. Note I used a Caffe model and coverted it to DLC for SNPE use. I used Samsung Tab S3 for testing.
What could cause the differences?
Thanks!
Best regard,
Harry
Her are a few things to consider in terms of why results might vary from run-time to run-time (e.g. CPU vs. GPU) or might vary in terms of training framework vs. SNPE.
Given that many layers (e.g. convolution) often involve a huge number of floating point math operations that may not execute in exactly the same order (e.g. due to many things being done in parallel, especially on the GPU), you will see some differences just because of the order in which all the calculations are done. Typically, this won't result in huge differences but can be noticeable (e.g. you likely would see the same top few classification results but the scores might be different).
If you choose to quantize the weight and bias data at model conversion, this can lead to some differences as well (especially when comparing against the origin training framework, which quite often is using full-precision floating point).
The GPU run-time uses 32-bit floating point for the math but loads weights as 16-bit floating point values (to reduce memory bandwidth and increase speed) which will lead to differences when comparing things to the CPU run-time (which loads 32-bit weights) and/or the origin training framework. The DSP uses 8-bit fixed point values, so it will likely vary even more there.
How much things should or will vary is hard to say as it is also model dependent. Typically, we encourage users to experiment (e.g. try converting a model without quantization) if they see accuracy issues.
In terms of what do SNPE layers look like relative to TensorFlow or Caffe, in most cases they look the same. For example, convolution is convolution regardless of platform. Where differences can show up is in things like how is padding done in cases in which things don't line up exactly right. Using padding as an example, internally, SNPE can support a variety of padding styles and the model converter will indicate which one to use to match the origin system. In other words, we typically will match what the training framework does as much as possible.
Hope this helps at least give part of an answer.
Thanks Bspinar. That really helps me understand more about the processing.