Hi I have tried to run the snpe_bench.py benchmarking tool on an RB5 with the custom config file that follows:
{
"Name":"XXXX",
"HostRootPath": "XXXX/data/benchmarking/depthmap",
"HostResultsDir":"XXXX/data/benchmarking/depthmap/results",
"DevicePath":"/data/local/tmp/testing/depthmap/benchmark",
"Devices":["ZTR10S1600EG"],
"Runs":10,
"Model": {
"Name": "DEPTHMAP",
"Dlc": "XXXX/models/dlc/final_model.dlc",
"Data": [
"XXXX/data/input"
],
"InputList": "XXXX/data/input/target_input_list.txt"
},
"Runtimes":["CPU","DSP","AIP"],
"Measurements": ["timing","mem"]
}
However, the benchmarking fails bcause it looks for an arm architecture and clang6.0 compiler, which can't be changed in the config file, whereas I was only able to run snpe-net-run on the RB5 with an aarch64-linux-gcc4.9 architecture:
export SNPE_TARGET_ARCH=aarch64-linux-gcc4.9
The error is below:
python3 snpe_bench.py -c depthmap_config.json -a
2021-06-10 16:56:50,577 - INFO - snpe_bench: Running snpe_bench with {'config_file': 'depthmap_config.json', 'output_base_dir_override': None, 'device_id_override': None, 'host_name': None, 'run_on_all_connected_devices_override': True, 'device_os_type_override': 'android', 'debug': False, 'sleep': 0, 'userbuffer_mode': '', 'perfprofile': '', 'profilinglevel': '', 'generate_json': False, 'profile_mode': False, 'enable_init_cache': False}
2021-06-10 16:56:50,582 - INFO - snpebm.snpebm_config: Overriding device id to ['ZTR10S1600EG'], instead of ['ZTR10S1600EG'] from config file
2021-06-10 16:56:50,582 - INFO - snpe_bench:
--CONFIG--
Name:Depthmap
HostRootPath:XXXX/data/benchmarking/depthmap
HostResultsDir:XXXX/data/benchmarking/depthmap/results
Devices:['ZTR10S1600EG']
DevicePath:/data/local/tmp/testing/depthmap/benchmark
HostName:localhost
Runs:10
Model:Name:DEPTHMAP
Model:Dlc:XXXX/models/dlc/final_model.dlc
Model:Data:['XXXX/data/input']
Model:UDO:None
Model:InputDimensions:['x:0:1,640,320,6', 'x_1:0:1,640,320,1', 'x_2:0:1,640,320,1']
Runtimes:['CPU', 'CPU_ub_float', 'DSP', 'DSP_ub_float', 'DSP_ub_tf8', 'AIP', 'AIP_ub_float', 'AIP_ub_tf8']
Architectures:['arm']
Compiler:clang6.0
C++ Standard Library:libc++_shared.so
Measurements:['timing', 'mem']
PerfProfile:high_performance
ProfilingLevel:basic
CpuFallback:False
UnsignedPD:False
UBFloatOutput:False
BufferTypes:All
--END CONFIG--
2021-06-10 16:57:05,564 - INFO - snpebm.snpebm_md5: md5 command to be used: /usr/bin/md5sum
2021-06-10 16:57:05,565 - INFO - snpebm.snpebm_md5: Perform MD5 check on files on device
2021-06-10 16:57:05,619 - INFO - snpebm.snpebm_md5: snpe-net-run not present on device at
/data/local/tmp/testing/depthmap/benchmark/artifacts/arm-android-clang6.0/bin, copying
2021-06-10 16:57:05,937 - INFO - snpebm.snpebm_md5: libSNPE.so not present on device at
/data/local/tmp/testing/depthmap/benchmark/artifacts/arm-android-clang6.0/lib, copying
2021-06-10 16:57:07,698 - INFO - snpebm.snpebm_md5: libsnpe_dsp_domains_v2.so not present on device at
/data/local/tmp/testing/depthmap/benchmark/artifacts/arm-android-clang6.0/lib, copying
2021-06-10 16:57:07,913 - INFO - snpebm.snpebm_md5: libsnpe_dsp_domains_v3.so not present on device at
/data/local/tmp/testing/depthmap/benchmark/artifacts/arm-android-clang6.0/lib, copying
2021-06-10 16:57:08,125 - INFO - snpebm.snpebm_md5: libhta.so not present on device at
/data/local/tmp/testing/depthmap/benchmark/artifacts/arm-android-clang6.0/lib, copying
2021-06-10 16:57:09,180 - INFO - snpebm.snpebm_md5: libc++_shared.so not present on device at
/data/local/tmp/testing/depthmap/benchmark/artifacts/arm-android-clang6.0/lib, copying
2021-06-10 16:57:09,578 - INFO - snpebm.snpebm_md5: libsnpe_dsp_v65_domains_v2_skel.so not present on device at
/data/local/tmp/testing/depthmap/benchmark/artifacts/dsp/lib, copying
2021-06-10 16:57:10,295 - INFO - snpebm.snpebm_md5: libsnpe_dsp_v66_domains_v2_skel.so not present on device at
/data/local/tmp/testing/depthmap/benchmark/artifacts/dsp/lib, copying
2021-06-10 16:57:10,998 - INFO - snpebm.snpebm_md5: libsnpe_dsp_v68_domains_v3_skel.so not present on device at
/data/local/tmp/testing/depthmap/benchmark/artifacts/dsp/lib, copying...
2021-06-11 10:08:54,693 - INFO - snpebm.snpebm_md5: md5 command to be used: /usr/bin/md5sum
2021-06-11 10:08:54,693 - INFO - snpebm.snpebm_md5: Perform MD5 check on files on device
2021-06-11 10:08:58,120 - INFO - snpe_bench: Running on CPU_timing
2021-06-11 10:08:58,120 - INFO - snpe_bench: Run 1
2021-06-11 10:09:12,669 - ERROR - snpebm.snpebm_device: ['/data/local/tmp/testing/depthmap/benchmark/artifacts/arm-android-clang6.0/bin/snpe-bench_cmds.sh: line 6: /data/local/tmp/testing/depthmap/benchmark/artifacts/arm-android-clang6.0/bin/snpe-net-run: No such file or directory']
2021-06-11 10:09:12,669 - ERROR - snpebm.snpebm_device: ['/data/local/tmp/testing/depthmap/benchmark/artifacts/arm-android-clang6.0/bin/snpe-bench_cmds.sh: line 6: /data/local/tmp/testing/depthmap/benchmark/artifacts/arm-android-clang6.0/bin/snpe-net-run: No such file or directory']
2021-06-11 10:09:12,669 - WARNING - snpe_bench: Failed to perform benchmark for CPU_timing.
2021-06-11 10:09:12,669 - INFO - snpe_bench: Running on CPU_mem
2021-06-11 10:09:12,669 - INFO - snpe_bench: Run 1
2021-06-11 10:09:12,791 - INFO - snpebm.snpebm_device: starting memory capture in a parallel process
2021-06-11 10:09:12,791 - INFO - snpebm.snpebm_device: Android platform
2021-06-11 10:09:27,122 - ERROR - snpebm.snpebm_device: ['/data/local/tmp/testing/depthmap/benchmark/artifacts/arm-android-clang6.0/bin/snpe-bench_cmds.sh: line 6: /data/local/tmp/testing/depthmap/benchmark/artifacts/arm-android-clang6.0/bin/snpe-net-run: No such file or directory']
2021-06-11 10:09:27,123 - ERROR - snpebm.snpebm_device: ['/data/local/tmp/testing/depthmap/benchmark/artifacts/arm-android-clang6.0/bin/snpe-bench_cmds.sh: line 6: /data/local/tmp/testing/depthmap/benchmark/artifacts/arm-android-clang6.0/bin/snpe-net-run: No such file or directory']
2021-06-11 10:09:27,123 - WARNING - snpe_bench: Failed to perform benchmark for CPU_mem.
2021-06-11 10:09:27,123 - INFO - snpebm.snpebm_device: memory capture is terminated
2021-06-11 10:09:27,135 - INFO - snpe_bench: Running on CPU_ub_float_timing
2021-06-11 10:09:27,135 - INFO - snpe_bench: Run 1
2021-06-11 10:09:41,665 - ERROR - snpebm.snpebm_device: ['/data/local/tmp/testing/depthmap/benchmark/artifacts/arm-android-clang6.0/bin/snpe-bench_cmds.sh: line 6: /data/local/tmp/testing/depthmap/benchmark/artifacts/arm-android-clang6.0/bin/snpe-net-run: No such file or directory']
2021-06-11 10:09:41,665 - ERROR - snpebm.snpebm_device: ['/data/local/tmp/testing/depthmap/benchmark/artifacts/arm-android-clang6.0/bin/snpe-bench_cmds.sh: line 6: /data/local/tmp/testing/depthmap/benchmark/artifacts/arm-android-clang6.0/bin/snpe-net-run: No such file or directory']
2021-06-11 10:09:41,665 - WARNING - snpe_bench: Failed to perform benchmark for CPU_ub_float_timing.
2021-06-11 10:09:41,665 - INFO - snpe_bench: Running on CPU_ub_float_mem
2021-06-11 10:09:41,665 - INFO - snpe_bench: Run 1
2021-06-11 10:09:41,828 - INFO - snpebm.snpebm_device: starting memory capture in a parallel process
2021-06-11 10:09:41,828 - INFO - snpebm.snpebm_device: Android platform
2021-06-11 10:09:56,160 - ERROR - snpebm.snpebm_device: ['/data/local/tmp/testing/depthmap/benchmark/artifacts/arm-android-clang6.0/bin/snpe-bench_cmds.sh: line 6: /data/local/tmp/testing/depthmap/benchmark/artifacts/arm-android-clang6.0/bin/snpe-net-run: No such file or directory']
2021-06-11 10:09:56,160 - ERROR - snpebm.snpebm_device: ['/data/local/tmp/testing/depthmap/benchmark/artifacts/arm-android-clang6.0/bin/snpe-bench_cmds.sh: line 6: /data/local/tmp/testing/depthmap/benchmark/artifacts/arm-android-clang6.0/bin/snpe-net-run: No such file or directory']
2021-06-11 10:09:56,160 - WARNING - snpe_bench: Failed to perform benchmark for CPU_ub_float_mem.
2021-06-11 10:09:56,160 - INFO - snpebm.snpebm_device: memory capture is terminated
2021-06-11 10:09:56,172 - INFO - snpe_bench: Running on DSP_timing
2021-06-11 10:09:56,172 - INFO - snpe_bench: Run 1
2021-06-11 10:10:10,654 - ERROR - snpebm.snpebm_device: ['/data/local/tmp/testing/depthmap/benchmark/artifacts/arm-android-clang6.0/bin/snpe-bench_cmds.sh: line 6: /data/local/tmp/testing/depthmap/benchmark/artifacts/arm-android-clang6.0/bin/snpe-net-run: No such file or directory']
2021-06-11 10:10:10,654 - ERROR - snpebm.snpebm_device: ['/data/local/tmp/testing/depthmap/benchmark/artifacts/arm-android-clang6.0/bin/snpe-bench_cmds.sh: line 6: /data/local/tmp/testing/depthmap/benchmark/artifacts/arm-android-clang6.0/bin/snpe-net-run: No such file or directory']
2021-06-11 10:10:10,654 - WARNING - snpe_bench: Failed to perform benchmark for DSP_timing.
2021-06-11 10:10:10,654 - INFO - snpe_bench: Running on DSP_mem
2021-06-11 10:10:10,654 - INFO - snpe_bench: Run 1
2021-06-11 10:10:10,797 - INFO - snpebm.snpebm_device: starting memory capture in a parallel process
2021-06-11 10:10:10,797 - INFO - snpebm.snpebm_device: Android platform
2021-06-11 10:10:25,133 - ERROR - snpebm.snpebm_device: ['/data/local/tmp/testing/depthmap/benchmark/artifacts/arm-android-clang6.0/bin/snpe-bench_cmds.sh: line 6: /data/local/tmp/testing/depthmap/benchmark/artifacts/arm-android-clang6.0/bin/snpe-net-run: No such file or directory']
2021-06-11 10:10:25,133 - ERROR - snpebm.snpebm_device: ['/data/local/tmp/testing/depthmap/benchmark/artifacts/arm-android-clang6.0/bin/snpe-bench_cmds.sh: line 6: /data/local/tmp/testing/depthmap/benchmark/artifacts/arm-android-clang6.0/bin/snpe-net-run: No such file or directory']
2021-06-11 10:10:25,133 - WARNING - snpe_bench: Failed to perform benchmark for DSP_mem.
2021-06-11 10:10:25,133 - INFO - snpebm.snpebm_device: memory capture is terminated
2021-06-11 10:10:25,145 - INFO - snpe_bench: Running on DSP_ub_float_timing
2021-06-11 10:10:25,145 - INFO - snpe_bench: Run 1
2021-06-11 10:10:39,670 - ERROR - snpebm.snpebm_device: ['/data/local/tmp/testing/depthmap/benchmark/artifacts/arm-android-clang6.0/bin/snpe-bench_cmds.sh: line 6: /data/local/tmp/testing/depthmap/benchmark/artifacts/arm-android-clang6.0/bin/snpe-net-run: No such file or directory']
2021-06-11 10:10:39,670 - ERROR - snpebm.snpebm_device: ['/data/local/tmp/testing/depthmap/benchmark/artifacts/arm-android-clang6.0/bin/snpe-bench_cmds.sh: line 6: /data/local/tmp/testing/depthmap/benchmark/artifacts/arm-android-clang6.0/bin/snpe-net-run: No such file or directory']
2021-06-11 10:10:39,670 - WARNING - snpe_bench: Failed to perform benchmark for DSP_ub_float_timing.
2021-06-11 10:10:39,670 - INFO - snpe_bench: Running on DSP_ub_float_mem
2021-06-11 10:10:39,670 - INFO - snpe_bench: Run 1
2021-06-11 10:10:39,831 - INFO - snpebm.snpebm_device: starting memory capture in a parallel process
2021-06-11 10:10:39,832 - INFO - snpebm.snpebm_device: Android platform
2021-06-11 10:10:54,177 - ERROR - snpebm.snpebm_device: ['/data/local/tmp/testing/depthmap/benchmark/artifacts/arm-android-clang6.0/bin/snpe-bench_cmds.sh: line 6: /data/local/tmp/testing/depthmap/benchmark/artifacts/arm-android-clang6.0/bin/snpe-net-run: No such file or directory']
2021-06-11 10:10:54,178 - ERROR - snpebm.snpebm_device: ['/data/local/tmp/testing/depthmap/benchmark/artifacts/arm-android-clang6.0/bin/snpe-bench_cmds.sh: line 6: /data/local/tmp/testing/depthmap/benchmark/artifacts/arm-android-clang6.0/bin/snpe-net-run: No such file or directory']
2021-06-11 10:10:54,178 - WARNING - snpe_bench: Failed to perform benchmark for DSP_ub_float_mem.
2021-06-11 10:10:54,178 - INFO - snpebm.snpebm_device: memory capture is terminated
2021-06-11 10:10:54,200 - INFO - snpe_bench: Running on DSP_ub_tf8_timing
2021-06-11 10:10:54,200 - INFO - snpe_bench: Run 1
2021-06-11 10:11:08,787 - ERROR - snpebm.snpebm_device: ['/data/local/tmp/testing/depthmap/benchmark/artifacts/arm-android-clang6.0/bin/snpe-bench_cmds.sh: line 6: /data/local/tmp/testing/depthmap/benchmark/artifacts/arm-android-clang6.0/bin/snpe-net-run: No such file or directory']
2021-06-11 10:11:08,787 - ERROR - snpebm.snpebm_device: ['/data/local/tmp/testing/depthmap/benchmark/artifacts/arm-android-clang6.0/bin/snpe-bench_cmds.sh: line 6: /data/local/tmp/testing/depthmap/benchmark/artifacts/arm-android-clang6.0/bin/snpe-net-run: No such file or directory']
2021-06-11 10:11:08,787 - WARNING - snpe_bench: Failed to perform benchmark for DSP_ub_tf8_timing.
2021-06-11 10:11:08,787 - INFO - snpe_bench: Running on DSP_ub_tf8_mem
2021-06-11 10:11:08,787 - INFO - snpe_bench: Run 1
2021-06-11 10:11:08,932 - INFO - snpebm.snpebm_device: starting memory capture in a parallel process
2021-06-11 10:11:08,932 - INFO - snpebm.snpebm_device: Android platform
2021-06-11 10:11:23,268 - ERROR - snpebm.snpebm_device: ['/data/local/tmp/testing/depthmap/benchmark/artifacts/arm-android-clang6.0/bin/snpe-bench_cmds.sh: line 6: /data/local/tmp/testing/depthmap/benchmark/artifacts/arm-android-clang6.0/bin/snpe-net-run: No such file or directory']
2021-06-11 10:11:23,268 - ERROR - snpebm.snpebm_device: ['/data/local/tmp/testing/depthmap/benchmark/artifacts/arm-android-clang6.0/bin/snpe-bench_cmds.sh: line 6: /data/local/tmp/testing/depthmap/benchmark/artifacts/arm-android-clang6.0/bin/snpe-net-run: No such file or directory']
2021-06-11 10:11:23,268 - WARNING - snpe_bench: Failed to perform benchmark for DSP_ub_tf8_mem.
2021-06-11 10:11:23,269 - INFO - snpebm.snpebm_device: memory capture is terminated
2021-06-11 10:11:23,280 - INFO - snpe_bench: Running on AIP_timing
2021-06-11 10:11:23,281 - INFO - snpe_bench: Run 1
2021-06-11 10:11:37,820 - ERROR - snpebm.snpebm_device: ['/data/local/tmp/testing/depthmap/benchmark/artifacts/arm-android-clang6.0/bin/snpe-bench_cmds.sh: line 6: /data/local/tmp/testing/depthmap/benchmark/artifacts/arm-android-clang6.0/bin/snpe-net-run: No such file or directory']
2021-06-11 10:11:37,820 - ERROR - snpebm.snpebm_device: ['/data/local/tmp/testing/depthmap/benchmark/artifacts/arm-android-clang6.0/bin/snpe-bench_cmds.sh: line 6: /data/local/tmp/testing/depthmap/benchmark/artifacts/arm-android-clang6.0/bin/snpe-net-run: No such file or directory']
2021-06-11 10:11:37,820 - WARNING - snpe_bench: Failed to perform benchmark for AIP_timing.
2021-06-11 10:11:37,820 - INFO - snpe_bench: Running on AIP_mem
2021-06-11 10:11:37,820 - INFO - snpe_bench: Run 1
2021-06-11 10:11:37,982 - INFO - snpebm.snpebm_device: starting memory capture in a parallel process
2021-06-11 10:11:37,982 - INFO - snpebm.snpebm_device: Android platform
2021-06-11 10:11:52,307 - ERROR - snpebm.snpebm_device: ['/data/local/tmp/testing/depthmap/benchmark/artifacts/arm-android-clang6.0/bin/snpe-bench_cmds.sh: line 6: /data/local/tmp/testing/depthmap/benchmark/artifacts/arm-android-clang6.0/bin/snpe-net-run: No such file or directory']
2021-06-11 10:11:52,307 - ERROR - snpebm.snpebm_device: ['/data/local/tmp/testing/depthmap/benchmark/artifacts/arm-android-clang6.0/bin/snpe-bench_cmds.sh: line 6: /data/local/tmp/testing/depthmap/benchmark/artifacts/arm-android-clang6.0/bin/snpe-net-run: No such file or directory']
2021-06-11 10:11:52,307 - WARNING - snpe_bench: Failed to perform benchmark for AIP_mem.
2021-06-11 10:11:52,307 - INFO - snpebm.snpebm_device: memory capture is terminated
2021-06-11 10:11:52,318 - INFO - snpe_bench: Running on AIP_ub_float_timing
2021-06-11 10:11:52,318 - INFO - snpe_bench: Run 1
2021-06-11 10:12:06,807 - ERROR - snpebm.snpebm_device: ['/data/local/tmp/testing/depthmap/benchmark/artifacts/arm-android-clang6.0/bin/snpe-bench_cmds.sh: line 6: /data/local/tmp/testing/depthmap/benchmark/artifacts/arm-android-clang6.0/bin/snpe-net-run: No such file or directory']
2021-06-11 10:12:06,807 - ERROR - snpebm.snpebm_device: ['/data/local/tmp/testing/depthmap/benchmark/artifacts/arm-android-clang6.0/bin/snpe-bench_cmds.sh: line 6: /data/local/tmp/testing/depthmap/benchmark/artifacts/arm-android-clang6.0/bin/snpe-net-run: No such file or directory']
2021-06-11 10:12:06,807 - WARNING - snpe_bench: Failed to perform benchmark for AIP_ub_float_timing.
2021-06-11 10:12:06,807 - INFO - snpe_bench: Running on AIP_ub_float_mem
2021-06-11 10:12:06,807 - INFO - snpe_bench: Run 1
2021-06-11 10:12:06,949 - INFO - snpebm.snpebm_device: starting memory capture in a parallel process
2021-06-11 10:12:06,949 - INFO - snpebm.snpebm_device: Android platform
2021-06-11 10:12:21,286 - ERROR - snpebm.snpebm_device: ['/data/local/tmp/testing/depthmap/benchmark/artifacts/arm-android-clang6.0/bin/snpe-bench_cmds.sh: line 6: /data/local/tmp/testing/depthmap/benchmark/artifacts/arm-android-clang6.0/bin/snpe-net-run: No such file or directory']
2021-06-11 10:12:21,287 - ERROR - snpebm.snpebm_device: ['/data/local/tmp/testing/depthmap/benchmark/artifacts/arm-android-clang6.0/bin/snpe-bench_cmds.sh: line 6: /data/local/tmp/testing/depthmap/benchmark/artifacts/arm-android-clang6.0/bin/snpe-net-run: No such file or directory']
2021-06-11 10:12:21,287 - WARNING - snpe_bench: Failed to perform benchmark for AIP_ub_float_mem.
2021-06-11 10:12:21,287 - INFO - snpebm.snpebm_device: memory capture is terminated
2021-06-11 10:12:21,300 - INFO - snpe_bench: Running on AIP_ub_tf8_timing
2021-06-11 10:12:21,300 - INFO - snpe_bench: Run 1
2021-06-11 10:12:35,846 - ERROR - snpebm.snpebm_device: ['/data/local/tmp/testing/depthmap/benchmark/artifacts/arm-android-clang6.0/bin/snpe-bench_cmds.sh: line 6: /data/local/tmp/testing/depthmap/benchmark/artifacts/arm-android-clang6.0/bin/snpe-net-run: No such file or directory']
2021-06-11 10:12:35,846 - ERROR - snpebm.snpebm_device: ['/data/local/tmp/testing/depthmap/benchmark/artifacts/arm-android-clang6.0/bin/snpe-bench_cmds.sh: line 6: /data/local/tmp/testing/depthmap/benchmark/artifacts/arm-android-clang6.0/bin/snpe-net-run: No such file or directory']
2021-06-11 10:12:35,846 - WARNING - snpe_bench: Failed to perform benchmark for AIP_ub_tf8_timing.
2021-06-11 10:12:35,847 - INFO - snpe_bench: Running on AIP_ub_tf8_mem
2021-06-11 10:12:35,847 - INFO - snpe_bench: Run 1
2021-06-11 10:12:36,019 - INFO - snpebm.snpebm_device: starting memory capture in a parallel process
2021-06-11 10:12:36,019 - INFO - snpebm.snpebm_device: Android platform
2021-06-11 10:12:50,344 - ERROR - snpebm.snpebm_device: ['/data/local/tmp/testing/depthmap/benchmark/artifacts/arm-android-clang6.0/bin/snpe-bench_cmds.sh: line 6: /data/local/tmp/testing/depthmap/benchmark/artifacts/arm-android-clang6.0/bin/snpe-net-run: No such file or directory']
2021-06-11 10:12:50,345 - ERROR - snpebm.snpebm_device: ['/data/local/tmp/testing/depthmap/benchmark/artifacts/arm-android-clang6.0/bin/snpe-bench_cmds.sh: line 6: /data/local/tmp/testing/depthmap/benchmark/artifacts/arm-android-clang6.0/bin/snpe-net-run: No such file or directory']
2021-06-11 10:12:50,345 - WARNING - snpe_bench: Failed to perform benchmark for AIP_ub_tf8_mem.
2021-06-11 10:12:50,345 - INFO - snpebm.snpebm_device: memory capture is terminated
2021-06-11 10:12:50,356 - ERROR - snpe_bench: None of the selected benchmarks ran, therefore no results reported
How would I be able to work around the above?
Thanks.
Need to use:
So append -t le64_gcc4.9 to bench.py command to use with RB5 as follows:
python3 snpe_bench.py -c config.json -t le64_gcc4.9