Hi,
I am working with a model for which we want to benchmark (snpe_bench.py) 2 distinct setups: 1) AIP with 8 bit quantisation and 2) AIP with A16W8.
Unfortunately our model has a couple of layers that cannot run in HTA, thus we use enable_hta and hta_partitions to remove these layers. The quantize step is as follows:
1) AIP with 8b
This approach works fine for the AIP runtime.
2) AIP with A16W8
Unfortunately it fails when in A16W8 mode. Furthermore there seems to be no CPU fallback, as both these models run fine on CPU.
So down to my questions:
Q1) Is there any way to tag certain layers to execute on a specific target? I.e., the HTA incompatible layers i could point to use CPU.
Q2) Is there any way i can edit the aip.metadata files in the .dlc? Is there a format guide somewhere?
Thanks for your time!
Below is the error we get for the AIP_timing:
2021-03-04 20:07:49,287 - INFO - snpe_bench: Running on AIP_timing2021-03-04 20:07:49,287 - INFO - snpe_bench: Run 12021-03-04 20:08:07,713 - ERROR - snpebm.snpebm_device: ['Translating kernel ioctl error 25 to npu sts: -1000', 'Translating kernel ioctl error 25 to npu sts: -1000', 'get_resp_queue_size(): warning kernel does not support this property', 'Translating kernel ioctl error 25 to npu sts: -1000', 'Translating kernel ioctl error 25 to npu sts: -1000', 'NPU User Driver: npu_read_info 0', 'Translating kernel ioctl error 25 to npu sts: -1000', 'Translating kernel ioctl error 25 to npu sts: -1000', 'get_resp_queue_size(): warning kernel does not support this property', 'Translating kernel ioctl error 25 to npu sts: -1000', 'Translating kernel ioctl error 25 to npu sts: -1000', 'Translating kernel ioctl error 25 to npu sts: -1000', 'Translating kernel ioctl error 25 to npu sts: -1000', 'get_resp_queue_size(): warning kernel does not support this property', 'Translating kernel ioctl error 25 to npu sts: -1000', 'Translating kernel ioctl error 25 to npu sts: -1000', 'Translating kernel ioctl error 25 to npu sts: -1000', 'Translating kernel ioctl error 25 to npu sts: -1000', 'Translating kernel ioctl error 25 to npu sts: -1000', 'Translating kernel ioctl error 25 to npu sts: -1000', 'NPU driver built on: Nov 12 2020 15:26:59', 'Translating kernel ioctl error 25 to npu sts: -1000', 'Translating kernel ioctl error 25 to npu sts: -1000', 'get_resp_queue_size(): warning kernel does not support this property', 'Translating kernel ioctl error 25 to npu sts: -1000', 'Translating kernel ioctl error 25 to npu sts: -1000', 'Metadata version: 2.8.0', 'NNC Revision: 786dd2a', 'NNC Branch: npu-nnc', 'NNC Timestamp: Fri Aug 16 14:37:15 2019', 'NNC Built On: 2019-08-16T15:28:15', 'ParseNpuMetaBufferDescriptor layer: [0] -> LAYER_TYPE_PARAM_IN_BUF[0] - [0x00020000 - 86400]', 'ParseNpuMetaBufferDescriptor layer: [14] -> LAYER_TYPE_PARAM_OUT_BUF[0] - [0x001c1000 - 921600]', 'npu_load_network(): perf mode = 5 flags = 0x200000', '* NPU_Stats: npu_load_network(): NPU + kernel : 162.37 ms', 'npu_load_network(): network handle = 0x10001', '* NPU_Stats: npu_load_network(): 170.06 ms', '* NPU_Stats: npu_alloc_buffer_v2(): 0.40 ms sts=0', '* NPU_Stats: npu_unload_network(): NPU + kernel : 155.55 ms', '* NPU_Stats: npu_unload_network(): 180.45 ms', 'error_code=1410; error_message=AIP runtime system error. error_code=1410; error_message=AIP runtime system error. Unsupported quantize type NPU_QUANTIZE_TYPE_NONE. Error code: -1005; error_component=AIP Runtime; line_no=1098; thread_id=-348186256; error_component=AIP Runtime; line_no=263; thread_id=-310352748']