Hi,
It is stated in the Neural SDK that `depth_to_space` (or `PixelShuffle` in PyTorch) (https://developer.qualcomm.com/sites/default/files/docs/snpe/network_lay...) is supported on the DSP/AIP. This seems incorrect based on a simple test case.
Dependencies:
- snpe-1.66.0.3729
- torch==1.11
Complete Example Code:
import torch
class PixelShuffleModule(torch.nn.Module):
def __init__(self):
super().__init__()
self.ps = torch.nn.PixelShuffle(upscale_factor=2**3)
def forward(self, x):
return self.ps(x)
channels = 128
input_shape = [1,channels,64,64]
psModule = PixelShuffleModule()
input = torch.randn(input_shape)
def trace_and_save(module, inputs, filename):
torchscript_model = torch.jit.trace(module, inputs)
torchscript_model.save(filename)
trace_and_save(psModule, [input], 'ps.pt')
Using `snep-pytorch-to-dlc`:
> snpe-pytorch-to-dlc -i ps.pt --input_dim 'input_0' 1,128,64,64 --debug
When this executes, PixelShuffle is reformulated as Reshape_0, Permute_0, Reshape_1, Permute_1. This is an issue, since Reshape_0 is a 6D tensor, which the DSP/AIP does not support!
"Layer is not supported in AIP. Layer reshape_0: Maximum number of output dimensions(4) exceeded. 6 output dimension were given for AIP runtime"
Is the documentation incorrect, or is my conversion incorrect? Can someone please help to clarify
Full trace:
2022-10-31 13:49:13,235 - 2972 - WARNING - Untyped Tensor found, assume it is float32
2022-10-31 13:49:13,244 - 190 - DEBUG_2 -
2022-10-31 13:49:13,244 - 190 - DEBUG_2 - ##### NEW OP Translation #####
2022-10-31 13:49:13,245 - 183 - DEBUG_1 - Added buffer named input_0 of shape [1, 128, 64, 64]
2022-10-31 13:49:13,245 - 190 - DEBUG_2 -
2022-10-31 13:49:13,245 - 190 - DEBUG_2 -
2022-10-31 13:49:13,245 - 190 - DEBUG_2 - ##### NEW OP Translation #####
2022-10-31 13:49:13,245 - 190 - DEBUG_2 -
2022-10-31 13:49:13,245 - 190 - DEBUG_2 -
2022-10-31 13:49:13,245 - 190 - DEBUG_2 - ##### NEW OP Translation #####
2022-10-31 13:49:13,245 - 190 - DEBUG_2 -
2022-10-31 13:49:13,245 - 190 - DEBUG_2 -
2022-10-31 13:49:13,245 - 190 - DEBUG_2 - ##### NEW OP Translation #####
2022-10-31 13:49:13,245 - 183 - DEBUG_1 -
2022-10-31 13:49:13,246 - 183 - DEBUG_1 - Relay Op name reshape
2022-10-31 13:49:13,246 - 190 - DEBUG_2 - op_type in get_translation reshape
2022-10-31 13:49:13,246 - 197 - DEBUG_3 - Reshape Op attribute new shape [1, 2, 8, 8, 64, 64]
2022-10-31 13:49:13,246 - 197 - DEBUG_3 - Reshape Op input shape [1, 128, 64, 64]
2022-10-31 13:49:13,246 - 183 - DEBUG_1 - Op reshape_0 Type Reshape inputs ['input_0']
2022-10-31 13:49:13,246 - 183 - DEBUG_1 - Op reshape_0 Type Reshape outputs ['reshape_0_0']
2022-10-31 13:49:13,247 - 183 - DEBUG_1 -
2022-10-31 13:49:13,247 - 183 - DEBUG_1 - Added OpNode with name reshape_0, in_names ['input_0'], out_names ['reshape_0_0']
2022-10-31 13:49:13,247 - 183 - DEBUG_1 - Added buffer named reshape_0_0 of shape [1, 2, 8, 8, 64, 64]
2022-10-31 13:49:13,247 - 190 - DEBUG_2 -
2022-10-31 13:49:13,247 - 190 - DEBUG_2 -
2022-10-31 13:49:13,247 - 190 - DEBUG_2 - ##### NEW OP Translation #####
2022-10-31 13:49:13,247 - 183 - DEBUG_1 -
2022-10-31 13:49:13,247 - 183 - DEBUG_1 - Relay Op name transpose
2022-10-31 13:49:13,248 - 190 - DEBUG_2 - op_type in get_translation transpose
2022-10-31 13:49:13,248 - 197 - DEBUG_3 - axes [0, 1, 4, 2, 5, 3]
2022-10-31 13:49:13,248 - 183 - DEBUG_1 - Op permute_0 Type Transpose inputs ['reshape_0_0']
2022-10-31 13:49:13,248 - 183 - DEBUG_1 - Op permute_0 Type Transpose outputs ['permute_0_0']
2022-10-31 13:49:13,248 - 183 - DEBUG_1 -
2022-10-31 13:49:13,248 - 183 - DEBUG_1 - Added OpNode with name permute_0, in_names ['reshape_0_0'], out_names ['permute_0_0']
2022-10-31 13:49:13,249 - 183 - DEBUG_1 - Added buffer named permute_0_0 of shape [1, 2, 64, 8, 64, 8]
2022-10-31 13:49:13,249 - 190 - DEBUG_2 -
2022-10-31 13:49:13,249 - 190 - DEBUG_2 -
2022-10-31 13:49:13,249 - 190 - DEBUG_2 - ##### NEW OP Translation #####
2022-10-31 13:49:13,249 - 183 - DEBUG_1 -
2022-10-31 13:49:13,249 - 183 - DEBUG_1 - Relay Op name reshape
2022-10-31 13:49:13,249 - 190 - DEBUG_2 - op_type in get_translation reshape
2022-10-31 13:49:13,250 - 197 - DEBUG_3 - Reshape Op attribute new shape [1, 2, 512, 512]
2022-10-31 13:49:13,250 - 197 - DEBUG_3 - Reshape Op input shape [1, 2, 64, 8, 64, 8]
2022-10-31 13:49:13,250 - 183 - DEBUG_1 - Op reshape_1 Type Reshape inputs ['permute_0_0']
2022-10-31 13:49:13,250 - 183 - DEBUG_1 - Op reshape_1 Type Reshape outputs ['reshape_1_0']
2022-10-31 13:49:13,250 - 183 - DEBUG_1 -
2022-10-31 13:49:13,250 - 183 - DEBUG_1 - Added OpNode with name reshape_1, in_names ['permute_0_0'], out_names ['reshape_1_0']
2022-10-31 13:49:13,250 - 183 - DEBUG_1 - Added buffer named reshape_1_0 of shape [1, 2, 512, 512]
2022-10-31 13:49:13,250 - 190 - DEBUG_2 -
2022-10-31 13:49:13,250 - 190 - DEBUG_2 -
2022-10-31 13:49:13,251 - 190 - DEBUG_2 - ##### NEW OP Translation #####
2022-10-31 13:49:13,251 - 190 - DEBUG_2 -
2022-10-31 13:49:13,251 - 190 - DEBUG_2 - Applying graph Optimizations...
2022-10-31 13:49:13,251 - 190 - DEBUG_2 - Evaluating to match Sequence ['constant', 'elementwise_product', 'constant', 'elementwise_sum']...
2022-10-31 13:49:13,251 - 190 - DEBUG_2 - Found 0 match(es)
2022-10-31 13:49:13,251 - 190 - DEBUG_2 - Evaluating to match Sequence ['reduce_l2', 'elementwise_sum', 'expand', 'elementwise_div']...
2022-10-31 13:49:13,252 - 190 - DEBUG_2 - Found 0 match(es)
2022-10-31 13:49:13,252 - 190 - DEBUG_2 - Evaluating to match Sequence ['reduce_l2', 'elementwise_sum', 'elementwise_div']...
2022-10-31 13:49:13,252 - 190 - DEBUG_2 - Found 0 match(es)
2022-10-31 13:49:13,252 - 190 - DEBUG_2 - Evaluating to match Sequence ['reduce_l2', 'elementwise_div']...
2022-10-31 13:49:13,252 - 190 - DEBUG_2 - Found 0 match(es)
2022-10-31 13:49:13,252 - 190 - DEBUG_2 - Evaluating to match Sequence ['matmul', 'elementwise_sum']...
2022-10-31 13:49:13,253 - 190 - DEBUG_2 - Found 0 match(es)
2022-10-31 13:49:13,253 - 190 - DEBUG_2 - Evaluating to match Sequence ['elementwise_min', 'elementwise_product', 'elementwise_max', 'elementwise_sum']...
2022-10-31 13:49:13,253 - 190 - DEBUG_2 - Found 0 match(es)
2022-10-31 13:49:13,253 - 190 - DEBUG_2 - Evaluating to match Sequence ['elementwise_max', 'elementwise_min', 'elementwise_product', 'elementwise_sum']...
2022-10-31 13:49:13,253 - 190 - DEBUG_2 - Found 0 match(es)
2022-10-31 13:49:13,253 - 190 - DEBUG_2 - Evaluating to match Sequence ['fake_non_max_suppression', 'constant', 'Gather', 'Reshape']...
2022-10-31 13:49:13,254 - 190 - DEBUG_2 - Found 0 match(es)
2022-10-31 13:49:13,254 - 190 - DEBUG_2 - Evaluating to match Sequence ['fake_non_max_suppression', 'StridedSlice']...
2022-10-31 13:49:13,254 - 190 - DEBUG_2 - Found 0 match(es)
2022-10-31 13:49:13,254 - 190 - DEBUG_2 - Evaluating to match Sequence ['Pad']...
2022-10-31 13:49:13,254 - 190 - DEBUG_2 - Found 0 match(es)
2022-10-31 13:49:13,254 - 190 - DEBUG_2 - Evaluating to match Sequence ['Concat']...
2022-10-31 13:49:13,255 - 190 - DEBUG_2 - Found 0 match(es)
2022-10-31 13:49:13,255 - 190 - DEBUG_2 - Evaluating to match Sequence ['Concat']...
2022-10-31 13:49:13,255 - 190 - DEBUG_2 - Found 0 match(es)
2022-10-31 13:49:13,255 - 190 - DEBUG_2 - Evaluating to match Sequence ['Reshape']...
2022-10-31 13:49:13,255 - 190 - DEBUG_2 - Sequence pattern ['Reshape'] matched, but not input/output buffers for node reshape_0 of type Reshape in sequence.
2022-10-31 13:49:13,256 - 190 - DEBUG_2 - Sequence pattern ['Reshape'] matched, but not input/output buffers for node reshape_1 of type Reshape in sequence.
2022-10-31 13:49:13,256 - 190 - DEBUG_2 - Found 0 match(es)
2022-10-31 13:49:13,256 - 190 - DEBUG_2 - Evaluating to match Sequence ['Reshape', 'Transpose', 'Reshape']...
2022-10-31 13:49:13,256 - 190 - DEBUG_2 - Found 0 match(es)
2022-10-31 13:49:13,256 - 190 - DEBUG_2 - Evaluating to match Sequence ['elementwise_div', 'erf', 'elementwise_sum', 'elementwise_product', 'elementwise_product']...
2022-10-31 13:49:13,257 - 190 - DEBUG_2 - Found 0 match(es)
2022-10-31 13:49:13,257 - 190 - DEBUG_2 - Evaluating to match Sequence ['elementwise_product', 'elementwise_div', 'erf', 'elementwise_sum', 'elementwise_product']...
2022-10-31 13:49:13,257 - 190 - DEBUG_2 - Found 0 match(es)
2022-10-31 13:49:13,257 - 190 - DEBUG_2 - Evaluating to match Sequence ['elementwise_sum', 'Neuron', 'elementwise_product', 'elementwise_product']...
2022-10-31 13:49:13,257 - 190 - DEBUG_2 - Found 0 match(es)
2022-10-31 13:49:13,257 - 190 - DEBUG_2 - Evaluating to match Sequence ['elementwise_sum', 'Neuron', 'elementwise_product', 'elementwise_product']...
2022-10-31 13:49:13,258 - 190 - DEBUG_2 - Found 0 match(es)
2022-10-31 13:49:13,258 - 190 - DEBUG_2 - Evaluating to match Sequence ['elementwise_sum', 'Neuron', 'elementwise_product', 'elementwise_div']...
2022-10-31 13:49:13,258 - 190 - DEBUG_2 - Found 0 match(es)
2022-10-31 13:49:13,258 - 190 - DEBUG_2 - Evaluating to match Sequence ['elementwise_sum', 'Neuron', 'elementwise_div', 'elementwise_product']...
2022-10-31 13:49:13,258 - 190 - DEBUG_2 - Found 0 match(es)
2022-10-31 13:49:13,258 - 190 - DEBUG_2 - Evaluating to match Sequence ['reduce_mean', 'elementwise_sub', 'elementwise_power', 'reduce_mean', 'elementwise_sum', 'elementwise_unary_sqrt', 'elementwise_div', 'elementwise_product', 'elementwise_sum']...
2022-10-31 13:49:13,259 - 190 - DEBUG_2 - Found 0 match(es)
2022-10-31 13:49:13,259 - 190 - DEBUG_2 - Evaluating to match Sequence ['Reshape', 'reduce_mean', 'Reshape', 'elementwise_sub', 'elementwise_power', 'Reshape', 'reduce_mean', 'Reshape', 'elementwise_sum', 'elementwise_unary_sqrt', 'elementwise_div', 'elementwise_product', 'elementwise_sum']...
2022-10-31 13:49:13,259 - 190 - DEBUG_2 - Found 0 match(es)
2022-10-31 13:49:13,259 - 190 - DEBUG_2 - Evaluating to match Sequence ['Reshape', 'Transpose', 'Reshape', 'Transpose', 'Reshape', 'Transpose', 'Reshape']...
2022-10-31 13:49:13,259 - 190 - DEBUG_2 - Found 0 match(es)
2022-10-31 13:49:13,259 - 190 - DEBUG_2 - Evaluating to match Sequence ['scale']...
2022-10-31 13:49:13,260 - 190 - DEBUG_2 - Found 0 match(es)
2022-10-31 13:49:13,260 - 190 - DEBUG_2 - Evaluating to match Sequence ['scale']...
2022-10-31 13:49:13,260 - 190 - DEBUG_2 - Found 0 match(es)
2022-10-31 13:49:13,260 - 190 - DEBUG_2 - Evaluating to match Sequence ['elementwise_product']...
2022-10-31 13:49:13,260 - 190 - DEBUG_2 - Found 0 match(es)
2022-10-31 13:49:13,260 - 190 - DEBUG_2 - Evaluating to match Sequence ['Conv2d']...
2022-10-31 13:49:13,261 - 190 - DEBUG_2 - Found 0 match(es)
2022-10-31 13:49:13,261 - 190 - DEBUG_2 - Evaluating to match Sequence ['DepthWiseConv2d']...
2022-10-31 13:49:13,261 - 190 - DEBUG_2 - Found 0 match(es)
2022-10-31 13:49:13,261 - 190 - DEBUG_2 - Evaluating to match Sequence ['TransposeConv2d']...
2022-10-31 13:49:13,261 - 190 - DEBUG_2 - Found 0 match(es)
2022-10-31 13:49:13,261 - 190 - DEBUG_2 - Evaluating to match Sequence ['elementwise_div']...
2022-10-31 13:49:13,261 - 190 - DEBUG_2 - Found 0 match(es)
2022-10-31 13:49:13,261 - 190 - DEBUG_2 - Evaluating to match Sequence ['Conv2d']...
2022-10-31 13:49:13,262 - 190 - DEBUG_2 - Found 0 match(es)
2022-10-31 13:49:13,262 - 190 - DEBUG_2 - Evaluating to match Sequence ['DepthWiseConv2d']...
2022-10-31 13:49:13,262 - 190 - DEBUG_2 - Found 0 match(es)
2022-10-31 13:49:13,262 - 190 - DEBUG_2 - Evaluating to match Sequence ['TransposeConv2d']...
2022-10-31 13:49:13,262 - 190 - DEBUG_2 - Found 0 match(es)
2022-10-31 13:49:13,262 - 190 - DEBUG_2 - Evaluating to match Sequence ['elementwise_sum']...
2022-10-31 13:49:13,262 - 190 - DEBUG_2 - Found 0 match(es)
2022-10-31 13:49:13,263 - 190 - DEBUG_2 - Evaluating to match Sequence ['Conv2d']...
2022-10-31 13:49:13,263 - 190 - DEBUG_2 - Found 0 match(es)
2022-10-31 13:49:13,263 - 190 - DEBUG_2 - Evaluating to match Sequence ['DepthWiseConv2d']...
2022-10-31 13:49:13,263 - 190 - DEBUG_2 - Found 0 match(es)
2022-10-31 13:49:13,263 - 190 - DEBUG_2 - Evaluating to match Sequence ['TransposeConv2d']...
2022-10-31 13:49:13,263 - 190 - DEBUG_2 - Found 0 match(es)
2022-10-31 13:49:13,263 - 190 - DEBUG_2 - Evaluating to match Sequence ['elementwise_sub']...
2022-10-31 13:49:13,263 - 190 - DEBUG_2 - Found 0 match(es)
2022-10-31 13:49:13,264 - 190 - DEBUG_2 - Evaluating to match Sequence ['Conv2d']...
2022-10-31 13:49:13,264 - 190 - DEBUG_2 - Found 0 match(es)
2022-10-31 13:49:13,264 - 190 - DEBUG_2 - Evaluating to match Sequence ['DepthWiseConv2d']...
2022-10-31 13:49:13,264 - 190 - DEBUG_2 - Found 0 match(es)
2022-10-31 13:49:13,264 - 190 - DEBUG_2 - Evaluating to match Sequence ['TransposeConv2d']...
2022-10-31 13:49:13,264 - 190 - DEBUG_2 - Found 0 match(es)
2022-10-31 13:49:13,264 - 190 - DEBUG_2 - Evaluating to match Sequence ['Conv2d']...
2022-10-31 13:49:13,265 - 190 - DEBUG_2 - Found 0 match(es)
2022-10-31 13:49:13,265 - 190 - DEBUG_2 - Evaluating to match Sequence ['DepthWiseConv2d']...
2022-10-31 13:49:13,265 - 190 - DEBUG_2 - Found 0 match(es)
2022-10-31 13:49:13,265 - 190 - DEBUG_2 - Evaluating to match Sequence ['fully_connected']...
2022-10-31 13:49:13,265 - 190 - DEBUG_2 - Found 0 match(es)
2022-10-31 13:49:13,265 - 190 - DEBUG_2 - Evaluating to match Sequence ['Reshape']...
2022-10-31 13:49:13,266 - 190 - DEBUG_2 - Sequence pattern ['Reshape'] matched, but not input/output buffers for node reshape_0 of type Reshape in sequence.
2022-10-31 13:49:13,266 - 190 - DEBUG_2 - Sequence pattern ['Reshape'] matched, but not input/output buffers for node reshape_1 of type Reshape in sequence.
2022-10-31 13:49:13,266 - 190 - DEBUG_2 - Found 0 match(es)
2022-10-31 13:49:13,266 - 190 - DEBUG_2 - Evaluating to match Sequence ['Transpose', 'Transpose']...
2022-10-31 13:49:13,267 - 190 - DEBUG_2 - Found 0 match(es)
2022-10-31 13:49:13,267 - 190 - DEBUG_2 - Evaluating to match Sequence ['elementwise_product']...
2022-10-31 13:49:13,267 - 190 - DEBUG_2 - Found 0 match(es)
2022-10-31 13:49:13,267 - 190 - DEBUG_2 - Evaluating to match Sequence ['elementwise_sum']...
2022-10-31 13:49:13,267 - 190 - DEBUG_2 - Found 0 match(es)
2022-10-31 13:49:13,268 - 197 - DEBUG_3 - no validation target specified. Using defaults.
2022-10-31 13:49:13,268 - 214 - INFO - INFO_DLC_SAVE_LOCATION: Saving model at /ps.dlc
/snpe-1.66.0.3729/lib/python/qti/aisw/converters/backend/ir_to_dlc.py:1260: RuntimeWarning: error_code=1400; error_message=Layer is not supported in AIP. Layer reshape_0: Maximum number of output dimensions(4) exceeded. 6 output dimension were given for AIP runtime.; error_component=AIP Runtime; line_no=140; thread_id=139925898090304
node.output_names[0])
/snpe-1.66.0.3729/lib/python/qti/aisw/converters/backend/ir_to_dlc.py:1049: RuntimeWarning: info_code=802; message=Layer parameter value is invalid in GPU. Layer permute_0: input tensor must be 5D, 4D, 3D or 2D.; component=GPU Runtime; line_no=499; thread_id=139925898090304
node.output_names[0])
/snpe-1.66.0.3729/lib/python/qti/aisw/converters/backend/ir_to_dlc.py:1260: RuntimeWarning: info_code=802; message=Layer parameter value is invalid in GPU. Layer reshape_1 : output width = 512, depth = 512 width * depth (packed) = 65536 exceeds maximum image width 16384 for Adreno A650; component=GPU Runtime; line_no=1113; thread_id=139925898090304
node.output_names[0])
2022-10-31 13:49:13,269 - 214 - INFO - INFO_CONVERSION_SUCCESS: Conversion completed successfully
I meet a same problem!
same issues!!!
you can use snpe1.10 to solve the pixelshuffle problem.
but when a conv() fallow the pixelshuffle(), it does not work, now i use nspe1.12 can run.