Forums - Define my interp layer according to UDL Tutorial

Hi, My caffe model has a Interpolation Layer which is not supported by SNPE. So I followed the UDL Tutorial. When I use  <code>snpe-dlc-info -i myinterp.net</code>  to view the model details, Part of the result is as follow:

| 153 | psp.pool4.interp         | user_defined    | psp.pool4.conv/relu.psp.pool4.conv           | psp.pool4.interp                             | 1x12x20x24   | blob_size: 12                    |

and the Last line of result is :

Total parameters: 95497 (<strong>0 MB assuming single precision float</strong>)
Total MACs per inference: 71M (100%)
Converter command: snpe-caffe-to-dlc-udl verbose=False encoding=bgr enable_strict_validation=False disable_batchnorm_folding=False input_types=None model_version=None validation_target=[] enable_preprocessing=True input_size=None copyright_file=None input_layers=None
DLC created with converter version: 1.23.1.245

Note that my interp's blob size is only 12, which I think is not right obviously. Also it said 0 MB assuimg single precision float, which is also wrong. I think there is something wrong with MyUdlLayers.py and snpe-caffe-to-dlc-udl. I changed the 2 files and the main part of files are as following:

MyUdlLayers.py:

<code>import struct

class LayerType:
    MY_CUSTOM_SCALE_LAYER = 1
    MY_INTERP_LAYER = 2
    MY_ANOTHER_LAYER = 3


class MyCustomScaleLayerParam:
    def __init__(self):
        self.type = LayerType.MY_CUSTOM_SCALE_LAYER
        self.bias_term = None
        self.weights_dim = []
        self.weights_data = []

    def Serialize(self):
        packed = struct.pack('i', self.type)
        packed += struct.pack('?', self.bias_term)
        packed += struct.pack('I%sI' % len(self.weights_dim),
                              len(self.weights_dim), *self.weights_dim)
        packed += struct.pack('I%sf' % len(self.weights_data),
                              len(self.weights_data), *self.weights_data)
        return packed


# interp layer shape, these params should be changed to fit the target model
INTERP_HEIGHT=12
INTERP_WIDTH=20
class MyInterpLayerParam:
    def __init__(self):
        self.type = LayerType.MY_INTERP_LAYER
        self.height = INTERP_HEIGHT
        self.width = INTERP_WIDTH

    def Serialize(self):
        packed = struct.pack('i', self.type)
        packed += struct.pack('i', self.height)
        packed += struct.pack('i', self.width)
        return packed</code>

snpe-caffe-to-dlc-udl:

<code>################################ interp layer
# interp layer shape, these params should be changed to fit the target model
INTERP_OUT_N = 1   # interp layer out shape batch
INTERP_OUT_H = 12  # interp layer out shape height
INTERP_OUT_W = 20  # interp layer out shape width
INTERP_OUT_C = 24  # interp layer out shape channel

class UdlBlobMyInterp(object):
    """
    Wrapper class for MyInterp layer blob
    """
    def __init__(self, layer):
        # MyInterp layer reuses the Caffe Interp layer params
        caffe_params = layer.interp_param

        # Initialize the SNPE params
        print 'before MyUdlLayers.MyInterpLayerParam()'
        snpe_params = MyUdlLayers.MyInterpLayerParam()
        print 'After MyUdlLayers.MyInterpLayerParam()'

        # fill the params
        print 'caffe_params.height: ', caffe_params.height
        print 'caffe_params.width: ', caffe_params.width
        snpe_params.height = caffe_params.height
        snpe_params.width = caffe_params.width

        self._blob = snpe_params.Serialize()
        self._size = len(self._blob)

    def getBlob(self):
        return self._blob

    def getSize(self):
        return self._size

def udl_myinterp_func(layer, input_dims):
    """
    Conversion callback function for MyInterp layer
    """
    print 'layer.name:', layer.name
    print 'input_dims:', input_dims # tensor shape: batch X height X width X channel
    # Initialize blob for our custom layer with the wrapper class
    blob = UdlBlobMyInterp(layer)
    print 'Init UdlBlobMyInterp done!'

    # output dims for MyInterp layer
    #output_dims = [[1, 12, 20, 48]] # tensor shape: batch X height X width X channel
    output_dims = [[INTERP_OUT_N, INTERP_OUT_H, INTERP_OUT_W, INTERP_OUT_C]] # tensor shape: batch X height X width X channel
    return snpe_udl_utils.UdlBlobOutput(blob=blob, out_dims=output_dims)

# Instance of Udl class for myinterp layer
udl_myinterp = snpe_udl_utils.Udl(udl_myinterp_func)

# Add SNPE udl's expected input axis order for 4D input and its output axis order
udl_myinterp.addAxisOrder([AxisAnnotation.BATCH,AxisAnnotation.HEIGHT,AxisAnnotation.WIDTH,AxisAnnotation.CHANNEL],
                              [AxisAnnotation.BATCH,AxisAnnotation.HEIGHT,AxisAnnotation.WIDTH,AxisAnnotation.CHANNEL])
# Add SNPE udl's expected input axis order for 3D input and its output axis order
udl_myinterp.addAxisOrder([AxisAnnotation.BATCH,AxisAnnotation.HEIGHT,AxisAnnotation.WIDTH],
                              [AxisAnnotation.BATCH,AxisAnnotation.HEIGHT,AxisAnnotation.WIDTH])
# Add SNPE udl's expected input axis order for 2D input and its output axis order
udl_myinterp.addAxisOrder([AxisAnnotation.BATCH,AxisAnnotation.CHANNEL],
                              [AxisAnnotation.BATCH,AxisAnnotation.CHANNEL])

# As Caffe supports batch dimension, we have an additional dimension here
# Add Caffe udl's expected input axis order for 4D input and its output axis order
udl_myinterp.addSrcAxisOrder([AxisAnnotation.BATCH, AxisAnnotation.CHANNEL, AxisAnnotation.HEIGHT,AxisAnnotation.WIDTH],
                                 [AxisAnnotation.BATCH, AxisAnnotation.CHANNEL, AxisAnnotation.HEIGHT,AxisAnnotation.WIDTH])
# Add Caffe udl's expected input axis order for 3D input and its output axis order
udl_myinterp.addSrcAxisOrder( [AxisAnnotation.CHANNEL,AxisAnnotation.HEIGHT,AxisAnnotation.WIDTH],
                                  [AxisAnnotation.CHANNEL,AxisAnnotation.HEIGHT,AxisAnnotation.WIDTH])
# Add Caffe udl's expected input axis order for 2D input and its output axis order
udl_myinterp.addSrcAxisOrder([AxisAnnotation.BATCH, AxisAnnotation.CHANNEL],
                                 [AxisAnnotation.BATCH, AxisAnnotation.CHANNEL])

# UDL layer name to UDL class map
udl_supported_types = {
    'MyCustomScale': udl_mycustomscale,
    'Interp': udl_myinterp
}

</code>

Could anyone can tell me where is the bug? I also want to know what data should be defined in MyInterpLayerParam? Because interp layer has no weight or bias, so I did not define the 2 variables.

Thanks in advance ^_^