Forums - Pytorch BERT model conversion

Hi,

I'm trying to convert a pytorch BERT model(neuralmind/bert-base-portuguese-cased), trained with max lenght = 256, to a DLC Container. For tracing I used:

inputs = (torch.randint(0,29794,[1,256]),torch.zeros(1,256), torch.zeros(1,256))
traced = torch.jit.trace(torch.load("model.pt"),inputs)
traced.save("traced.pt")

But when running the snpe command I always get errors. I tried two approaches:

snpe-pytorch-to-dlc \
--input_network bert_wrapper_256.pt \
--input_dim 'input' 1,256 \
--input_dtype 'input' 'int32' \
--input_dim 'input_ids' 1,256 \
--input_dtype 'input_ids' 'int32' \
--input_dim 'attention_mask' 1,256 \
--input_dtype 'attention_mask' 'int32' \
--output_path model_dtype.dlc

That Returns:

2023-07-25 13:16:53,925 - 230 - ERROR - Encountered Error: validateQnnOpConfig: Failed QNN validation for model.bert.subtract_0 with error

Traceback (most recent call last):
  File "/home/rafael/snpe/2.12.0.230626/bin/x86_64-linux-clang/snpe-pytorch-to-dlc", line 65, in main
    backend.save(optimized_graph)
  File "/home/rafael/snpe/2.12.0.230626/lib/python/qti/aisw/converters/backend/ir_to_dlc.py", line 161, in save
    ir_graph = self.get_ir_graph(graph)
  File "/home/rafael/snpe/2.12.0.230626/lib/python/qti/aisw/converters/qnn_backend/qnn_backend_base.py", line 526, in get_ir_graph
    raise e
  File "/home/rafael/snpe/2.12.0.230626/lib/python/qti/aisw/converters/qnn_backend/qnn_backend_base.py", line 521, in get_ir_graph
    QnnTranslations.apply_method_to_all_ops(BackendTranslationBase.ADD_OP_TO_BACKEND, graph, self)
  File "/home/rafael/snpe/2.12.0.230626/lib/python/qti/aisw/converters/common/converter_ir/translation.py", line 71, in apply_method_to_all_ops
    self.apply_method_to_op(node.op.type, method_name, node, graph, *args, **kwargs)
  File "/home/rafael/snpe/2.12.0.230626/lib/python/qti/aisw/converters/common/converter_ir/translation.py", line 51, in apply_method_to_op
    return translation.apply_method(method_name, *args, **kwargs)
  File "/home/rafael/snpe/2.12.0.230626/lib/python/qti/aisw/converters/common/converter_ir/translation.py", line 18, in apply_method
    return self.indexed_methods[method_name](*args, **kwargs)
  File "/home/rafael/snpe/2.12.0.230626/lib/python/qti/aisw/converters/qnn_backend/qnn_translations.py", line 1208, in add_op_to_backend
    backend.add_node(node.op.name, op_type,
  File "/home/rafael/snpe/2.12.0.230626/lib/python/qti/aisw/converters/backend/ir_to_dlc.py", line 114, in add_node
    if not self.model.add_node(node_name, node_type, node_package_name, tensor_params, scalar_params,
RuntimeError: validateQnnOpConfig: Failed QNN validation for model.bert.subtract_0 with error

And using:

snpe-pytorch-to-dlc \
--input_network bert_wrapper_256.pt \
--input_dim 'input' 1,256 \
--input_dim 'input_ids' 1,256 \
--input_dim 'attention_mask' 1,256 \
--output_path model_dtype.dlc

That I get:

Traceback (most recent call last):
  File "/home/rafael/snpe/2.12.0.230626/bin/x86_64-linux-clang/snpe-pytorch-to-dlc", line 65, in main
    backend.save(optimized_graph)
  File "/home/rafael/snpe/2.12.0.230626/lib/python/qti/aisw/converters/backend/ir_to_dlc.py", line 161, in save
    ir_graph = self.get_ir_graph(graph)
  File "/home/rafael/snpe/2.12.0.230626/lib/python/qti/aisw/converters/qnn_backend/qnn_backend_base.py", line 526, in get_ir_graph
    raise e
  File "/home/rafael/snpe/2.12.0.230626/lib/python/qti/aisw/converters/qnn_backend/qnn_backend_base.py", line 521, in get_ir_graph
    QnnTranslations.apply_method_to_all_ops(BackendTranslationBase.ADD_OP_TO_BACKEND, graph, self)
  File "/home/rafael/snpe/2.12.0.230626/lib/python/qti/aisw/converters/common/converter_ir/translation.py", line 71, in apply_method_to_all_ops
    self.apply_method_to_op(node.op.type, method_name, node, graph, *args, **kwargs)
  File "/home/rafael/snpe/2.12.0.230626/lib/python/qti/aisw/converters/common/converter_ir/translation.py", line 51, in apply_method_to_op
    return translation.apply_method(method_name, *args, **kwargs)
  File "/home/rafael/snpe/2.12.0.230626/lib/python/qti/aisw/converters/common/converter_ir/translation.py", line 18, in apply_method
    return self.indexed_methods[method_name](*args, **kwargs)
  File "/home/rafael/snpe/2.12.0.230626/lib/python/qti/aisw/converters/qnn_backend/qnn_translations.py", line 414, in add_op_to_backend
    backend.add_tensor(node.op.name, node.output_names[0], qnn_definitions.QNN_TENSOR_TYPE_STATIC,
  File "/home/rafael/snpe/2.12.0.230626/lib/python/qti/aisw/converters/backend/ir_to_dlc.py", line 106, in add_tensor
    if not self.model.add_tensor(node_name, tensor_info, is_quantizable=is_quantizable):
ValueError: IrStaticTensor: IrStaticTensor (model.bert.pooler.constant_0) must specify correct dims!

Anyone have already solved this problem? Am I missing some steps?