Forums - ONNX convert failed with GPT-NEO model

5 posts / 0 new
or Register
Last post
ONNX convert failed with GPT-NEO model
yaoxingwei1
Join Date: 11 Jul 23
Posts: 3
Posted: Tue, 2023-07-11 00:47
Top

Hi,

I try to convert GPT-NEO ONNX model to model.cpp with qnn-onnx-convert tool. But trigger failed:

Quote:
KeyError: 'Graph has no buffer 1239, referred to as input for split_0'
2023-07-11 15:18:48,710 - 209 - ERROR - Node : 'Graph has no buffer 1239, referred to as input for split_0'
 
After debug, I found the root cause is onnxsim will add new split OP after optimized. But convert tool can't handle the new OP with parameters. The buffer issue will be triggered as above.
 
Quote:
DEBUG: [MatMul]
['/transformer/h.0/ln_1/Add_1_output_0', '1234']
['1235']
DEBUG: [Split]
['1235', '1239']
['1236', '1237', '1238']
DEBUG: [Reshape]
['1236', '/transformer/h.0/attn/attention/Concat_output_0']
['/transformer/h.0/attn/attention/Reshape_output_0']
DEBUG: [Transpose]
['/transformer/h.0/attn/attention/Reshape_output_0']
['/transformer/h.0/attn/attention/Transpose_output_0']
DEBUG: [Reshape]
['1237', '/transformer/h.0/attn/attention/Concat_output_0']
['/transformer/h.0/attn/attention/Reshape_1_output_0']
DEBUG: [Reshape]
['1238', '/transformer/h.0/attn/attention/Concat_output_0']
['/transformer/h.0/attn/attention/Reshape_2_output_0']
DEBUG: [Transpose]
['/transformer/h.0/attn/attention/Reshape_2_output_0']
['/transformer/h.0/attn/attention/Transpose_2_output_0']
 
If add parameter "-n", the convert tool will work ok with original model structure. But maybe the convert tool should have the feature that handle new add OP after onnxsim optimized.
 
Test SDK version: qnn-2.11.0/2.12.0, snpe-2.11.0
 
Thanks.
  • Up0
  • Down0
Re: ONNX convert failed with GPT-NEO model
yunxqin
Join Date: 2 Mar 23
Posts: 44
Posted: Tue, 2023-07-11 20:07
Top

Dear developer,

Could you please share the model with us, and we will test the model to find the reason for you.

BR.

Yunxiang

  • Up0
  • Down0
Re: ONNX convert failed with GPT-NEO model
yaoxingwei1
Join Date: 11 Jul 23
Posts: 3
Posted: Tue, 2023-07-11 21:01
Top

Hi Yunxiang,

The model is about 600MB. Is there any data server can be used to share the model ?

Regards

Xingwei

 

 

 

  • Up0
  • Down0
Re: ONNX convert failed with GPT-NEO model
yunxqin
Join Date: 2 Mar 23
Posts: 44
Posted: Tue, 2023-07-11 23:34
Top

Dear developer,

You can push the model to github and I will pull it locally. 

BR.

Yunxiang

  • Up0
  • Down0
Re: ONNX convert failed with GPT-NEO model
yaoxingwei1
Join Date: 11 Jul 23
Posts: 3
Posted: Sun, 2023-07-16 19:58
Top
Hi yunxiang,
 
 
There are 2 issue. Reproduce step:
 
1. Run:
    $ qnn-onnx-converter -i gpt-neo.onnx -d input_ids 1,256 -d attention_mask 1,256 --input_list input_list.txt
 
Trigger error: 
Quote:
KeyError: 'Graph has no buffer 1239, referred to as input for split_0'
2023-07-11 15:18:48,710 - 209 - ERROR - Node : 'Graph has no buffer 1239, referred to as input for split_0'
 
2. Run:
    $ qnn-onnx-converter -i gpt-neo-125m.onnx -d input_ids 1,256 -d attention_mask 1,256 -n --input_list Input_list.txt
    $ qnn-model-lib-generator -c gpt-neo-125m.cpp -t x86_64-linux-clang -b gpt-neo-125m.bin
    $ qnn-net-run --backend libQnnCpu.so --model libs/x86_64-linux-clang/libgpt-neo-125m.so --input_list input_list.txt --profiling_level detailed --output_dir gpt-neo-quant
 
Trigger fail:
Quote:
[ ERROR ] OpConfig validation failed for ElementWiseSelect
[ ERROR ] QnnModel::addNode() adding node _transformer_h_0_attn_attention_Where failed.
[ ERROR ] gpt_neo_125m.addNode(QNN_OPCONFIG_VERSION_1, "_transformer_h_0_attn_attention_Where", "qti.aisw", "ElementWiseSelect", nullptr, 0, inputs__transformer_h_0_attn_attention_Where, 3, outputs__transformer_h_0_attn_attention_Where, 1 ) expected MODEL_NO_ERROR, got MODEL_GRAPH_ERROR
 
Regards
Xingwei
  • Up0
  • Down0
or Register

Opinions expressed in the content posted here are the personal opinions of the original authors, and do not necessarily reflect those of Qualcomm Incorporated or its subsidiaries (“Qualcomm”). The content is provided for informational purposes only and is not meant to be an endorsement or representation by Qualcomm or any other party. This site may also provide links or references to non-Qualcomm sites and resources. Qualcomm makes no representations, warranties, or other commitments whatsoever about any non-Qualcomm sites or third-party resources that may be referenced, accessible from, or linked to this site.