Forums - can't find qnn-genai-transformer-composer

6 posts / 0 new
or Register
Last post
can't find qnn-genai-transformer-composer
f10.liu
Join Date: 22 Mar 18
Posts: 15
Posted: Mon, 2024-04-01 23:14
Top

I follow the document "QNN GenAI Transformer SDK Download and Utilization".

Download qualcomm_ai_engine_direct --version 2.19.0.240124, but I can't find qnn-genai-transformer-composer.

Could you help me?

 

I want to run tiny-llama on Snapdragon_8_gen_3, do you have some guide?

 

  • Up0
  • Down0
Re: can't find qnn-genai-transformer-composer
sanjjey.a.sanjjey
Join Date: 17 May 22
Posts: 57
Posted: Thu, 2024-04-04 00:24
Top

Hi,

May I know, what are you trying to do?

Is it about model conversion using qnn to run it on Snapdragon_8_gen_3?

 

  • Up0
  • Down0
Re: can't find qnn-genai-transformer-composer
f10.liu
Join Date: 22 Mar 18
Posts: 15
Posted: Mon, 2024-04-08 22:02
Top

Yes, I want to use model conversion of qnn to convert tiny LLAMA and run it on Snapdragon_8_gen_3

  • Up0
  • Down0
Re: can't find qnn-genai-transformer-composer
f10.liu
Join Date: 22 Mar 18
Posts: 15
Posted: Tue, 2024-04-09 18:43
Top

Is  document "QNN GenAI Transformer SDK Download and Utilization" is not used? 

Or do you have another guide to run LLM model on Snapdragon_8_gen_3?

  • Up0
  • Down0
Re: can't find qnn-genai-transformer-composer
sanjjey.a.sanjjey
Join Date: 17 May 22
Posts: 57
Posted: Wed, 2024-04-17 23:16
Top

Hi, 
For model conversion using qnn you can check the below mentioned link,
https://docs.qualcomm.com/bundle/publicresource/topics/80-63442-50/tutor...

  • Up0
  • Down0
Re: can't find qnn-genai-transformer-composer
f10.liu
Join Date: 22 Mar 18
Posts: 15
Posted: Sun, 2024-04-28 20:36
Top
Thanks for your sharing the link. I convert inceptionV3 successfully, but for LLAMA, download from Hugging Face, the model files are:
1.config.json
2.generation_config.json
3.pytorch_model.bin
4.tokenizer.model
5.tokenizer_config.json
for command as the link "${QNN_SDK_ROOT}/bin/x86_64-linux-clang/qnn-pytorch-converter", how to assign --input_network  --input_dim and --input_list?
I mean does LLM(Transformer model) convert tool is different with CNN model?
 
  • Up0
  • Down0
or Register

Opinions expressed in the content posted here are the personal opinions of the original authors, and do not necessarily reflect those of Qualcomm Incorporated or its subsidiaries (“Qualcomm”). The content is provided for informational purposes only and is not meant to be an endorsement or representation by Qualcomm or any other party. This site may also provide links or references to non-Qualcomm sites and resources. Qualcomm makes no representations, warranties, or other commitments whatsoever about any non-Qualcomm sites or third-party resources that may be referenced, accessible from, or linked to this site.