Add QNN support for Gemma3-4b-it #125
Conversation
| def run(args: argparse.Namespace): | ||
| logger.info("Loading model...") | ||
| config = og.Config(args.model_path) | ||
| if args.execution_provider != "follow_config": |
There was a problem hiding this comment.
should we be providing some error/warning messages if either of these conditions are not true
| if args.image_paths: | ||
| image_paths = args.image_paths | ||
| else: | ||
| image_paths = [str(Path(__file__).parent / "images" / "dog.jpg")] |
There was a problem hiding this comment.
do we want to guarantee that this exists? or should we just require args.image_paths in non-interactive
| if args.prompt: | ||
| text = args.prompt | ||
| else: | ||
| text = "What is shown in this image?" |
There was a problem hiding this comment.
same comment here regarding default vs. requiring args.prompt
|
|
||
| # Construct the "messages" argument passed to apply_chat_template | ||
| messages = [] | ||
| if model.type == "phi3v": |
There was a problem hiding this comment.
should we remove the phi3v version of things since this is the gemma example?
| uv pip install setuptools | ||
|
|
||
| # Requires installation of uv | ||
| uv pip install -r ../requirements.txt |
There was a problem hiding this comment.
we need to look at qnn_req.txt and confirm whether that needs to be added here
| @@ -0,0 +1,379 @@ | |||
| { | |||
| "cells": [ | |||
There was a problem hiding this comment.
I'm a little hesitant to say we need both the notebook and the README.md/env_setup.sh script
maybe we can look to consolidate things so that we don't have duplicate commands in multiple places and have to essentially maintain multiple copies of the same instructions (e.g. remove some commands from README, have the notebook just call the env_setup.sh script for some stuff, etc.)
|
|
||
| - **`env_setup.sh`** - Automated environment setup script (Linux only) | ||
| - **`gemma3-4b-text-qnn-config.json`** - Olive configuration for optimizing the text component | ||
| - **`gemma3-4b-vision-qnn-config.json`** - Olive configuration for optimizing the vision component |
| @@ -0,0 +1,7 @@ | |||
| coloredlogs | |||
| flatbuffers | |||
| numpy >= 1.21.6 | |||
There was a problem hiding this comment.
need to look at whether we can just move these into requirements.txt?
|
Please add a description to the PR and resolve all open conversations. |
prudhvi-qti
force-pushed
the
dev/prudhvi-qti/gemma3
branch
from
e0ff1ef to
e81cb34
Compare
3 participants
No description provided.