Update iOS LLM Application #505
Conversation
vraspar
commented
Dec 18, 2024
vraspar
commented
- Update App to use new GenAI API
- Add ability to select model path from app to allow using multiple models in single build
vraspar
force-pushed
the
vraspar/llm-ios-app-update
branch
from
a6c98fc to
6102533
Compare
There was a problem hiding this comment.
Copilot reviewed 3 out of 7 changed files in this pull request and generated no comments.
Files not reviewed (4)
- mobile/examples/phi-3/ios/LocalLLM/LocalLLM/ContentView.swift: Language not supported
- mobile/examples/phi-3/ios/LocalLLM/LocalLLM/FolderPicker.swift: Language not supported
- mobile/examples/phi-3/ios/LocalLLM/LocalLLM/GenAIGenerator.h: Language not supported
- mobile/examples/phi-3/ios/LocalLLM/LocalLLM/GenAIGenerator.mm: Language not supported
mobile/examples/phi-3/ios/LocalLLM/LocalLLM.xcodeproj/project.pbxproj
Outdated
Show resolved
Hide resolved
mobile/examples/phi-3/ios/LocalLLM/LocalLLM/header/onnxruntime_c_api.h
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
can this file have a more meaningful name?
| /* Begin PBXFileSystemSynchronizedBuildFileExceptionSet section */ | ||
| 8AC409CD2DADE0EC00388525 /* PBXFileSystemSynchronizedBuildFileExceptionSet */ = { | ||
| isa = PBXFileSystemSynchronizedBuildFileExceptionSet; | ||
| membershipExceptions = ( | ||
| "LLama-3.2-1B-int4-acc_4-gqa-webgpu-fp16/genai_config.json", | ||
| "LLama-3.2-1B-int4-acc_4-gqa-webgpu-fp16/model.onnx", | ||
| "LLama-3.2-1B-int4-acc_4-gqa-webgpu-fp16/model.onnx.data", | ||
| "LLama-3.2-1B-int4-acc_4-gqa-webgpu-fp16/special_tokens_map.json", | ||
| "LLama-3.2-1B-int4-acc_4-gqa-webgpu-fp16/tokenizer_config.json", | ||
| "LLama-3.2-1B-int4-acc_4-gqa-webgpu-fp16/tokenizer.json", | ||
| ); | ||
| target = 51D4C8D12BFD22D70029FCEA /* LocalLLM */; | ||
| }; | ||
| /* End PBXFileSystemSynchronizedBuildFileExceptionSet section */ |
There was a problem hiding this comment.
what's this section for? looks like it specifies llama3.2 specific files
| ); | ||
| MARKETING_VERSION = 1.0; | ||
| PRODUCT_BUNDLE_IDENTIFIER = ai.onnxruntime.genai.demo.LocalLLM; | ||
| PRODUCT_BUNDLE_IDENTIFIER = ai.onnxruntime.genai.demo.LocalLL; |
There was a problem hiding this comment.
the identifier probably doesn't need to change
| } | ||
|
|
||
| - (void)setModelFolderPath:(NSString*)modelPath { | ||
| @synchronized(self) { |
There was a problem hiding this comment.
should we also synchronize access to self->model and self->tokenizer from generate? I'm not sure this is the right level that we should be synchronizing at. is there a way we can ensure that generate and setModelFolderPath don't get called at the same time?
perhaps the constructor can take the model path and we just create a new GenAIGenerator if the model path changes?
