Feat/coreml #308
Feat/coreml #308
Conversation
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| #else | ||
| package func getModel(modelURL: URL) -> Zenz? { | ||
| self.zenzStatus = "zenz-v2 model unavailable on this platform" | ||
| return nil |
There was a problem hiding this comment.
When ZenzaiCoreML is not set, getModel now always marks the model as unavailable and returns nil, but the rest of the pipeline still checks zenzaiMode.enabled. On Zenzai/ZenzaiCPU builds this means the llama backend is never loaded and convertToLattice falls through to the non-zenz path, so enabling zenzaiMode no longer produces any zenz candidates or next-character predictions even though the trait and weight URL are provided—a regression from the previous behavior.
Useful? React with 👍 / 👎.
Adds support for CoreML-based Zenzai conversion, leveraging the `ZenzCoreMLBackend` for improved performance and efficiency on compatible devices. This includes: - Integration of the CoreML model for stateful generation. - A new `ZenzContext+CoreML.swift` to manage the CoreML inference process. - Updates to `Package.swift` to include `swift-transformers` dependency. - Conditional compilation to switch between CoreML and CPU/Metal implementations based on the `ZenzaiCoreML` flag. - Adds tokenizer files and marks mlpackage for LFS. The changes aim to provide a faster and more resource-efficient Zenzai conversion method on Apple platforms with Neural Engine support.
This reverts commit 63f31ac.
Cause of swift-transformers package uses combine, and it not compatible with windows
Enables building the application with CoreML support for enhanced performance on Apple devices. This allows users to leverage CoreML acceleration by using the `--zenzai-coreml` flag during installation.
Removes the explicit LFS bootstrap step from CI workflows as it is no longer required due to changes in how dependencies are managed. This simplifies the CI configuration and reduces build times. Adds documentation for development setup in multiple languages, covering build instructions, testing and devcontainer usage. This enhances the developer experience by providing clear and comprehensive instructions in English, Japanese and Korean. Improves Zenzai integration, adding CoreML support and cross-platform compatibility with Swift 6 concurrency. This allows for high-precision neural Kana-Kanji conversion on Apple devices using CoreML, while ensuring compatibility and performance on both Darwin and Linux platforms.
Skyline-23
force-pushed
the
feat/coreml
branch
from
5a07f7d to
ddfa88d
Compare
1 participant
PR Summary (feat/coreml)
Overview
This branch isolates CoreML integration (no actor refactor) and makes it buildable/usable via the CLI with a CoreML-only flag. Sync APIs remain primary; async wrappers are thin bridges.
Key changes
ZenzandZenzCoreMLServicemarked@unchecked Sendableto unblock async bridging.blockingAsyncnow accepts@Sendableclosures viaTask.detached; CoreML personalization handle is mapped explicitly to a tuple.ZenzaiCoreMLtrait is used, macOS linker gets a-platform_version macos 15.0 15.0hint to silence xcframework version warnings while keeping the default macOS 13 baseline for non-CoreML builds.--zenzai-coreml(plus optional--debug).AzooKeyKanaKanjiConverter_KanaKanjiConverterModuleWithDefaultDictionary.bundleinto/usr/local/bin/.llama.framework; for Zenzai/ZenzaiCPU builds, still expects it.Add Support Page for English/Korean
How to build and install (CoreML)
Known warnings / notes
updateIfRequired(options:)inKanaKanjiConverteris deprecated but behavior is unchanged; can be refactored later.intentioninInputTable.swift, redundanttryinZenzContext+CoreML.swift) with no behavioral impact.