Skip to content

Commit cc632b2

Browse files
sanchitmonga22cursoragentclaude
authored
[Swift-example][Playground] [Swift-SDK] Introducing YapRun (#400)
* feat: implement Voice Dictation feature for iOS keyboard extension - Enabled local binaries for development in Package.swift. - Added VoiceDictationManagementView and ViewModel for managing dictation settings and history. - Introduced FlowSessionManager to handle audio recording and transcription flow. - Created shared constants and data bridge for inter-process communication between the main app and keyboard extension. - Updated Info.plist for keyboard extension with necessary permissions and configurations. - Integrated deep linking to start dictation sessions from the keyboard. This commit lays the groundwork for a fully functional voice dictation feature in the RunAnywhereAI iOS app. * some fixes * it works * UI updates * adding yaprun app * UI/UX update * adding macos app * Refactor color handling and UI components for adaptive design - Removed hardcoded color schemes in multiple views, replacing them with adaptive colors from AppColors. - Updated ContentView, ModelCardView, NotepadView, and various onboarding steps to use AppColors for text and background styles. - Added a new AppTab enum for managing tab selection in the iOS app. - Introduced a new network client entitlement in the app's entitlements file. - Ensured that the app respects the system's light/dark mode settings across various views. * tests * Implement WhisperKit support for speech-to-text functionality - Added WhisperKit as a new library for speech-to-text processing using Apple Neural Engine. - Introduced WhisperKitRuntime target and integrated it into the main application. - Updated ModelRegistry to include WhisperKit models and their respective handling. - Enhanced RunAnywhere SDK to support model registration and discovery for WhisperKit. - Updated various components to utilize WhisperKit for STT, ensuring compatibility with existing frameworks. - Refactored model management to accommodate WhisperKit's directory-based model structure. * minor fix * minor changes * Release v0.19.2: Add WhisperKit module + persistence fixes - Added RunAnywhereWhisperKit SPM product for Apple Neural Engine STT - Changed WhisperKit dependency from local path to remote URL for SPM consumers - Fixed model download persistence: added RAC_FRAMEWORK_WHISPERKIT to discovery scan - Fixed registerModel() race condition with flushPendingRegistrations() - Emitted Swift STT events for WhisperKit model load/unload - Updated checksums for v0.19.2 xcframeworks (iOS + macOS) Co-authored-by: Cursor <cursoragent@cursor.com> * addressing comments + renaming * changes * updates to whisper kit * Release v0.19.3: Fix WhisperKit telemetry + STT framework tracking - Fixed framework_to_string() in telemetry to serialize whisperkit_coreml, coreml, mlx - Fixed STT component to configure actual_framework before model load - Pass framework from ModelInfo through CppBridge.STT.loadModel() - Updated checksums for v0.19.3 xcframeworks (iOS + macOS) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * minor things * minor changes * Release v0.19.4: Fix framework Info.plist for App Store validation - Added CFBundleVersion to build-ios.sh Info.plist templates (RACommons, backends) - Added CFBundleVersion + CFBundleShortVersionString to onnxruntime macOS plist in create-onnxruntime-xcframework.sh - Added MinimumOSVersion to pre-built onnxruntime iOS Info.plists - Updated Package.swift version to 0.19.4 with new checksums - Updated sdk/runanywhere-swift/VERSION to 0.19.4 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Release v0.19.4: Set useLocalBinaries = false for SPM consumers Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Release v0.19.5: Fix App Store validation by using library-format xcframeworks - Restructure xcframeworks from .framework format to plain .a library format to prevent SPM/Xcode from embedding static libraries as dynamic frameworks - Split onnxruntime into iOS (static) and macOS (dynamic) xcframeworks since xcframeworks can't mix library types - Make all library products explicitly .static in Package.swift - Add build phase to strip SPM-generated framework stubs from app bundle - Update build-ios.sh and create-onnxruntime-xcframework.sh to produce library-format xcframeworks going forward This fixes: "Invalid Bundle - does not support minimum OS Version" and "Upload Symbols Failed" errors during App Store validation. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * minor changes * Fix v0.19.5: Remove type: .static from library products to fix linker errors type: .static on SPM library products prevents binary target linker flags from being forwarded to consuming apps, causing "Undefined symbol: _rac_*" errors when archiving. Reverting to automatic library type resolves this. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Fix v0.19.5: Use framework-format xcframeworks for remote SPM Library-format xcframeworks (.a files) cause linker errors when consumed by Xcode apps via remote SPM. Reverted release zips to framework-format (.framework bundles) which properly propagate linker flags. Local dev still uses library-format via useLocalBinaries=true. The YapRun strip build phase handles App Store embedding. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Fix v0.19.5: Use versioned framework structure for macOS slices macOS frameworks require Versions/A/ directory structure with symlinks. The flat iOS-style structure caused linker errors on macOS builds. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Release v0.19.6: Fix remote SPM with versioned macOS framework structure - Framework-format xcframeworks for proper SPM linker flag propagation - Versioned macOS framework structure (Versions/A/ with symlinks) - Flat iOS framework structure (standard) - Fixes undefined symbol errors on both iOS and macOS remote SPM builds Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Fix missing ONNXRuntimeBinary target in Package.swift Replace stale "ONNXRuntimeBinary" dependency (which doesn't exist) with platform-conditional ONNXRuntimeiOSBinary/ONNXRuntimemacOSBinary targets. This was preventing SPM from resolving the package graph. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * adding 0.19.6 * Add linker stubs for unimplemented LoRA and RAG symbols The LoRA functions (rac_llm_component_*_lora) are declared in headers but not yet implemented in the RACommons C++ library. The RAG functions (rac_rag_*) are implemented in RABackendRAG which is excluded from remote releases. Both cause undefined symbol linker errors. Add weak stub implementations in shim.c that return RAC_ERROR_NOT_IMPLEMENTED. When the real backends are available, the strong definitions will override these stubs automatically. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Add voice agent/WhisperKit stubs, fix YapRun linker settings - Add weak stubs for rac_voice_agent_* and rac_whisperkit_coreml_stt_set_callbacks (not yet in RACommons binary) - Add ARCHS=arm64 to YapRun (xcframeworks are arm64-only) - Fix OTHER_LDFLAGS to use $(inherited) + -all_load array format (required to force-link all symbols from SPM binary targets) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * mac app upload * minor updates * minor updates --------- Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2 parents 4d1ff04 + ddc4a3e commit cc632b2

File tree

192 files changed

+22111
-273
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

192 files changed

+22111
-273
lines changed

.gitignore

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -391,3 +391,4 @@ sdk/runanywhere-react-native/packages/rag/ios/.testlocal
391391
# Node
392392
node_modules/
393393
/tools/
394+
*.trace

Package.resolved

Lines changed: 36 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

Package.swift

Lines changed: 58 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -37,11 +37,11 @@ import Foundation
3737
// ./scripts/build-swift.sh --set-remote (sets useLocalBinaries = false)
3838
//
3939
// =============================================================================
40-
let useLocalBinaries = true // Toggle: true for local dev, false for release
40+
let useLocalBinaries = false // Toggle: true for local dev, false for release
4141

4242
// Version for remote XCFrameworks (used when testLocal = false)
4343
// Updated automatically by CI/CD during releases
44-
let sdkVersion = "0.19.1"
44+
let sdkVersion = "0.19.6"
4545

4646
// RAG binary is only available in local dev mode until the release artifact is published.
4747
// In remote mode, the RAG xcframework zip + checksum don't exist yet, so including the
@@ -80,6 +80,13 @@ let package = Package(
8080
targets: ["LlamaCPPRuntime"]
8181
),
8282

83+
// =================================================================
84+
// WhisperKit Backend - adds STT via Apple Neural Engine
85+
// =================================================================
86+
.library(
87+
name: "RunAnywhereWhisperKit",
88+
targets: ["WhisperKitRuntime"]
89+
),
8390
] + ragProducts(),
8491
dependencies: [
8592
.package(url: "https://github.com/apple/swift-crypto.git", from: "3.0.0"),
@@ -91,6 +98,8 @@ let package = Package(
9198
.package(url: "https://github.com/getsentry/sentry-cocoa", from: "8.40.0"),
9299
// ml-stable-diffusion for CoreML-based image generation
93100
.package(url: "https://github.com/apple/ml-stable-diffusion.git", from: "1.1.0"),
101+
// WhisperKit for Neural Engine STT
102+
.package(url: "https://github.com/argmaxinc/WhisperKit.git", from: "0.9.0"),
94103
],
95104
targets: [
96105
// =================================================================
@@ -118,7 +127,11 @@ let package = Package(
118127
// =================================================================
119128
.target(
120129
name: "ONNXBackend",
121-
dependencies: ["RABackendONNXBinary"],
130+
dependencies: [
131+
"RABackendONNXBinary",
132+
.target(name: "ONNXRuntimeiOSBinary", condition: .when(platforms: [.iOS])),
133+
.target(name: "ONNXRuntimemacOSBinary", condition: .when(platforms: [.macOS])),
134+
],
122135
path: "sdk/runanywhere-swift/Sources/ONNXRuntime/include",
123136
publicHeadersPath: "."
124137
),
@@ -159,7 +172,8 @@ let package = Package(
159172
"RunAnywhere",
160173
"ONNXBackend",
161174
"RABackendONNXBinary",
162-
"ONNXRuntimeBinary",
175+
.target(name: "ONNXRuntimeiOSBinary", condition: .when(platforms: [.iOS])),
176+
.target(name: "ONNXRuntimemacOSBinary", condition: .when(platforms: [.macOS])),
163177
],
164178
path: "sdk/runanywhere-swift/Sources/ONNXRuntime",
165179
exclude: ["include"],
@@ -192,6 +206,22 @@ let package = Package(
192206
]
193207
),
194208

209+
// =================================================================
210+
// WhisperKit Runtime Backend (Apple Neural Engine STT)
211+
// =================================================================
212+
.target(
213+
name: "WhisperKitRuntime",
214+
dependencies: [
215+
"RunAnywhere",
216+
.product(name: "WhisperKit", package: "whisperkit"),
217+
],
218+
path: "sdk/runanywhere-swift/Sources/WhisperKitRuntime",
219+
linkerSettings: [
220+
.linkedFramework("CoreML"),
221+
.linkedFramework("Accelerate"),
222+
]
223+
),
224+
195225
// =================================================================
196226
// RunAnywhere unit tests (e.g. AudioCaptureManager – Issue #198)
197227
// =================================================================
@@ -289,14 +319,19 @@ func binaryTargets() -> [Target] {
289319
),
290320
]
291321

292-
// Local combined ONNX Runtime xcframework (iOS + macOS)
293-
// Created by: cd sdk/runanywhere-swift && ./scripts/create-onnxruntime-xcframework.sh
294-
// targets.append(
295-
// .binaryTarget(
296-
// name: "ONNXRuntimeBinary",
297-
// path: "sdk/runanywhere-swift/Binaries/onnxruntime.xcframework"
298-
// )
299-
// )
322+
// ONNX Runtime xcframeworks - split by platform
323+
// iOS: static library format (not embedded in app bundle)
324+
// macOS: dynamic framework format (embedded in app bundle)
325+
targets.append(contentsOf: [
326+
.binaryTarget(
327+
name: "ONNXRuntimeiOSBinary",
328+
path: "sdk/runanywhere-swift/Binaries/onnxruntime-ios.xcframework"
329+
),
330+
.binaryTarget(
331+
name: "ONNXRuntimemacOSBinary",
332+
path: "sdk/runanywhere-swift/Binaries/onnxruntime-macos.xcframework"
333+
),
334+
])
300335

301336
return targets
302337
} else {
@@ -309,22 +344,27 @@ func binaryTargets() -> [Target] {
309344
.binaryTarget(
310345
name: "RACommonsBinary",
311346
url: "https://github.com/RunanywhereAI/runanywhere-sdks/releases/download/v\(sdkVersion)/RACommons-v\(sdkVersion).zip",
312-
checksum: "f6bc152b1689d7549d6a7b5e692f6babb0efc44fe334c0e60acfc0c12d848c44"
347+
checksum: "40ea84cf054f59fbc65e87d92550d4acb2bcbf433041438822c6b30985e3db24"
313348
),
314349
.binaryTarget(
315350
name: "RABackendLlamaCPPBinary",
316351
url: "https://github.com/RunanywhereAI/runanywhere-sdks/releases/download/v\(sdkVersion)/RABackendLLAMACPP-v\(sdkVersion).zip",
317-
checksum: "ba150fd924f71c2137d6cad0a294c3f9c2da5bc748b547cced87bc0910a9b327"
352+
checksum: "314dddb242caf3d2d0b19c0f919c35187023c6c66cc861de741d071faddbf58b"
318353
),
319354
.binaryTarget(
320355
name: "RABackendONNXBinary",
321356
url: "https://github.com/RunanywhereAI/runanywhere-sdks/releases/download/v\(sdkVersion)/RABackendONNX-v\(sdkVersion).zip",
322-
checksum: "00b28c0542ab25585c534b4e33ddacd4a1d24447aa8c2178949aad89eb56cb1f"
357+
checksum: "809e2510da49f71f6d019e77bcc0a7e12e967f3b739ba0b9eea7adb77936edc0"
358+
),
359+
.binaryTarget(
360+
name: "ONNXRuntimeiOSBinary",
361+
url: "https://github.com/RunanywhereAI/runanywhere-sdks/releases/download/v\(sdkVersion)/onnxruntime-ios-v\(sdkVersion).zip",
362+
checksum: "310022d76a16b2d2d106577a1aa84a9e608c721bb6221c4ba47bf962a88bd9fd"
323363
),
324364
.binaryTarget(
325-
name: "ONNXRuntimeBinary",
326-
url: "https://github.com/RunanywhereAI/runanywhere-sdks/releases/download/v\(sdkVersion)/onnxruntime-v\(sdkVersion).zip",
327-
checksum: "e0180262bd1b10fcda95aaf9aac595af5e6819bd454312b6fc8ffc3828db239f"
365+
name: "ONNXRuntimemacOSBinary",
366+
url: "https://github.com/RunanywhereAI/runanywhere-sdks/releases/download/v\(sdkVersion)/onnxruntime-macos-v\(sdkVersion).zip",
367+
checksum: "f73db9dc09012325b35fd3da74de794a75f4e9971d9b923af0805d6ab1dfc243"
328368
),
329369
]
330370

0 commit comments

Comments
 (0)