Hybrid Flexibility: Native Models > Local Models > Remote Models (Per Feature - Text Gen, Tool Calling, STT, TTS)

[Matt2012]

I wonder if this is currently possible or on the roadmap:

The idea is a ProviderChain per feature (TextGen, ToolCalling, STT, TTS), with three tiers:

1. Platform Native (Apple Foundation Models, Gemini Nano on Android)
2. Local (llama.cpp, Whisper ONNX, Piper ONNX)
3. Remote (cloud API — your backend or third-party)

Each tier can be enabled/disabled, and the SDK resolves the best available provider at runtime. Library developers set sensible defaults; app developers can override at init; end users can override at runtime (if you expose the settings surface).

RunAnywhere.initialize(
    config = RunAnywhereConfig {
        textGeneration {
            providers = ProviderChain.of(
                NativeTextProvider(),
                LocalTextProvider("llama-3.2-1b"),
                RemoteTextProvider()
            )
            userOverridable = UserOverridePolicy.full()
        }
        toolCalling {
            providers = ProviderChain.remoteOnly()
            userOverridable = UserOverridePolicy.none()
        }
        speechToText {
            providers = ProviderChain.of(
                NativeSTTProvider(),
                LocalSTTProvider("whisper-base"),
                RemoteSTTProvider()
            )
            userOverridable = UserOverridePolicy.full()
        }
        textToSpeech {
            providers = ProviderChain.of(
                NativeTTSProvider(),
                LocalTTSProvider("piper-en-gb"),
                RemoteTTSProvider()
            )
            userOverridable = UserOverridePolicy.full()
        }
    }
)

// User overrides (from app settings UI)
RunAnywhere.userPreferences.apply {
    textGeneration.preferredTier = ProviderTier.REMOTE   // wants best quality
    speechToText.preferredTier = ProviderTier.LOCAL      // wants privacy
    textToSpeech.preferredTier = ProviderTier.NATIVE     // uses platform voice
    // toolCalling locked to remote — user cannot change
}