Add LocalChatClientWithTools sample — on-device AI with Apple Intelligence tool calling

[mattleibow]

What this sample shows

A complete .NET MAUI chat app that runs entirely on-device using Apple Intelligence — no API keys, no cloud services, no cost. It demonstrates how to give an on-device LLM real capabilities through function calling (tools) and multi-turn conversation history using the standard IChatClient interface from Microsoft.Extensions.AI.

Screenshots (iOS)

Landing page	Calculator tool	Weather (multi-turn)	Multi-tool calls

What Apple Intelligence can do with tools

The AI can autonomously decide which tool(s) to call based on the user's natural language input:

• 🌤️ Weather — "What's the weather in Seattle?" → AI calls get_weather, geocodes the city, fetches live forecast from open-meteo.com
• 🔢 Calculator — "Calculate 15% tip on $47.50" → AI calls calculate with the right math expression
• 📁 Files — "List files in my Documents folder" → AI calls list_files with the resolved path
• ⚙️ System Info — "How much storage do I have?" → AI calls get_system_info and reports battery, memory, storage
• ⏲️ Timer — "Set a 5-minute timer" → AI calls set_timer with duration and title
• 🧩 Multi-tool — "How much storage do I have, and how much would I have left after downloading a 9.3 GB file?" → AI chains get_system_info + calculate automatically
• 🔄 Multi-turn — "What is 100 + 50?" → 150, then "Add 25 to that" → 175 — the AI remembers prior messages, tool calls, and results

How the sample is built

Layer	What it shows
MauiProgram.cs	How to register AppleIntelligenceChatClient as an IChatClient with UseFunctionInvocation() for automatic tool dispatch
Services/Tools/*.cs	How to create tools using AIFunctionFactory.Create with strongly-typed methods, [Description] attributes for the AI to understand parameters, and AOT-compatible JSON serialization
ChatViewModel.cs	How to maintain conversation history (user messages, assistant text, tool calls, tool results), pass tools via ChatOptions.Tools, stream responses with GetStreamingResponseAsync, and observe tool invocations in real-time
MainPage.xaml	Chat UI with DataTemplateSelector for different message types, responsive FlexLayout landing page, and tool-call indicators with expandable raw JSON

Key design decisions

• IChatClient abstraction — The sample uses the standard Microsoft.Extensions.AI interface, so swapping Apple Intelligence for OpenAI/Ollama/etc. is a one-line DI change
• AIFunctionFactory.Create pattern — Tools are plain C# methods with [Description] attributes, not custom base classes. This is the recommended approach for .NET AI tools
• Multi-turn conversation — Full conversation history (including FunctionCallContent and FunctionResultContent) is accumulated and sent with each request, enabling follow-up questions like "double that" or "what about Tokyo?"
• On-device only — All LLM inference stays on the device. Weather data comes from the free open-meteo.com API (no key needed). No other network calls
• AOT-compatible — All JSON serialization uses source-generated JsonSerializerContext via ToolJsonContext

Platform support

• ✅ iOS 26.0+ and macCatalyst 26.0+ (requires Apple Intelligence enabled)
• ❌ Windows / Android — throws PlatformNotSupportedException at startup with a clear message

How to run

# iOS simulator
dotnet build -f net10.0-ios -t:Run

# macCatalyst
dotnet build -f net10.0-maccatalyst -t:Run

Requires .NET 10 SDK.

Relationship to existing samples

This is the on-device companion to the existing ChatClientWithTools sample (which uses Azure OpenAI). Same tool set, same UI patterns, but zero cloud dependencies.

[Copilot]

Pull request overview

Adds a new .NET MAUI sample app (LocalChatClientWithTools) that demonstrates fully on-device chat using Apple Intelligence via Microsoft.Extensions.AI, including function/tool calling and multi-turn conversation history.

Changes:

• Introduces a streaming chat UI + MVVM view models that maintain conversation history and surface tool-invocation progress.
• Adds a set of tools (weather, calculator, file listing, system info, timer) implemented via AIFunctionFactory.Create with AOT-friendly JSON source generation.
• Adds platform/project scaffolding, assets, and documentation for iOS/macCatalyst-focused execution (with explicit unsupported-platform behavior).

Reviewed changes

Copilot reviewed 42 out of 53 changed files in this pull request and generated 13 comments.

Show a summary per file

File	Description
10.0/AI/LocalChatClientWithTools/src/ViewModels/ToolCallMessageViewModel.cs	Tool-call message VM (expandable raw JSON + completion state).
10.0/AI/LocalChatClientWithTools/src/ViewModels/TextMessageViewModel.cs	Text message VM for user/assistant/error display.
10.0/AI/LocalChatClientWithTools/src/ViewModels/ChatViewModel.cs	Core chat orchestration: streaming responses, tool wrapping, history tracking.
10.0/AI/LocalChatClientWithTools/src/ViewModels/ChatMessageViewModel.cs	Base VM shared fields (user flag, timestamp).
10.0/AI/LocalChatClientWithTools/src/Services/Tools/WeatherTool.cs	Weather tool (geocode + forecast via open-meteo).
10.0/AI/LocalChatClientWithTools/src/Services/Tools/ToolJsonContext.cs	JSON source-gen context for tool schemas/results.
10.0/AI/LocalChatClientWithTools/src/Services/Tools/TimerTool.cs	Timer tool that schedules a one-shot timer and alerts when done.
10.0/AI/LocalChatClientWithTools/src/Services/Tools/SystemInfoTool.cs	Device/battery/storage info tool using MAUI Essentials + DriveInfo.
10.0/AI/LocalChatClientWithTools/src/Services/Tools/FileOperationsTool.cs	File listing tool with common path resolution and entry limiting.
10.0/AI/LocalChatClientWithTools/src/Services/Tools/CalculatorTool.cs	Calculator tool using preprocessing + DataTable.Compute.
10.0/AI/LocalChatClientWithTools/src/Resources/Styles/Styles.xaml	UI styles for the sample (controls, layout defaults).
10.0/AI/LocalChatClientWithTools/src/Resources/Styles/Colors.xaml	Color palette + chat-specific colors.
10.0/AI/LocalChatClientWithTools/src/Resources/Splash/splash.svg	Splash asset for the sample.
10.0/AI/LocalChatClientWithTools/src/Resources/Raw/AboutAssets.txt	Default MAUI raw-assets placeholder doc.
10.0/AI/LocalChatClientWithTools/src/Resources/AppIcon/appiconfg.svg	App icon foreground.
10.0/AI/LocalChatClientWithTools/src/Resources/AppIcon/appicon.svg	App icon background.
10.0/AI/LocalChatClientWithTools/src/Properties/launchSettings.json	Launch profile for local debugging.
10.0/AI/LocalChatClientWithTools/src/Platforms/iOS/Resources/PrivacyInfo.xcprivacy	iOS privacy manifest additions for accessed APIs.
10.0/AI/LocalChatClientWithTools/src/Platforms/iOS/Program.cs	iOS entry point.
10.0/AI/LocalChatClientWithTools/src/Platforms/iOS/Info.plist	iOS Info.plist configuration.
10.0/AI/LocalChatClientWithTools/src/Platforms/iOS/AppDelegate.cs	iOS MAUI app delegate.
10.0/AI/LocalChatClientWithTools/src/Platforms/Windows/app.manifest	Windows manifest (template scaffolding).
10.0/AI/LocalChatClientWithTools/src/Platforms/Windows/Package.appxmanifest	Windows packaging manifest (template scaffolding).
10.0/AI/LocalChatClientWithTools/src/Platforms/Windows/App.xaml.cs	Windows app host bootstrap.
10.0/AI/LocalChatClientWithTools/src/Platforms/Windows/App.xaml	Windows XAML app host.
10.0/AI/LocalChatClientWithTools/src/Platforms/MacCatalyst/Program.cs	macCatalyst entry point.
10.0/AI/LocalChatClientWithTools/src/Platforms/MacCatalyst/Info.plist	macCatalyst Info.plist configuration.
10.0/AI/LocalChatClientWithTools/src/Platforms/MacCatalyst/Entitlements.plist	macCatalyst entitlements (sandbox + network client).
10.0/AI/LocalChatClientWithTools/src/Platforms/MacCatalyst/AppDelegate.cs	macCatalyst MAUI app delegate.
10.0/AI/LocalChatClientWithTools/src/Platforms/Android/Resources/values/colors.xml	Android colors resource file.
10.0/AI/LocalChatClientWithTools/src/Platforms/Android/MainApplication.cs	Android application host bootstrap.
10.0/AI/LocalChatClientWithTools/src/Platforms/Android/MainActivity.cs	Android activity host bootstrap.
10.0/AI/LocalChatClientWithTools/src/Platforms/Android/AndroidManifest.xml	Android manifest (includes internet permissions for weather).
10.0/AI/LocalChatClientWithTools/src/MauiProgram.cs	DI setup: tools, chat client builder, platform gating.
10.0/AI/LocalChatClientWithTools/src/MainPage.xaml.cs	Page code-behind wiring entry completion to Send command.
10.0/AI/LocalChatClientWithTools/src/MainPage.xaml	Chat UI (empty prompts, message list, tool call indicator, input area).
10.0/AI/LocalChatClientWithTools/src/LocalChatClientWithTools.sln	Sample solution file.
10.0/AI/LocalChatClientWithTools/src/LocalChatClientWithTools.csproj	MAUI project targeting net10 TFMs + packages.
10.0/AI/LocalChatClientWithTools/src/GlobalXmlns.cs	Global XAML xmlns mappings (implicit xmlns).
10.0/AI/LocalChatClientWithTools/src/ChatTemplateSelector.cs	Template selector for text vs tool-call messages.
10.0/AI/LocalChatClientWithTools/src/AppShell.xaml.cs	Shell code-behind.
10.0/AI/LocalChatClientWithTools/src/AppShell.xaml	Shell layout + main route.
10.0/AI/LocalChatClientWithTools/src/App.xaml.cs	App bootstrap + root window creation.
10.0/AI/LocalChatClientWithTools/src/App.xaml	Resource dictionary merges (colors + styles).
10.0/AI/LocalChatClientWithTools/images/calculator.png	Screenshot asset for README.
10.0/AI/LocalChatClientWithTools/README.md	Sample documentation and usage instructions.

[Copilot]

Pull request overview

Adds a new .NET MAUI sample app (“LocalChatClientWithTools”) demonstrating on-device Apple Intelligence chat via Microsoft.Extensions.AI with automatic tool/function invocation, streaming UI updates, and a set of built-in tools (weather, calculator, files, system info, timer).

Changes:

• Introduces the MAUI app shell/page/viewmodels for a chat UI with tool-call indicators and streaming responses.
• Adds tool implementations (weather, calculator, file listing, system info, timers) with source-generated JSON serialization for AOT compatibility.
• Adds platform/project scaffolding (TFMs, icons/splash, styles, manifests/plists) and documentation for running the sample.

Reviewed changes

Copilot reviewed 42 out of 53 changed files in this pull request and generated 7 comments.

Show a summary per file

File	Description
10.0/AI/LocalChatClientWithTools/src/ViewModels/ToolCallMessageViewModel.cs	ViewModel for tool-call status + expandable raw JSON payload.
10.0/AI/LocalChatClientWithTools/src/ViewModels/TextMessageViewModel.cs	ViewModel for text chat messages (user/assistant/error).
10.0/AI/LocalChatClientWithTools/src/ViewModels/ChatViewModel.cs	Core chat flow: history, streaming responses, tool observability wrappers.
10.0/AI/LocalChatClientWithTools/src/ViewModels/ChatMessageViewModel.cs	Base message VM with user flag + timestamp.
10.0/AI/LocalChatClientWithTools/src/Services/Tools/WeatherTool.cs	Weather tool using open-meteo geocoding + current forecast.
10.0/AI/LocalChatClientWithTools/src/Services/Tools/ToolJsonContext.cs	Source-generated JSON context for tool schema/results.
10.0/AI/LocalChatClientWithTools/src/Services/Tools/TimerTool.cs	Timer tool creating one-shot timers and alerting on completion.
10.0/AI/LocalChatClientWithTools/src/Services/Tools/SystemInfoTool.cs	System info tool (battery/storage/device details).
10.0/AI/LocalChatClientWithTools/src/Services/Tools/FileOperationsTool.cs	File listing tool with path resolution and entry limiting.
10.0/AI/LocalChatClientWithTools/src/Services/Tools/CalculatorTool.cs	Calculator tool with expression preprocessing + evaluation.
10.0/AI/LocalChatClientWithTools/src/Resources/Styles/Styles.xaml	App-wide control styles and visual states.
10.0/AI/LocalChatClientWithTools/src/Resources/Styles/Colors.xaml	Color palette + chat UI color resources.
10.0/AI/LocalChatClientWithTools/src/Resources/Splash/splash.svg	Splash asset.
10.0/AI/LocalChatClientWithTools/src/Resources/Raw/AboutAssets.txt	Default MAUI raw assets guidance file.
10.0/AI/LocalChatClientWithTools/src/Resources/AppIcon/appiconfg.svg	App icon foreground vector.
10.0/AI/LocalChatClientWithTools/src/Resources/AppIcon/appicon.svg	App icon base/background vector.
10.0/AI/LocalChatClientWithTools/src/Properties/launchSettings.json	Launch profile scaffolding.
10.0/AI/LocalChatClientWithTools/src/Platforms/iOS/Resources/PrivacyInfo.xcprivacy	iOS privacy manifest for required accessed APIs.
10.0/AI/LocalChatClientWithTools/src/Platforms/iOS/Program.cs	iOS entry point.
10.0/AI/LocalChatClientWithTools/src/Platforms/iOS/Info.plist	iOS plist scaffold.
10.0/AI/LocalChatClientWithTools/src/Platforms/iOS/AppDelegate.cs	iOS app delegate.
10.0/AI/LocalChatClientWithTools/src/Platforms/Windows/app.manifest	Windows app manifest scaffold.
10.0/AI/LocalChatClientWithTools/src/Platforms/Windows/Package.appxmanifest	Windows packaging manifest scaffold.
10.0/AI/LocalChatClientWithTools/src/Platforms/Windows/App.xaml.cs	WinUI app bootstrap.
10.0/AI/LocalChatClientWithTools/src/Platforms/Windows/App.xaml	WinUI XAML app definition.
10.0/AI/LocalChatClientWithTools/src/Platforms/MacCatalyst/Program.cs	MacCatalyst entry point.
10.0/AI/LocalChatClientWithTools/src/Platforms/MacCatalyst/Info.plist	MacCatalyst plist scaffold.
10.0/AI/LocalChatClientWithTools/src/Platforms/MacCatalyst/Entitlements.plist	MacCatalyst entitlements (sandbox + network).
10.0/AI/LocalChatClientWithTools/src/Platforms/MacCatalyst/AppDelegate.cs	MacCatalyst app delegate.
10.0/AI/LocalChatClientWithTools/src/Platforms/Android/Resources/values/colors.xml	Android colors resource.
10.0/AI/LocalChatClientWithTools/src/Platforms/Android/MainApplication.cs	Android application bootstrap.
10.0/AI/LocalChatClientWithTools/src/Platforms/Android/MainActivity.cs	Android activity bootstrap.
10.0/AI/LocalChatClientWithTools/src/Platforms/Android/AndroidManifest.xml	Android manifest + network permissions (for weather).
10.0/AI/LocalChatClientWithTools/src/MauiProgram.cs	DI setup: chat client, tools, platform gating.
10.0/AI/LocalChatClientWithTools/src/MainPage.xaml.cs	Main page code-behind (Entry Completed -> Send).
10.0/AI/LocalChatClientWithTools/src/MainPage.xaml	Chat UI (EmptyView prompts, templates, tool-call expansion).
10.0/AI/LocalChatClientWithTools/src/LocalChatClientWithTools.sln	New solution file for the sample.
10.0/AI/LocalChatClientWithTools/src/LocalChatClientWithTools.csproj	New MAUI project file + package references.
10.0/AI/LocalChatClientWithTools/src/GlobalXmlns.cs	Global XAML namespace mapping for implicit xmlns usage.
10.0/AI/LocalChatClientWithTools/src/ChatTemplateSelector.cs	DataTemplateSelector for text vs tool-call messages.
10.0/AI/LocalChatClientWithTools/src/AppShell.xaml.cs	Shell code-behind.
10.0/AI/LocalChatClientWithTools/src/AppShell.xaml	Shell structure (routes MainPage).
10.0/AI/LocalChatClientWithTools/src/App.xaml.cs	App bootstrap and window creation.
10.0/AI/LocalChatClientWithTools/src/App.xaml	Resource dictionary merge (colors + styles).
10.0/AI/LocalChatClientWithTools/images/calculator.png	Screenshot asset used in README.
10.0/AI/LocalChatClientWithTools/README.md	Sample documentation (overview, tools, run instructions).

[PureWeen]

🔍 Multi-Model Code Review — PR #745 (LocalChatClientWithTools)

CI Status: ✅ All checks passing

Consensus Findings (flagged by 2+ of 5 models)

---

🔴 CRITICAL — ClearChat race condition crashes streaming

File: src/ViewModels/ChatViewModel.cs — ClearChat() method

ClearChat has no guard against being called while Send() is actively streaming. Because Send() is async and yields back to the main thread at every await foreach iteration, the user can tap "Clear" between two streaming updates. This calls Messages.Clear() and _conversationHistory.Clear(). When Send() resumes, every subsequent Messages[^1] access will throw IndexOutOfRangeException — and the catch block itself will re-throw, crashing the app.

Fix: Guard ClearChat against concurrent execution:

[RelayCommand]
private void ClearChat()
{
    if (IsProcessing) return;
    Messages.Clear();
    _conversationHistory.Clear();
}

---

🟡 MODERATE — catch block writes Messages[^1] before placeholder is guaranteed to exist

File: src/ViewModels/ChatViewModel.cs — Send() catch block

The catch unconditionally executes Messages[^1] = new TextMessageViewModel { IsError = true, ... }. If an exception fires between the user-message Add and the "⏳" placeholder Add (e.g., a binding throw on InputText = string.Empty), Messages[^1] is the user's own message, which gets silently overwritten with the error. If Messages is empty (cleared concurrently — see above), this throws a secondary exception masking the original.

---

🟡 MODERATE — FileOperationsTool passes arbitrary rooted paths directly to DirectoryInfo

File: src/Services/Tools/FileOperationsTool.cs — ResolveCommonPath() (4/5 model consensus)

_ => Path.IsPathRooted(path ?? string.Empty) ? path! : ...

The AI model (or a prompt-injected payload) can supply any absolute path such as /private/var, /etc, or ~/Library and the tool will enumerate its contents without restriction. On macCatalyst the app sandbox is considerably looser than iOS. The resolved path should be validated against a set of known allowed root directories before use.

---

🟢 MINOR — EvaluatePowerOperator silently fails for parenthesized or expression bases

File: src/Services/Tools/CalculatorTool.cs — EvaluatePowerOperator()

The base-scanning loop only walks backward through digits and . characters. An expression like (2+3)^2 produces baseStart >= baseEnd (the ) is not a digit), hits the break, and returns the expression with ^ still in it. DataTable.Compute does not support ^, so it throws and the calculator returns "error". The PR's own [Description] attribute advertises 2^3 as a supported example — but the implementation silently fails for any expression base that isn't a bare number literal.

---

Single-Model Findings (below consensus threshold — noted for author awareness)

Issue	Notes
textBuilder not cleared after FunctionCallContent — streamed text after a tool call is prefixed with pre-tool text	Real functional bug worth fixing
OperationCanceledException swallowed in WeatherTool.GetWeatherAsync — cancellation silently returns a fake result	Should re-throw
JsonDocument not disposed in BuildRawJson — ArrayPool memory not returned on large payloads	Use using var
WeatherTool URL uses culture-sensitive double formatting ({latitude:F4}) — commas in locale-specific cultures break the API URL	Use CultureInfo.InvariantCulture

---

Recommended Action: ⚠️ Request Changes

1. Guard ClearChat with if (IsProcessing) return;
2. Sandbox paths in ResolveCommonPath to allowed directories (or document that arbitrary paths are intentional)
3. Fix EvaluatePowerOperator to handle parenthesized bases, or update the [Description] to not advertise 2^3 syntax
4. Clear textBuilder when a FunctionCallContent is received to prevent cross-tool text bleed

Review generated by multi-model consensus (2× Claude Opus 4.6, Claude Sonnet 4.6, Gemini 3 Pro, GPT-5.3-Codex)

[kubaflo]

Reviewed by Claude Opus 4.6, Gemini 3 Pro, and GPT-5.2 Codex. Only substantive bugs, security issues, and logic errors are reported below.

---

File: ChatViewModel. Send()+ClearChat()cs ###
Flagged by: Opus

ClearChat has no CanExecute guard, so it can run mid-stream. When Messages.Clear() executes during the await foreach loop, subsequent Messages[^1] access throws IndexOutOfRangeException. The catch block also uses Messages[^1], so the error handler itself double fault, app terminates.crashes

Suggested fix: Add CanExecute = nameof(CanClearChat) returning !IsProcessing, or guard Messages[^1] accesses with bounds checks in both the streaming loop and catch block.

---

File: MauiProgram. #else branchcs ###
Flagged by: Gemini, Codex

The #else branch throws PlatformNotSupportedException unconditionally, but the .csproj still targets net10.0-android and net10.0-windows. Users building for those platforms get a fatal crash with no UI feedback.

Suggested fix: Either remove the unsupported TFMs from the .csproj, or replace the throw with a graceful fallback (register a no-op IChatClient, show an "unsupported platform" message in the UI).

---

File: ChatViewModel. streaming loopcs ###
Flagged by: Gemini

more text, all text is merged into a single bubble. Combined with OnToolInvoking inserting at Count - 1, the pre-tool and post-tool text appear as one combined message after the tool indicator, breaking chronological order.

Suggested fix: When a FunctionCallContent is encountered, finalize the current text bubble, reset textBuilder, and start a fresh TextMessageViewModel for any subsequent text.

---

File: ChatViewModel. InvokeCoreAsync catch blockcs ###
Flagged by: Opus

The manual JSON construction only escapes " and \n:

result = $"{{\"error\": \"{ex.GetType().Name}: {ex.Message.Replace("\"", "\\\"").Replace("\n", " ")}\"}}";

Backslashes, \r, \t, and control characters in exception messages produce invalid JSON. This garbles the tool result sent back to the LLM and can cause JsonDocument.Parse() to fail in BuildRawJson.

Suggested fix: Use JsonSerializer.Serialize(new { error = $"{ex.GetType().Name}: {ex.Message}" }) for proper escaping.

---

Summary

Finding	Severity	Models
Incomplete JSON escaping in error path		Text accumulation across tool boundaries
Overall this is a well-structured sample with good AOT-safe JSON serialization, proper IHttpClientFactory usage, ConcurrentDictionary for timer thread safety, and clean MVVM architecture. The issues above are the only high-signal findings across all three models.