Skip to content
Merged
Show file tree
Hide file tree
Changes from 33 commits
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
92eca2c
Explicit EP download and registration
baijumeswani Mar 27, 2026
20912a3
Address pull-request review comments
baijumeswani Mar 28, 2026
3434e77
Give a model name by default
baijumeswani Mar 28, 2026
2bcdf7b
More review comments
baijumeswani Mar 28, 2026
ab7bf01
Copilot review feedback
baijumeswani Mar 31, 2026
e33e2f0
Update packages
baijumeswani Mar 31, 2026
3ad6368
Merge branch 'main' of https://github.com/microsoft/Foundry-Local int…
baijumeswani Mar 31, 2026
2e0aece
Update rust format
baijumeswani Mar 31, 2026
2248e6c
Revert model info caching
baijumeswani Mar 31, 2026
b1c8812
Invalidate cache on download and register eps
baijumeswani Mar 31, 2026
ee80da5
Merge branch 'main' of https://github.com/microsoft/Foundry-Local int…
baijumeswani Mar 31, 2026
011d6e3
Add per-EP progress callbacks to all SDKs and samples
bmehta001 Mar 31, 2026
eb8196d
Update READMEs and regenerate API docs for EP progress callbacks
bmehta001 Mar 31, 2026
449bed0
Update Core package versions to 0.9.0-dev-20260331T004032-a768b6a
bmehta001 Mar 31, 2026
b92986e
Address Copilot PR review feedback
bmehta001 Mar 31, 2026
af1a953
Merge remote-tracking branch 'origin/main' into bhamehta/per-ep-progr…
bmehta001 Mar 31, 2026
8ec354b
Fix duplicate JS method, regenerate docs, fix XML doc warning
bmehta001 Mar 31, 2026
a2abc1f
Add curly braces to single-line if/for statements
bmehta001 Mar 31, 2026
4df9814
Remove parallel-download-test from tracking
bmehta001 Mar 31, 2026
b94993a
Restore dropped TODO comment and blank line in Catalog.cs
bmehta001 Mar 31, 2026
7832d09
Fix review issues: return EpDownloadResult from progress overloads
bmehta001 Mar 31, 2026
6fe6c25
Remove parallel-download-test from tracking (again)
bmehta001 Mar 31, 2026
066adca
Fix consistency issues across Python and JS SDKs
bmehta001 Mar 31, 2026
e4c639d
Remove parallel-download-test from tracking
bmehta001 Mar 31, 2026
94a6aa0
Clean up generated C# docs: remove inherited-member noise
bmehta001 Mar 31, 2026
1153614
Unify EP download into single async method with optional callback
bmehta001 Apr 1, 2026
6d7feea
Remove unrelated generated doc files
bmehta001 Apr 1, 2026
774afa0
Fix Rust formatting: extra space in catalog.rs
bmehta001 Apr 1, 2026
cd01eee
Fix Python docstring: remove stale 'blocking' wording
bmehta001 Apr 1, 2026
58ecbb7
Update sdk/cs/docs/api/microsoft.ai.foundry.local.icatalog.md
bmehta001 Apr 1, 2026
7b1ae67
Fix PR review comments: async JS tests and typo fix
bmehta001 Apr 1, 2026
b8ce509
Merge remote-tracking branch 'origin/main' into bhamehta/per-ep-progr…
bmehta001 Apr 1, 2026
d58217d
Add overloads for EP download APIs, revert Python __init__.py
bmehta001 Apr 1, 2026
40dade6
Revert version references to match main
bmehta001 Apr 1, 2026
f622c45
Fix Rust formatting: split .await to separate line
bmehta001 Apr 1, 2026
fbe2a99
Fix LiveAudioTranscription sample: replace removed EnsureEpsDownloade…
bmehta001 Apr 1, 2026
f72f3ae
Regenerate C# and JS SDK docs for overload signatures
bmehta001 Apr 1, 2026
a51ef0e
Merge origin/main into bhamehta/per-ep-progress-v2
bmehta001 Apr 1, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions samples/cs/Directory.Packages.props
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,8 @@
<OnnxRuntimeVersion>1.23.2</OnnxRuntimeVersion>
</PropertyGroup>
<ItemGroup>
<PackageVersion Include="Microsoft.AI.Foundry.Local" Version="0.9.0-dev" />
<PackageVersion Include="Microsoft.AI.Foundry.Local.WinML" Version="0.9.0-dev-20260324" />
<PackageVersion Include="Microsoft.AI.Foundry.Local" Version="0.9.0-dev-20260401T031220-dab355a0" />
<PackageVersion Include="Microsoft.AI.Foundry.Local.WinML" Version="0.9.0-dev-20260401T031220-dab355a0" />
<PackageVersion Include="Betalgo.Ranul.OpenAI" Version="9.1.1" />
<PackageVersion Include="Microsoft.Extensions.Logging" Version="9.0.10" />
<PackageVersion Include="Microsoft.Extensions.Logging.Console" Version="9.0.10" />
Expand Down
3 changes: 2 additions & 1 deletion samples/cs/audio-transcription-example/Program.cs
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@
// EP packages include dependencies and may be large.
// Download is only required again if a new version of the EP is released.
// For cross platform builds there is no dynamic EP download and this will return immediately.
await Utils.RunWithSpinner("Registering execution providers", mgr.EnsureEpsDownloadedAsync());
await Utils.RunWithSpinner("Registering execution providers", mgr.DownloadAndRegisterEpsAsync());
// </init>


Expand Down Expand Up @@ -56,6 +56,7 @@ await model.DownloadAsync(progress =>
// <transcription>
// Get an audio client
var audioClient = await model.GetAudioClientAsync();
audioClient.Settings.Language = "en";

// Get a transcription with streaming outputs
var audioFile = args.Length > 0 ? args[0] : Path.Combine(AppContext.BaseDirectory, "Recording.mp3");
Expand Down
2 changes: 1 addition & 1 deletion samples/cs/foundry-local-web-server/Program.cs
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@
// EP packages include dependencies and may be large.
// Download is only required again if a new version of the EP is released.
// For cross platform builds there is no dynamic EP download and this will return immediately.
await Utils.RunWithSpinner("Registering execution providers", mgr.EnsureEpsDownloadedAsync());
await Utils.RunWithSpinner("Registering execution providers", mgr.DownloadAndRegisterEpsAsync());
// </init>


Expand Down
7 changes: 2 additions & 5 deletions samples/cs/model-management-example/Program.cs
Original file line number Diff line number Diff line change
Expand Up @@ -16,11 +16,8 @@
var mgr = FoundryLocalManager.Instance;


// Ensure that any Execution Provider (EP) downloads run and are completed.
// EP packages include dependencies and may be large.
// Download is only required again if a new version of the EP is released.
// For cross platform builds there is no dynamic EP download and this will return immediately.
await Utils.RunWithSpinner("Registering execution providers", mgr.EnsureEpsDownloadedAsync());
// Download and register all execution providers.
await Utils.RunWithSpinner("Registering execution providers", mgr.DownloadAndRegisterEpsAsync());


// Model catalog operations
Expand Down
36 changes: 34 additions & 2 deletions samples/cs/native-chat-completions/Program.cs
Original file line number Diff line number Diff line change
Expand Up @@ -19,11 +19,43 @@
var mgr = FoundryLocalManager.Instance;


// Ensure that any Execution Provider (EP) downloads run and are completed.
// Discover available execution providers and their registration status.
var eps = mgr.DiscoverEps();
Console.WriteLine("Available execution providers:");
foreach (var ep in eps)
{
Console.WriteLine($" {ep.Name} (registered: {ep.IsRegistered})");
}

// Download and register all execution providers with per-EP progress.
// EP packages include dependencies and may be large.
// Download is only required again if a new version of the EP is released.
// For cross platform builds there is no dynamic EP download and this will return immediately.
await Utils.RunWithSpinner("Registering execution providers", mgr.EnsureEpsDownloadedAsync());
if (eps.Length > 0)
{
int maxNameLen = eps.Max(e => e.Name.Length);
string currentEp = "";
await mgr.DownloadAndRegisterEpsAsync((epName, percent) =>
{
if (epName != currentEp)
{
if (currentEp != "")
{
Console.WriteLine();
}
currentEp = epName;
}
Console.Write($"\r {epName.PadRight(maxNameLen)} {percent,6:F1}%");
if (percent >= 100)
{
Console.WriteLine();
}
});
}
else
{
Console.WriteLine("No execution providers to download.");
}
// </init>


Expand Down
2 changes: 1 addition & 1 deletion samples/cs/tool-calling-foundry-local-sdk/Program.cs
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@
// EP packages include dependencies and may be large.
// Download is only required again if a new version of the EP is released.
// For cross platform builds there is no dynamic EP download and this will return immediately.
await Utils.RunWithSpinner("Registering execution providers", mgr.EnsureEpsDownloadedAsync());
await Utils.RunWithSpinner("Registering execution providers", mgr.DownloadAndRegisterEpsAsync());

Check failure on line 29 in samples/cs/tool-calling-foundry-local-sdk/Program.cs

View workflow job for this annotation

GitHub Actions / cs-samples (macos)

'FoundryLocalManager' does not contain a definition for 'DownloadAndRegisterEpsAsync' and no accessible extension method 'DownloadAndRegisterEpsAsync' accepting a first argument of type 'FoundryLocalManager' could be found (are you missing a using directive or an assembly reference?)
// </init>


Expand Down Expand Up @@ -59,7 +59,7 @@

// Get a chat client
var chatClient = await model.GetChatClientAsync();
chatClient.Settings.ToolChoice = ToolChoice.Required; // Force the model to make a tool call

Check failure on line 62 in samples/cs/tool-calling-foundry-local-sdk/Program.cs

View workflow job for this annotation

GitHub Actions / cs-samples (macos)

'OpenAIChatClient.ChatSettings' does not contain a definition for 'ToolChoice' and no accessible extension method 'ToolChoice' accepting a first argument of type 'OpenAIChatClient.ChatSettings' could be found (are you missing a using directive or an assembly reference?)


// Prepare messages
Expand Down
7 changes: 2 additions & 5 deletions samples/cs/tool-calling-foundry-local-web-server/Program.cs
Original file line number Diff line number Diff line change
Expand Up @@ -21,11 +21,8 @@
var mgr = FoundryLocalManager.Instance;


// Ensure that any Execution Provider (EP) downloads run and are completed.
// EP packages include dependencies and may be large.
// Download is only required again if a new version of the EP is released.
// For cross platform builds there is no dynamic EP download and this will return immediately.
await Utils.RunWithSpinner("Registering execution providers", mgr.EnsureEpsDownloadedAsync());
// Download and register all execution providers.
await Utils.RunWithSpinner("Registering execution providers", mgr.DownloadAndRegisterEpsAsync());

Check failure on line 25 in samples/cs/tool-calling-foundry-local-web-server/Program.cs

View workflow job for this annotation

GitHub Actions / cs-samples (macos)

'FoundryLocalManager' does not contain a definition for 'DownloadAndRegisterEpsAsync' and no accessible extension method 'DownloadAndRegisterEpsAsync' accepting a first argument of type 'FoundryLocalManager' could be found (are you missing a using directive or an assembly reference?)

Check failure on line 25 in samples/cs/tool-calling-foundry-local-web-server/Program.cs

View workflow job for this annotation

GitHub Actions / cs-samples (macos)

'FoundryLocalManager' does not contain a definition for 'DownloadAndRegisterEpsAsync' and no accessible extension method 'DownloadAndRegisterEpsAsync' accepting a first argument of type 'FoundryLocalManager' could be found (are you missing a using directive or an assembly reference?)


// Get the model catalog
Expand Down
29 changes: 29 additions & 0 deletions samples/js/native-chat-completions/app.js
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,35 @@ const manager = FoundryLocalManager.create({
// </init>
console.log('✓ SDK initialized successfully');

// Discover available execution providers and their registration status.
const eps = manager.discoverEps();
console.log('\nAvailable execution providers:');
for (const ep of eps) {
console.log(` ${ep.name} (registered: ${ep.isRegistered})`);
}

// Download and register all execution providers with per-EP progress.
// EP packages include dependencies and may be large.
// Download is only required again if a new version of the EP is released.
if (eps.length > 0) {
const maxNameLen = Math.max(...eps.map(e => e.name.length));
let currentEp = '';
await manager.downloadAndRegisterEps((epName, percent) => {
if (epName !== currentEp) {
if (currentEp !== '') {
process.stdout.write('\n');
}
currentEp = epName;
}
process.stdout.write(`\r ${epName.padEnd(maxNameLen)} ${percent.toFixed(1).padStart(5)}%`);
if (percent >= 100) {
process.stdout.write('\n');
}
});
} else {
console.log('No execution providers to download.');
}

// <model_setup>
// Get the model object
const modelAlias = 'qwen2.5-0.5b'; // Using an available model from the list above
Expand Down
49 changes: 44 additions & 5 deletions sdk/cs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,21 +48,60 @@ dotnet build src/Microsoft.AI.Foundry.Local.csproj /p:UseWinML=true

### Triggering EP download

EP download can be time-consuming. Call `DownloadAndRegisterEpsAsync` early (after initialization) to separate the download step from catalog access:
EP management is explicit via two methods:

- **`DiscoverEps()`** — returns an array of `EpInfo` describing each available EP and whether it is already registered.
- **`DownloadAndRegisterEpsAsync(names?, progressCallback?, ct?)`** — downloads and registers the specified EPs (or all available EPs if no names are given). Returns an `EpDownloadResult`. Overloads are provided so you can pass just a callback without specifying names.

```csharp
// Initialize the manager first (see Quick Start)
await FoundryLocalManager.CreateAsync(
new Configuration { AppName = "my-app" },
NullLogger.Instance);

await FoundryLocalManager.Instance.DownloadAndRegisterEpsAsync();
var mgr = FoundryLocalManager.Instance;

// Now catalog access won't trigger an EP download
var catalog = await FoundryLocalManager.Instance.GetCatalogAsync();
// Discover what EPs are available
var eps = mgr.DiscoverEps();
foreach (var ep in eps)
{
Console.WriteLine($"{ep.Name} — registered: {ep.IsRegistered}");
}

// Download and register all EPs
var result = await mgr.DownloadAndRegisterEpsAsync();
Console.WriteLine($"Success: {result.Success}, Status: {result.Status}");

// Or download only specific EPs
var result2 = await mgr.DownloadAndRegisterEpsAsync(new[] { eps[0].Name });
```

#### Per-EP download progress

Pass an optional `Action<string, double>` callback to receive `(epName, percent)` updates
as each EP downloads (`percent` is 0–100):

```csharp
string currentEp = "";
await mgr.DownloadAndRegisterEpsAsync((epName, percent) =>
{
if (epName != currentEp)
{
if (currentEp != "")
{
Console.WriteLine();
}
currentEp = epName;
}
Console.Write($"\r {epName} {percent,6:F1}%");
if (percent >= 100)
{
Console.WriteLine();
}
});
```

If you skip this step, EPs are downloaded automatically the first time you access the catalog. Once cached, subsequent calls are fast.
Catalog access no longer blocks on EP downloads. Call `DownloadAndRegisterEpsAsync` explicitly when you need hardware-accelerated execution providers.

## Quick Start

Expand Down
14 changes: 12 additions & 2 deletions sdk/cs/docs/api/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,10 @@

[DeviceType](./microsoft.ai.foundry.local.devicetype.md)

[EpDownloadResult](./microsoft.ai.foundry.local.epdownloadresult.md)

[EpInfo](./microsoft.ai.foundry.local.epinfo.md)

[FoundryLocalException](./microsoft.ai.foundry.local.foundrylocalexception.md)

[FoundryLocalManager](./microsoft.ai.foundry.local.foundrylocalmanager.md)
Expand All @@ -22,8 +26,6 @@

[ModelSettings](./microsoft.ai.foundry.local.modelsettings.md)

[ModelVariant](./microsoft.ai.foundry.local.modelvariant.md)

[OpenAIAudioClient](./microsoft.ai.foundry.local.openaiaudioclient.md)

[OpenAIChatClient](./microsoft.ai.foundry.local.openaichatclient.md)
Expand All @@ -39,3 +41,11 @@
[AsyncLock](./microsoft.ai.foundry.local.detail.asynclock.md)

[CoreInteropRequest](./microsoft.ai.foundry.local.detail.coreinteroprequest.md)

## Microsoft.AI.Foundry.Local.OpenAI

[LiveAudioTranscriptionResponse](./microsoft.ai.foundry.local.openai.liveaudiotranscriptionresponse.md)

[LiveAudioTranscriptionSession](./microsoft.ai.foundry.local.openai.liveaudiotranscriptionsession.md)

[ResponseFormatExtended](./microsoft.ai.foundry.local.openai.responseformatextended.md)
63 changes: 63 additions & 0 deletions sdk/cs/docs/api/microsoft.ai.foundry.local.epdownloadresult.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
# EpDownloadResult

Namespace: Microsoft.AI.Foundry.Local

Result of an explicit EP download and registration operation.

```csharp
public record EpDownloadResult : System.IEquatable`1[[Microsoft.AI.Foundry.Local.EpDownloadResult, Microsoft.AI.Foundry.Local, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null]]
```

## Properties

### Property Value

[Type](https://docs.microsoft.com/en-us/dotnet/api/system.type)<br>

### **Success**

True if all requested EPs were successfully downloaded and registered.

```csharp
public bool Success { get; set; }
```

#### Property Value

[Boolean](https://docs.microsoft.com/en-us/dotnet/api/system.boolean)<br>

### **Status**

Human-readable status message.

```csharp
public string Status { get; set; }
```

#### Property Value

[String](https://docs.microsoft.com/en-us/dotnet/api/system.string)<br>

### **RegisteredEps**

Names of EPs that were successfully registered.

```csharp
public String[] RegisteredEps { get; set; }
```

#### Property Value

[String[]](https://docs.microsoft.com/en-us/dotnet/api/system.string)<br>

### **FailedEps**

Names of EPs that failed to register.

```csharp
public String[] FailedEps { get; set; }
```

#### Property Value

[String[]](https://docs.microsoft.com/en-us/dotnet/api/system.string)<br>
39 changes: 39 additions & 0 deletions sdk/cs/docs/api/microsoft.ai.foundry.local.epinfo.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
# EpInfo

Namespace: Microsoft.AI.Foundry.Local

Describes a discoverable execution provider bootstrapper.

```csharp
public record EpInfo : System.IEquatable`1[[Microsoft.AI.Foundry.Local.EpInfo, Microsoft.AI.Foundry.Local, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null]]
```

## Properties

### Property Value

[Type](https://docs.microsoft.com/en-us/dotnet/api/system.type)<br>

### **Name**

The identifier of the bootstrapper/execution provider (e.g. "CUDAExecutionProvider").

```csharp
public string Name { get; set; }
```

#### Property Value

[String](https://docs.microsoft.com/en-us/dotnet/api/system.string)<br>

### **IsRegistered**

True if this EP has already been successfully downloaded and registered.

```csharp
public bool IsRegistered { get; set; }
```

#### Property Value

[Boolean](https://docs.microsoft.com/en-us/dotnet/api/system.boolean)<br>
Loading
Loading