Namespace: Microsoft.AI.Foundry.Local
public class FoundryLocalManager : System.IDisposableInheritance Object → FoundryLocalManager
Implements IDisposable
Attributes NullableContextAttribute, NullableAttribute
public static bool IsInitialized { get; }public static FoundryLocalManager Instance { get; }Bound Urls if the web service has been started. Null otherwise. See FoundryLocalManager.StartWebServiceAsync(Nullable<CancellationToken>).
public String[] Urls { get; private set; }Create the FoundryLocalManager singleton instance.
public static Task CreateAsync(Configuration configuration, ILogger logger, Nullable<CancellationToken> ct)configuration Configuration
Configuration to use.
logger ILogger
Application logger to use.
Use Microsoft.Extensions.Logging.NullLogger.Instance if you wish to ignore log output from the SDK.
ct Nullable<CancellationToken>
Optional cancellation token for the initialization.
Task
Task creating the instance.
Get the model catalog instance.
public Task<ICatalog> GetCatalogAsync(Nullable<CancellationToken> ct)ct Nullable<CancellationToken>
Optional cancellation token.
Task<ICatalog>
The model catalog.
Remarks:
The catalog is populated on first use and returns models based on currently available execution providers. To ensure all hardware-accelerated models are listed, call FoundryLocalManager.DownloadAndRegisterEpsAsync(IEnumerable<String>, Action<String, Double>, Nullable<CancellationToken>) first to register execution providers, then access the catalog.
Start the optional web service. This will provide an OpenAI-compatible REST endpoint that supports /v1/chat_completions /v1/models to list downloaded models /v1/models/{model_id} to get model details
FoundryLocalManager.Urls is populated with the actual bound Urls after startup.
public Task StartWebServiceAsync(Nullable<CancellationToken> ct)ct Nullable<CancellationToken>
Optional cancellation token.
Task
Task starting the web service.
Stops the web service if started.
public Task StopWebServiceAsync(Nullable<CancellationToken> ct)ct Nullable<CancellationToken>
Optional cancellation token.
Task
Task stopping the web service.
Discovers all available execution provider bootstrappers. Returns metadata about each EP including whether it is already registered.
public EpInfo[] DiscoverEps()EpInfo[]
Array of EP bootstrapper info describing available EPs.
DownloadAndRegisterEpsAsync(IEnumerable<String>, Action<String, Double>, Nullable<CancellationToken>)
Downloads and registers execution providers.
public Task<EpDownloadResult> DownloadAndRegisterEpsAsync(IEnumerable<string> names, Action<string, double> progressCallback, Nullable<CancellationToken> ct)names IEnumerable<String>
Optional subset of EP bootstrapper names to download (as returned by FoundryLocalManager.DiscoverEps()).
If null or empty, all discoverable EPs are downloaded.
progressCallback Action<String, Double>
Optional callback invoked as each EP downloads. Parameters are (epName, percentComplete) where percentComplete is 0-100.
ct Nullable<CancellationToken>
Optional cancellation token.
Task<EpDownloadResult>
Result describing which EPs succeeded and which failed.
Remarks:
Catalog and model requests use whatever EPs are currently registered and do not block on EP downloads. After downloading new EPs, re-fetch the model catalog to include models requiring the newly registered EPs.
protected void Dispose(bool disposing)disposing Boolean
public void Dispose()