Skip to content

Commit 43e1e38

Browse files
committed
review feedback addressed
1 parent 0bbab1c commit 43e1e38

17 files changed

+83
-31
lines changed

articles/ai-foundry-local/concepts/foundry-local-architecture.md

Lines changed: 14 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
---
2-
title: Foundry Local Architecture
2+
title: Foundry Local architecture
33
titleSuffix: Foundry Local
44
description: Learn about the architecture and components of Foundry Local
55
manager: scottpolly
@@ -11,7 +11,7 @@ ms.author: samkemp
1111
author: samuel100
1212
---
1313

14-
# Foundry Local Architecture
14+
# Foundry Local architecture
1515

1616
Foundry Local enables efficient, secure, and scalable AI model inference directly on your devices. This article explains the core components of Foundry Local and how they work together to deliver AI capabilities.
1717

@@ -27,13 +27,13 @@ Key benefits of Foundry Local include:
2727
> - **Offline Operation**: Work without an internet connection in remote or disconnected environments.
2828
> - **Seamless Integration**: Easily incorporate into existing development workflows for smooth adoption.
2929
30-
## Key Components
30+
## Key components
3131

3232
The Foundry Local architecture consists of these main components:
3333

34-
:::image type="content" source="../media/architecture/foundry-local-arch.png" alt-text="Foundry Local Architecture Diagram":::
34+
:::image type="content" source="../media/architecture/foundry-local-arch.png" alt-text="Diagram of Foundry Local Architecture":::
3535

36-
### Foundry Local Service
36+
### Foundry Local service
3737

3838
The Foundry Local Service is an OpenAI-compatible REST server that provides a standard interface for working with the inference engine and managing models. Developers use this API to send requests, run models, and get results programmatically.
3939

@@ -42,7 +42,7 @@ The Foundry Local Service is an OpenAI-compatible REST server that provides a st
4242
- Connect Foundry Local to your custom applications
4343
- Execute models through HTTP requests
4444

45-
### ONNX Runtime
45+
### ONNX runtime
4646

4747
The ONNX Runtime is a core component that executes AI models. It runs optimized ONNX models efficiently on local hardware like CPUs, GPUs, or NPUs.
4848

@@ -53,11 +53,11 @@ The ONNX Runtime is a core component that executes AI models. It runs optimized
5353
- Delivers best-in-class performance
5454
- Supports quantized models for faster inference
5555

56-
### Model Management
56+
### Model management
5757

5858
Foundry Local provides robust tools for managing AI models, ensuring that they're readily available for inference and easy to maintain. Model management is handled through the **Model Cache** and the **Command-Line Interface (CLI)**.
5959

60-
#### Model Cache
60+
#### Model cache
6161

6262
The model cache stores downloaded AI models locally on your device, which ensures models are ready for inference without needing to download them repeatedly. You can manage the cache using either the Foundry CLI or REST API.
6363

@@ -67,29 +67,29 @@ The model cache stores downloaded AI models locally on your device, which ensure
6767
- `foundry cache remove <model-name>`: Removes a specific model from the cache
6868
- `foundry cache cd <path>`: Changes the storage location for cached models
6969

70-
#### Model Lifecycle
70+
#### Model lifecycle
7171

7272
1. **Download**: Get models from the Azure AI Foundry model catalog and save them to your local disk.
7373
2. **Load**: Load models into the Foundry Local service memory for inference. Set a TTL (time-to-live) to control how long the model stays in memory (default: 10 minutes).
7474
3. **Run**: Execute model inference for your requests.
7575
4. **Unload**: Remove models from memory to free up resources when no longer needed.
7676
5. **Delete**: Remove models from your local cache to reclaim disk space.
7777

78-
#### Model Compilation using Olive
78+
#### Model compilation using Olive
7979

8080
Before models can be used with Foundry Local, they must be compiled and optimized in the [ONNX](https://onnx.ai) format. Microsoft provides a selection of published models in the Azure AI Foundry Model Catalog that are already optimized for Foundry Local. However, you aren't limited to those models - by using [Olive](https://microsoft.github.io/Olive/). Olive is a powerful framework for preparing AI models for efficient inference. It converts models into the ONNX format, optimizes their graph structure, and applies techniques like quantization to improve performance on local hardware.
8181

8282
> [!TIP]
83-
> To learn more about compiling models for Foundry Local, read [How to compile Hugging Face models to run on Foundry Local](../how-to/how-to-compile-hf-models.md).
83+
> To learn more about compiling models for Foundry Local, read [How to compile Hugging Face models to run on Foundry Local](../how-to/how-to-compile-huggingface-models.md).
8484
85-
### Hardware Abstraction Layer
85+
### Hardware abstraction layer
8686

8787
The hardware abstraction layer ensures that Foundry Local can run on various devices by abstracting the underlying hardware. To optimize performance based on the available hardware, Foundry Local supports:
8888

8989
- **multiple _execution providers_**, such as NVIDIA CUDA, AMD, Qualcomm, Intel.
9090
- **multiple _device types_**, such as CPU, GPU, NPU.
9191

92-
### Developer Experiences
92+
### Developer experiences
9393

9494
The Foundry Local architecture is designed to provide a seamless developer experience, enabling easy integration and interaction with AI models.
9595
Developers can choose from various interfaces to interact with the system, including:
@@ -107,7 +107,7 @@ The Foundry CLI is a powerful tool for managing models, the inference engine, an
107107
> [!TIP]
108108
> To learn more about the CLI commands, read [Foundry Local CLI Reference](../reference/reference-cli.md).
109109
110-
#### Inferencing SDK Integration
110+
#### Inferencing SDK integration
111111

112112
Foundry Local supports integration with various SDKs, such as the OpenAI SDK, enabling developers to use familiar programming interfaces to interact with the local inference engine.
113113

articles/ai-foundry-local/get-started.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -93,5 +93,5 @@ foundry cache --help
9393
- [Explore the Foundry Local documentation](index.yml)
9494
- [Learn about best practices and troubleshooting](reference/reference-best-practice.md)
9595
- [Explore the Foundry Local API reference](reference/reference-catalog-api.md)
96-
- [Learn how to compile Hugging Face models](how-to/how-to-compile-hf-models.md)
96+
- [Learn how to compile Hugging Face models](how-to/how-to-compile-huggingface-models.md)
9797
File renamed without changes.

articles/ai-foundry-local/how-to/integrate-with-inference-sdks.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
---
2-
title: Integrate with Inference SDKs
2+
title: Integrate with inference SDKs
33
titleSuffix: Foundry Local
44
description: This article provides instructions on how to integrate Foundry Local with common Inferencing SDKs.
55
manager: scottpolly
@@ -12,7 +12,7 @@ zone_pivot_groups: azure-ai-model-catalog-samples-chat
1212
author: samuel100
1313
---
1414

15-
# Integrate Foundry Local with Inferencing SDKs
15+
# Integrate Foundry Local with inferencing SDKs
1616

1717
Foundry Local provides a REST API endpoint that makes it easy to integrate with various inferencing SDKs and programming languages. This guide shows you how to connect your applications to locally running AI models using popular SDKs.
1818

@@ -48,5 +48,5 @@ When Foundry Local is running, it exposes an OpenAI-compatible REST API endpoint
4848

4949
## Next steps
5050

51-
- [How to compile Hugging Face models to run on Foundry Local](how-to-compile-hf-models.md)
51+
- [How to compile Hugging Face models to run on Foundry Local](how-to-compile-huggingface-models.md)
5252
- [Explore the Foundry Local CLI reference](../reference/reference-cli.md)

articles/ai-foundry-local/includes/integrate-examples/csharp.md

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,12 @@
1+
---
2+
ms.service: azure-ai-foundry
3+
ms.custom: build-2025
4+
ms.topic: reference
5+
ms.date: 05/02/2025
6+
ms.author: maanavdalal
7+
author: maanavd
8+
---
9+
110
## Basic Integration
211

312
```csharp

articles/ai-foundry-local/includes/integrate-examples/javascript.md

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,12 @@
1+
---
2+
ms.service: azure-ai-foundry
3+
ms.custom: build-2025
4+
ms.topic: reference
5+
ms.date: 05/02/2025
6+
ms.author: maanavdalal
7+
author: maanavd
8+
---
9+
110
## Using the OpenAI Node.js SDK
211

312
```javascript

articles/ai-foundry-local/includes/integrate-examples/python.md

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,12 @@
1+
---
2+
ms.service: azure-ai-foundry
3+
ms.custom: build-2025
4+
ms.topic: reference
5+
ms.date: 05/02/2025
6+
ms.author: maanavdalal
7+
author: maanavd
8+
---
9+
110
## Using the OpenAI SDK
211

312
```python

articles/ai-foundry-local/includes/integrate-examples/rest.md

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,12 @@
1+
---
2+
ms.service: azure-ai-foundry
3+
ms.custom: build-2025
4+
ms.topic: reference
5+
ms.date: 05/02/2025
6+
ms.author: maanavdalal
7+
author: maanavd
8+
---
9+
110
## Basic Request
211

312
For quick tests or integrations with command line scripts:

articles/ai-foundry-local/includes/sdk-examples/csharp.md

Lines changed: 0 additions & 3 deletions
This file was deleted.

articles/ai-foundry-local/includes/sdk-examples/javascript.md

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,13 @@
1+
---
2+
ms.service: azure-ai-foundry
3+
ms.custom: build-2025
4+
ms.topic: reference
5+
ms.date: 05/02/2025
6+
ms.author: maanavdalal
7+
author: maanavd
8+
---
9+
10+
111
## JavaScript SDK Reference
212

313
### Installation

0 commit comments

Comments
 (0)