MicrosoftDocs
diff --git a/‎articles/ai-foundry/foundry-local/concepts/foundry-local-architecture.md
Lines changed: 132 additions & 0 deletions b/‎articles/ai-foundry/foundry-local/concepts/foundry-local-architecture.md
Lines changed: 132 additions & 0 deletions
diff --git a/‎articles/ai-foundry/foundry-local/get-started.md
Lines changed: 104 additions & 0 deletions b/‎articles/ai-foundry/foundry-local/get-started.md
Lines changed: 104 additions & 0 deletions
diff --git a/‎articles/ai-foundry/foundry-local/how-to/how-to-chat-application-with-open-web-ui.md
Lines changed: 59 additions & 0 deletions b/‎articles/ai-foundry/foundry-local/how-to/how-to-chat-application-with-open-web-ui.md
Lines changed: 59 additions & 0 deletions
@@ -0,0 +1,132 @@
+---
+title: Foundry Local architecture
+titleSuffix: Foundry Local
+description: Learn about the architecture and components of Foundry Local
+manager: scottpolly
+ms.service: azure-ai-foundry
+ms.custom: build-2025
+ms.topic: concept-article
+ms.date: 02/12/2025
+ms.author: samkemp
+author: samuel100
+---
+
+# Foundry Local architecture
+
+Foundry Local enables efficient, secure, and scalable AI model inference directly on your devices. This article explains the core components of Foundry Local and how they work together to deliver AI capabilities.
+
+Key benefits of Foundry Local include:
+
+> [!div class="checklist"]
+>
+> - **Low Latency**: Run models locally to minimize processing time and deliver faster results.
+> - **Data Privacy**: Process sensitive data locally without sending it to the cloud, helping meet data protection requirements.
+> - **Flexibility**: Support for diverse hardware configurations lets you choose the optimal setup for your needs.
+> - **Scalability**: Deploy across various devices, from laptops to servers, to suit different use cases.
+> - **Cost-Effectiveness**: Reduce cloud computing costs, especially for high-volume applications.
+> - **Offline Operation**: Work without an internet connection in remote or disconnected environments.
+> - **Seamless Integration**: Easily incorporate into existing development workflows for smooth adoption.
+
+## Key components
+
+The Foundry Local architecture consists of these main components:
+
+:::image type="content" source="../media/architecture/foundry-local-arch.png" alt-text="Diagram of Foundry Local Architecture.":::
+
+### Foundry Local service
+
+The Foundry Local Service includes an OpenAI-compatible REST server that provides a standard interface for working with the inference engine. It's also possible to manage models over REST. Developers use this API to send requests, run models, and get results programmatically.
+
+- **Endpoint**: The endpoint is *dynamically allocated* when the service starts. You can find the endpoint by running the `foundry service status` command. When using Foundry Local in your applications, we recommend using the SDK that automatically handles the endpoint for you. For more details on how to use the Foundry Local SDK, read the [Integrated inferencing SDKs with Foundry Local](../how-to/how-to-integrate-with-inference-sdks.md) article.
+- **Use Cases**:
+  - Connect Foundry Local to your custom applications
+  - Execute models through HTTP requests
+
+### ONNX runtime
+
+The ONNX Runtime is a core component that executes AI models. It runs optimized ONNX models efficiently on local hardware like CPUs, GPUs, or NPUs.
+
+**Features**:
+
+- Works with multiple hardware providers (NVIDIA, AMD, Intel, Qualcomm) and device types (NPUs, CPUs, GPUs)
+- Offers a consistent interface for running across models different hardware
+- Delivers best-in-class performance
+- Supports quantized models for faster inference
+
+### Model management
+
+Foundry Local provides robust tools for managing AI models, ensuring that they're readily available for inference and easy to maintain. Model management is handled through the **Model Cache** and the **Command-Line Interface (CLI)**.
+
+#### Model cache
+
+The model cache stores downloaded AI models locally on your device, which ensures models are ready for inference without needing to download them repeatedly. You can manage the cache using either the Foundry CLI or REST API.
+
+- **Purpose**: Speeds up inference by keeping models locally available
+- **Key Commands**:
+  - `foundry cache list`: Shows all models in your local cache
+  - `foundry cache remove <model-name>`: Removes a specific model from the cache
+  - `foundry cache cd <path>`: Changes the storage location for cached models
+
+#### Model lifecycle
+
+1. **Download**: Download models from the Azure AI Foundry model catalog and save them to your local disk.
+2. **Load**: Load models into the Foundry Local service memory for inference. Set a TTL (time-to-live) to control how long the model stays in memory (default: 10 minutes).
+3. **Run**: Execute model inference for your requests.
+4. **Unload**: Remove models from memory to free up resources when no longer needed.
+5. **Delete**: Remove models from your local cache to reclaim disk space.
+
+#### Model compilation using Olive
+
+Before models can be used with Foundry Local, they must be compiled and optimized in the [ONNX](https://onnx.ai) format. Microsoft provides a selection of published models in the Azure AI Foundry Model Catalog that are already optimized for Foundry Local. However, you aren't limited to those models - by using [Olive](https://microsoft.github.io/Olive/). Olive is a powerful framework for preparing AI models for efficient inference. It converts models into the ONNX format, optimizes their graph structure, and applies techniques like quantization to improve performance on local hardware.
+
+> [!TIP]
+> To learn more about compiling models for Foundry Local, read [How to compile Hugging Face models to run on Foundry Local](../how-to/how-to-compile-hugging-face-models.md).
+
+### Hardware abstraction layer
+
+The hardware abstraction layer ensures that Foundry Local can run on various devices by abstracting the underlying hardware. To optimize performance based on the available hardware, Foundry Local supports:
+
+- **multiple _execution providers_**, such as NVIDIA CUDA, AMD, Qualcomm, Intel.
+- **multiple _device types_**, such as CPU, GPU, NPU.
+
+### Developer experiences
+
+The Foundry Local architecture is designed to provide a seamless developer experience, enabling easy integration and interaction with AI models.
+Developers can choose from various interfaces to interact with the system, including:
+
+#### Command-Line Interface (CLI)
+
+The Foundry CLI is a powerful tool for managing models, the inference engine, and the local cache.
+
+**Examples**:
+
+- `foundry model list`: Lists all available models in the local cache.
+- `foundry model run <model-name>`: Runs a model.
+- `foundry service status`: Checks the status of the service.
+
+> [!TIP]
+> To learn more about the CLI commands, read [Foundry Local CLI Reference](../reference/reference-cli.md).
+
+#### Inferencing SDK integration
+
+Foundry Local supports integration with various SDKs, such as the OpenAI SDK, enabling developers to use familiar programming interfaces to interact with the local inference engine.
+
+- **Supported SDKs**: Python, JavaScript, C#, and more.
+
+> [!TIP]
+> To learn more about integrating with inferencing SDKs, read [Integrate inferencing SDKs with Foundry Local](../how-to/how-to-integrate-with-inference-sdks.md).
+
+#### AI Toolkit for Visual Studio Code
+
+The AI Toolkit for Visual Studio Code provides a user-friendly interface for developers to interact with Foundry Local. It allows users to run models, manage the local cache, and visualize results directly within the IDE.
+
+- **Features**:
+  - Model management: Download, load, and run models from within the IDE.
+  - Interactive console: Send requests and view responses in real-time.
+  - Visualization tools: Graphical representation of model performance and results.
+
+## Next Steps
+
+- [Get started with Foundry Local](../get-started.md)
+- [Integrate inferencing SDKs with Foundry Local](../how-to/how-to-integrate-with-inference-sdks.md)
+- [Foundry Local CLI Reference](../reference/reference-cli.md)
@@ -0,0 +1,104 @@
+---
+title: Get started with Foundry Local
+titleSuffix: Foundry Local
+description: Learn how to install, configure, and run your first AI model with Foundry Local
+manager: scottpolly
+keywords: Azure AI services, cognitive, AI models, local inference
+ms.service: azure-ai-foundry
+ms.topic: quickstart
+ms.date: 02/20/2025
+ms.reviewer: samkemp
+ms.author: samkemp
+author: samuel100
+ms.custom: build-2025
+#customer intent: As a developer, I want to get started with Foundry Local so that I can run AI models locally.
+---
+
+# Get started with Foundry Local
+
+This guide walks you through setting up Foundry Local to run AI models on your device. 
+
+## Prerequisites
+
+Your system must meet the following requirements to run Foundry Local:
+
+- **Operating System**: Windows 10 (x64), Windows 11 (x64/ARM), macOS.
+- **Hardware**: Minimum 8GB RAM, 3GB free disk space. Recommended 16GB RAM, 15GB free disk space.
+- **Network**: Internet connection for initial model download (optional for offline use)
+- **Acceleration (optional)**: NVIDIA GPU (2,000 series or newer), AMD GPU (6,000 series or newer), Qualcomm Snapdragon X Elite (8GB or more of memory), or Apple silicon.
+
+Also, ensure you have administrative privileges to install software on your device.
+
+## Quickstart
+
+Get started with Foundry Local quickly:
+
+1. [**Download Foundry Local Installer**](https://aka.ms/foundry-local-installer) and **install** by following the on-screen prompts. 
+    > [!TIP]
+    > If you're installing on Windows, you can also use `winget` to install Foundry Local. Open a terminal window and run the following command:
+    >
+    > ```powershell
+    > winget install Microsoft.FoundryLocal
+    > ```
+1. **Run your first model** Open a terminal window and run the following command to run a model: 
+
+    ```bash
+    foundry model run deepseek-r1-1.5b 
+    ```
+    
+    The model downloads - which can take a few minutes, depending on your internet speed - and the model runs. Once the model is running, you can interact with it using the command line interface (CLI). For example, you can ask:
+
+    ```text
+    Why is the sky blue?
+    ```
+
+    You should see a response from the model in the terminal:
+    :::image type="content" source="media/get-started-output.png" alt-text="Screenshot of output from foundry local run command." lightbox="media/get-started-output.png":::
+
+
+> [!TIP]
+> You can replace `deepseek-r1-1.5b` with any model name from the catalog (see `foundry model list` for available models). Foundry Local downloads the model variant that best matches your system's hardware and software configuration. For example, if you have an NVIDIA GPU, it downloads the CUDA version of the model. If you have a Qualcomm NPU, it downloads the NPU variant. If you have no GPU or NPU, it downloads the CPU version.
+
+## Explore commands
+
+The Foundry CLI organizes commands into these main categories:
+
+- **Model**: Commands for managing and running models.
+- **Service**: Commands for managing the Foundry Local service.
+- **Cache**: Commands for managing the local model cache (downloaded models on local disk).
+
+View all available commands with:
+
+```bash
+foundry --help
+```
+
+To view available **model** commands, run:
+
+```bash
+foundry model --help
+```
+To view available **service** commands, run:
+
+```bash
+foundry service --help
+```
+
+To view available **cache** commands, run:
+
+```bash
+foundry cache --help
+```
+
+> [!TIP]
+> For a complete guide to all CLI commands and their usage, see the [Foundry Local CLI Reference](reference/reference-cli.md).
+
+
+## Next steps
+
+- [Integrate inferencing SDKs with Foundry Local](how-to/how-to-integrate-with-inference-sdks.md)
+- [Explore the Foundry Local documentation](index.yml)
+- [Learn about best practices and troubleshooting](reference/reference-best-practice.md)
+- [Explore the Foundry Local API reference](reference/reference-catalog-api.md)
+- [Learn Compile Hugging Face models](how-to/how-to-compile-hugging-face-models.md)
+
@@ -0,0 +1,59 @@
+---
+title: Integrate Open Web UI with Foundry Local
+titleSuffix: Foundry Local
+description: Learn how to create a chat application using Foundry Local and Open Web UI
+manager: scottpolly
+keywords: Azure AI services, cognitive, AI models, local inference
+ms.service: azure-ai-foundry
+ms.topic: how-to
+ms.date: 02/20/2025
+ms.reviewer: samkemp
+ms.author: samkemp
+author: samuel100
+ms.custom: build-2025
+#customer intent: As a developer, I want to get started with Foundry Local so that I can run AI models locally.
+---
+
+# Integrate Open Web UI with Foundry Local
+
+This tutorial shows you how to create a chat application using Foundry Local and Open Web UI. When you finish, you have a working chat interface running entirely on your local device.
+
+## Prerequisites
+
+Before you start this tutorial, you need:
+
+- **Foundry Local** installed on your computer. Read the [Get started with Foundry Local](../get-started.md) guide for installation instructions.
+
+## Set up Open Web UI for chat
+
+1. **Install Open Web UI** by following the instructions from the [Open Web UI GitHub repository](https://github.com/open-webui/open-webui).
+
+2. **Launch Open Web UI** with this command in your terminal:
+
+   ```bash
+   open-webui serve
+   ```
+
+3. Open your web browser and go to [http://localhost:8080](http://localhost:8080).
+
+4. **Connect Open Web UI to Foundry Local**:
+
+   1. Select **Settings** in the navigation menu
+   2. Select **Connections**
+   3. Select **Manage Direct Connections**
+   4. Select the **+** icon to add a connection
+   5. For the **URL**, enter `http://localhost:PORT/v1` where `PORT` is replaced with the port of the Foundry Local endpoint, which you can find using the CLI command `foundry service status`. Note, that Foundry Local dynamically assigns a port, so it's not always the same.
+   6. Type any value (like `test`) for the API Key, since it can't be empty.
+   7. Save your connection
+
+5. **Start chatting with your model**:
+   1. Your loaded models appear in the dropdown at the top
+   2. Select any model from the list
+   3. Type your message in the input box at the bottom
+
+That's it! You're now chatting with an AI model running entirely on your local device.
+
+## Next steps
+
+- [Integrate inferencing SDKs with Foundry Local](how-to-integrate-with-inference-sdks.md)
+- [Compile Hugging Face models to run on Foundry Local](../how-to/how-to-compile-hugging-face-models.md)