Skip to content

Updated langchain-azure-ai usage #560

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
+47 −47
Open

Updated langchain-azure-ai usage #560

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
94 changes: 47 additions & 47 deletions articles/ai-foundry/how-to/develop/langchain.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
title: Develop application with LangChain and Azure AI Foundry
title: Develop applications with LangChain and Azure AI Foundry
titleSuffix: Azure AI Foundry
description: This article explains how to use LangChain with models deployed in Azure AI Foundry portal to build advance intelligent applications.
description: Learn how to use LangChain with models deployed in Azure AI Foundry to build advanced, intelligent applications.
ms.service: azure-ai-foundry
ms.custom:
- ignite-2024
Expand All @@ -15,31 +15,31 @@ author: sdgilley

# Develop applications with LangChain and Azure AI Foundry

LangChain is a development ecosystem that makes as easy possible for developers to build applications that reason. The ecosystem is composed by multiple components. Most of the them can be used by themselves, allowing you to pick and choose whichever components you like best.
LangChain is a developer ecosystem that makes it easier to build reasoning applications. It includes multiple components, and most of them can be used independently, allowing you to pick and choose the pieces you need.

Models deployed to [Azure AI Foundry](https://ai.azure.com/?cid=learnDocs) can be used with LangChain in two ways:

- **Using the Azure AI Model Inference API:** All models deployed to Azure AI Foundry support the [Model Inference API](../../../ai-foundry/model-inference/reference/reference-model-inference-api.md), which offers a common set of functionalities that can be used for most of the models in the catalog. The benefit of this API is that, since it's the same for all the models, changing from one to another is as simple as changing the model deployment being use. No further changes are required in the code. When working with LangChain, install the extensions `langchain-azure-ai`.
- **Use the Azure AI Model Inference API:** All models deployed in Azure AI Foundry support the [Model Inference API](../../../ai-foundry/model-inference/reference/reference-model-inference-api.md), which offers a common set of capabilities across most models in the catalog. Because the API is consistent, switching models is as simple as changing the deployment; no code changes are required. With LangChain, install the `langchain-azure-ai` integration.

- **Using the model's provider specific API:** Some models, like OpenAI, Cohere, or Mistral, offer their own set of APIs and extensions for LangChain. Those extensions might include specific functionalities that the model support and hence are suitable if you want to exploit them. When working with LangChain, install the extension specific for the model you want to use, like `langchain-openai` or `langchain-cohere`.
- **Use the model provider’s API:** Some models, such as OpenAI, Cohere, or Mistral, offer their own APIs and LangChain extensions. These extensions might include model-specific capabilities and are suitable if you need to use them. Install the extension for your chosen model, such as `langchain-openai` or `langchain-cohere`.

In this tutorial, you learn how to use the packages `langchain-azure-ai` to build applications with LangChain.
This tutorial shows how to use the `langchain-azure-ai` package with LangChain.

## Prerequisites

To run this tutorial, you need:

* [!INCLUDE [azure-subscription](../../includes/azure-subscription.md)]

* A model deployment supporting the [Model Inference API](https://aka.ms/azureai/modelinference) deployed. In this example, we use a `Mistral-Large-2411` deployment in the [Foundry Models](../../../ai-foundry/model-inference/overview.md).
* A model deployment that supports the [Model Inference API](https://aka.ms/azureai/modelinference). This article uses a `Mistral-Large-2411` deployment available in the [Azure AI Foundry model catalog](../../../ai-foundry/model-inference/overview.md).
* Python 3.9 or later installed, including pip.
* LangChain installed. You can do it with:
* LangChain installed. You can install it with:

```bash
pip install langchain
```

* In this example, we're working with the Model Inference API, hence we install the following packages:
* Install the Azure AI Foundry integration:

```bash
pip install -U langchain-azure-ai
Expand All @@ -49,7 +49,7 @@ To run this tutorial, you need:

[!INCLUDE [set-endpoint](../../includes/set-endpoint.md)]

Once configured, create a client to connect with the chat model by using the `init_chat_model`. For Azure OpenAI models, configure the client as indicated at [Using Azure OpenAI models](#using-azure-openai-models).
After configuration, create a client to connect to the chat model using `init_chat_model`. For Azure OpenAI models, see [Use Azure OpenAI models](#use-azure-openai-models).

```python
from langchain.chat_models import init_chat_model
Expand All @@ -63,7 +63,7 @@ You can also use the class `AzureAIChatCompletionsModel` directly.


> [!CAUTION]
> **Breaking change:** Parameter `model_name` was renamed `model` in version `0.1.3`.
> **Breaking change:** Parameter `model_name` was renamed to `model` in version `0.1.3`.

You can use the following code to create the client if your endpoint supports Microsoft Entra ID:

Expand All @@ -82,7 +82,7 @@ model = AzureAIChatCompletionsModel(
> [!NOTE]
> When using Microsoft Entra ID, make sure that the endpoint was deployed with that authentication method and that you have the required permissions to invoke it.

If you're planning to use asynchronous calling, it's a best practice to use the asynchronous version for the credentials:
If you plan to use asynchronous calls, use the asynchronous version of the credentials:

```python
from azure.identity.aio import (
Expand All @@ -97,7 +97,7 @@ model = AzureAIChatCompletionsModel(
)
```

If your endpoint is serving one model, like with the serverless API deployment, you don't have to indicate `model` parameter:
If your endpoint serves a single model (for example, serverless API deployments), omit the `model` parameter:

```python
import os
Expand All @@ -109,13 +109,13 @@ model = AzureAIChatCompletionsModel(
)
```

## Use chat completions models
## Use chat completion models

Let's first use the model directly. `ChatModels` are instances of LangChain `Runnable`, which means they expose a standard interface for interacting with them. To call the model, we can pass in a list of messages to the `invoke` method.
Use the model directly. `ChatModels` are instances of the LangChain `Runnable` interface, which provides a standard way to interact with them. To call the model, pass a list of messages to the `invoke` method.

[!notebook-python[](~/azureai-samples-main/scenarios/langchain/getting-started-with-langchain-chat-models.ipynb?name=human_message)]

You can also compose operations as needed in **chains**. Let's now use a prompt template to translate sentences:
Compose operations as needed in chains. Use a prompt template to translate sentences:

```python
from langchain_core.output_parsers import StrOutputParser
Expand All @@ -127,43 +127,43 @@ prompt_template = ChatPromptTemplate.from_messages(
)
```

As you can see from the prompt template, this chain has a `language` and `text` input. Now, let's create an output parser:
This chain takes `language` and `text` inputs. Now, create an output parser:

[!notebook-python[](~/azureai-samples-main/scenarios/langchain/getting-started-with-langchain-chat-models.ipynb?name=create_output_parser)]

We can now combine the template, model, and the output parser from above using the pipe (`|`) operator:
Combine the template, model, and output parser using the pipe (`|`) operator:

```python
chain = prompt_template | model | parser
```

To invoke the chain, identify the inputs required and provide values using the `invoke` method:
Invoke the chain by providing `language` and `text` values using the `invoke` method:

```python
chain.invoke({"language": "italian", "text": "hi"})
```

### Chaining multiple LLMs together
### Chain multiple LLMs together

Models deployed to Azure AI Foundry support the Model Inference API, which is standard across all the models. Chain multiple LLM operations based on the capabilities of each model so you can optimize for the right model based on capabilities.
Because models in Azure AI Foundry expose a common Model Inference API, you can chain multiple LLM operations and choose the model best suited to each step.

In the following example, we create two model clients. One is a producer and another one is a verifier. To make the distinction clear, we're using a multi-model endpoint like the [Foundry Models API](../../model-inference/overview.md) and hence we're passing the parameter `model` to use a `Mistral-Large` and a `Mistral-Small` model, quoting the fact that **producing content is more complex than verifying it**.
In the following example, we create two model clients: one producer and one verifier. To make the distinction clear, we use a multi-model endpoint such as the [Model Inference API](../../model-inference/overview.md) and pass the `model` parameter to use `Mistral-Large` for generation and `Mistral-Small` for verification. Producing content generally requires a larger model, while verification can use a smaller one.

[!notebook-python[](~/azureai-samples-main/scenarios/langchain/getting-started-with-langchain-chat-models.ipynb?name=create_producer_verifier)]


> [!TIP]
> Explore the model card of each of the models to understand the best use cases for each model.
> Review the model card for each model to understand the best use cases.

The following example generates a poem written by an urban poet:

[!notebook-python[](~/azureai-samples-main/scenarios/langchain/getting-started-with-langchain-chat-models.ipynb?name=generate_poem)]

Now let's chain the pieces:
Chain the pieces:

[!notebook-python[](~/azureai-samples-main/scenarios/langchain/getting-started-with-langchain-chat-models.ipynb?name=create_chain)]

The previous chain returns the output of the step `verifier` only. Since we want to access the intermediate result generated by the `producer`, in LangChain you need to use a `RunnablePassthrough` object to also output that intermediate step.
The previous chain returns only the output of the `verifier` step. To access the intermediate result generated by the `producer`, use a `RunnablePassthrough` to output that intermediate step.

```python
from langchain_core.runnables import RunnablePassthrough, RunnableParallel
Expand All @@ -174,44 +174,45 @@ verify_poem = verifier_template | verifier | parser
chain = generate_poem | RunnableParallel(poem=RunnablePassthrough(), verification=RunnablePassthrough() | verify_poem)
```

To invoke the chain, identify the inputs required and provide values using the `invoke` method:
Invoke the chain using the `invoke` method:

[!notebook-python[](~/azureai-samples-main/scenarios/langchain/getting-started-with-langchain-chat-models.ipynb?name=invoke_chain)]


## Use embeddings models
## Use embedding models

In the same way, you create an LLM client, you can connect to an embeddings model. In the following example, we're setting the environment variable to now point to an embeddings model:
Create an embeddings client similarly. Set the environment variables to point to an embeddings model:

```bash
export AZURE_INFERENCE_ENDPOINT="<your-model-endpoint-goes-here>"
export AZURE_INFERENCE_CREDENTIAL="<your-key-goes-here>"
```

Then create the client:
Create the client:

[!notebook-python[](~/azureai-samples-main/scenarios/langchain/getting-started-with-langchain-embeddings.ipynb?name=create_embed_model_client)]

The following example shows a simple example using a vector store in memory:
Use an in-memory vector store:

[!notebook-python[](~/azureai-samples-main/scenarios/langchain/getting-started-with-langchain-embeddings.ipynb?name=create_vector_store)]


Let's add some documents:
Add documents:

[!notebook-python[](~/azureai-samples-main/scenarios/langchain/getting-started-with-langchain-embeddings.ipynb?name=add_documents)]


Let's search by similarity:
Search by similarity:

[!notebook-python[](~/azureai-samples-main/scenarios/langchain/getting-started-with-langchain-embeddings.ipynb?name=search_similarity)]

## Using Azure OpenAI models
## Use Azure OpenAI models

If you're using Azure OpenAI models with `langchain-azure-ai` package, use the following URL:
When using Azure OpenAI models with the `langchain-azure-ai` package, use the following endpoint format:

```python
from langchain_azure_ai.chat_models import AzureAIChatCompletionsModel
import os

llm = AzureAIChatCompletionsModel(
endpoint="https://<resource>.openai.azure.com/openai/v1",
Expand All @@ -222,14 +223,14 @@ llm = AzureAIChatCompletionsModel(

## Debugging and troubleshooting

If you need to debug your application and understand the requests sent to the models in Azure AI Foundry, you can use the debug capabilities of the integration as follows:
If you need to debug your application and understand the requests sent to models in Azure AI Foundry, use the integration’s debug capabilities:

First, configure logging to the level you are interested in:
First, configure logging to the desired level:

[!notebook-python[](~/azureai-samples-main/scenarios/langchain/getting-started-with-langchain-chat-models.ipynb?name=configure_logging)]


To see the payloads of the requests, when instantiating the client, pass the argument `logging_enable`=`True` to the `client_kwargs`:
To see request payloads, pass `logging_enable=True` in `client_kwargs` when instantiating the client:

[!notebook-python[](~/azureai-samples-main/scenarios/langchain/getting-started-with-langchain-chat-models.ipynb?name=create_client_with_logging)]

Expand All @@ -238,7 +239,7 @@ Use the client as usual in your code.

## Tracing

You can use the tracing capabilities in Azure AI Foundry by creating a tracer. Logs are stored in Azure Application Insights and can be queried at any time using Azure Monitor or Azure AI Foundry portal. Each AI Hub has an Azure Application Insights associated with it.
Use tracing in Azure AI Foundry by creating a tracer. Logs are stored in Azure Application Insights and can be queried at any time using Azure Monitor or the Azure AI Foundry portal. Each AI hub has an associated Azure Application Insights instance.

### Get your instrumentation connection string

Expand Down Expand Up @@ -282,27 +283,28 @@ You can configure your application to send telemetry to Azure Application Insigh

### Configure tracing for Azure AI Foundry

The following code creates a tracer connected to the Azure Application Insights behind a project in Azure AI Foundry. Notice that the parameter `enable_content_recording` is set to `True`. This enables the capture of the inputs and outputs of the entire application as well as the intermediate steps. Such is helpful when debugging and building applications, but you might want to disable it on production environments. It defaults to the environment variable `AZURE_TRACING_GEN_AI_CONTENT_RECORDING_ENABLED`:
The following code creates a tracer connected to the Azure Application Insights behind an Azure AI Foundry project. The `enable_content_recording` parameter is set to `True`, which captures inputs and outputs across the application, including intermediate steps. This is helpful when debugging and building applications, but you might want to disable it in production environments. You can also control this via the `AZURE_TRACING_GEN_AI_CONTENT_RECORDING_ENABLED` environment variable:

```python
from langchain_azure_ai.callbacks.tracers import AzureAIInferenceTracer
from langchain_azure_ai.callbacks.tracers import AzureAIOpenTelemetryTracer

tracer = AzureAIInferenceTracer(
azure_tracer = AzureAIOpenTelemetryTracer(
connection_string=application_insights_connection_string,
enable_content_recording=True,
name="LangChain sample application",
)
```

To configure tracing with your chain, indicate the value config in the `invoke` operation as a callback:
Pass the tracer via `config` in the `invoke` operation:

```python
chain.invoke({"topic": "living in a foreign country"}, config={"callbacks": [tracer]})
chain.invoke({"topic": "living in a foreign country"}, config={"callbacks": [azure_tracer]})
```

To configure the chain itself for tracing, use the `.with_config()` method:

```python
chain = chain.with_config({"callbacks": [tracer]})
chain = chain.with_config({"callbacks": [azure_tracer]})
```

Then use the `invoke()` method as usual:
Expand All @@ -319,9 +321,7 @@ To see traces:

2. Navigate to **Tracing** section.

3. Identify the trace you have created. It may take a couple of seconds for the trace to show.

:::image type="content" source="../../media/how-to/develop-langchain/langchain-portal-tracing-example.png" alt-text="A screenshot showing the trace of a chain." lightbox="../../media/how-to/develop-langchain/langchain-portal-tracing-example.png":::
3. Identify the trace you created. It may take a few seconds to appear.

Learn more about [how to visualize and manage traces](visualize-traces.md).

Expand Down