Skip to content

Commit ba00799

Browse files
committed
docs: migrate customize_models guide to modern factory APIs
Replace legacy LangChain wrapper pattern with llm_factory and embedding_factory using LiteLLM provider for all cloud providers. Changes: Updated examples - Removed LangChain dependencies (langchain_openai, langchain_google_vertexai, langchain_aws) in favor of litellm - Fixed Azure model naming to use deployment names (azure/{deployment-name}) - Removed unnecessary interface="modern" parameter (auto-detected) Benefits: - Consistent pattern across all cloud providers - Simpler code with fewer dependencies - No LangChain required - Uses modern Ragas factory APIs with Instructor and LiteLLM All syntax verified against LiteLLM and Instructor documentation.
1 parent 1385aa6 commit ba00799

File tree

1 file changed

+79
-107
lines changed

1 file changed

+79
-107
lines changed
Lines changed: 79 additions & 107 deletions
Original file line numberDiff line numberDiff line change
@@ -1,170 +1,142 @@
11
## Customize Models
22

3-
Ragas may use a LLM and or Embedding for evaluation and synthetic data generation. Both of these models can be customised according to you availability.
3+
Ragas may use a LLM and or Embedding for evaluation and synthetic data generation. Both of these models can be customised according to your availability.
44

5-
!!! note
6-
Ragas supports all the [LLMs](https://python.langchain.com/docs/integrations/chat/) and [Embeddings](https://python.langchain.com/docs/integrations/text_embedding/) available in LangChain
5+
Ragas provides factory functions (`llm_factory` and `embedding_factory`) that support multiple providers:
76

8-
- `BaseRagasLLM` and `BaseRagasEmbeddings` are the base classes Ragas uses internally for LLMs and Embeddings. Any custom LLM or Embeddings should be a subclass of these base classes.
7+
- **Direct provider support**: OpenAI, Anthropic, Google
8+
- **Other providers via LiteLLM**: Azure OpenAI, AWS Bedrock, Google Vertex AI, and 100+ other providers
99

10-
- If you are using LangChain, you can pass the LangChain LLM and Embeddings directly and Ragas will wrap it with `LangchainLLMWrapper` or `LangchainEmbeddingsWrapper` as needed.
10+
The factory functions use the [Instructor](https://python.useinstructor.com/) library for structured outputs and [LiteLLM](https://docs.litellm.ai/) for unified access to multiple LLM providers.
1111

1212
## Examples
1313

14-
- [Azure OpenAI](#azure-openai)
15-
- [Google Vertex](#google-vertex)
16-
- [AWS Bedrock](#aws-bedrock)
14+
- [Customize Models](#customize-models)
15+
- [Examples](#examples)
16+
- [Azure OpenAI](#azure-openai)
17+
- [Google Vertex](#google-vertex)
18+
- [AWS Bedrock](#aws-bedrock)
1719

1820

1921
### Azure OpenAI
2022

2123
```bash
22-
pip install langchain_openai
24+
pip install litellm
2325
```
2426

2527
```python
26-
27-
from langchain_openai.chat_models import AzureChatOpenAI
28-
from langchain_openai.embeddings import AzureOpenAIEmbeddings
29-
from ragas.llms import LangchainLLMWrapper
30-
from ragas.embeddings import LangchainEmbeddingsWrapper
28+
import litellm
29+
from ragas.llms import llm_factory
30+
from ragas.embeddings.base import embedding_factory
3131

3232
azure_configs = {
33-
"base_url": "https://<your-endpoint>.openai.azure.com/",
33+
"api_base": "https://<your-endpoint>.openai.azure.com/",
34+
"api_key": "your-api-key",
35+
"api_version": "2024-02-15-preview",
3436
"model_deployment": "your-deployment-name",
35-
"model_name": "your-model-name",
36-
"embedding_deployment": "your-deployment-name",
37-
"embedding_name": "text-embedding-ada-002", # most likely
37+
"embedding_deployment": "your-embedding-deployment-name",
3838
}
3939

40-
41-
azure_llm = AzureChatOpenAI(
42-
openai_api_version="2023-05-15",
43-
azure_endpoint=azure_configs["base_url"],
44-
azure_deployment=azure_configs["model_deployment"],
45-
model=azure_configs["model_name"],
46-
validate_base_url=False,
40+
# Configure LiteLLM for Azure OpenAI
41+
litellm.api_base = azure_configs["api_base"]
42+
litellm.api_key = azure_configs["api_key"]
43+
litellm.api_version = azure_configs["api_version"]
44+
45+
# Create LLM using llm_factory with litellm provider
46+
# Note: Use deployment name, not model name for Azure
47+
azure_llm = llm_factory(
48+
f"azure/{azure_configs['model_deployment']}",
49+
provider="litellm",
50+
client=litellm,
4751
)
4852

49-
# init the embeddings for answer_relevancy, answer_correctness and answer_similarity
50-
azure_embeddings = AzureOpenAIEmbeddings(
51-
openai_api_version="2023-05-15",
52-
azure_endpoint=azure_configs["base_url"],
53-
azure_deployment=azure_configs["embedding_deployment"],
54-
model=azure_configs["embedding_name"],
53+
# Create embeddings using embedding_factory
54+
# Note: Embeddings use the global litellm configuration set above
55+
azure_embeddings = embedding_factory(
56+
"litellm",
57+
model=f"azure/{azure_configs['embedding_deployment']}",
5558
)
56-
57-
azure_llm = LangchainLLMWrapper(azure_llm)
58-
azure_embeddings = LangchainEmbeddingsWrapper(azure_embeddings)
5959
```
6060
Yay! Now you are ready to use ragas with Azure OpenAI endpoints
6161

6262
### Google Vertex
6363

6464
```bash
65-
!pip install langchain_google_vertexai
65+
pip install litellm google-cloud-aiplatform
6666
```
6767

6868
```python
69-
import google.auth
70-
from langchain_google_vertexai import ChatVertexAI, VertexAIEmbeddings
71-
from ragas.llms import LangchainLLMWrapper
72-
from ragas.embeddings import LangchainEmbeddingsWrapper
73-
from langchain_core.outputs import LLMResult, ChatGeneration
69+
import litellm
70+
import os
71+
from ragas.llms import llm_factory
72+
from ragas.embeddings.base import embedding_factory
7473

7574
config = {
7675
"project_id": "<your-project-id>",
76+
"location": "us-central1", # e.g., "us-central1", "us-east1"
7777
"chat_model_id": "gemini-1.5-pro-002",
7878
"embedding_model_id": "text-embedding-005",
7979
}
8080

81-
# authenticate to GCP
82-
creds, _ = google.auth.default(quota_project_id=config["project_id"])
81+
# Set environment variables for Vertex AI
82+
os.environ["VERTEXAI_PROJECT"] = config["project_id"]
83+
os.environ["VERTEXAI_LOCATION"] = config["location"]
8384

84-
# create LangChain LLM and Embeddings
85-
vertextai_llm = ChatVertexAI(
86-
credentials=creds,
87-
model_name=config["chat_model_id"],
88-
)
89-
vertextai_embeddings = VertexAIEmbeddings(
90-
credentials=creds, model_name=config["embedding_model_id"]
85+
# Create LLM using llm_factory with litellm provider
86+
vertex_llm = llm_factory(
87+
f"vertex_ai/{config['chat_model_id']}",
88+
provider="litellm",
89+
client=litellm,
9190
)
9291

93-
# Create a custom is_finished_parser to capture Gemini generation completion signals
94-
def gemini_is_finished_parser(response: LLMResult) -> bool:
95-
is_finished_list = []
96-
for g in response.flatten():
97-
resp = g.generations[0][0]
98-
99-
# Check generation_info first
100-
if resp.generation_info is not None:
101-
finish_reason = resp.generation_info.get("finish_reason")
102-
if finish_reason is not None:
103-
is_finished_list.append(
104-
finish_reason in ["STOP", "MAX_TOKENS"]
105-
)
106-
continue
107-
108-
# Check response_metadata as fallback
109-
if isinstance(resp, ChatGeneration) and resp.message is not None:
110-
metadata = resp.message.response_metadata
111-
if metadata.get("finish_reason"):
112-
is_finished_list.append(
113-
metadata["finish_reason"] in ["STOP", "MAX_TOKENS"]
114-
)
115-
elif metadata.get("stop_reason"):
116-
is_finished_list.append(
117-
metadata["stop_reason"] in ["STOP", "MAX_TOKENS"]
118-
)
119-
120-
# If no finish reason found, default to True
121-
if not is_finished_list:
122-
is_finished_list.append(True)
123-
124-
return all(is_finished_list)
125-
126-
127-
vertextai_llm = LangchainLLMWrapper(vertextai_llm, is_finished_parser=gemini_is_finished_parser)
128-
vertextai_embeddings = LangchainEmbeddingsWrapper(vertextai_embeddings)
92+
# Create embeddings using embedding_factory
93+
# Note: Embeddings use the environment variables set above
94+
vertex_embeddings = embedding_factory(
95+
"litellm",
96+
model=f"vertex_ai/{config['embedding_model_id']}",
97+
)
12998
```
13099
Yay! Now you are ready to use ragas with Google VertexAI endpoints
131100

132101
### AWS Bedrock
133102

134103
```bash
135-
pip install langchain_aws
104+
pip install litellm
136105
```
137106

138107
```python
139-
from langchain_aws import ChatBedrockConverse
140-
from langchain_aws import BedrockEmbeddings
141-
from ragas.llms import LangchainLLMWrapper
142-
from ragas.embeddings import LangchainEmbeddingsWrapper
108+
import litellm
109+
import os
110+
from ragas.llms import llm_factory
111+
from ragas.embeddings.base import embedding_factory
143112

144113
config = {
145-
"credentials_profile_name": "your-profile-name", # E.g "default"
146-
"region_name": "your-region-name", # E.g. "us-east-1"
147-
"llm": "your-llm-model-id", # E.g "anthropic.claude-3-5-sonnet-20241022-v2:0"
148-
"embeddings": "your-embedding-model-id", # E.g "amazon.titan-embed-text-v2:0"
114+
"region_name": "us-east-1", # E.g. "us-east-1"
115+
"llm": "anthropic.claude-3-5-sonnet-20241022-v2:0", # Your LLM model ID
116+
"embeddings": "amazon.titan-embed-text-v2:0", # Your embedding model ID
149117
"temperature": 0.4,
150118
}
151119

152-
bedrock_llm = ChatBedrockConverse(
153-
credentials_profile_name=config["credentials_profile_name"],
154-
region_name=config["region_name"],
155-
base_url=f"https://bedrock-runtime.{config['region_name']}.amazonaws.com",
156-
model=config["llm"],
120+
# Set AWS credentials as environment variables
121+
# Option 1: Use AWS credentials file (~/.aws/credentials)
122+
# Option 2: Set environment variables directly
123+
os.environ["AWS_REGION_NAME"] = config["region_name"]
124+
# os.environ["AWS_ACCESS_KEY_ID"] = "your-access-key"
125+
# os.environ["AWS_SECRET_ACCESS_KEY"] = "your-secret-key"
126+
127+
# Create LLM using llm_factory with litellm provider
128+
bedrock_llm = llm_factory(
129+
f"bedrock/{config['llm']}",
130+
provider="litellm",
131+
client=litellm,
157132
temperature=config["temperature"],
158133
)
159134

160-
# init the embeddings
161-
bedrock_embeddings = BedrockEmbeddings(
162-
credentials_profile_name=config["credentials_profile_name"],
163-
region_name=config["region_name"],
164-
model_id=config["embeddings"],
135+
# Create embeddings using embedding_factory
136+
# Note: Embeddings use the environment variables set above
137+
bedrock_embeddings = embedding_factory(
138+
"litellm",
139+
model=f"bedrock/{config['embeddings']}",
165140
)
166-
167-
bedrock_llm = LangchainLLMWrapper(bedrock_llm)
168-
bedrock_embeddings = LangchainEmbeddingsWrapper(bedrock_embeddings)
169141
```
170142
Yay! Now you are ready to use ragas with AWS Bedrock endpoints

0 commit comments

Comments
 (0)