|
1 | 1 | ## Customize Models |
2 | 2 |
|
3 | | -Ragas may use a LLM and or Embedding for evaluation and synthetic data generation. Both of these models can be customised according to you availability. |
| 3 | +Ragas may use a LLM and or Embedding for evaluation and synthetic data generation. Both of these models can be customised according to your availability. |
4 | 4 |
|
5 | | -!!! note |
6 | | - Ragas supports all the [LLMs](https://python.langchain.com/docs/integrations/chat/) and [Embeddings](https://python.langchain.com/docs/integrations/text_embedding/) available in LangChain |
| 5 | +Ragas provides factory functions (`llm_factory` and `embedding_factory`) that support multiple providers: |
7 | 6 |
|
8 | | -- `BaseRagasLLM` and `BaseRagasEmbeddings` are the base classes Ragas uses internally for LLMs and Embeddings. Any custom LLM or Embeddings should be a subclass of these base classes. |
| 7 | +- **Direct provider support**: OpenAI, Anthropic, Google |
| 8 | +- **Other providers via LiteLLM**: Azure OpenAI, AWS Bedrock, Google Vertex AI, and 100+ other providers |
9 | 9 |
|
10 | | -- If you are using LangChain, you can pass the LangChain LLM and Embeddings directly and Ragas will wrap it with `LangchainLLMWrapper` or `LangchainEmbeddingsWrapper` as needed. |
| 10 | +The factory functions use the [Instructor](https://python.useinstructor.com/) library for structured outputs and [LiteLLM](https://docs.litellm.ai/) for unified access to multiple LLM providers. |
11 | 11 |
|
12 | 12 | ## Examples |
13 | 13 |
|
14 | | -- [Azure OpenAI](#azure-openai) |
15 | | -- [Google Vertex](#google-vertex) |
16 | | -- [AWS Bedrock](#aws-bedrock) |
| 14 | +- [Customize Models](#customize-models) |
| 15 | +- [Examples](#examples) |
| 16 | + - [Azure OpenAI](#azure-openai) |
| 17 | + - [Google Vertex](#google-vertex) |
| 18 | + - [AWS Bedrock](#aws-bedrock) |
17 | 19 |
|
18 | 20 |
|
19 | 21 | ### Azure OpenAI |
20 | 22 |
|
21 | 23 | ```bash |
22 | | -pip install langchain_openai |
| 24 | +pip install litellm |
23 | 25 | ``` |
24 | 26 |
|
25 | 27 | ```python |
26 | | - |
27 | | -from langchain_openai.chat_models import AzureChatOpenAI |
28 | | -from langchain_openai.embeddings import AzureOpenAIEmbeddings |
29 | | -from ragas.llms import LangchainLLMWrapper |
30 | | -from ragas.embeddings import LangchainEmbeddingsWrapper |
| 28 | +import litellm |
| 29 | +from ragas.llms import llm_factory |
| 30 | +from ragas.embeddings.base import embedding_factory |
31 | 31 |
|
32 | 32 | azure_configs = { |
33 | | - "base_url": "https://<your-endpoint>.openai.azure.com/", |
| 33 | + "api_base": "https://<your-endpoint>.openai.azure.com/", |
| 34 | + "api_key": "your-api-key", |
| 35 | + "api_version": "2024-02-15-preview", |
34 | 36 | "model_deployment": "your-deployment-name", |
35 | | - "model_name": "your-model-name", |
36 | | - "embedding_deployment": "your-deployment-name", |
37 | | - "embedding_name": "text-embedding-ada-002", # most likely |
| 37 | + "embedding_deployment": "your-embedding-deployment-name", |
38 | 38 | } |
39 | 39 |
|
40 | | - |
41 | | -azure_llm = AzureChatOpenAI( |
42 | | - openai_api_version="2023-05-15", |
43 | | - azure_endpoint=azure_configs["base_url"], |
44 | | - azure_deployment=azure_configs["model_deployment"], |
45 | | - model=azure_configs["model_name"], |
46 | | - validate_base_url=False, |
| 40 | +# Configure LiteLLM for Azure OpenAI |
| 41 | +litellm.api_base = azure_configs["api_base"] |
| 42 | +litellm.api_key = azure_configs["api_key"] |
| 43 | +litellm.api_version = azure_configs["api_version"] |
| 44 | + |
| 45 | +# Create LLM using llm_factory with litellm provider |
| 46 | +# Note: Use deployment name, not model name for Azure |
| 47 | +azure_llm = llm_factory( |
| 48 | + f"azure/{azure_configs['model_deployment']}", |
| 49 | + provider="litellm", |
| 50 | + client=litellm, |
47 | 51 | ) |
48 | 52 |
|
49 | | -# init the embeddings for answer_relevancy, answer_correctness and answer_similarity |
50 | | -azure_embeddings = AzureOpenAIEmbeddings( |
51 | | - openai_api_version="2023-05-15", |
52 | | - azure_endpoint=azure_configs["base_url"], |
53 | | - azure_deployment=azure_configs["embedding_deployment"], |
54 | | - model=azure_configs["embedding_name"], |
| 53 | +# Create embeddings using embedding_factory |
| 54 | +# Note: Embeddings use the global litellm configuration set above |
| 55 | +azure_embeddings = embedding_factory( |
| 56 | + "litellm", |
| 57 | + model=f"azure/{azure_configs['embedding_deployment']}", |
55 | 58 | ) |
56 | | - |
57 | | -azure_llm = LangchainLLMWrapper(azure_llm) |
58 | | -azure_embeddings = LangchainEmbeddingsWrapper(azure_embeddings) |
59 | 59 | ``` |
60 | 60 | Yay! Now you are ready to use ragas with Azure OpenAI endpoints |
61 | 61 |
|
62 | 62 | ### Google Vertex |
63 | 63 |
|
64 | 64 | ```bash |
65 | | -!pip install langchain_google_vertexai |
| 65 | +pip install litellm google-cloud-aiplatform |
66 | 66 | ``` |
67 | 67 |
|
68 | 68 | ```python |
69 | | -import google.auth |
70 | | -from langchain_google_vertexai import ChatVertexAI, VertexAIEmbeddings |
71 | | -from ragas.llms import LangchainLLMWrapper |
72 | | -from ragas.embeddings import LangchainEmbeddingsWrapper |
73 | | -from langchain_core.outputs import LLMResult, ChatGeneration |
| 69 | +import litellm |
| 70 | +import os |
| 71 | +from ragas.llms import llm_factory |
| 72 | +from ragas.embeddings.base import embedding_factory |
74 | 73 |
|
75 | 74 | config = { |
76 | 75 | "project_id": "<your-project-id>", |
| 76 | + "location": "us-central1", # e.g., "us-central1", "us-east1" |
77 | 77 | "chat_model_id": "gemini-1.5-pro-002", |
78 | 78 | "embedding_model_id": "text-embedding-005", |
79 | 79 | } |
80 | 80 |
|
81 | | -# authenticate to GCP |
82 | | -creds, _ = google.auth.default(quota_project_id=config["project_id"]) |
| 81 | +# Set environment variables for Vertex AI |
| 82 | +os.environ["VERTEXAI_PROJECT"] = config["project_id"] |
| 83 | +os.environ["VERTEXAI_LOCATION"] = config["location"] |
83 | 84 |
|
84 | | -# create LangChain LLM and Embeddings |
85 | | -vertextai_llm = ChatVertexAI( |
86 | | - credentials=creds, |
87 | | - model_name=config["chat_model_id"], |
88 | | -) |
89 | | -vertextai_embeddings = VertexAIEmbeddings( |
90 | | - credentials=creds, model_name=config["embedding_model_id"] |
| 85 | +# Create LLM using llm_factory with litellm provider |
| 86 | +vertex_llm = llm_factory( |
| 87 | + f"vertex_ai/{config['chat_model_id']}", |
| 88 | + provider="litellm", |
| 89 | + client=litellm, |
91 | 90 | ) |
92 | 91 |
|
93 | | -# Create a custom is_finished_parser to capture Gemini generation completion signals |
94 | | -def gemini_is_finished_parser(response: LLMResult) -> bool: |
95 | | - is_finished_list = [] |
96 | | - for g in response.flatten(): |
97 | | - resp = g.generations[0][0] |
98 | | - |
99 | | - # Check generation_info first |
100 | | - if resp.generation_info is not None: |
101 | | - finish_reason = resp.generation_info.get("finish_reason") |
102 | | - if finish_reason is not None: |
103 | | - is_finished_list.append( |
104 | | - finish_reason in ["STOP", "MAX_TOKENS"] |
105 | | - ) |
106 | | - continue |
107 | | - |
108 | | - # Check response_metadata as fallback |
109 | | - if isinstance(resp, ChatGeneration) and resp.message is not None: |
110 | | - metadata = resp.message.response_metadata |
111 | | - if metadata.get("finish_reason"): |
112 | | - is_finished_list.append( |
113 | | - metadata["finish_reason"] in ["STOP", "MAX_TOKENS"] |
114 | | - ) |
115 | | - elif metadata.get("stop_reason"): |
116 | | - is_finished_list.append( |
117 | | - metadata["stop_reason"] in ["STOP", "MAX_TOKENS"] |
118 | | - ) |
119 | | - |
120 | | - # If no finish reason found, default to True |
121 | | - if not is_finished_list: |
122 | | - is_finished_list.append(True) |
123 | | - |
124 | | - return all(is_finished_list) |
125 | | - |
126 | | - |
127 | | -vertextai_llm = LangchainLLMWrapper(vertextai_llm, is_finished_parser=gemini_is_finished_parser) |
128 | | -vertextai_embeddings = LangchainEmbeddingsWrapper(vertextai_embeddings) |
| 92 | +# Create embeddings using embedding_factory |
| 93 | +# Note: Embeddings use the environment variables set above |
| 94 | +vertex_embeddings = embedding_factory( |
| 95 | + "litellm", |
| 96 | + model=f"vertex_ai/{config['embedding_model_id']}", |
| 97 | +) |
129 | 98 | ``` |
130 | 99 | Yay! Now you are ready to use ragas with Google VertexAI endpoints |
131 | 100 |
|
132 | 101 | ### AWS Bedrock |
133 | 102 |
|
134 | 103 | ```bash |
135 | | -pip install langchain_aws |
| 104 | +pip install litellm |
136 | 105 | ``` |
137 | 106 |
|
138 | 107 | ```python |
139 | | -from langchain_aws import ChatBedrockConverse |
140 | | -from langchain_aws import BedrockEmbeddings |
141 | | -from ragas.llms import LangchainLLMWrapper |
142 | | -from ragas.embeddings import LangchainEmbeddingsWrapper |
| 108 | +import litellm |
| 109 | +import os |
| 110 | +from ragas.llms import llm_factory |
| 111 | +from ragas.embeddings.base import embedding_factory |
143 | 112 |
|
144 | 113 | config = { |
145 | | - "credentials_profile_name": "your-profile-name", # E.g "default" |
146 | | - "region_name": "your-region-name", # E.g. "us-east-1" |
147 | | - "llm": "your-llm-model-id", # E.g "anthropic.claude-3-5-sonnet-20241022-v2:0" |
148 | | - "embeddings": "your-embedding-model-id", # E.g "amazon.titan-embed-text-v2:0" |
| 114 | + "region_name": "us-east-1", # E.g. "us-east-1" |
| 115 | + "llm": "anthropic.claude-3-5-sonnet-20241022-v2:0", # Your LLM model ID |
| 116 | + "embeddings": "amazon.titan-embed-text-v2:0", # Your embedding model ID |
149 | 117 | "temperature": 0.4, |
150 | 118 | } |
151 | 119 |
|
152 | | -bedrock_llm = ChatBedrockConverse( |
153 | | - credentials_profile_name=config["credentials_profile_name"], |
154 | | - region_name=config["region_name"], |
155 | | - base_url=f"https://bedrock-runtime.{config['region_name']}.amazonaws.com", |
156 | | - model=config["llm"], |
| 120 | +# Set AWS credentials as environment variables |
| 121 | +# Option 1: Use AWS credentials file (~/.aws/credentials) |
| 122 | +# Option 2: Set environment variables directly |
| 123 | +os.environ["AWS_REGION_NAME"] = config["region_name"] |
| 124 | +# os.environ["AWS_ACCESS_KEY_ID"] = "your-access-key" |
| 125 | +# os.environ["AWS_SECRET_ACCESS_KEY"] = "your-secret-key" |
| 126 | + |
| 127 | +# Create LLM using llm_factory with litellm provider |
| 128 | +bedrock_llm = llm_factory( |
| 129 | + f"bedrock/{config['llm']}", |
| 130 | + provider="litellm", |
| 131 | + client=litellm, |
157 | 132 | temperature=config["temperature"], |
158 | 133 | ) |
159 | 134 |
|
160 | | -# init the embeddings |
161 | | -bedrock_embeddings = BedrockEmbeddings( |
162 | | - credentials_profile_name=config["credentials_profile_name"], |
163 | | - region_name=config["region_name"], |
164 | | - model_id=config["embeddings"], |
| 135 | +# Create embeddings using embedding_factory |
| 136 | +# Note: Embeddings use the environment variables set above |
| 137 | +bedrock_embeddings = embedding_factory( |
| 138 | + "litellm", |
| 139 | + model=f"bedrock/{config['embeddings']}", |
165 | 140 | ) |
166 | | - |
167 | | -bedrock_llm = LangchainLLMWrapper(bedrock_llm) |
168 | | -bedrock_embeddings = LangchainEmbeddingsWrapper(bedrock_embeddings) |
169 | 141 | ``` |
170 | 142 | Yay! Now you are ready to use ragas with AWS Bedrock endpoints |
0 commit comments