Skip to content

Commit 8019f89

Browse files
vrushankportkeyvrushankportkey
authored
Merge pull request #380 from Portkey-AI/small-fixes
librechat web search docs
2 parents 8c268a0 + 7a0ed04 commit 8019f89

File tree

2 files changed

+302
-1
lines changed

2 files changed

+302
-1
lines changed

‎docs.json

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -383,7 +383,13 @@
383383
"integrations/libraries/cline",
384384
"integrations/libraries/goose",
385385
"integrations/libraries/janhq",
386-
"integrations/libraries/librechat",
386+
{
387+
"group": "Librechat",
388+
"pages": [
389+
"integrations/libraries/librechat",
390+
"guides/use-cases/librechat-web-search"
391+
]
392+
},
387393
"integrations/libraries/n8n",
388394
"integrations/libraries/codex",
389395
"integrations/libraries/openwebui",
@@ -847,6 +853,7 @@
847853
"group": "Use Cases",
848854
"pages": [
849855
"guides/use-cases",
856+
"guides/use-cases/librechat-web-search",
850857
"guides/use-cases/run-batch-evals",
851858
"guides/use-cases/few-shot-prompting",
852859
"guides/use-cases/enforcing-json-schema-with-anyscale-and-together",

‎guides/use-cases/librechat-web-search.mdx

Lines changed: 294 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,294 @@
1+
---
2+
title: "Web Search for Any LLM in LibreChat"
3+
---
4+
5+
> Enable real-time internet search capabilities for any LLM in LibreChat using Portkey's Exa integration
6+
7+
Transform your LibreChat experience by adding web search capabilities to any LLM - whether it's GPT-4, Claude, Llama, or any of the 250+ models supported by Portkey. This guide shows you how to combine LibreChat, Portkey, and Exa to create a powerful AI chat interface with real-time internet access.
8+
9+
<Note>
10+
This integration builds upon the basic [LibreChat + Portkey setup](/integrations/apps/librechat). Please complete that integration first before proceeding.
11+
</Note>
12+
13+
## What You'll Get
14+
15+
By following this guide, your LibreChat installation will have:
16+
17+
- ✅ **Web search for any LLM** - Not just OpenAI's browsing models
18+
- ✅ **Real-time information** - Access to current events, latest data, and up-to-date facts
19+
- ✅ **All Portkey features** - Observability, caching, fallbacks, and more
20+
- ✅ **Unified interface** - One LibreChat setup for all your AI needs
21+
- ✅ **250+ LLM support** - Use web search with any model through Portkey
22+
23+
## How It Works
24+
25+
<Steps>
26+
<Step title="User asks a question in LibreChat">
27+
When you send a message requiring current information
28+
</Step>
29+
<Step title="Request routes through Portkey">
30+
Portkey intercepts the request and triggers the Exa plugin
31+
</Step>
32+
<Step title="Exa searches the web">
33+
Relevant search results are fetched from across the internet
34+
</Step>
35+
<Step title="Context enhancement">
36+
Search results are added to your prompt as additional context
37+
</Step>
38+
<Step title="LLM responds with current information">
39+
Your chosen LLM now has access to real-time data to answer accurately
40+
</Step>
41+
</Steps>
42+
43+
## Prerequisites
44+
45+
Before starting, ensure you have:
46+
47+
1. ✅ [LibreChat installed and running](https://www.librechat.ai/docs/installation)
48+
2. ✅ [Basic Portkey + LibreChat integration](/integrations/apps/librechat) completed
49+
3. ✅ [Portkey account](https://app.portkey.ai) with API key
50+
4. ✅ [Exa account](https://exa.ai) with API key
51+
52+
## Setup Guide
53+
54+
### Step 1: Enable Exa Plugin in Portkey
55+
56+
First, activate the Exa plugin in your Portkey account:
57+
58+
1. Log into your [Portkey dashboard](https://app.portkey.ai)
59+
2. Navigate to `Settings` → `Plugins` in the sidebar
60+
3. Find **Exa** in the list of available plugins
61+
4. Click **Enable** and enter your Exa API key
62+
5. Save your settings
63+
64+
<Frame>
65+
<img src="/images/product/exa-plugin.png" height="40" />
66+
</Frame>
67+
68+
### Step 2: Create an Exa Guardrail
69+
70+
Next, create a guardrail that will add web search to your requests:
71+
72+
1. Go to the `Guardrails` page in Portkey
73+
2. Click `Create New Guardrail`
74+
3. Search for **"Exa Online Search"** and click `Add`
75+
4. Configure the following parameters:
76+
77+
```json
78+
{
79+
"context_prefix": "<web_search_context>",
80+
"context_suffix": "</web_search_context>",
81+
"num_results": 3,
82+
"timeout": 10000
83+
}
84+
```
85+
86+
<Info>
87+
**Recommended Settings:**
88+
- **Number of Results**: 3-5 (balances information vs token usage)
89+
- **Timeout**: 10000ms (10 seconds)
90+
- **Include/Exclude Domains**: Leave empty for general use, or specify trusted sources
91+
</Info>
92+
93+
5. Set the action to `passthrough` (default)
94+
6. Save the guardrail and copy the **Guardrail ID**
95+
96+
### Step 3: Create a Config with Web Search
97+
98+
Now create a Portkey config that includes your Exa guardrail:
99+
100+
1. Navigate to `Configs` in the Portkey dashboard
101+
2. Click `Create New Config`
102+
3. Add your configuration:
103+
104+
<CodeGroup>
105+
```json Using Virtual Key
106+
{
107+
"virtual_key": "YOUR_PROVIDER_VIRTUAL_KEY",
108+
"input_guardrails": ["your_exa_guardrail_id"],
109+
"override_params": {
110+
"model": "gpt-4o" // or any model you prefer
111+
}
112+
}
113+
```
114+
115+
```json Using Provider API Key
116+
{
117+
"provider": "openai",
118+
"api_key": "YOUR_PROVIDER_API_KEY",
119+
"input_guardrails": ["your_exa_guardrail_id"],
120+
"override_params": {
121+
"model": "gpt-4o"
122+
}
123+
}
124+
```
125+
</CodeGroup>
126+
127+
4. Save the config and note the **Config ID** (e.g., `pc-websearch-xxx`)
128+
129+
### Step 4: Update Your LibreChat Configuration
130+
131+
Finally, update your LibreChat setup to use the web-search enabled config:
132+
133+
1. Edit your `librechat.yaml` file:
134+
135+
```yaml librechat.yaml
136+
version: 1.1.4
137+
cache: true
138+
endpoints:
139+
custom:
140+
- name: "Portkey with Web Search"
141+
apiKey: "dummy"
142+
baseURL: ${PORTKEY_GATEWAY_URL}
143+
headers:
144+
x-portkey-api-key: "${PORTKEY_API_KEY}"
145+
x-portkey-config: "pc-websearch-xxx" # Your config ID from Step 3
146+
models:
147+
default: ["gpt-4o", "claude-3-sonnet", "llama-3.1-70b"]
148+
fetch: true
149+
titleConvo: true
150+
titleModel: "current_model"
151+
summarize: false
152+
summaryModel: "current_model"
153+
forcePrompt: false
154+
modelDisplayLabel: "Portkey:WebSearch"
155+
```
156+
157+
2. Restart your LibreChat instance
158+
159+
### Step 5: Test Your Setup
160+
161+
1. Open LibreChat in your browser
162+
2. Select **"Portkey with Web Search"** as your endpoint
163+
3. Try asking questions that require current information:
164+
165+
```
166+
"What happened in the tech industry today?"
167+
"What's the current price of Bitcoin?"
168+
"Who won the latest sports championship?"
169+
"What are the latest AI model releases?"
170+
```
171+
172+
You should see responses with up-to-date information pulled from the web!
173+
174+
## Advanced Configuration
175+
176+
### Domain Filtering
177+
178+
For specialized use cases, you can limit search results to specific domains:
179+
180+
```json
181+
{
182+
"virtual_key": "YOUR_VIRTUAL_KEY",
183+
"input_guardrails": ["your_exa_guardrail_id"],
184+
"override_params": {
185+
"model": "gpt-4o"
186+
}
187+
}
188+
```
189+
190+
In your Exa guardrail settings:
191+
- **Include Domains**: `["arxiv.org", "nature.com", "pubmed.ncbi.nlm.nih.gov"]` (for academic research)
192+
- **Exclude Domains**: `["reddit.com", "twitter.com"]` (to avoid social media)
193+
194+
### Multiple Configurations
195+
196+
Create different configs for different use cases:
197+
198+
```yaml librechat.yaml
199+
endpoints:
200+
custom:
201+
- name: "AI Research Assistant"
202+
apiKey: "dummy"
203+
baseURL: ${PORTKEY_GATEWAY_URL}
204+
headers:
205+
x-portkey-api-key: "${PORTKEY_API_KEY}"
206+
x-portkey-config: "pc-research-xxx" # Config with academic domains
207+
models:
208+
default: ["gpt-4o"]
209+
modelDisplayLabel: "Research:WebSearch"
210+
211+
- name: "News & Current Events"
212+
apiKey: "dummy"
213+
baseURL: ${PORTKEY_GATEWAY_URL}
214+
headers:
215+
x-portkey-api-key: "${PORTKEY_API_KEY}"
216+
x-portkey-config: "pc-news-xxx" # Config with news domains
217+
models:
218+
default: ["claude-3-sonnet"]
219+
modelDisplayLabel: "News:WebSearch"
220+
```
221+
222+
### Combining with Other Portkey Features
223+
224+
Enhance your web-search enabled config with additional Portkey features:
225+
226+
```json
227+
{
228+
"virtual_key": "YOUR_VIRTUAL_KEY",
229+
"input_guardrails": ["your_exa_guardrail_id"],
230+
"output_guardrails": ["content_safety_guardrail_id"],
231+
"cache": {
232+
"mode": "semantic",
233+
"max_age": 3600
234+
},
235+
"retry": {
236+
"attempts": 3
237+
},
238+
"override_params": {
239+
"model": "gpt-4o",
240+
"temperature": 0.7
241+
}
242+
}
243+
```
244+
245+
## Monitoring Web Search Usage
246+
247+
Track your web-search enhanced conversations in the Portkey dashboard:
248+
249+
1. Navigate to **Logs** in Portkey
250+
2. Filter by config ID to see web-search requests
251+
3. Click on individual logs to see:
252+
- The original user query
253+
- Web search results added as context
254+
- Token usage (including search context)
255+
- Response time and costs
256+
257+
<Frame>
258+
<img src="/images/product/EXA-LOGS.png" alt="Web Search Logs" />
259+
</Frame>
260+
261+
## Best Practices
262+
263+
<CardGroup cols={2}>
264+
<Card title="Token Management" icon="coins">
265+
Monitor token usage as web search adds context. Adjust the number of search results based on your needs and budget.
266+
</Card>
267+
268+
<Card title="Model Selection" icon="robot">
269+
Some models handle web context better than others. Test different models to find the best fit for your use case.
270+
</Card>
271+
272+
<Card title="Query Optimization" icon="magnifying-glass">
273+
Not every query needs web search. Consider creating separate endpoints for general chat vs. current information needs.
274+
</Card>
275+
276+
<Card title="Caching Strategy" icon="database">
277+
Use semantic caching for frequently asked current events questions to reduce API calls and costs.
278+
</Card>
279+
</CardGroup>
280+
281+
282+
## Next Steps
283+
284+
- **Try different models** with web search to compare performance
285+
- **Monitor costs** using Portkey's analytics dashboard
286+
- **Create specialized configs** for your team's specific use cases
287+
- **Set up access controls** with user-specific API keys
288+
289+
## Support
290+
291+
Need help? Contact us:
292+
- Email: [email protected]
293+
- [Documentation](https://docs.portkey.ai)
294+
- [Discord Community](https://portkey.sh/discord-report)

0 commit comments

Comments
 (0)