Skip to content

Commit 9faa29b

Browse files
ppradospprados
authored
Merge branch 'langchain-ai:master' into pprados/01-prepare
2 parents 33b69b3 + 187131c commit 9faa29b

File tree

24 files changed

+774
-113
lines changed

24 files changed

+774
-113
lines changed

‎docs/docs/how_to/callbacks_custom_events.ipynb‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,7 @@
1515
"- [Astream Events API](/docs/concepts/streaming/#astream_events) the `astream_events` method will surface custom callback events.\n",
1616
":::\n",
1717
"\n",
18-
"In some situations, you may want to dipsatch a custom callback event from within a [Runnable](/docs/concepts/runnables) so it can be surfaced\n",
18+
"In some situations, you may want to dispatch a custom callback event from within a [Runnable](/docs/concepts/runnables) so it can be surfaced\n",
1919
"in a custom callback handler or via the [Astream Events API](/docs/concepts/streaming/#astream_events).\n",
2020
"\n",
2121
"For example, if you have a long running tool with multiple steps, you can dispatch custom events between the steps and use these custom events to monitor progress.\n",

‎docs/docs/integrations/document_loaders/index.mdx‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -45,7 +45,7 @@ The below document loaders allow you to load documents from your favorite cloud
4545

4646
## Social Platforms
4747

48-
The below document loaders allow you to load documents from differnt social media platforms.
48+
The below document loaders allow you to load documents from different social media platforms.
4949

5050
<CategoryTable category="social_loaders"/>
5151

‎docs/docs/integrations/document_loaders/pull_md.ipynb‎

Lines changed: 140 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,140 @@
1+
{
2+
"cells": [
3+
{
4+
"cell_type": "raw",
5+
"metadata": {},
6+
"source": [
7+
"---\n",
8+
"sidebar_label: PullMdLoader\n",
9+
"---"
10+
]
11+
},
12+
{
13+
"cell_type": "markdown",
14+
"metadata": {},
15+
"source": [
16+
"# PullMdLoader\n",
17+
"\n",
18+
"Loader for converting URLs into Markdown using the pull.md service.\n",
19+
"\n",
20+
"This package implements a [document loader](/docs/concepts/document_loaders/) for web content. Unlike traditional web scrapers, PullMdLoader can handle web pages built with dynamic JavaScript frameworks like React, Angular, or Vue.js, converting them into Markdown without local rendering.\n",
21+
"\n",
22+
"## Overview\n",
23+
"### Integration details\n",
24+
"\n",
25+
"| Class | Package | Local | Serializable | JS Support |\n",
26+
"| :--- | :--- | :---: | :---: | :---: |\n",
27+
"| PullMdLoader | langchain-pull-md | ✅ | ✅ | ❌ |\n"
28+
]
29+
},
30+
{
31+
"cell_type": "markdown",
32+
"metadata": {},
33+
"source": [
34+
"## Setup\n",
35+
"\n",
36+
"### Installation\n",
37+
"\n",
38+
"```bash\n",
39+
"pip install langchain-pull-md\n",
40+
"```"
41+
]
42+
},
43+
{
44+
"cell_type": "markdown",
45+
"metadata": {},
46+
"source": [
47+
"### Initialization"
48+
]
49+
},
50+
{
51+
"cell_type": "code",
52+
"execution_count": 10,
53+
"metadata": {},
54+
"outputs": [],
55+
"source": [
56+
"from langchain_pull_md.markdown_loader import PullMdLoader\n",
57+
"\n",
58+
"# Instantiate the loader with a URL\n",
59+
"loader = PullMdLoader(url=\"https://example.com\")"
60+
]
61+
},
62+
{
63+
"cell_type": "markdown",
64+
"metadata": {},
65+
"source": [
66+
"### Load"
67+
]
68+
},
69+
{
70+
"cell_type": "code",
71+
"execution_count": 11,
72+
"metadata": {},
73+
"outputs": [],
74+
"source": [
75+
"documents = loader.load()"
76+
]
77+
},
78+
{
79+
"cell_type": "code",
80+
"execution_count": 12,
81+
"metadata": {},
82+
"outputs": [
83+
{
84+
"data": {
85+
"text/plain": [
86+
"{'source': 'https://example.com',\n",
87+
" 'page_content': '# Example Domain\\nThis domain is used for illustrative examples in documents. You may use this domain in literature without prior coordination or asking for permission.'}"
88+
]
89+
},
90+
"execution_count": 12,
91+
"metadata": {},
92+
"output_type": "execute_result"
93+
}
94+
],
95+
"source": [
96+
"documents[0].metadata"
97+
]
98+
},
99+
{
100+
"cell_type": "markdown",
101+
"metadata": {},
102+
"source": [
103+
"## Lazy Load\n",
104+
"\n",
105+
"No lazy loading is implemented. `PullMdLoader` performs a real-time conversion of the provided URL into Markdown format whenever the `load` method is called."
106+
]
107+
},
108+
{
109+
"cell_type": "markdown",
110+
"metadata": {},
111+
"source": [
112+
"## API reference:\n",
113+
"\n",
114+
"- [GitHub](https://github.com/chigwell/langchain-pull-md)\n",
115+
"- [PyPi](https://pypi.org/project/langchain-pull-md/)"
116+
]
117+
}
118+
],
119+
"metadata": {
120+
"kernelspec": {
121+
"display_name": "Python 3 (ipykernel)",
122+
"language": "python",
123+
"name": "python3"
124+
},
125+
"language_info": {
126+
"codemirror_mode": {
127+
"name": "ipython",
128+
"version": 3
129+
},
130+
"file_extension": ".py",
131+
"mimetype": "text/x-python",
132+
"name": "python",
133+
"nbconvert_exporter": "python",
134+
"pygments_lexer": "ipython3",
135+
"version": "3.10.12"
136+
}
137+
},
138+
"nbformat": 4,
139+
"nbformat_minor": 5
140+
}

‎docs/docs/integrations/providers/cratedb.mdx‎

Lines changed: 66 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -27,7 +27,7 @@ docker run --name=cratedb --rm \
2727
[free trial][CrateDB Cloud Console].
2828

2929
### Install Client
30-
Install the most recent version of the `langchain-cratedb` package
30+
Install the most recent version of the [langchain-cratedb] package
3131
and a few others that are needed for this tutorial.
3232
```bash
3333
pip install --upgrade langchain-cratedb langchain-openai unstructured
@@ -120,13 +120,78 @@ message_history = CrateDBChatMessageHistory(
120120
message_history.add_user_message("hi!")
121121
```
122122

123+
### Full Cache
124+
The standard / full cache avoids invoking the LLM when the supplied
125+
prompt is exactly the same as one encountered already.
126+
See also [CrateDBCache Example].
127+
128+
To use the full cache in your applications:
129+
```python
130+
import sqlalchemy as sa
131+
from langchain.globals import set_llm_cache
132+
from langchain_openai import ChatOpenAI, OpenAIEmbeddings
133+
from langchain_cratedb import CrateDBCache
134+
135+
# Configure cache.
136+
engine = sa.create_engine("crate://crate@localhost:4200/?schema=testdrive")
137+
set_llm_cache(CrateDBCache(engine))
138+
139+
# Invoke LLM conversation.
140+
llm = ChatOpenAI(
141+
model_name="chatgpt-4o-latest",
142+
temperature=0.7,
143+
)
144+
print()
145+
print("Asking with full cache:")
146+
answer = llm.invoke("What is the answer to everything?")
147+
print(answer.content)
148+
```
149+
150+
### Semantic Cache
151+
152+
The semantic cache allows users to retrieve cached prompts based on semantic
153+
similarity between the user input and previously cached inputs. It also avoids
154+
invoking the LLM when not needed.
155+
See also [CrateDBSemanticCache Example].
156+
157+
To use the semantic cache in your applications:
158+
```python
159+
import sqlalchemy as sa
160+
from langchain.globals import set_llm_cache
161+
from langchain_openai import ChatOpenAI, OpenAIEmbeddings
162+
from langchain_cratedb import CrateDBSemanticCache
163+
164+
# Configure embeddings.
165+
embeddings = OpenAIEmbeddings(model="text-embedding-3-small")
166+
167+
# Configure cache.
168+
engine = sa.create_engine("crate://crate@localhost:4200/?schema=testdrive")
169+
set_llm_cache(
170+
CrateDBSemanticCache(
171+
embedding=embeddings,
172+
connection=engine,
173+
search_threshold=1.0,
174+
)
175+
)
176+
177+
# Invoke LLM conversation.
178+
llm = ChatOpenAI(model_name="chatgpt-4o-latest")
179+
print()
180+
print("Asking with semantic cache:")
181+
answer = llm.invoke("What is the answer to everything?")
182+
print(answer.content)
183+
```
184+
123185

124186
[all features of CrateDB]: https://cratedb.com/docs/guide/feature/
125187
[CrateDB]: https://cratedb.com/database
126188
[CrateDB Cloud]: https://cratedb.com/database/cloud
127189
[CrateDB Cloud Console]: https://console.cratedb.cloud/?utm_source=langchain&utm_content=documentation
128190
[CrateDB installation options]: https://cratedb.com/docs/guide/install/
191+
[CrateDBCache Example]: https://github.com/crate/langchain-cratedb/blob/main/examples/basic/cache.py
192+
[CrateDBSemanticCache Example]: https://github.com/crate/langchain-cratedb/blob/main/examples/basic/cache.py
129193
[CrateDBChatMessageHistory Tutorial]: https://github.com/crate/cratedb-examples/blob/main/topic/machine-learning/llm-langchain/conversational_memory.ipynb
130194
[CrateDBLoader Tutorial]: https://github.com/crate/cratedb-examples/blob/main/topic/machine-learning/llm-langchain/document_loader.ipynb
131195
[CrateDBVectorStore Tutorial]: https://github.com/crate/cratedb-examples/blob/main/topic/machine-learning/llm-langchain/vector_search.ipynb
196+
[langchain-cratedb]: https://pypi.org/project/langchain-cratedb/
132197
[using LangChain with CrateDB]: https://cratedb.com/docs/guide/integrate/langchain/

‎docs/docs/integrations/providers/dappier.mdx‎

Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,48 @@
1+
# Dappier
2+
3+
[Dappier](https://dappier.com) connects any LLM or your Agentic AI to
4+
real-time, rights-cleared, proprietary data from trusted sources,
5+
making your AI an expert in anything. Our specialized models include
6+
Real-Time Web Search, News, Sports, Financial Stock Market Data,
7+
Crypto Data, and exclusive content from premium publishers. Explore a
8+
wide range of data models in our marketplace at
9+
[marketplace.dappier.com](https://marketplace.dappier.com).
10+
11+
[Dappier](https://dappier.com) delivers enriched, prompt-ready, and
12+
contextually relevant data strings, optimized for seamless integration
13+
with LangChain. Whether you're building conversational AI, recommendation
14+
engines, or intelligent search, Dappier's LLM-agnostic RAG models ensure
15+
your AI has access to verified, up-to-date data—without the complexity of
16+
building and managing your own retrieval pipeline.
17+
18+
## Installation and Setup
19+
20+
Install ``langchain-dappier`` and set environment variable
21+
``DAPPIER_API_KEY``.
22+
23+
```bash
24+
pip install -U langchain-dappier
25+
export DAPPIER_API_KEY="your-api-key"
26+
```
27+
28+
We also need to set our Dappier API credentials, which can be generated at
29+
the [Dappier site.](https://platform.dappier.com/profile/api-keys).
30+
31+
We can find the supported data models by heading over to the
32+
[Dappier marketplace.](https://platform.dappier.com/marketplace)
33+
34+
## Chat models
35+
36+
See a [usage example](/docs/integrations/chat/dappier).
37+
38+
```python
39+
from langchain_community.chat_models import ChatDappierAI
40+
```
41+
42+
## Retriever
43+
44+
See a [usage example](/docs/integrations/retrievers/dappier).
45+
46+
```python
47+
from langchain_dappier import DappierRetriever
48+
```

‎docs/docs/integrations/providers/dappierai.mdx‎

Lines changed: 0 additions & 18 deletions
This file was deleted.

‎docs/docs/integrations/providers/pull-md.mdx‎

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,42 @@
1+
# PullMd Loader
2+
3+
>[PullMd](https://pull.md/) is a service that converts web pages into Markdown format. The `langchain-pull-md` package utilizes this service to convert URLs, especially those rendered with JavaScript frameworks like React, Angular, or Vue.js, into Markdown without the need for local rendering.
4+
5+
## Installation and Setup
6+
7+
To get started with `langchain-pull-md`, you need to install the package via pip:
8+
9+
```bash
10+
pip install langchain-pull-md
11+
```
12+
13+
See the [usage example](/docs/integrations/document_loaders/pull_md) for detailed integration and usage instructions.
14+
15+
## Document Loader
16+
17+
The `PullMdLoader` class in `langchain-pull-md` provides an easy way to convert URLs to Markdown. It's particularly useful for loading content from modern web applications for use within LangChain's processing capabilities.
18+
19+
```python
20+
from langchain_pull_md import PullMdLoader
21+
22+
# Initialize the loader with a URL of a JavaScript-rendered webpage
23+
loader = PullMdLoader(url='https://example.com')
24+
25+
# Load the content as a Document
26+
documents = loader.load()
27+
28+
# Access the Markdown content
29+
for document in documents:
30+
print(document.page_content)
31+
```
32+
33+
This loader supports any URL and is particularly adept at handling sites built with dynamic JavaScript, making it a versatile tool for markdown extraction in data processing workflows.
34+
35+
## API Reference
36+
37+
For a comprehensive guide to all available functions and their parameters, visit the [API reference](https://github.com/chigwell/langchain-pull-md).
38+
39+
## Additional Resources
40+
41+
- [GitHub Repository](https://github.com/chigwell/langchain-pull-md)
42+
- [PyPi Package](https://pypi.org/project/langchain-pull-md/)

0 commit comments

Comments
 (0)