-
Notifications
You must be signed in to change notification settings - Fork 19.2k
docs(huggingface): add chat usage guide and regression tests for init_chat_model #33194
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Closed
Accidental-MVP
wants to merge
9
commits into
langchain-ai:master
from
Accidental-MVP:tests/hf-init-chat-model
+224
−0
Closed
Changes from all commits
Commits
Show all changes
9 commits
Select commit
Hold shift + click to select a range
0c521e3
test(chat): add xfail regression tests for Hugging Face init_chat_model
Accidental-MVP aa7ba0f
docs(huggingface): add pipeline→wrap guide; note upcoming init_chat_m…
Accidental-MVP 3f9f2e9
test(huggingface): remove requires marker to avoid extended-tests ins…
Accidental-MVP 8ed6195
docs(huggingface): conform chat page to required headers
Accidental-MVP 47ff6b7
test(huggingface): ruff fixes; remove requires marker for extended tests
Accidental-MVP 9de7872
tests(huggingface): fix docstring formatting to satisfy D205
Accidental-MVP 9e19a82
test(huggingface): update monkeypatching to use setattr for module at…
Accidental-MVP 882e9f0
test(huggingface): add noqa comments for B010 linting issues in monke…
Accidental-MVP 4ebf335
Merge branch 'master' into tests/hf-init-chat-model
Accidental-MVP File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,86 @@ | ||
--- | ||
title: Hugging Face (chat) | ||
sidebar_label: Hugging Face | ||
--- | ||
|
||
## Overview | ||
|
||
This page shows how to use Hugging Face models as chat models in LangChain. | ||
|
||
## Setup | ||
|
||
Install the required packages: | ||
|
||
```bash | ||
pip install langchain-huggingface transformers | ||
``` | ||
|
||
> For Hugging Face pipelines, prefer `max_new_tokens` (not `max_tokens`). The pipeline will use CPU/GPU automatically depending on availability. | ||
|
||
## Instantiation | ||
|
||
### Option 1 (works today): pipeline → wrap with `ChatHuggingFace` | ||
|
||
```python | ||
from transformers import pipeline | ||
from langchain_huggingface import ChatHuggingFace | ||
|
||
# Create a text-generation pipeline (CPU/GPU as available) | ||
pipe = pipeline( | ||
"text-generation", | ||
model="microsoft/Phi-3-mini-4k-instruct", | ||
do_sample=False, # deterministic | ||
max_new_tokens=128, # HF uses max_new_tokens (not max_tokens) | ||
) | ||
|
||
# Wrap the pipeline as a LangChain chat model | ||
llm = ChatHuggingFace(llm=pipe) | ||
``` | ||
|
||
### Option 2 (coming after fix): `init_chat_model(..., model_provider="huggingface")` | ||
|
||
Once available in your version, you can initialize via `init_chat_model`: | ||
|
||
```python | ||
from langchain.chat_models import init_chat_model | ||
|
||
llm = init_chat_model( | ||
model="microsoft/Phi-3-mini-4k-instruct", | ||
model_provider="huggingface", | ||
task="text-generation", | ||
do_sample=False, | ||
max_new_tokens=128, | ||
) | ||
``` | ||
|
||
> If your version doesn’t support this yet, use **Option 1** above. | ||
|
||
## Invocation | ||
|
||
```python | ||
msg = llm.invoke("Say hi in one sentence.") | ||
print(msg.content) | ||
``` | ||
|
||
## Chaining | ||
|
||
```python | ||
from langchain_core.prompts import ChatPromptTemplate | ||
|
||
prompt = ChatPromptTemplate.from_messages([ | ||
("system", "You are helpful."), | ||
("human", "{question}"), | ||
]) | ||
|
||
chain = prompt | llm | ||
result = chain.invoke({"question": "What is the capital of France?"}) | ||
print(result.content) | ||
``` | ||
|
||
## API reference | ||
|
||
- `langchain_huggingface.ChatHuggingFace` | ||
- `transformers.pipeline` (Hugging Face) | ||
- `langchain.chat_models.init_chat_model` (when available for Hugging Face) | ||
|
||
|
138 changes: 138 additions & 0 deletions
138
libs/langchain/tests/unit_tests/chat_models/test_init_chat_model_hf.py
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,138 @@ | ||
import sys | ||
import types | ||
from importlib import util as import_util | ||
from types import SimpleNamespace | ||
from typing import Any, Optional | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We want to handle this via integration tests rather than in chat models. Could you remove the unit test? |
||
|
||
import pytest | ||
|
||
from langchain.chat_models import init_chat_model | ||
|
||
|
||
@pytest.fixture | ||
def hf_fakes(monkeypatch: pytest.MonkeyPatch) -> SimpleNamespace: | ||
"""Install fakes for Hugging Face and transformers. | ||
|
||
Capture call arguments and simulate module presence to test initialization | ||
behavior, including current failure modes. | ||
""" | ||
pipeline_calls: list[tuple[str, dict[str, Any]]] = [] | ||
init_calls: list[dict[str, Any]] = [] | ||
|
||
# Fake transformers.pipeline | ||
def fake_pipeline(task: str, **kwargs: Any) -> SimpleNamespace: | ||
pipeline_calls.append((task, dict(kwargs))) | ||
return SimpleNamespace(_kind="dummy_hf_pipeline") | ||
|
||
transformers_mod = types.ModuleType("transformers") | ||
setattr(transformers_mod, "pipeline", fake_pipeline) # noqa: B010 | ||
monkeypatch.setitem(sys.modules, "transformers", transformers_mod) | ||
|
||
# Fake langchain_huggingface.ChatHuggingFace that REQUIRES `llm` | ||
class FakeChatHuggingFace: | ||
def __init__(self, *, llm: object, **kwargs: Any) -> None: | ||
init_calls.append({"llm": llm, "kwargs": dict(kwargs)}) | ||
self._llm = llm | ||
self._kwargs = kwargs | ||
|
||
# Build full package path: | ||
# langchain_huggingface.chat_models.huggingface | ||
hf_pkg = types.ModuleType("langchain_huggingface") | ||
hf_pkg.__path__ = [] # mark as package | ||
|
||
hf_chat_models_pkg = types.ModuleType("langchain_huggingface.chat_models") | ||
hf_chat_models_pkg.__path__ = [] # mark as package | ||
|
||
hf_chat_hf_mod = types.ModuleType( | ||
"langchain_huggingface.chat_models.huggingface", | ||
) | ||
setattr(hf_chat_hf_mod, "ChatHuggingFace", FakeChatHuggingFace) # noqa: B010 | ||
|
||
# Also expose at package root for top-level imports | ||
setattr(hf_pkg, "ChatHuggingFace", FakeChatHuggingFace) # noqa: B010 | ||
|
||
monkeypatch.setitem(sys.modules, "langchain_huggingface", hf_pkg) | ||
monkeypatch.setitem( | ||
sys.modules, | ||
"langchain_huggingface.chat_models", | ||
hf_chat_models_pkg, | ||
) | ||
monkeypatch.setitem( | ||
sys.modules, | ||
"langchain_huggingface.chat_models.huggingface", | ||
hf_chat_hf_mod, | ||
) | ||
|
||
# Ensure _check_pkg sees both packages as installed | ||
orig_find_spec = import_util.find_spec | ||
|
||
def fake_find_spec(name: str) -> Optional[object]: | ||
if name in { | ||
"transformers", | ||
"langchain_huggingface", | ||
"langchain_huggingface.chat_models", | ||
"langchain_huggingface.chat_models.huggingface", | ||
}: | ||
return object() | ||
return orig_find_spec(name) | ||
|
||
monkeypatch.setattr("importlib.util.find_spec", fake_find_spec) | ||
|
||
return SimpleNamespace( | ||
pipeline_calls=pipeline_calls, | ||
init_calls=init_calls, | ||
) | ||
|
||
|
||
def test_hf_current_bug_basic_raises_typeerror( | ||
hf_fakes: SimpleNamespace, | ||
) -> None: | ||
"""Current behavior raises TypeError when using Hugging Face provider. | ||
|
||
init_chat_model constructs ChatHuggingFace without ``llm`` and never builds | ||
a pipeline. Verify that explicitly. | ||
""" | ||
with pytest.raises(TypeError): | ||
_ = init_chat_model( | ||
"huggingface:microsoft/Phi-3-mini-4k-instruct", | ||
task="text-generation", | ||
temperature=0, | ||
) | ||
# Buggy path should not touch transformers.pipeline | ||
assert not hf_fakes.pipeline_calls, "pipeline should NOT be called" | ||
|
||
|
||
def test_hf_current_bug_max_tokens_case_raises_typeerror( | ||
hf_fakes: SimpleNamespace, | ||
) -> None: | ||
"""Same failure when passing ``max_tokens``. | ||
|
||
Should raise and avoid constructing a pipeline. | ||
""" | ||
with pytest.raises(TypeError): | ||
_ = init_chat_model( | ||
model="mistralai/Mistral-7B-Instruct-v0.2", | ||
model_provider="huggingface", | ||
task="text-generation", | ||
max_tokens=42, | ||
) | ||
assert not hf_fakes.pipeline_calls, "pipeline should NOT be called" | ||
|
||
|
||
def test_hf_current_bug_timeout_retries_case_raises_typeerror( | ||
hf_fakes: SimpleNamespace, | ||
) -> None: | ||
"""Same failure when passing ``timeout``/``max_retries``. | ||
|
||
Should raise and avoid constructing a pipeline. | ||
""" | ||
with pytest.raises(TypeError): | ||
_ = init_chat_model( | ||
model="microsoft/Phi-3-mini-4k-instruct", | ||
model_provider="huggingface", | ||
task="text-generation", | ||
temperature=0.1, | ||
timeout=7, | ||
max_retries=3, | ||
) | ||
assert not hf_fakes.pipeline_calls, "pipeline should NOT be called" |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could you add these to the new langchain docs?
https://docs.langchain.com/oss/python/contributing/integrations-langchain