@@ -3,7 +3,21 @@ title: Hugging Face (chat)
3
3
sidebar_label : Hugging Face
4
4
---
5
5
6
- ## Chat models with Hugging Face
6
+ ## Overview
7
+
8
+ This page shows how to use Hugging Face models as chat models in LangChain.
9
+
10
+ ## Setup
11
+
12
+ Install the required packages:
13
+
14
+ ``` bash
15
+ pip install langchain-huggingface transformers
16
+ ```
17
+
18
+ > For Hugging Face pipelines, prefer ` max_new_tokens ` (not ` max_tokens ` ). The pipeline will use CPU/GPU automatically depending on availability.
19
+
20
+ ## Instantiation
7
21
8
22
### Option 1 (works today): pipeline → wrap with ` ChatHuggingFace `
9
23
@@ -15,24 +29,17 @@ from langchain_huggingface import ChatHuggingFace
15
29
pipe = pipeline(
16
30
" text-generation" ,
17
31
model = " microsoft/Phi-3-mini-4k-instruct" ,
18
- do_sample = False , # deterministic (similar to temperature=0)
32
+ do_sample = False , # deterministic
19
33
max_new_tokens = 128 , # HF uses max_new_tokens (not max_tokens)
20
34
)
21
35
22
36
# Wrap the pipeline as a LangChain chat model
23
37
llm = ChatHuggingFace(llm = pipe)
24
-
25
- print (llm.invoke(" Say hi in one sentence." ).content)
26
38
```
27
39
28
- :::note
29
- - ** Install** : ` pip install langchain-huggingface transformers `
30
- - For Hugging Face pipelines prefer ` max_new_tokens ` (not ` max_tokens ` ).
31
- :::
32
-
33
40
### Option 2 (coming after fix): ` init_chat_model(..., model_provider="huggingface") `
34
41
35
- Once available, you’ll be able to initialize via ` init_chat_model ` :
42
+ Once available in your version, you can initialize via ` init_chat_model ` :
36
43
37
44
``` python
38
45
from langchain.chat_models import init_chat_model
@@ -44,10 +51,36 @@ llm = init_chat_model(
44
51
do_sample = False ,
45
52
max_new_tokens = 128 ,
46
53
)
54
+ ```
47
55
48
- print (llm.invoke(" Say hi in one sentence." ).content)
56
+ > If your version doesn’t support this yet, use ** Option 1** above.
57
+
58
+ ## Invocation
59
+
60
+ ``` python
61
+ msg = llm.invoke(" Say hi in one sentence." )
62
+ print (msg.content)
63
+ ```
64
+
65
+ ## Chaining
66
+
67
+ ``` python
68
+ from langchain_core.prompts import ChatPromptTemplate
69
+
70
+ prompt = ChatPromptTemplate.from_messages([
71
+ (" system" , " You are helpful." ),
72
+ (" human" , " {question} " ),
73
+ ])
74
+
75
+ chain = prompt | llm
76
+ result = chain.invoke({" question" : " What is the capital of France?" })
77
+ print (result.content)
49
78
```
50
79
51
- This path depends on a bug fix tracked for Hugging Face chat initialization. If your version doesn’t support it yet, use Option 1 above.
80
+ ## API reference
81
+
82
+ - ` langchain_huggingface.ChatHuggingFace `
83
+ - ` transformers.pipeline ` (Hugging Face)
84
+ - ` langchain.chat_models.init_chat_model ` (when available for Hugging Face)
52
85
53
86
0 commit comments