Skip to content

Commit f288843

Browse files
committed
add phi4 usage
1 parent a9e9b23 commit f288843

File tree

1 file changed

+102
-1
lines changed

1 file changed

+102
-1
lines changed

docs/apps/15_streamlit_chat_slm.md

Lines changed: 102 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@
66
# Run Ollama server
77
$ ollama serve
88

9-
# Install dependencies
9+
# Pull Phi3 model
1010
$ ollama pull phi3
1111

1212
# Run a simple chat with Ollama
@@ -17,8 +17,109 @@ $ poetry run python apps/15_streamlit_chat_slm/summarize.py
1717

1818
# Run streamlit app
1919
$ poetry run python -m streamlit run apps/15_streamlit_chat_slm/main.py
20+
21+
# List models
22+
$ ollama list
23+
NAME ID SIZE MODIFIED
24+
phi3:latest 4f2222927938 2.2 GB 3 minutes ago
25+
phi4:latest ac896e5b8b34 9.1 GB 55 minutes ago
26+
```
27+
28+
## Usage
29+
30+
### Use Phi3 model
31+
32+
```shell
33+
# Pull Phi3 model
34+
$ ollama pull phi3
35+
36+
# Measure time to run the chat
37+
$ time poetry run python apps/15_streamlit_chat_slm/chat.py \
38+
--model phi3 \
39+
--prompt "hello"
40+
{
41+
"content": "Hello! How can I help you today?",
42+
"additional_kwargs": {},
43+
"response_metadata": {
44+
"model": "phi3",
45+
"created_at": "2025-01-09T03:30:05.706397262Z",
46+
"message": {
47+
"role": "assistant",
48+
"content": ""
49+
},
50+
"done_reason": "stop",
51+
"done": true,
52+
"total_duration": 540964618,
53+
"load_duration": 5078297,
54+
"prompt_eval_count": 22,
55+
"prompt_eval_duration": 229000000,
56+
"eval_count": 10,
57+
"eval_duration": 305000000
58+
},
59+
"type": "ai",
60+
"name": null,
61+
"id": "run-54f0d2a6-b3d4-4ef5-b009-0c72b23297ac-0",
62+
"example": false,
63+
"tool_calls": [],
64+
"invalid_tool_calls": [],
65+
"usage_metadata": {
66+
"input_tokens": 22,
67+
"output_tokens": 10,
68+
"total_tokens": 32
69+
}
70+
}
71+
poetry run python apps/15_streamlit_chat_slm/chat.py --model phi3 --prompt 1.57s user 0.16s system 68% cpu 2.515 total
2072
```
2173

74+
### Use Phi4 model
75+
76+
```shell
77+
# Pull Phi4 model
78+
$ ollama pull phi4
79+
80+
# Measure time to run the chat
81+
$ time poetry run python apps/15_streamlit_chat_slm/chat.py \
82+
--model phi4 \
83+
--prompt "hello"
84+
{
85+
"content": "Hello! How can I assist you today? If you have any questions or need information, feel free to let me know. 😊",
86+
"additional_kwargs": {},
87+
"response_metadata": {
88+
"model": "phi4",
89+
"created_at": "2025-01-09T03:16:19.661532868Z",
90+
"message": {
91+
"role": "assistant",
92+
"content": ""
93+
},
94+
"done_reason": "stop",
95+
"done": true,
96+
"total_duration": 10662476906,
97+
"load_duration": 10769327,
98+
"prompt_eval_count": 23,
99+
"prompt_eval_duration": 426000000,
100+
"eval_count": 28,
101+
"eval_duration": 10223000000
102+
},
103+
"type": "ai",
104+
"name": null,
105+
"id": "run-16375018-116e-422f-afd4-84692af42bd6-0",
106+
"example": false,
107+
"tool_calls": [],
108+
"invalid_tool_calls": [],
109+
"usage_metadata": {
110+
"input_tokens": 23,
111+
"output_tokens": 28,
112+
"total_tokens": 51
113+
}
114+
}
115+
poetry run python apps/15_streamlit_chat_slm/chat.py --model phi4 --prompt 1.48s user 0.12s system 12% cpu 12.455 total
116+
```
117+
118+
Note:
119+
120+
- Update Ollama to the latest version to run phi4 model
121+
- To use Ollama on WSL2, you may need to enable systemd. For more information, see [Use systemd to manage Linux services with WSL](https://learn.microsoft.com/en-us/windows/wsl/systemd#how-to-enable-systemd)
122+
22123
# References
23124

24125
- [ChatOllama](https://python.langchain.com/docs/integrations/chat/ollama/)

0 commit comments

Comments
 (0)