Skip to content

Commit ea3b600

Browse files
committed
add phi4 usage
1 parent a9e9b23 commit ea3b600

File tree

1 file changed

+45
-0
lines changed

1 file changed

+45
-0
lines changed

docs/apps/15_streamlit_chat_slm.md

Lines changed: 45 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,51 @@ $ poetry run python apps/15_streamlit_chat_slm/summarize.py
1919
$ poetry run python -m streamlit run apps/15_streamlit_chat_slm/main.py
2020
```
2121

22+
## Usage
23+
24+
### Use Phi4 model
25+
26+
```shell
27+
$ time poetry run python apps/15_streamlit_chat_slm/chat.py --model phi4 --prompt "hello"
28+
{
29+
"content": "Hello! How can I assist you today? If you have any questions or need information, feel free to let me know. 😊",
30+
"additional_kwargs": {},
31+
"response_metadata": {
32+
"model": "phi4",
33+
"created_at": "2025-01-09T03:16:19.661532868Z",
34+
"message": {
35+
"role": "assistant",
36+
"content": ""
37+
},
38+
"done_reason": "stop",
39+
"done": true,
40+
"total_duration": 10662476906,
41+
"load_duration": 10769327,
42+
"prompt_eval_count": 23,
43+
"prompt_eval_duration": 426000000,
44+
"eval_count": 28,
45+
"eval_duration": 10223000000
46+
},
47+
"type": "ai",
48+
"name": null,
49+
"id": "run-16375018-116e-422f-afd4-84692af42bd6-0",
50+
"example": false,
51+
"tool_calls": [],
52+
"invalid_tool_calls": [],
53+
"usage_metadata": {
54+
"input_tokens": 23,
55+
"output_tokens": 28,
56+
"total_tokens": 51
57+
}
58+
}
59+
poetry run python apps/15_streamlit_chat_slm/chat.py --model phi4 --prompt 1.48s user 0.12s system 12% cpu 12.455 total
60+
```
61+
62+
Note:
63+
64+
- Update Ollama to the latest version to run phi4 model
65+
- To use Ollama on WSL2, you may need to enable systemd. For more information, see [Use systemd to manage Linux services with WSL](https://learn.microsoft.com/en-us/windows/wsl/systemd#how-to-enable-systemd)
66+
2267
# References
2368

2469
- [ChatOllama](https://python.langchain.com/docs/integrations/chat/ollama/)

0 commit comments

Comments
 (0)