Skip to content

Commit bb7bc0e

Browse files
authored
Update README.md
1 parent f28efde commit bb7bc0e

File tree

1 file changed

+1
-0
lines changed

1 file changed

+1
-0
lines changed

README.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -145,6 +145,7 @@ chmod +x run_streamlit.sh
145145
---
146146

147147
## Note
148+
- Latest scripts in this repo will query all your models at same time. If you want quick output, load your ollama models in your machine before starting any streamlit commands. By default ollama loads model for 5 mins and unloads them automatically after 5 mins. To keep them for longer time, use OLLAMA_KEEP_ALIVE ollama parameter and set time in mins (Ex: 30m) or hours (Ex: 4h). So, this script will query all models at same time and as the models are loaded already, output will be faster.
148149
- Vertical view denotes prompt and models are in vertical layout to the left. Horiztontal view denotes prompt and models are in horizontal layout.
149150
- In vertical view, you can move prompt and models window to the right as needed and move them to the left, to give more space to your output window.
150151
- Using this streamlit site you can run multiple LLMs at same time. But if your results shows one after the other, you should set OLLAMA_MAX_LOADED_MODELS = 2 (or any number as your hardware supports). Refer to Ollama documentation on how to use it in your OS version.

0 commit comments

Comments
 (0)