Options and Runtime Params

[BWC4WIFE]

Do these options override what is set in the Modelfile? Is there anyway to check which params are actually being used ?

Additionally -- some params (TFS Z) if manually set in the modelfile are "unknown" for an Ollama instance --

Is there a doc or a place we can get more information on the function of each parameter -- for instance Vocab Only, MMAP, NUMA

[fmaclen]

By default Hollama also relies on the default values Ollama applies. They used to document this in modelfile.md in greater detail but it looks like they updated it at some point. There is some extra info api.md.

The currently placeholder values you see in the model settings page in Hollama were sourced from that original Ollama docs.

On that note, I'd like to add some tooltips below each of those options to explain what they do: #256

I ran this question by Perplexity and this is what it came up with.

Note: haven't double checked these.

Model Generation Parameters

Parameter	Description	Default Value
Num keep	Controls how many tokens from the initial prompt to retain in the context window	Auto (typically 4)
Seed	Sets the random number seed for generation; using a specific number makes the model generate consistent text for the same prompt	Random (-1)
Num predict	Maximum number of tokens to predict when generating text	128
Top K	Reduces probability of nonsense by limiting token selection to the K most likely tokens; higher values (e.g., 100) give more diverse answers, lower values (e.g., 10) produce more conservative text	40
Top P	Works with top-k for nucleus sampling; higher values (e.g., 0.95) lead to more diverse text, lower values (e.g., 0.5) generate more focused text	0.9
Min P	Alternative to top_p that ensures balance of quality and variety; represents minimum probability for token consideration relative to most likely token	0.0
TFS Z	Tail-free sampling to reduce impact of less probable tokens; higher value (e.g., 2.0) reduces impact more, 1.0 disables this setting	1.0
Typical P	Controls how closely the model follows typical text patterns	Auto (1.0)
Repeat last N	Sets how far back the model looks to prevent repetition	64 tokens
Temperature	Controls randomness and creativity; higher values produce more creative but potentially less coherent text	0.8
Repeat penalty	How strongly to penalize repetitions; higher values (e.g., 1.5) penalize repetitions more strongly	1.1
Presence penalty	Reduces likelihood of repeating the same content	Auto (0.0)
Frequency penalty	Reduces repetition of frequent tokens	Auto (0.0)
Mirostat	Sampling method to control perplexity (0 = disabled, 1 = Mirostat, 2 = Mirostat 2.0)	0
Mirostat Tau	Controls balance between coherence and diversity; lower values result in more focused text	5.0
Mirostat ETA	Influences how quickly algorithm responds to feedback; lower learning rate = slower adjustments	0.1
Stop sequence	Patterns that will cause the model to stop generating text	Auto
Penalize newline	Whether to penalize newline characters	Unchecked (false)

Runtime Options

Parameter	Description	Default Value
Context size	Sets the size of the context window used to generate the next token	2048 tokens
Num batch	Prompt processing maximum batch size	Auto (512)
Num GPU	Number of layers to send to GPU(s)	Auto (-1, set dynamically)
Main GPU	Controls which GPU is used for small tensors when using multiple GPUs	Auto (0)
Number of threads	Number of CPU threads to use during computation	Auto (0 = runtime decides)
NUMA	Whether to use Non-Uniform Memory Access	Unchecked (false)
Low VRAM	Option for systems with limited video memory	Unchecked (false)
F16 KV	Whether to use 16-bit floating point for key-value operations	Unchecked (true by default)
Vocab only	Load only the vocabulary, not the weights	Unchecked (false)
Use MMAP	Whether to memory-map the model file; improves memory usage efficiency but may cause performance issues if model is larger than available RAM	Unchecked
Use MLOCK	Locks the model in memory, preventing it from being swapped out	Unchecked (false)