llama-run : fix context size (ggml-org#11094)

ericcurtin · tinglou · commit 015493573297 · 2025-02-13T23:04:04.000+08:00
Set `n_ctx` equal to `n_batch` in `Opt` class. Now context size is
a more reasonable 2048.

Signed-off-by: Eric Curtin &lt;ecurtin@redhat.com&gt;
diff --git a/examples/run/run.cpp b/examples/run/run.cpp
@@ -83,6 +83,7 @@ class Opt {
         }
 
         ctx_params.n_batch        = context_size >= 0 ? context_size : context_size_default;
+        ctx_params.n_ctx          = ctx_params.n_batch;
         model_params.n_gpu_layers = ngl >= 0 ? ngl : ngl_default;
         temperature               = temperature >= 0 ? temperature : temperature_default;
 

Original file line number	Diff line number	Diff line change
`@@ -83,6 +83,7 @@ class Opt {`
`83`	`83`	`}`
`84`	`84`
`85`	`85`	`ctx_params.n_batch = context_size >= 0 ? context_size : context_size_default;`
	`86`	`+ ctx_params.n_ctx = ctx_params.n_batch;`
`86`	`87`	`model_params.n_gpu_layers = ngl >= 0 ? ngl : ngl_default;`
`87`	`88`	`temperature = temperature >= 0 ? temperature : temperature_default;`
`88`	`89`