Add runtime images without models

[sozercan]

Summary

Closes #719

• Add "runner" images that contain only the inference runtime (LocalAI + backends) without model weights — users pass a model reference at docker run time and it is downloaded at container startup
• Runner mode detected implicitly when an aikitfile has backends but no models
• Refactor copyModels() to extract writeConfig() for reuse in runner path
• Add runner build logic (runner.go): isRunnerMode(), entrypoint script generation with backend-specific download logic (GGUF for llama-cpp, HF model config for diffusers/vllm), dependency installation (curl, huggingface-cli)
• Update NewImageConfig() to set /usr/local/bin/aikit-runner entrypoint and add runner labels
• Add 4 runner aikitfile definitions: llama-cpp-cpu, llama-cpp-cuda, diffusers-cuda, vllm-cuda
• Add CI: docker-test-runner workflow (runs on push/PR, tests CPU runner) and docker-test-runner-gpu workflow (manual trigger, tests all GPU runners on self-hosted GPU)
• Add unit tests for runner mode detection and script generation

Usage:

docker run -p 8080:8080 <runner-image> unsloth/gemma-3-1b-it-GGUF
docker run --gpus all -p 8080:8080 <cuda-runner-image> Qwen/Qwen2.5-0.5B-Instruct

Test plan

• go test ./... — all existing + new unit tests pass
• golangci-lint run ./... — 0 lint issues
• docker-test-runner CI workflow builds CPU runner and validates chat completions
• docker-test-runner-gpu CI workflow builds GPU runners and validates inference (manual trigger)
• Build a runner image locally: docker buildx build -t runner-test:latest -f runners/llama-cpp-cpu.yaml .
• Run with a small model: docker run -p 8080:8080 runner-test:latest unsloth/gemma-3-1b-it-GGUF
• Verify volume caching: docker run -p 8080:8080 -v model-cache:/models runner-test:latest unsloth/gemma-3-1b-it-GGUF (skips download on second run)

[Copilot]

Pull request overview

Adds a new “runner image” mode to build runtime-only AIKit images (LocalAI + backend dependencies) without bundling model weights, so users can pass a model reference at docker run time and have it downloaded on container startup.

Changes:

• Introduces runner-mode detection (backends present, models absent) and a generated /usr/local/bin/aikit-runner entrypoint script that downloads/models/configures at runtime.
• Refactors config writing out of copyModels() into writeConfig() and wires runner-mode build flow to skip model copying while still installing runner dependencies.
• Adds runner aikitfile definitions plus CPU/GPU GitHub Actions workflows to build and validate runner images, along with unit tests.

Reviewed changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
runners/llama-cpp-cpu.yaml	New runner aikitfile for CPU llama-cpp runtime-only image.
runners/llama-cpp-cuda.yaml	New runner aikitfile for CUDA llama-cpp runtime-only image.
runners/diffusers-cuda.yaml	New runner aikitfile for CUDA diffusers runtime-only image.
runners/vllm-cuda.yaml	New runner aikitfile for CUDA vLLM runtime-only image.
pkg/build/build_test.go	Extends inference config validation tests to accept runner-mode configs (backends without models).
pkg/aikit2llb/inference/runner.go	Implements runner-mode detection, dependency installation, entrypoint install, and script generation for runtime downloads/config.
pkg/aikit2llb/inference/runner_test.go	Adds unit tests for runner-mode detection and generated script content for supported backends.
pkg/aikit2llb/inference/image.go	Sets runner entrypoint and labels in image config when runner mode is active.
pkg/aikit2llb/inference/convert.go	Adds runner-mode build path, and factors config creation into reusable writeConfig().
.github/workflows/test-docker-runner.yaml	New CI workflow to build/test the CPU runner image and validate chat completions.
.github/workflows/test-docker-runner-gpu.yaml	New manual CI workflow to build/test GPU runner images on self-hosted GPU runners.

Comments suppressed due to low confidence (1)

.github/workflows/test-docker-runner-gpu.yaml:92

• Similarly, jq '.data[0].url' will output null on failures and the -z check won’t catch it. Use jq -e to assert .data[0].url is a non-null string (and/or validate .data|length>0) so the diffusers runner test fails when image generation fails.

          url=$(echo "$result" | jq '.data[0].url')
          if [ -z "$url" ]; then
            exit 1
          fi

[Copilot]

The dependency install shell chain mixes && and || without grouping. As written, an apt-get failure can be masked because the || pip install ... branch may run and the overall command can still exit 0 after apt-get clean. Consider grouping the pip fallback only (e.g., wrap the two pip install attempts in parentheses) so apt-get failures still fail the build.

Suggested change

	utils.Sh("apt-get update && apt-get install --no-install-recommends -y curl python3 python3-pip && pip install --break-system-packages huggingface-hub[cli] 2>/dev/null || pip install huggingface-hub[cli] && apt-get clean"),
	utils.Sh("apt-get update && apt-get install --no-install-recommends -y curl python3 python3-pip && (pip install --break-system-packages huggingface-hub[cli] 2>/dev/null || pip install huggingface-hub[cli]) && apt-get clean"),

[Copilot]

The runner dependency install step unconditionally uses apt-get, but utils.AppleSiliconBase is Fedora-based (no apt). Runner mode with runtime: applesilicon will fail at build time. Consider branching on runtime (or detecting available package manager) and using dnf on Apple Silicon, or explicitly disallowing runner mode for that runtime in validation with a clear error.

Suggested change

	s = s.Run(
	utils.Sh("apt-get update && apt-get install --no-install-recommends -y curl python3 python3-pip && pip install --break-system-packages huggingface-hub[cli] 2>/dev/null || pip install huggingface-hub[cli] && apt-get clean"),
	installerCmd := `if command -v apt-get >/dev/null 2>&1; then \
	apt-get update && apt-get install --no-install-recommends -y curl python3 python3-pip && \
	pip install --break-system-packages huggingface-hub[cli] 2>/dev/null || pip install huggingface-hub[cli] && \
	apt-get clean; \
	elif command -v dnf >/dev/null 2>&1; then \
	dnf install -y curl python3 python3-pip && \
	pip install --break-system-packages huggingface-hub[cli] 2>/dev/null || pip install huggingface-hub[cli] && \
	dnf clean all; \
	else \
	echo "No supported package manager (apt-get or dnf) found; cannot install runner dependencies." >&2; \
	exit 1; \
	fi`
	s = s.Run(
	utils.Sh(installerCmd),

[Copilot]

LOCAL_AI_ARGS+=("${EXTRA_ARGS[@]+"${EXTRA_ARGS[@]}"}") does not append extra args correctly: the nested quotes become literal text, so multiple extra args can collapse into a single element containing quotes/spaces. With set -u, it’s safer to append conditionally based on array length (e.g., only append when ${#EXTRA_ARGS[@]} > 0) and then use LOCAL_AI_ARGS+=("${EXTRA_ARGS[@]}").

Suggested change

	sb.WriteString(`LOCAL_AI_ARGS+=("${EXTRA_ARGS[@]+"${EXTRA_ARGS[@]}"}")
	sb.WriteString(`if ((${#EXTRA_ARGS[@]})); then
	LOCAL_AI_ARGS+=("${EXTRA_ARGS[@]}")
	fi

[Copilot]

writeConfig builds a shell command with echo -n "%s" using the raw YAML string. This will break for configs containing quotes/newlines and also allows shell injection during the image build. Prefer writing the file via llb.Mkfile (or a here-doc that avoids interpolation) so the config content is treated as data, not shell syntax.

[Copilot]

This workflow’s validation can false-pass: jq '.choices' returns the string null on error responses, which is non-empty so [ -z "$choices" ] won’t fail. Use jq -e to assert a non-null, non-empty array (and/or explicitly fail if .error is present) so CI reliably detects runner failures.

Suggested change

	echo $result
	choices=$(echo "$result" | jq '.choices')
	if [ -z "$choices" ]; then
	exit 1
	fi
	echo "$result"
	echo "$result" | jq -e '
	if (.error? != null) then
	error("error field present in response")
	elif (.choices | type != "array" or (.choices | length) == 0) then
	error("choices must be a non-empty array")
	else
	.
	end
	' > /dev/null

[Copilot]

The JSON assertion here can false-pass: jq '.choices' yields null (non-empty string) if the API returns an error object, so the [ -z "$choices" ] check won’t catch failures. Consider jq -e '.choices and (.choices|type=="array") and (.choices|length>0)' (and fail on .error) to make the GPU runner test meaningful.

This issue also appears on line 89 of the same file.