Skip to content
Merged
Show file tree
Hide file tree
Changes from 13 commits
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
2a8e3aa
fix(onboard): offer Ollama upgrade when host version too old
laitingsheng May 25, 2026
d6c7199
refactor(onboard): extract Ollama install menu to satisfy entrypoint …
laitingsheng May 25, 2026
c39e816
refactor(onboard): brew upgrade on macOS, extract Ollama version helpers
laitingsheng May 25, 2026
cd08178
fix(onboard): force system upgrade and verify Ollama version after in…
laitingsheng May 25, 2026
4be19f8
fix(onboard): verify upgrade via running daemon and reject user-local…
laitingsheng May 25, 2026
e7d34ed
fix(onboard): consider running daemon version when deciding Ollama up…
laitingsheng May 25, 2026
262bbea
fix(onboard): scope Ollama daemon upgrade check to local host and lab…
laitingsheng May 25, 2026
c5b23c1
fix(onboard): pin resolved Ollama host after install and label stale …
laitingsheng May 25, 2026
7612370
fix(onboard): start local Ollama daemon when only Windows-host probe …
laitingsheng May 25, 2026
5c43b26
fix(onboard): re-probe loopback fresh instead of trusting cached reso…
laitingsheng May 25, 2026
3f2b8b5
fix(onboard): guard env=system upgrade against missing sudo and unblo…
laitingsheng May 25, 2026
373b05f
fix(onboard): stop stale Ollama daemon on macOS upgrade; drop unused …
laitingsheng May 25, 2026
8497696
fix(onboard): stop stale Linux Ollama daemon before relaunch on non-s…
laitingsheng May 25, 2026
0ecfbc0
Merge branch 'main' into fix/4178-ollama-version-upgrade
cv May 25, 2026
74b8723
refactor(onboard): split Ollama Linux upgrade helpers
cv May 25, 2026
ba1b68f
Merge branch 'main' into fix/4178-ollama-version-upgrade
cv May 25, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions docs/inference/use-local-inference.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,12 @@ The onboard wizard detects Ollama automatically when it is installed or running

If Ollama is installed but not running, NemoClaw starts it for you.
On macOS and Linux, the wizard can also offer to install Ollama when it is not present.
When the host Ollama is below the minimum version NemoClaw expects for its starter models (currently `0.7.0`), the wizard surfaces an explicit **Upgrade Ollama** entry in the provider menu instead of silently reusing the older daemon, and the express setup path resolves to that entry.
The wizard inspects both the CLI binary (`ollama --version`) and the locally running daemon (`/api/version` on `:11434`) so the upgrade entry still appears when only one side is stale, for example a fresh user-local binary paired with the original system daemon.
The gate skips Windows-host Ollama reached from WSL via `host.docker.internal`; the separate **Use / Start / Install Ollama on Windows host** entries handle that case and run their own actions on the Windows side.
The wizard then runs the platform's install/upgrade path: `brew upgrade ollama` on macOS, the official `https://ollama.com/install.sh` on Linux.
Upgrades on Linux always take the sudo-driven system path because the sudo-free user-local fallback would leave the existing system daemon on `:11434` serving the stale binary; if sudo is not available in a non-interactive run, NemoClaw refuses to silently downgrade the path and asks you to rerun interactively or upgrade Ollama manually.
After an upgrade finishes, NemoClaw re-probes the running daemon's `/api/version` and fails the run if the daemon still reports below the minimum. Fresh installs skip this re-probe because the bundled installers ship a daemon at or above the minimum.
On WSL, the wizard can use, start, restart, or install Ollama on the Windows host through PowerShell interop.

#### Linux Install Modes
Expand Down
2 changes: 1 addition & 1 deletion docs/reference/commands.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -1256,7 +1256,7 @@ Set them before running `nemoclaw onboard`.
| `NEMOCLAW_REASONING` | `true` or `false` | Overrides the model's reasoning-mode flag in the built OpenClaw config. |
| `NEMOCLAW_AGENT_HEARTBEAT_EVERY` | duration with `s`, `m`, or `h` suffix (for example `30m`, `1h`, or `0m`) | Overrides `agents.defaults.heartbeat.every` in the built OpenClaw config. Set `0m` to disable periodic agent turns. |
| `NEMOCLAW_OLLAMA_REQUIRE_TOOLS` | `0` to disable, anything else to keep the default | When set to `0`, skips the Ollama tool-calling capability check during local-inference onboarding. |
| `NEMOCLAW_OLLAMA_INSTALL_MODE` | `system`, `user`, or empty/unset | Pins the Linux Ollama install location. `system` runs the official `https://ollama.com/install.sh` (sudo, writes to `/usr/local`, configures systemd). `user` extracts the release tarball to `${HOME}/.local` without sudo and launches the daemon manually (no systemd; manual restart after reboot). Empty/unset auto-detects: root or passwordless `sudo` selects `system`; a non-interactive run without passwordless `sudo` selects `user`; an interactive shell falls back to `system` so the official installer can prompt for the password. Any other value is rejected. |
| `NEMOCLAW_OLLAMA_INSTALL_MODE` | `system`, `user`, or empty/unset | Pins the Linux Ollama install location. `system` runs the official `https://ollama.com/install.sh` (sudo, writes to `/usr/local`, configures systemd). `user` extracts the release tarball to `${HOME}/.local` without sudo and launches the daemon manually (no systemd; manual restart after reboot). Empty/unset auto-detects: root or passwordless `sudo` selects `system`; a non-interactive run without passwordless `sudo` selects `user`; an interactive shell falls back to `system` so the official installer can prompt for the password. Any other value is rejected. On the upgrade path the helper rejects `user` because a user-local install cannot replace the system daemon on `:11434`, and it also rejects `system` under `NEMOCLAW_NON_INTERACTIVE=1` when passwordless `sudo` is unavailable (the installer would hang on a hidden sudo prompt); the run exits with an actionable diagnostic instead. |
| `NEMOCLAW_PROXY_HOST` | hostname or IP | Overrides the sandbox-side outbound HTTP proxy host. Defaults to `10.200.0.1`. |
| `NEMOCLAW_PROXY_PORT` | integer port | Overrides the sandbox-side outbound HTTP proxy port. Defaults to `3128`. |
| `NEMOCLAW_OPENSHELL_BIN` | path | Overrides the `openshell` binary the CLI invokes. Defaults to `openshell` (resolved via `PATH`). |
Expand Down
7 changes: 7 additions & 0 deletions src/lib/inference/local.ts
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,13 @@ export const QWEN3_6_OLLAMA_MODEL = assertRegistryTag("qwen3.6:35b");

export type RunCaptureFn = (cmd: string | string[], opts?: { ignoreError?: boolean }) => string;

export {
getInstalledOllamaVersion,
getRunningOllamaDaemonVersion,
isOllamaVersionAtLeast,
MIN_OLLAMA_VERSION,
} from "./ollama-version";

export type RunCaptureExFn = (cmd: string[]) => CaptureResult;

// Hosts that the WSL-side onboard CLI tries when probing Ollama. Native Linux
Expand Down
72 changes: 72 additions & 0 deletions src/lib/inference/ollama-version.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
// SPDX-License-Identifier: Apache-2.0

import { describe, expect, it } from "vitest";

import {
getInstalledOllamaVersion,
getRunningOllamaDaemonVersion,
isOllamaVersionAtLeast,
MIN_OLLAMA_VERSION,
} from "../../../dist/lib/inference/ollama-version";

describe("Ollama version detection", () => {
it("parses 'ollama version is X.Y.Z' output", () => {
const capture = () => "ollama version is 0.6.2";
expect(getInstalledOllamaVersion(capture)).toBe("0.6.2");
});

it("returns null when ollama --version produces no output", () => {
const capture = () => "";
expect(getInstalledOllamaVersion(capture)).toBeNull();
});

it("returns null when ollama --version output has no version", () => {
const capture = () => "ollama: command not found";
expect(getInstalledOllamaVersion(capture)).toBeNull();
});

it("treats null/missing versions as below the minimum", () => {
expect(isOllamaVersionAtLeast(null, MIN_OLLAMA_VERSION)).toBe(false);
});

it("treats 0.6.2 as below the 0.7.0 floor", () => {
expect(isOllamaVersionAtLeast("0.6.2", "0.7.0")).toBe(false);
});

it("treats 0.7.0 as meeting the 0.7.0 floor", () => {
expect(isOllamaVersionAtLeast("0.7.0", "0.7.0")).toBe(true);
});

it("treats 0.24.0 as above the 0.7.0 floor", () => {
expect(isOllamaVersionAtLeast("0.24.0", "0.7.0")).toBe(true);
});

it("treats 1.0.0 as above the 0.7.0 floor", () => {
expect(isOllamaVersionAtLeast("1.0.0", "0.7.0")).toBe(true);
});

it("returns false for unparseable version components", () => {
expect(isOllamaVersionAtLeast("not-a-version", "0.7.0")).toBe(false);
});

it("reads the running daemon version from /api/version", () => {
const capture = () => '{"version":"0.24.0"}';
expect(getRunningOllamaDaemonVersion(capture)).toBe("0.24.0");
});

it("returns null when the daemon endpoint is unreachable", () => {
const capture = () => "";
expect(getRunningOllamaDaemonVersion(capture)).toBeNull();
});

it("returns null when the daemon payload is not JSON", () => {
const capture = () => "ollama is running";
expect(getRunningOllamaDaemonVersion(capture)).toBeNull();
});

it("returns null when the daemon payload omits a version field", () => {
const capture = () => '{"status":"ok"}';
expect(getRunningOllamaDaemonVersion(capture)).toBeNull();
});
});
91 changes: 91 additions & 0 deletions src/lib/inference/ollama-version.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
// SPDX-License-Identifier: Apache-2.0

/**
* Ollama version detection helpers. Kept separate from the larger
* `inference/local.ts` so the version-floor logic can evolve without
* dragging the rest of the local-inference helpers along.
*/

const { runCapture } = require("../runner");
import { OLLAMA_PORT } from "../core/ports";

export type OllamaVersionRunCapture = (
cmd: readonly string[],
opts?: { ignoreError?: boolean },
) => string;

/**
* Minimum Ollama version NemoClaw expects when reusing an existing host
* Ollama. Older Ollama runners crash loading newer starter models because
* their GGUF parsers predate the model format. Bump this when starter-model
* recipes adopt a newer GGUF feature.
*/
export const MIN_OLLAMA_VERSION = "0.7.0";

export function getInstalledOllamaVersion(
runCaptureImpl?: OllamaVersionRunCapture,
): string | null {
const capture = runCaptureImpl ?? runCapture;
const out = capture(["ollama", "--version"], { ignoreError: true });
if (!out) return null;
const match = out.match(/(\d+)\.(\d+)\.(\d+)/);
return match ? match[0] : null;
}

/**
* Probe the running Ollama daemon's version via `/api/version`. This is
* the version NemoClaw will actually drive for inference — the CLI binary
* on `PATH` may be newer than the daemon that still owns `:11434` (for
* example after a user-local install while the original systemd unit is
* still running). Returns `null` when the daemon is unreachable or the
* response payload is not parseable.
*/
export function getRunningOllamaDaemonVersion(
runCaptureImpl?: OllamaVersionRunCapture,
endpoint: string = `http://127.0.0.1:${OLLAMA_PORT}/api/version`,
): string | null {
Comment thread
coderabbitai[bot] marked this conversation as resolved.
const capture = runCaptureImpl ?? runCapture;
const out = capture(
[
"curl",
"-sf",
"--connect-timeout",
"2",
"--max-time",
"5",
endpoint,
],
{ ignoreError: true },
);
if (!out) return null;
let parsed: unknown;
try {
parsed = JSON.parse(out);
} catch {
return null;
}
if (!parsed || typeof parsed !== "object") return null;
const raw = (parsed as { version?: unknown }).version;
if (typeof raw !== "string") return null;
const match = raw.match(/(\d+)\.(\d+)\.(\d+)/);
return match ? match[0] : null;
}

export function isOllamaVersionAtLeast(
version: string | null,
minimum: string,
): boolean {
if (!version) return false;
const parts = version.split(".").map((v) => Number.parseInt(v, 10));
const min = minimum.split(".").map((v) => Number.parseInt(v, 10));
const len = Math.max(parts.length, min.length);
for (let i = 0; i < len; i += 1) {
const a = parts[i] ?? 0;
const b = min[i] ?? 0;
if (Number.isNaN(a) || Number.isNaN(b)) return false;
if (a > b) return true;
if (a < b) return false;
}
return true;
}
66 changes: 32 additions & 34 deletions src/lib/onboard.ts
Original file line number Diff line number Diff line change
Expand Up @@ -105,6 +105,9 @@ const {
const {
installOllamaOnLinux,
}: typeof import("./onboard/install-ollama-linux") = require("./onboard/install-ollama-linux");
const {
installOllamaOnMacOS,
}: typeof import("./onboard/install-ollama-macos") = require("./onboard/install-ollama-macos");
const crypto = require("node:crypto");
const fs = require("fs");
const os = require("os");
Expand Down Expand Up @@ -183,6 +186,9 @@ const {
validateOllamaModel,
validateLocalProvider,
} = localInference;
const { resolveOllamaInstallMenuEntry, assertOllamaUpgradeApplied } = require(
"./onboard/ollama-install-menu",
);
const {
ensureOllamaAuthProxy,
getOllamaProxyToken,
Expand Down Expand Up @@ -4219,7 +4225,10 @@ async function selectAndValidateOllamaModel(
},
);
if (validation.retry === "selection") return { outcome: "back-to-selection" };
if (!validation.ok) continue;
if (!validation.ok) {
if (isNonInteractive()) process.exit(1);
continue;
}
// Ollama's /v1/responses endpoint does not produce correctly formatted
// tool calls — force chat completions like vLLM/NIM.
if (validation.api !== "openai-completions") {
Expand Down Expand Up @@ -4375,19 +4384,15 @@ async function setupNim(
label: "Install Ollama on Windows host (recommended)",
});
}
// Without any Ollama, offer to install one locally as a fallback (e.g. when
// the NVIDIA API server is down and cloud keys are unavailable).
if (!hasOllama && !ollamaRunning && !hasWindowsOllama) {
if (process.platform === "darwin") {
options.push({ key: "install-ollama", label: "Install Ollama (macOS)" });
} else if (process.platform === "linux") {
if (isWsl()) {
options.push({ key: "install-ollama", label: "Install Ollama (WSL Linux)" });
} else {
options.push({ key: "install-ollama", label: "Install Ollama (Linux)" });
}
}
}
const ollamaInstallMenu = resolveOllamaInstallMenuEntry({
hasOllama,
ollamaRunning,
hasWindowsOllama,
ollamaHost,
platform: process.platform,
isWsl: isWsl(),
});
if (ollamaInstallMenu.entry) options.push(ollamaInstallMenu.entry);

// Model Router: complexity-based routing via blueprint config.
const blueprintRouterCfg = loadBlueprintProfile("routed");
Expand Down Expand Up @@ -5265,26 +5270,19 @@ async function setupNim(
break;
} else if (selected.key === "install-ollama") {
if (!checkOllamaPortsOrWarn()) continue selectionLoop;
if (process.platform === "darwin") {
console.log(" Installing Ollama via Homebrew...");
run(["brew", "install", "ollama"], { ignoreError: true });
// brew install doesn't auto-start a service; launch directly.
// Shell required: backgrounding (&), env var prefix, output redirection.
console.log(" Starting Ollama...");
runShell(`OLLAMA_HOST=127.0.0.1:${OLLAMA_PORT} ollama serve > /dev/null 2>&1 &`, {
ignoreError: true,
});
if (!waitForHttp(`http://127.0.0.1:${OLLAMA_PORT}/`, 10)) {
console.error(` Ollama did not become ready on :${OLLAMA_PORT} within timeout.`);
if (isNonInteractive()) process.exit(1);
continue selectionLoop;
}
} else {
const installResult = installOllamaOnLinux({ isNonInteractive });
if (!installResult.ok) {
if (isNonInteractive()) process.exit(1);
continue selectionLoop;
}
const isUpgrade = ollamaInstallMenu.hasUpgradableOllama;
const installResult = process.platform === "darwin"
? installOllamaOnMacOS({ isNonInteractive, isUpgrade })
: installOllamaOnLinux({ isNonInteractive, isUpgrade });
if (!installResult.ok) {
if (isNonInteractive()) process.exit(1);
continue selectionLoop;
}
const upgradeCheck = assertOllamaUpgradeApplied(ollamaInstallMenu);
if (!upgradeCheck.ok) {
console.error(` ${upgradeCheck.message}`);
if (isNonInteractive()) process.exit(1);
continue selectionLoop;
}
if (shouldFrontOllamaWithProxy()) {
if (!startOllamaAuthProxy()) process.exit(1);
Expand Down
Loading
Loading