You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
feat: auto-bootstrap missing DMR and MLX host components on harbor up
- dmr_install_components: auto-installs docker-model-plugin via apt/dnf
on Linux, or enables Model Runner via Docker Desktop on macOS
- dmr_install_runner: installs the DMR runtime when available
- harbor up dmr now bootstraps gracefully instead of erroring out
- harbor up mlx auto-installs mlx-serve via uv on macOS when missing
- Updated all three backend docs to describe the automatic setup
Copy file name to clipboardExpand all lines: .facts
+3Lines changed: 3 additions & 0 deletions
Original file line number
Diff line number
Diff line change
@@ -83,6 +83,9 @@
83
83
- label: cli > routines > routines/utils.ts has explicit TypeScript annotations for shared utility exports so deno check routines/models.ts is not blocked by implicit-any diagnostics
84
84
command: deno check routines/models.ts
85
85
tags: [spec, routines-utils, implemented]
86
+
- label: services > dmr/mlx > harbor up dmr and harbor up mlx automatically bootstrap missing host-managed backend components before starting their Harbor proxy services
Copy file name to clipboardExpand all lines: docs/2.2.22-Backend-Docker-Model-Runner.md
+2-2Lines changed: 2 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -13,7 +13,7 @@ DMR is especially useful on Apple Silicon because Metal inference runs on the ma
13
13
harbor up dmr
14
14
```
15
15
16
-
When `HARBOR_DMR_MANAGE_HOST=true`, Harbor checks for the Docker Model Runner CLI, enables the TCP endpoint when configured, and pulls `HARBOR_DMR_MODEL` before starting the proxy.
16
+
When `HARBOR_DMR_MANAGE_HOST=true`, Harbor automatically installs or enables missing Docker Model Runner host components before starting the proxy. On Linux this uses the official `docker-model-plugin` package through `apt` or `dnf` when the `docker model`CLI is missing; on Docker Desktop systems Harbor uses Docker Desktop's Model Runner enablement command when it is available. Harbor then installs the runner runtime when supported, enables the TCP endpoint when configured, and pulls `HARBOR_DMR_MODEL`.
17
17
18
18
Start a frontend against DMR:
19
19
@@ -96,7 +96,7 @@ harbor logs dmr
96
96
97
97
##### Docker Model Runner CLI is missing
98
98
99
-
Install or update Docker Desktop and verify:
99
+
`harbor up dmr` attempts automatic setup when host management is enabled. If setup cannot proceed on your platform, install or update Docker Desktop, or install Docker's `docker-model-plugin` package on Linux, then verify:
Copy file name to clipboardExpand all lines: docs/2.2.23-Backend-MLX.md
+3-9Lines changed: 3 additions & 9 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -9,19 +9,13 @@ This service is intended for Apple Silicon systems. On other platforms the proxy
9
9
10
10
#### Starting
11
11
12
-
Install `mlx-serve` on the host first:
13
-
14
-
```bash
15
-
uv tool install mlx-serve
16
-
```
17
-
18
-
Then start the Harbor service:
12
+
Start the Harbor service:
19
13
20
14
```bash
21
15
harbor up mlx
22
16
```
23
17
24
-
When `HARBOR_MLX_MANAGE_HOST=true`, Harbor renders `services/mlx/models.yaml` from the template, starts `mlx-serve` from `services/mlx/`, and starts the Harbor proxy.
18
+
When `HARBOR_MLX_MANAGE_HOST=true`, Harbor automatically installs missing MLX host components before starting the proxy. If `mlx-serve` is missing on macOS, Harbor installs it with `uv tool install mlx-serve`; if `uv` is missing and Homebrew is available, Harbor installs `uv` first. Harbor then renders `services/mlx/models.yaml` from the template, starts `mlx-serve` from `services/mlx/`, and starts the Harbor proxy.
25
19
26
20
Start a frontend against MLX:
27
21
@@ -119,7 +113,7 @@ harbor logs mlx
119
113
120
114
##### mlx-serve is not installed
121
115
122
-
Install it on the macOS host:
116
+
`harbor up mlx` automatically installs `mlx-serve` on macOS when host management is enabled. If automatic setup cannot proceed, install `uv` and `mlx-serve` manually:
Copy file name to clipboardExpand all lines: docs/8.5-OpenAI-Compatible-Local-LLM-Backends.md
+5-1Lines changed: 5 additions & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -75,6 +75,8 @@ harbor up dmr
75
75
76
76
On Apple Silicon, DMR is the preferred Docker-managed path for host-native Metal inference. Harbor starts a proxy container at `http://dmr:8080/v1`; Docker Model Runner itself stays on the host.
77
77
78
+
When host management is enabled, `harbor up dmr` also attempts to bootstrap missing Docker Model Runner components before the proxy starts.
79
+
78
80
### MLX
79
81
80
82
[MLX](./2.2.23-Backend-MLX.md) is useful when you specifically want Apple's MLX runtime on an Apple Silicon Mac. Harbor manages `mlx-serve` on the host and exposes it to the Compose network through a proxy container.
MLX acceleration does not run inside Harbor's Linux containers. The Harbor service owns lifecycle, config, docs, and integration; Metal inference runs on macOS.
88
90
91
+
When host management is enabled, `harbor up mlx` automatically installs `mlx-serve` on macOS if it is missing.
92
+
89
93
## Choose or Swap Backends
90
94
91
95
The default Harbor profile starts Ollama and Open WebUI. You can still start another backend directly:
@@ -188,7 +192,7 @@ Use Docker Model Runner when:
188
192
Use MLX when:
189
193
190
194
- You are on an Apple Silicon Mac and want direct MLX-backed inference.
191
-
- You are comfortable installing `mlx-serve`on the host.
195
+
- You want Harbor to manage `mlx-serve`startup and install it when it is missing.
192
196
- You want Harbor-managed config and proxying for a host-native MLX process.
193
197
194
198
For many users, the practical workflow is simple: start with Ollama, add llama.cpp when you want direct GGUF serving, use vLLM when serving throughput or model format support matters more, and use DMR or MLX on Apple Silicon when Metal acceleration is the goal.
0 commit comments