Skip to content

Commit edf17a6

Browse files
committed
Improve CLI docs: detail registries, sources, and new devbox verify/apply commands. Add apply to configure registries/sources and reconcile packages from lockfile. Add verify to ensure box matches lockfile. Update lockfile to include registries/sources. Update Docker client for verification/application.
1 parent 718e448 commit edf17a6

File tree

7 files changed

+674
-8
lines changed

7 files changed

+674
-8
lines changed

docs/src/content/docs/docs/cli.md

Lines changed: 73 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -286,7 +286,11 @@ devbox lock <project> [-o, --output <path>]
286286
- Installed package snapshots:
287287
- apt: manually installed packages pinned as `name=version`
288288
- pip: `pip freeze` output
289-
- npm/yarn/pnpm: globally installed packages as `name@version`
289+
- npm/yarn/pnpm: globally installed packages as `name@version` (Yarn global versions are detected from Yarn's global dir)
290+
- Registries and sources for reproducibility:
291+
- pip: `index-url` and `extra-index-url`
292+
- npm/yarn/pnpm: global registry URLs
293+
- apt: `sources.list` lines, snapshot base URL if present, and OS release codename
290294
- If `devbox.json` exists in the workspace, includes its `setup_commands` for context.
291295

292296
This snapshot is meant for sharing and audit. It does not currently drive `devbox up` automatically; continue to use `devbox.json` plus the simple `devbox.lock` command list for replay. A future `devbox restore` may apply `devbox.lock.json` directly.
@@ -328,15 +332,82 @@ devbox lock myproject -o ./env/devbox.lock.json
328332
"apt": ["git=1:2.34.1-..."],
329333
"pip": ["requests==2.32.3"],
330334
"npm": ["typescript@5.6.2"],
331-
"yarn": [],
335+
"yarn": ["eslint@9.1.0"],
332336
"pnpm": []
333337
},
338+
"registries": {
339+
"pip_index_url": "https://pypi.org/simple",
340+
"pip_extra_index_urls": ["https://mirror.example/simple"],
341+
"npm_registry": "https://registry.npmjs.org/",
342+
"yarn_registry": "https://registry.yarnpkg.com",
343+
"pnpm_registry": "https://registry.npmjs.org/"
344+
},
345+
"apt_sources": {
346+
"snapshot_url": "https://snapshot.debian.org/archive/debian/20240915T000000Z/",
347+
"sources_lists": [
348+
"deb https://snapshot.debian.org/archive/debian/20240915T000000Z/ bullseye main"
349+
],
350+
"pinned_release": "jammy"
351+
},
334352
"setup_commands": [
335353
"apt install -y python3 python3-pip"
336354
]
337355
}
338356
```
339357

358+
---
359+
360+
### `devbox verify`
361+
362+
Validate that the running box matches the `devbox.lock.json` exactly. Fails fast on any drift.
363+
364+
**Syntax:**
365+
```bash
366+
devbox verify <project>
367+
```
368+
369+
**Checks:**
370+
- Package sets: apt, pip, npm, yarn, pnpm (exact set match)
371+
- Registries: pip index/extra-index, npm/yarn/pnpm registry URLs
372+
- Apt sources: sources.list lines, snapshot base URL (if present), OS release codename
373+
374+
Returns non-zero on any mismatch and prints a concise drift report.
375+
376+
**Example:**
377+
```bash
378+
devbox verify myproject
379+
```
380+
381+
---
382+
383+
### `devbox apply`
384+
385+
Apply the `devbox.lock.json` to the running box: configure registries and apt sources, then reconcile package sets to match the lock.
386+
387+
**Syntax:**
388+
```bash
389+
devbox apply <project>
390+
```
391+
392+
**Behavior:**
393+
- Registries:
394+
- Writes `/etc/pip.conf` with `index-url`/`extra-index-url` from lock
395+
- Runs `npm/yarn/pnpm` config to set global registry URLs
396+
- Apt sources:
397+
- Backs up and rewrites `/etc/apt/sources.list`, clears `/etc/apt/sources.list.d/*.list`
398+
- Optionally sets a default release hint, then `apt update`
399+
- Reconciliation:
400+
- APT: install exact versions from lock, remove extras, autoremove
401+
- Pip: install missing exact versions, uninstall extras
402+
- npm/yarn/pnpm (global): add missing exact versions, remove extras
403+
404+
Exits non-zero if application fails at any step.
405+
406+
**Example:**
407+
```bash
408+
devbox apply myproject
409+
```
410+
340411
## Configuration Commands
341412

342413
---

docs/src/content/docs/docs/configuration.md

Lines changed: 8 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -120,12 +120,18 @@ This writes a JSON snapshot (by default to `<workspace>/devbox.lock.json`) that
120120
- Installed packages:
121121
- apt: manually installed packages pinned as `name=version`
122122
- pip: `pip freeze`
123-
- npm/yarn/pnpm: globally installed packages `name@version`
123+
- npm/yarn/pnpm: globally installed packages `name@version` (Yarn global versions are read from Yarn's global directory)
124+
- Registries and sources for reproducibility:
125+
- pip: `index-url` and `extra-index-url`
126+
- npm/yarn/pnpm: global registry URLs
127+
- apt: full `sources.list` lines, snapshot base URL if present, and OS release codename
124128
- Any `setup_commands` from your `devbox.json` (for context)
125129
126130
Usage notes:
127131
- Commit `devbox.lock.json` to your repository to share environment details with teammates.
128-
- This file is an authoritative snapshot for auditing/sharing. The current execution path for rebuilds remains `devbox.json` + the simple `devbox.lock` replay file. A future `devbox restore` may apply `devbox.lock.json` directly.
132+
- This file is an authoritative snapshot for auditing/sharing. The current execution path for rebuilds remains `devbox.json` + the simple `devbox.lock` replay file. You can now also use:
133+
- `devbox verify <project>` to validate a box matches the lock (fails fast on drift)
134+
- `devbox apply <project>` to configure registries/sources and reconcile package sets to the lock
129135
- Local app dependencies (e.g. non-global Node packages in your repo) are intentionally not included; rely on your project’s own lockfiles (package-lock.json, yarn.lock, pnpm-lock.yaml, requirements.txt/poetry.lock, etc.).
130136
131137
## Initialize with Configuration

internal/commands/apply.go

Lines changed: 254 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,254 @@
1+
package commands
2+
3+
import (
4+
"encoding/json"
5+
"fmt"
6+
"os"
7+
"path/filepath"
8+
"strings"
9+
10+
"github.com/spf13/cobra"
11+
)
12+
13+
type applyLockFile struct {
14+
Version int `json:"version"`
15+
Project string `json:"project"`
16+
BoxName string `json:"box_name"`
17+
Packages lockPackages `json:"packages"`
18+
Registries lockRegistries `json:"registries"`
19+
AptSources lockAptSources `json:"apt_sources"`
20+
}
21+
22+
var applyCmd = &cobra.Command{
23+
Use: "apply <project>",
24+
Short: "Apply devbox.lock.json: set registries and apt sources, then reconcile packages",
25+
Args: cobra.ExactArgs(1),
26+
RunE: func(cmd *cobra.Command, args []string) error {
27+
projectName := args[0]
28+
29+
cfg, err := configManager.Load()
30+
if err != nil {
31+
return fmt.Errorf("failed to load config: %w", err)
32+
}
33+
proj, ok := cfg.GetProject(projectName)
34+
if !ok {
35+
return fmt.Errorf("project '%s' not found", projectName)
36+
}
37+
38+
lockPath := filepath.Join(proj.WorkspacePath, "devbox.lock.json")
39+
data, err := os.ReadFile(lockPath)
40+
if err != nil {
41+
return fmt.Errorf("failed to read %s: %w", lockPath, err)
42+
}
43+
44+
var lf applyLockFile
45+
if err := json.Unmarshal(data, &lf); err != nil {
46+
return fmt.Errorf("invalid lockfile: %w", err)
47+
}
48+
49+
exists, err := dockerClient.BoxExists(proj.BoxName)
50+
if err != nil {
51+
return err
52+
}
53+
if !exists {
54+
return fmt.Errorf("box '%s' not found; run 'devbox up %s' first", proj.BoxName, projectName)
55+
}
56+
status, err := dockerClient.GetBoxStatus(proj.BoxName)
57+
if err != nil {
58+
return err
59+
}
60+
if status != "running" {
61+
if err := dockerClient.StartBox(proj.BoxName); err != nil {
62+
return fmt.Errorf("failed to start box: %w", err)
63+
}
64+
}
65+
66+
var applyCmds []string
67+
68+
if len(lf.AptSources.SourcesLists) > 0 {
69+
70+
heredoc := "cat > /etc/apt/sources.list <<'EOF'\n" + strings.Join(lf.AptSources.SourcesLists, "\n") + "\nEOF"
71+
applyCmds = append(applyCmds,
72+
"cp /etc/apt/sources.list /etc/apt/sources.list.bak 2>/dev/null || true",
73+
"rm -f /etc/apt/sources.list.d/*.list 2>/dev/null || true",
74+
heredoc,
75+
)
76+
}
77+
if lf.AptSources.PinnedRelease != "" {
78+
applyCmds = append(applyCmds, fmt.Sprintf("bash -lc 'echo APT::Default-Release \"%s\"; > /etc/apt/apt.conf.d/99defaultrelease'", escapeBash(lf.AptSources.PinnedRelease)))
79+
}
80+
if len(lf.AptSources.SourcesLists) > 0 {
81+
applyCmds = append(applyCmds, "apt update -y")
82+
}
83+
84+
if lf.Registries.PipIndexURL != "" || len(lf.Registries.PipExtraIndex) > 0 {
85+
var b strings.Builder
86+
b.WriteString("cat > /etc/pip.conf <<'EOF'\n[global]\n")
87+
if lf.Registries.PipIndexURL != "" {
88+
b.WriteString("index-url = ")
89+
b.WriteString(lf.Registries.PipIndexURL)
90+
b.WriteString("\n")
91+
}
92+
for _, u := range lf.Registries.PipExtraIndex {
93+
if strings.TrimSpace(u) == "" {
94+
continue
95+
}
96+
b.WriteString("extra-index-url = ")
97+
b.WriteString(u)
98+
b.WriteString("\n")
99+
}
100+
b.WriteString("EOF")
101+
applyCmds = append(applyCmds, b.String())
102+
}
103+
104+
if lf.Registries.NpmRegistry != "" {
105+
applyCmds = append(applyCmds, fmt.Sprintf("npm config set registry %s -g", lf.Registries.NpmRegistry))
106+
}
107+
if lf.Registries.YarnRegistry != "" {
108+
applyCmds = append(applyCmds, fmt.Sprintf("yarn config set npmRegistryServer %s -g", lf.Registries.YarnRegistry))
109+
}
110+
if lf.Registries.PnpmRegistry != "" {
111+
applyCmds = append(applyCmds, fmt.Sprintf("pnpm config set registry %s -g", lf.Registries.PnpmRegistry))
112+
}
113+
114+
if err := dockerClient.ExecuteSetupCommandsWithOutput(proj.BoxName, applyCmds, false); err != nil {
115+
return fmt.Errorf("failed applying registries/sources: %w", err)
116+
}
117+
118+
curApt, curPip, curNpm, curYarn, curPnpm := dockerClient.QueryPackagesParallel(proj.BoxName)
119+
120+
actions := buildReconcileActions(lf.Packages, curApt, curPip, curNpm, curYarn, curPnpm)
121+
if len(actions) > 0 {
122+
if err := dockerClient.ExecuteSetupCommandsWithOutput(proj.BoxName, actions, true); err != nil {
123+
return fmt.Errorf("failed to reconcile packages: %w", err)
124+
}
125+
}
126+
127+
fmt.Println("✅ Applied lockfile: registries/sources configured and packages reconciled")
128+
return nil
129+
},
130+
}
131+
132+
func escapeBash(s string) string {
133+
return strings.ReplaceAll(s, "'", "'\\''")
134+
}
135+
136+
func parseMap(list []string, sep string) map[string]string {
137+
m := map[string]string{}
138+
for _, line := range list {
139+
s := strings.TrimSpace(line)
140+
if s == "" {
141+
continue
142+
}
143+
if sep == "==" {
144+
if i := strings.Index(s, "=="); i != -1 {
145+
name := strings.ToLower(strings.TrimSpace(s[:i]))
146+
ver := strings.TrimSpace(s[i+2:])
147+
m[name] = ver
148+
}
149+
continue
150+
}
151+
if sep == "@" {
152+
153+
idx := strings.LastIndex(s, "@")
154+
if idx > 0 {
155+
name := strings.ToLower(strings.TrimSpace(s[:idx]))
156+
ver := strings.TrimSpace(s[idx+1:])
157+
m[name] = ver
158+
}
159+
continue
160+
}
161+
if sep == "=" {
162+
if i := strings.Index(s, "="); i != -1 {
163+
name := strings.ToLower(strings.TrimSpace(s[:i]))
164+
ver := strings.TrimSpace(s[i+1:])
165+
m[name] = ver
166+
}
167+
}
168+
}
169+
return m
170+
}
171+
172+
func keysNotIn(a, b map[string]string) []string {
173+
var out []string
174+
for k := range a {
175+
if _, ok := b[k]; !ok {
176+
out = append(out, k)
177+
}
178+
}
179+
return out
180+
}
181+
182+
func buildReconcileActions(lockPkgs lockPackages, curApt, curPip, curNpm, curYarn, curPnpm []string) []string {
183+
var cmds []string
184+
185+
lockA := parseMap(lockPkgs.Apt, "=")
186+
curA := parseMap(curApt, "=")
187+
lockP := parseMap(lockPkgs.Pip, "==")
188+
curP := parseMap(curPip, "==")
189+
lockN := parseMap(lockPkgs.Npm, "@")
190+
curN := parseMap(curNpm, "@")
191+
lockY := parseMap(lockPkgs.Yarn, "@")
192+
curY := parseMap(curYarn, "@")
193+
lockQ := parseMap(lockPkgs.Pnpm, "@")
194+
curQ := parseMap(curPnpm, "@")
195+
196+
var aptInstall []string
197+
for name, ver := range lockA {
198+
if curVer, ok := curA[name]; !ok || curVer != ver {
199+
aptInstall = append(aptInstall, fmt.Sprintf("%s=%s", name, ver))
200+
}
201+
}
202+
if len(aptInstall) > 0 {
203+
cmds = append(cmds, "apt update -y", "DEBIAN_FRONTEND=noninteractive apt-get install -y "+strings.Join(aptInstall, " "))
204+
}
205+
206+
for _, extra := range keysNotIn(curA, lockA) {
207+
cmds = append(cmds, fmt.Sprintf("apt-get remove -y %s", extra))
208+
}
209+
if len(keysNotIn(curA, lockA)) > 0 {
210+
cmds = append(cmds, "apt-get autoremove -y")
211+
}
212+
213+
for name, ver := range lockP {
214+
if curVer, ok := curP[name]; !ok || curVer != ver {
215+
cmds = append(cmds, fmt.Sprintf("python3 -m pip install %s==%s", name, ver))
216+
}
217+
}
218+
for _, extra := range keysNotIn(curP, lockP) {
219+
cmds = append(cmds, fmt.Sprintf("python3 -m pip uninstall -y %s", extra))
220+
}
221+
222+
for name, ver := range lockN {
223+
if curVer, ok := curN[name]; !ok || curVer != ver {
224+
cmds = append(cmds, fmt.Sprintf("npm i -g %s@%s", name, ver))
225+
}
226+
}
227+
for _, extra := range keysNotIn(curN, lockN) {
228+
cmds = append(cmds, fmt.Sprintf("npm rm -g %s", extra))
229+
}
230+
231+
for name, ver := range lockY {
232+
if curVer, ok := curY[name]; !ok || curVer != ver {
233+
cmds = append(cmds, fmt.Sprintf("yarn global add %s@%s", name, ver))
234+
}
235+
}
236+
for _, extra := range keysNotIn(curY, lockY) {
237+
cmds = append(cmds, fmt.Sprintf("yarn global remove %s", extra))
238+
}
239+
240+
for name, ver := range lockQ {
241+
if curVer, ok := curQ[name]; !ok || curVer != ver {
242+
cmds = append(cmds, fmt.Sprintf("pnpm add -g %s@%s", name, ver))
243+
}
244+
}
245+
for _, extra := range keysNotIn(curQ, lockQ) {
246+
cmds = append(cmds, fmt.Sprintf("pnpm remove -g %s", extra))
247+
}
248+
249+
return cmds
250+
}
251+
252+
func init() {
253+
rootCmd.AddCommand(applyCmd)
254+
}

0 commit comments

Comments
 (0)