Skip to content

Commit aecd1a7

Browse files
author
Federico De Ponte
committed
Add low-cost AI sidecar support
1 parent b255b7e commit aecd1a7

10 files changed

Lines changed: 692 additions & 3 deletions

File tree

.env.example

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -52,6 +52,9 @@ MAC_REVERSE_PORT=2222
5252
# MOTO_REMOTE_DIR=/opt/moto
5353

5454
# Optional keys exported into remote tmux sessions.
55+
# Prefer `ai-provider-key set <provider>` for sidecar keys. Do not store
56+
# Groq/OpenRouter/NVIDIA sidecar keys here unless your own workflow requires
57+
# environment-variable export.
5558
GOOGLE_GENERATIVE_AI_API_KEY=
5659
OPENROUTER_API_KEY=
5760
DEEPINFRA_API_KEY=

README.md

Lines changed: 34 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -43,6 +43,7 @@ The philosophy is simple:
4343
- [Mac iTerm Workflow](#mac-iterm-workflow)
4444
- [CLAUDE.md Templates](#claudemd-templates)
4545
- [Safety Hooks](#safety-hooks)
46+
- [Low-Cost AI Sidecars](#low-cost-ai-sidecars)
4647
- [Skills (40+)](#skills-40)
4748
- [Memory System](#memory-system)
4849
- [Server Infrastructure](#server-infrastructure)
@@ -136,6 +137,35 @@ Two templates encoding months of iteration on making Claude Code reliable:
136137

137138
---
138139

140+
## Low-Cost AI Sidecars
141+
142+
`moto` treats frontier agents as the control plane and routes bounded text work to cheaper sidecars. The goal is simple: spend premium model budget on judgment, orchestration, debugging, architecture, and final review; use cheaper/free models for narrow stateless work.
143+
144+
| Route | Typical use | Notes |
145+
|-------|-------------|-------|
146+
| Gemini free / OAuth wrapper | broad repo summaries, docs drafts, test plans | Good for large text-in/text-out analysis when privacy constraints allow |
147+
| Groq | single-file review, diff chunks, error logs | Fast stateless reviewer; prefer prompts with tight scope |
148+
| OpenRouter free | backup free route | Expect provider throttling and model changes |
149+
| NVIDIA NIM | hosted specialist sidecar | Use stronger models for difficult reasoning or code-specific second opinions |
150+
| Local Ollama on the remote box | private/offline bounded work | Slow on CPU; advisory only, not final correctness authority |
151+
152+
Recommended routing pattern:
153+
154+
- Use hosted sidecars for stateless research, summaries, diff review, and second opinions.
155+
- Use NVIDIA `deepseek-ai/deepseek-v4-pro` for high-depth reasoning, difficult code analysis, long-context synthesis, and planning.
156+
- Use NVIDIA `qwen/qwen3-coder-480b-a35b-instruct` for code-specific second opinions.
157+
- Use a local Ollama model only when privacy/offline locality matters and the prompt is self-contained.
158+
- Keep final authority with Claude Code, Codex, tests, screenshots, builds, and direct evidence.
159+
160+
Provider keys belong in a local secret store, never in repos or shell startup files. A practical implementation is:
161+
162+
- macOS: Keychain services such as `codex:GROQ_API_KEY`, `codex:NVIDIA_API_KEY`, `codex:OPENROUTER_API_KEY`
163+
- Linux remote: `~/.config/ai-sidecar/keys.json` with directory mode `700` and file mode `600`
164+
165+
See [`docs/architecture.md`](docs/architecture.md) for how this fits into the Mac + remote Linux runtime.
166+
167+
---
168+
139169
## Safety Hooks
140170

141171
17 hooks wired into `settings.json` that prevent Claude from doing damage. They run automatically on every tool call.
@@ -173,6 +203,9 @@ All hooks are pure bash (no external deps beyond `jq`). See [`claude/hooks/READM
173203
| `claude-rate-limit-watcher.sh` | Background rate limit monitor daemon |
174204
| `claude-switch.sh` | Switch between multiple Claude accounts |
175205
| `sync-claude-config.sh` | Sync CLAUDE.md across machines via SCP |
206+
| `ai-provider-key` | Store sidecar provider keys in Keychain or a 0600 Linux key file |
207+
| `ai-sidecar` | Call Groq, OpenRouter, or NVIDIA for bounded stateless text work |
208+
| `ai-sidecar-health` | Verify configured sidecar providers with tiny health checks |
176209

177210
---
178211

@@ -299,7 +332,7 @@ See [`mac/README.md`](mac/README.md) for the full setup.
299332
settings.json <- claude/settings.json (hooks + permissions, $HOME resolved)
300333
.mcp.json <- claude/.mcp.json (MCP servers)
301334
hooks/ <- claude/hooks/ (12 safety hooks)
302-
scripts/ <- claude/scripts/ (5 utility scripts)
335+
scripts/ <- claude/scripts/ (utility scripts + AI sidecars)
303336
commands/ <- claude/skills/ (40+ slash commands)
304337
metrics/ <- cost tracking output (costs.jsonl)
305338
projects/*/memory/ <- external brain (MEMORY.md + topic files)

claude/CLAUDE.md

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -46,6 +46,27 @@
4646
- Always use `timeout` for long tasks: `timeout 5m <command>`
4747
- Check for orphan processes: `pgrep -f "<your-heavy-processes>"`
4848

49+
### Low-Cost AI Sidecars
50+
51+
<!-- Customize: delete providers you do not use. Keep keys out of repos and shell startup files. -->
52+
53+
Use sidecars only for bounded stateless text work. Claude Code remains the control plane and final authority for tool use, verification, security, architecture, and production decisions.
54+
55+
| Task | Route |
56+
|------|-------|
57+
| Broad summaries, docs drafts, test planning | Gemini free / OAuth wrapper |
58+
| Single-file review, diff chunks, error logs | Groq |
59+
| Backup free route | OpenRouter free |
60+
| High-depth hosted reasoning and planning | NVIDIA `deepseek-ai/deepseek-v4-pro` |
61+
| Code-specific hosted second opinion | NVIDIA `qwen/qwen3-coder-480b-a35b-instruct` |
62+
| Private/offline bounded prompt on dev server | Local Ollama CPU model |
63+
64+
Do not use a local CPU model as final authority for code correctness, security decisions, architecture calls, long open-ended generation, repo-wide analysis, or tasks requiring tools/browser/tests.
65+
66+
Provider key storage pattern:
67+
- macOS: Keychain services such as `codex:GROQ_API_KEY`, `codex:NVIDIA_API_KEY`, `codex:OPENROUTER_API_KEY`
68+
- Linux dev server: `~/.config/ai-sidecar/keys.json` with directory mode `700` and file mode `600`
69+
4970
### Browser Automation
5071

5172
<!-- Customize: if you use browser automation, describe your Chrome/CDP setup here -->

claude/scripts/ai-provider-key

Lines changed: 186 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,186 @@
1+
#!/usr/bin/env python3
2+
"""Store provider API keys for moto sidecars without writing them to repos."""
3+
4+
from __future__ import annotations
5+
6+
import argparse
7+
import getpass
8+
import json
9+
import os
10+
import platform
11+
import subprocess
12+
import sys
13+
import tempfile
14+
from pathlib import Path
15+
16+
17+
PROVIDERS = {
18+
"gemini": "GEMINI_FREE_API_KEY",
19+
"groq": "GROQ_API_KEY",
20+
"openrouter": "OPENROUTER_API_KEY",
21+
"nvidia": "NVIDIA_API_KEY",
22+
}
23+
24+
25+
def store_path() -> Path:
26+
return Path(os.environ.get("AI_SIDECAR_DIR", Path.home() / ".config/ai-sidecar")) / "keys.json"
27+
28+
29+
def is_macos() -> bool:
30+
return platform.system() == "Darwin" and bool(shutil_which("security"))
31+
32+
33+
def shutil_which(name: str) -> str | None:
34+
from shutil import which
35+
36+
return which(name)
37+
38+
39+
def keychain_service(provider: str) -> str:
40+
return f"codex:{PROVIDERS[provider]}"
41+
42+
43+
def keychain_get(provider: str) -> str:
44+
result = subprocess.run(
45+
[
46+
"security",
47+
"find-generic-password",
48+
"-a",
49+
os.environ.get("USER", ""),
50+
"-s",
51+
keychain_service(provider),
52+
"-w",
53+
],
54+
check=False,
55+
capture_output=True,
56+
text=True,
57+
)
58+
return result.stdout.strip() if result.returncode == 0 else ""
59+
60+
61+
def keychain_set(provider: str, secret: str) -> None:
62+
subprocess.run(
63+
[
64+
"security",
65+
"add-generic-password",
66+
"-U",
67+
"-a",
68+
os.environ.get("USER", ""),
69+
"-s",
70+
keychain_service(provider),
71+
"-w",
72+
secret,
73+
],
74+
check=True,
75+
stdout=subprocess.DEVNULL,
76+
)
77+
78+
79+
def keychain_delete(provider: str) -> None:
80+
subprocess.run(
81+
[
82+
"security",
83+
"delete-generic-password",
84+
"-a",
85+
os.environ.get("USER", ""),
86+
"-s",
87+
keychain_service(provider),
88+
],
89+
check=False,
90+
stdout=subprocess.DEVNULL,
91+
stderr=subprocess.DEVNULL,
92+
)
93+
94+
95+
def ensure_file_store() -> Path:
96+
path = store_path()
97+
path.parent.mkdir(parents=True, exist_ok=True)
98+
path.parent.chmod(0o700)
99+
if not path.exists():
100+
path.write_text("{}\n")
101+
path.chmod(0o600)
102+
return path
103+
104+
105+
def read_file_store() -> dict[str, str]:
106+
path = ensure_file_store()
107+
try:
108+
data = json.loads(path.read_text())
109+
except json.JSONDecodeError:
110+
data = {}
111+
return {str(k): str(v) for k, v in data.items()}
112+
113+
114+
def write_file_store(data: dict[str, str]) -> None:
115+
path = ensure_file_store()
116+
fd, tmp = tempfile.mkstemp(dir=path.parent, prefix=".keys.", suffix=".tmp")
117+
with os.fdopen(fd, "w") as handle:
118+
json.dump(data, handle, indent=2, sort_keys=True)
119+
handle.write("\n")
120+
os.chmod(tmp, 0o600)
121+
os.replace(tmp, path)
122+
123+
124+
def get_key(provider: str) -> str:
125+
if is_macos():
126+
return keychain_get(provider)
127+
return read_file_store().get(provider, "")
128+
129+
130+
def set_key(provider: str, secret: str) -> None:
131+
if is_macos():
132+
keychain_set(provider, secret)
133+
else:
134+
data = read_file_store()
135+
data[provider] = secret
136+
write_file_store(data)
137+
138+
139+
def delete_key(provider: str) -> None:
140+
if is_macos():
141+
keychain_delete(provider)
142+
else:
143+
data = read_file_store()
144+
data.pop(provider, None)
145+
write_file_store(data)
146+
147+
148+
def main() -> int:
149+
parser = argparse.ArgumentParser(description=__doc__)
150+
parser.add_argument("command", choices=["set", "get", "delete", "list"])
151+
parser.add_argument("provider", nargs="?", choices=sorted(PROVIDERS))
152+
args = parser.parse_args()
153+
154+
if args.command != "list" and not args.provider:
155+
parser.error("provider is required")
156+
157+
if args.command == "set":
158+
secret = sys.stdin.readline().rstrip("\n") if not sys.stdin.isatty() else getpass.getpass(
159+
f"Enter {PROVIDERS[args.provider]}: "
160+
)
161+
if not secret:
162+
raise SystemExit("Empty key; nothing stored.")
163+
set_key(args.provider, secret)
164+
print(f"Stored {args.provider}.")
165+
return 0
166+
167+
if args.command == "get":
168+
secret = get_key(args.provider)
169+
if not secret:
170+
return 1
171+
print(secret)
172+
return 0
173+
174+
if args.command == "delete":
175+
delete_key(args.provider)
176+
print(f"Deleted {args.provider} if it existed.")
177+
return 0
178+
179+
for provider in sorted(PROVIDERS):
180+
if get_key(provider):
181+
print(provider)
182+
return 0
183+
184+
185+
if __name__ == "__main__":
186+
raise SystemExit(main())

0 commit comments

Comments
 (0)