diff --git a/DOCKER_DISCOVERY_FINDINGS.md b/DOCKER_DISCOVERY_FINDINGS.md new file mode 100644 index 0000000..5e43aab --- /dev/null +++ b/DOCKER_DISCOVERY_FINDINGS.md @@ -0,0 +1,174 @@ +# Docker Discovery Findings + +## Scope + +This note captures the investigation into why `comapeo-headless` is discoverable +from Android when run directly with Node on the host, but is not discoverable +when run through `docker compose`. + +## Short Answer + +On this machine, the Docker runtime is the blocker, not the CoMapeo daemon +logic. + +The direct host-node run is needed here because the Docker container does not +appear to share the real Wi-Fi LAN namespace that Android can reach. In this +environment, `network_mode: host` is not exposing the same effective network +view as the host process. + +## Findings + +### 1. The daemon startup path is healthy + +- The headless daemon reaches: + - config load + - root-key load + - `MapeoManager` init + - device-info write + - local peer discovery server startup + - mDNS advertisement call + - invite handler startup +- This means the failure is not a simple startup crash or missed invite + subscription. + +### 2. Bare Node on the host is visible to Android + +- Running the daemon directly with Node on the host made the device show up on + the phone. +- `avahi-browse -rt _comapeo._tcp` showed the host-node CoMapeo service on the + host LAN address. + +## 3. The container claims to advertise mDNS, but the LAN does not see it + +- With Docker Compose, the daemon logs report: + - local peer discovery server started + - mDNS service advertised + - daemon ready +- However, the expected `_comapeo._tcp` service from the container was not + visible as a usable LAN advertisement in the host browse results. +- That means `service.advertise()` succeeding inside the container is not + enough to conclude Android can discover the service. + +### 4. The container does not see the real Wi-Fi interface + +- From inside the running container, `os.networkInterfaces()` showed interfaces + such as: + - `tap0` with `10.0.2.100` + - `docker0` with `172.17.0.1` +- It did not show the host Wi-Fi interface/address that the phone is actually + using on the LAN. +- The working host-node path, by contrast, resolved to the real host-side LAN + address (`10.208.159.116` during this test session). + +### 5. This Docker environment is rootless + +- `docker info` reported rootless security options. +- `docker context ls` showed the daemon accessed through + `unix:///run/user/1000/docker.sock`. +- In this rootless setup, `network_mode: host` is not behaving like "true host + Wi-Fi networking" for mDNS discovery purposes. + +### 6. `@homebridge/ciao` is not the core bug, but it is affected by the runtime + +- `@homebridge/ciao` advertises on the interfaces it detects. +- If the container sees the wrong interfaces, it will advertise against the + wrong network view. +- So the app code can be correct and still fail to be discoverable when the + container runtime presents an unusable interface set. + +### 7. Attempted Docker-side workaround: publish through host Avahi + +- An Avahi-based workaround was attempted from inside the container so the host + would publish `_comapeo._tcp` instead of `ciao`. +- Result: failed. +- Observed error: + +```text +Failed to create client object: An unexpected D-Bus error occurred +``` + +- This failed even after testing the usual container D-Bus prerequisites such as + mounting the system bus socket and mounting the host machine-id. + +### 8. Current conclusion + +- On this machine, the Docker discovery path is not reliably fixable by a small + daemon code change alone. +- The primary issue is the Docker runtime/network model, not invite handling or + peer discovery server startup inside CoMapeo. + +## Why Running Directly On The Host Was Needed + +That step was needed because it isolates the variable that matters most: +whether the daemon logic is broken, or whether Docker is breaking LAN +visibility. + +Once the same daemon became visible on Android outside Docker, the problem +stopped being "CoMapeo headless discovery is broken" and became +"this Docker networking setup cannot expose the service correctly to the LAN." + +## Evidence Collected + +- Compose logs show successful startup and claimed mDNS advertisement. +- Host-node run was visible to Android. +- Host browse saw Android and host-node `_comapeo._tcp` services. +- Container interface view did not match the host Wi-Fi LAN interface. +- Docker runtime reported rootless mode. +- Avahi-from-container workaround failed with D-Bus client creation errors. + +## Recommended Next Directions + +### Direction A: Use host-node runtime on the target device + +This is the most direct path to a working deployment. + +- Run `comapeo-headless` directly with Node on the Pi. +- Manage it with `systemd` instead of Docker. +- This preserves the real LAN/mDNS environment and already proved workable in + testing. + +### Direction B: Re-test Docker only on a native rootful engine + +If Docker remains a hard requirement: + +- test on the actual Raspberry Pi target +- use native/rootful Docker Engine +- verify from inside the container that `os.networkInterfaces()` exposes the + real LAN interface and address + +If that check fails, discovery should be treated as unsupported in that Docker +runtime. + +### Direction C: Add runtime diagnostics for unsupported Docker discovery + +A pragmatic improvement would be to add startup diagnostics that log: + +- detected interfaces from `os.networkInterfaces()` +- whether the process appears to be in Docker +- whether Docker appears rootless +- a warning when the process cannot see a plausible LAN interface + +This would not fix discovery, but it would make the failure mode obvious. + +### Direction D: Host-side publication only if the runtime allows it + +If containerization is still required later: + +- prefer a host-side publisher or native host mDNS integration +- do not assume in-container `ciao` or in-container Avahi publication will work + on rootless/VM-backed setups + +This direction needs a runtime that can actually talk to the host publisher +cleanly, which was not true here. + +## Practical Recommendation + +For the current environment, stop trying to make Android discovery work through +this rootless Docker setup. + +The best next move is: + +1. run the daemon directly on the target host or Pi +2. use `systemd` for service management +3. only revisit Docker on a native/rootful engine where the container can prove + it sees the real LAN interface diff --git a/Dockerfile b/Dockerfile new file mode 100644 index 0000000..439a1c7 --- /dev/null +++ b/Dockerfile @@ -0,0 +1,72 @@ +# syntax=docker/dockerfile:1 +# +# CoMapeo Headless – multi-stage Docker image +# +# Primary target: linux/arm64 (Raspberry Pi 4/5 and similar) +# Also supports: linux/amd64 (local development / CI) +# +# Native modules (better-sqlite3, sodium-native) are compiled in the builder +# stage and copied directly to the runtime stage, so no build toolchain is +# needed at runtime. + +# --------------------------------------------------------------------------- # +# Stage 1: builder +# Full toolchain: npm ci (compiles native addons), tsc build, prune dev deps. +# --------------------------------------------------------------------------- # +FROM node:24-bookworm-slim AS builder + +WORKDIR /app + +# Build toolchain for native addons (better-sqlite3, sodium-native). +RUN apt-get update && apt-get install -y --no-install-recommends \ + python3 \ + make \ + g++ \ + && rm -rf /var/lib/apt/lists/* + +# Install ALL dependencies (needed to compile native addons and run tsc). +COPY package*.json ./ +RUN npm ci + +# Compile TypeScript source → dist/. +COPY tsconfig.json tsconfig.build.json ./ +COPY src ./src +RUN npm run build + +# Drop dev dependencies from node_modules in-place so we can copy a lean set +# of production modules to the runtime stage. +RUN npm prune --omit=dev + +# --------------------------------------------------------------------------- # +# Stage 2: runtime +# Minimal image: no build tools, pre-compiled native addons from builder. +# --------------------------------------------------------------------------- # +FROM node:24-bookworm-slim AS runtime + +WORKDIR /app + +# Copy pre-built production node_modules (includes compiled native addons). +COPY --from=builder /app/node_modules ./node_modules + +# Copy compiled JS output. +COPY --from=builder /app/dist ./dist + +# Copy package.json so Node can resolve package metadata. +COPY package.json ./ + +ENV NODE_ENV=production +ENV COMAPEO_DATA_DIR=/data + +# Data volume: CoMapeo state, SQLite databases, root key, readiness marker. +VOLUME ["/data"] + +# Health check: passes once the daemon writes /data/.ready after full startup. +# --start-period gives the daemon time to boot before failures are counted. +HEALTHCHECK \ + --interval=10s \ + --timeout=5s \ + --start-period=30s \ + --retries=3 \ + CMD test -f /data/.ready + +CMD ["node", "dist/daemon/index.js"] diff --git a/README.md b/README.md index 5be0bad..5d5d674 100644 --- a/README.md +++ b/README.md @@ -1,177 +1,51 @@ -# comapeo-local-server +# comapeo-headless Docker investigation -> Local-Server CoMapeo daemon for local servers +This branch keeps the Docker and Compose experiments for `comapeo-headless`. -[![Node Version](https://img.shields.io/badge/node-%3E%3D24.0.0-brightgreen)](https://nodejs.org) -[![License](https://img.shields.io/badge/license-AGPL--3.0-blue)](LICENSE) -[![TypeScript](https://img.shields.io/badge/TypeScript-5.9-blue)](https://www.typescriptlang.org/) +It is not the supported v1 release path. The supported release branch is +`main`, which documents direct host-Node deployment. -`comapeo-local-server` is a Node-first CoMapeo daemon designed for local-server devices. It enables continuous synchronization and peer discovery on Linux-based devices without a graphical interface. +## Scope -## Features +This branch preserves: -- **Local-Server Operation**: Runs as a background daemon without GUI requirements -- **Automatic Sync**: Continuous synchronization with CoMapeo peers over mDNS -- **Invite Management**: Auto-accepts project invitations (configurable) -- **HTTP API**: Fastify-based API for integration with other services -- **Zero Configuration**: Works out of the box with sensible defaults +- `Dockerfile` +- `docker-compose.yml` +- Docker readiness marker support +- Docker discovery notes in `DOCKER_DISCOVERY_FINDINGS.md` -## Quick Start +## Current status -Get up and running in under a minute: - -```bash -# Install dependencies -npm install - -# Create configuration -cp .env.example .env - -# Start the daemon -node --run start -``` - -That's it! The daemon will automatically discover and sync with other CoMapeo peers on your local network. - -## Requirements - -- **Node.js**: >= 24.0.0 -- **npm**: >= 10.0.0 -- **OS**: Linux (for mDNS peer discovery) -- **Network**: LAN environment for local peer discovery - -## Installation - -```bash -# Clone the repository -git clone -cd comapeo-local-server - -# Install dependencies -npm install -``` - -## Configuration - -Create a `.env` file from the example: - -```bash -cp .env.example .env -``` - -### Required Variables - -| Variable | Description | Example | -|----------|-------------|---------| -| `COMAPEO_DEVICE_NAME` | Device name shown to peers | `my-field-device` | - -### Optional Variables - -| Variable | Description | Default | -|----------|-------------|---------| -| `COMAPEO_DATA_DIR` | Storage root directory | `./data` | -| `COMAPEO_ROOT_KEY` | Hex-encoded root key override | - | -| `COMAPEO_AUTO_ACCEPT_INVITES` | Auto-accept project invitations | `true` | -| `COMAPEO_DEVICE_TYPE` | Device type identifier | `desktop` | -| `ONLINE_STYLE_URL` | Custom style URL override | - | -| `LOG_LEVEL` | Logging verbosity | `info` | - -### Example Configuration - -```bash -# .env -COMAPEO_DEVICE_NAME=field-device-01 -COMAPEO_AUTO_ACCEPT_INVITES=true -LOG_LEVEL=debug -``` - -## Usage - -### Development Mode - -```bash -# Run with TypeScript -node --run start -``` - -### Production Mode - -```bash -# Build for production -npm run build - -# Run production build -COMAPEO_DEVICE_NAME=my-device node --run start:prod -``` - -### Smoke Test - -Verify the daemon works correctly: - -```bash -node --run start:smoke -``` - -## Deployment - -For production deployments, run the daemon directly on the host and manage it with a service manager like `systemd`. This preserves the host LAN environment required for mDNS peer discovery. - -### Example systemd Service - -```ini -[Unit] -Description=CoMapeo Local-Server Daemon -After=network.target - -[Service] -Type=simple -User=mapeo -WorkingDirectory=/opt/comapeo-local-server -Environment="COMAPEO_DEVICE_NAME=field-device-01" -ExecStart=/usr/bin/node --run start:prod -Restart=always - -[Install] -WantedBy=multi-user.target -``` +The daemon starts in Docker, but the Docker runtime tested here did not provide +reliable LAN discovery for Android peers. The current findings point to the +runtime and network model, not the daemon bootstrap path itself. ## Verification -Run the test suite to verify your installation: - ```bash -# Run unit tests npm test - -# Run type checking npm run typecheck - -# Run smoke test -node --run start:smoke ``` -## Project Structure +For the investigation context and next directions, see +`DOCKER_DISCOVERY_FINDINGS.md`. -``` -comapeo-local-server/ -├── src/ -│ ├── config/ # Environment parsing and validation -│ ├── core/ # MapeoManager bootstrap and lifecycle -│ └── daemon/ # Daemon entry and invite handling -├── test/ # Unit and integration tests -└── scripts/ - └── apply-comapeo-core-sync-patch.mjs # Postinstall patches -``` +--- -## Contributing +## Docker Investigation (Preserved for Reference) -Contributions are welcome! Please ensure all tests pass and type checking completes before submitting a pull request. +This branch preserves Docker and Compose experiments for `comapeo-local-server`. Note that this is not the supported v1 release path - the supported release branch is `main`, which documents direct host-Node deployment. -```bash -npm test -npm run typecheck -``` +### Scope + +The investigation preserves: + +- `Dockerfile` +- Docker readiness marker support +- Docker discovery notes in `DOCKER_DISCOVERY_FINDINGS.md` + +### Current Status -## License +The daemon starts in Docker, but the Docker runtime tested here did not provide reliable LAN discovery for Android peers. The current findings point to the runtime and network model, not the daemon bootstrap path itself. -AGPL-3.0 - see LICENSE file for details. +For the investigation context and next directions, see `DOCKER_DISCOVERY_FINDINGS.md`. diff --git a/src/daemon/index.ts b/src/daemon/index.ts index 4acaf29..bbe0491 100644 --- a/src/daemon/index.ts +++ b/src/daemon/index.ts @@ -6,6 +6,7 @@ import { initCore } from '../core/index.js' import { configureLogging } from '../logging/index.js' import { startInviteHandler } from './invites.js' import { enableSyncForJoinedProjects, startAlwaysOnSync } from './sync.js' +import { markReady, clearReady } from './ready.js' const log = debug('comapeo:daemon') @@ -37,13 +38,15 @@ async function main() { () => enableSyncForJoinedProjects(core.manager), ) - // Signal readiness for smoke tests once startup is complete. + // Signal readiness: stdout marker for smoke tests + file marker for Docker healthcheck. + markReady(config.dataDir) process.stdout.write('READY\n') // ── Signal handling ─────────────────────────────────────────────────────── await new Promise((resolve) => { async function shutdown(signal: string) { log('Received %s, shutting down', signal) + clearReady(config.dataDir) inviteHandler.stop() alwaysOnSync.stop() await core.stop() diff --git a/src/daemon/ready.ts b/src/daemon/ready.ts new file mode 100644 index 0000000..5257d59 --- /dev/null +++ b/src/daemon/ready.ts @@ -0,0 +1,31 @@ +import { rmSync, writeFileSync } from 'node:fs' +import { join } from 'node:path' +import debug from 'debug' + +const log = debug('comapeo:daemon:ready') + +const READY_FILE_NAME = '.ready' + +/** + * Write the readiness marker file so the Docker healthcheck can detect that + * the daemon has fully started (config valid, storage ready, manager up). + */ +export function markReady(dataDir: string): void { + const readyPath = join(dataDir, READY_FILE_NAME) + writeFileSync(readyPath, String(Date.now()), { encoding: 'utf8' }) + log('Readiness marker written: %s', readyPath) +} + +/** + * Remove the readiness marker on clean shutdown so the container is not + * considered healthy after the daemon has exited. + */ +export function clearReady(dataDir: string): void { + const readyPath = join(dataDir, READY_FILE_NAME) + try { + rmSync(readyPath) + log('Readiness marker removed: %s', readyPath) + } catch { + // best effort – the file may not exist if startup failed before markReady + } +}