|
1 | | -# Synchronized Data |
| 1 | +# Synchronized Data |
| 2 | + |
| 3 | +Proxbox synchronizes data between Proxmox clusters and NetBox using a dual-mode sync architecture. The NetBox plugin triggers sync requests against the ProxBox FastAPI backend, which performs the actual object creation and updates. |
| 4 | + |
| 5 | +## Sync Modes |
| 6 | + |
| 7 | +### POST Polling (Traditional) |
| 8 | + |
| 9 | +When you click a sync button, the plugin sends a POST request to the backend and waits for a single JSON response containing the final result. This is the simplest mode and works well for quick syncs. |
| 10 | + |
| 11 | +### GET SSE Streaming (Real-Time Progress) |
| 12 | + |
| 13 | +For longer sync operations, the plugin uses Server-Sent Events (SSE) to stream progress updates in real time. When you click a sync button, the browser fetches a streaming endpoint, and the plugin proxies the backend's `text/event-stream` response back to the page. Progress updates appear immediately in the sync log without waiting for the entire operation to finish. |
| 14 | + |
| 15 | +The plugin prefers SSE streaming when available. Each sync button carries both a POST URL (`data-sync-url`) and a stream URL (`data-sync-stream-url`). The browser JavaScript uses the stream URL when present. |
| 16 | + |
| 17 | +## Stream Endpoints |
| 18 | + |
| 19 | +| Plugin Path | Backend Path | Description | |
| 20 | +|-------------|--------------|-------------| |
| 21 | +| `sync/devices/stream/` | `GET /dcim/devices/create/stream` | Stream device synchronization progress | |
| 22 | +| `sync/virtual-machines/stream/` | `GET /virtualization/virtual-machines/create/stream` | Stream VM synchronization progress | |
| 23 | +| `sync/full-update/stream/` | `GET /full-update/stream` | Stream full update (devices + VMs) progress | |
| 24 | + |
| 25 | +## Progress Messages |
| 26 | + |
| 27 | +SSE streaming provides granular per-object progress messages. For example, during a full update you might see: |
| 28 | + |
| 29 | +``` |
| 30 | +full-update: Starting devices synchronization. |
| 31 | +full-update: Processing device pve01 |
| 32 | +full-update: Synced device pve01 |
| 33 | +full-update: Processing device pve02 |
| 34 | +full-update: Synced device pve02 |
| 35 | +full-update: Devices synchronization finished. |
| 36 | +full-update: Starting virtual machines synchronization. |
| 37 | +full-update: Processing virtual_machine vm101 |
| 38 | +full-update: Synced virtual_machine vm101 |
| 39 | +full-update: Virtual machines synchronization finished. |
| 40 | +full-update: Full update sync completed. |
| 41 | +full-update: stream completed |
| 42 | +``` |
| 43 | + |
| 44 | +## SSE Event Format |
| 45 | + |
| 46 | +All stream endpoints return `Content-Type: text/event-stream` and emit three event types: |
| 47 | + |
| 48 | +- **step**: Progress frame with `step` (object kind), `status` (`started`, `progress`, `completed`), `message` (human-readable text), and `rowid` (object name/ID). |
| 49 | +- **error**: Error frame when an object fails to sync. Contains `step`, `error`, and `detail`. |
| 50 | +- **complete**: Final frame with `ok` (boolean) and `message`. Marks the end of the stream. |
| 51 | + |
| 52 | +## Failure Handling |
| 53 | + |
| 54 | +- If the backend returns an error during streaming, the plugin emits an SSE `error` frame with the failure details and continues with a `complete` frame. |
| 55 | +- If the stream proxy itself encounters an error (e.g., backend unreachable), it emits a fallback SSE error frame instead of returning a Django 500 page. |
| 56 | +- The browser JavaScript displays error details from both `error` frames and non-200 HTTP responses. |
| 57 | + |
| 58 | +## WebSocket Mode (Legacy) |
| 59 | + |
| 60 | +The backend also provides a WebSocket endpoint (`/ws`) for interactive sync. This predates the SSE streaming approach and sends the same per-object progress JSON over a bidirectional WebSocket channel. SSE streaming is now preferred for browser-based sync because it works with standard HTTP requests. |
0 commit comments