Merge pull request #302 from emersonfelipesp/v0.0.8

Release v0.0.8: multi-stage sync, vm snapshots, CI test alignment

Author: emersonfelipesp

.github/workflows/ci.yml

@@ -2,12 +2,32 @@ name: CI
 
 on:
   push:
-    branches: [develop, testing]
+    branches: ['**']
   pull_request:
-    branches: [develop, testing]
+    branches: ['**']
 
 jobs:
-  test:
+  lint:
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v6
+
+      - name: Set up Python
+        uses: actions/setup-python@v6
+        with:
+          python-version: "3.12"
+          cache: "pip"
+
+      - name: Install ruff
+        run: pip install ruff
+
+      - name: Run ruff (lint + format check)
+        run: |
+          ruff check .
+          ruff format --check .
+
+  compile:
     runs-on: ubuntu-latest
     steps:
       - name: Checkout
@@ -27,6 +47,24 @@ jobs:
       - name: Compile package
         run: python -m compileall netbox_proxbox tests
 
+  test:
+    runs-on: ubuntu-latest
+    needs: [lint, compile]
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v6
+
+      - name: Set up Python
+        uses: actions/setup-python@v6
+        with:
+          python-version: "3.12"
+          cache: "pip"
+
+      - name: Install dependencies
+        run: |
+          python -m pip install --upgrade pip
+          python -m pip install -e ".[test]"
+
       - name: Run mocked test suite
         run: pytest --cov=netbox_proxbox --cov-report=term-missing --cov-report=xml tests
 

.pre-commit-config.yaml

@@ -0,0 +1,33 @@
+repos:
+  - repo: local
+    hooks:
+      - id: compileall
+        name: compileall
+        entry: python -m compileall
+        language: system
+        pass_filenames: true
+        types: [python]
+        args: ["netbox_proxbox", "tests"]
+
+      - id: ruff
+        name: ruff
+        entry: ruff
+        language: system
+        pass_filenames: true
+        types: [python]
+        args: ["check", "."]
+
+      - id: ruff-format
+        name: ruff-format
+        entry: ruff
+        language: system
+        pass_filenames: true
+        types: [python]
+        args: ["format", "--check", "."]
+
+      - id: pytest
+        name: pytest
+        entry: pytest
+        language: system
+        pass_filenames: false
+        args: ["tests/"]

AGENTS.md

@@ -1,5 +1,31 @@
 # Agent Entry Points
 
+## Pre-commit Checklist
+
+**Before committing ANY change:**
+
+1. Run syntax check: `python -m compileall netbox_proxbox tests`
+2. Run linter: `rtk ruff check .`
+3. Run tests: `rtk pytest tests/`
+
+---
+
+## Framework stack preference
+
+When implementing or changing behavior, prefer solutions in this order:
+
+1. **NetBox plugin idioms** — Patterns used in this plugin and in NetBox’s plugin framework (`netbox.plugins`, plugin models, views, tables, filtersets, serializers as NetBox documents them).
+2. **NetBox core** — Built-in modules such as `utilities.forms`, `utilities.views`, `netbox.*` model and API bases, and DRF usage aligned with upstream NetBox.
+3. **Django** — Standard `django.*` APIs when NetBox does not expose an equivalent.
+
+Do **not** add new third-party PyPI dependencies to replace what NetBox or Django already provides (forms, widgets, routing, auth, REST patterns). Existing runtime deps in `pyproject.toml` (for example `requests`, `websockets`, optional CLI extras) are fine; avoid piling on more libraries for the same problems.
+
+## Security
+
+Use NetBox view mixins from `utilities.views` (`ConditionalLoginRequiredMixin`, `TokenConditionalLoginRequiredMixin`, `ContentTypePermissionRequiredMixin`) for custom routes; enforce object visibility with `QuerySet.restrict()`. Permission strings for ProxBox-specific operations are centralized in [`netbox_proxbox/views/proxbox_access.py`](./netbox_proxbox/views/proxbox_access.py). Details: [`CLAUDE.md`](./CLAUDE.md) (Security and permissions).
+
+---
+
 Read [`CLAUDE.md`](./CLAUDE.md) first for the plugin architecture and the full documentation map. Use the lower-level `CLAUDE.md` files when working in a specific directory or when changing only one layer of the plugin.
 
 ## CLAUDE.md Index

CLAUDE.md

@@ -1,8 +1,40 @@
 # netbox-proxbox Codebase Guide
 
+## Pre-commit Checklist
+
+**Before committing ANY change:**
+
+1. Run syntax check: `python -m compileall netbox_proxbox tests`
+2. Run linter: `rtk ruff check .`
+3. Run tests: `rtk pytest tests/`
+
+---
+
+## Framework stack preference
+
+Follow the same dependency order agents use (see [`AGENTS.md`](./AGENTS.md)):
+
+1. **NetBox plugin layer** — Reuse this plugin’s established patterns and NetBox’s plugin APIs (registration, plugin paths, `NetBoxModel` / `NetBoxModelViewSet`, tables and filtersets consistent with other plugin code here).
+2. **NetBox core** — Prefer `utilities.forms.fields`, `utilities.forms.widgets`, `utilities.views`, and other `utilities.*` / `netbox.*` primitives before inventing parallel implementations.
+3. **Django** — Use `django.forms`, `django.http`, ORM, and related stdlib-backed APIs when NetBox does not offer a specific helper.
+
+**Third-party packages:** Do not introduce new PyPI dependencies for capabilities NetBox or Django already cover. The project already declares `requests`, `websockets`, and optional CLI-related packages in [`pyproject.toml`](./pyproject.toml); add new deps only for integration needs that have no NetBox/Django path, not as shortcuts for UI or API patterns the core stack handles.
+
+**Example:** NetBox may remove or rename widgets (for example legacy Select2 helpers under `utilities.forms.widgets`). Prefer current NetBox field/widget pairs such as `DynamicModelMultipleChoiceField` with API-driven multi-select rather than pulling in extra front-end or Python widget libraries. For form layout and field choices in this plugin, see [`netbox_proxbox/forms/CLAUDE.md`](./netbox_proxbox/forms/CLAUDE.md).
+
+## Security and permissions
+
+- **Registered CRUD** (via `register_model_view` and `netbox.views.generic`) inherits NetBox `ObjectPermissionRequiredMixin`: model permissions plus `queryset.restrict()` for object-level rules.
+- **Custom views** should use `utilities.views.ConditionalLoginRequiredMixin` (respects `LOGIN_REQUIRED`) instead of Django’s unconditional `login_required`, and `TokenConditionalLoginRequiredMixin` where REST tokens should authenticate browser-style endpoints.
+- **Operational endpoints** (sync, SSE streams, schedule job, WebSocket bridge): `ContentTypePermissionRequiredMixin` with permissions defined in [`netbox_proxbox/views/proxbox_access.py`](./netbox_proxbox/views/proxbox_access.py) — typically `change` on `FastAPIEndpoint` for backend sync, `add` on `SyncProcess` for queued jobs, `view` on `FastAPIEndpoint` for read-only WebSocket test UI.
+- **Dashboard and JSON helpers**: plugin home requires at least one of `view` on `ProxmoxEndpoint` / `NetBoxEndpoint` / `FastAPIEndpoint` when the user is authenticated; endpoint lists use `.restrict(request.user, "view")`. Proxmox card and keepalive JSON resolve objects through restricted querysets (`get_object_or_404(...restrict(...))`). Tagged devices and VMs use `Device.objects.restrict` / `VirtualMachine.objects.restrict` before listing.
+- **Plugin REST API** remains on `NetBoxModelViewSet` with standard NetBox/DRF permission classes.
+
+---
+
 This repository packages the `netbox_proxbox` NetBox plugin. The plugin adds endpoint inventory for Proxmox, NetBox, and the companion ProxBox FastAPI backend; UI pages for sync operations and status checks; REST API endpoints for those models; and a small amount of browser-side JavaScript and styling for the plugin pages.
 
-The current plugin config lives in [`netbox_proxbox/__init__.py`](./netbox_proxbox/__init__.py). It declares plugin version `0.0.7` and NetBox compatibility `4.5.0` through `4.5.99`.
+The current plugin config lives in [`netbox_proxbox/__init__.py`](./netbox_proxbox/__init__.py). It declares plugin version `0.0.8` and NetBox compatibility `4.5.0` through `4.5.99`.
 
 ## Architecture Summary
 
@@ -15,6 +47,16 @@ The current plugin config lives in [`netbox_proxbox/__init__.py`](./netbox_proxb
 - Browser updates can flow over SSE streams or the existing WebSocket channel.
 - Templates and static assets are conventional Django plugin assets under `netbox_proxbox/templates/` and `netbox_proxbox/static/`.
 
+## Backend integration notes
+
+- **Single FastAPI row:** HTTP and WebSocket helpers such as `get_fastapi_request_context()` in [`netbox_proxbox/services/backend_proxy.py`](./netbox_proxbox/services/backend_proxy.py), `websocket_client`, and several dashboard views resolve the backend via `FastAPIEndpoint.objects.first()` (or the first row from a restricted queryset). If multiple FastAPI endpoints exist, whichever row sorts first is used; plan automation and operator docs accordingly.
+- **Background Proxbox sync jobs (RQ):** `ProxboxSyncJob` enqueues on NetBox’s **`default`** RQ queue (`RQ_QUEUE_DEFAULT`) so a stock **`manage.py rqworker`** (no queue arguments) picks them up. NetBox’s default worker only listens to **`high`**, **`default`**, and **`low`**; the extra django-rq queue **`netbox_proxbox.sync`** is still registered for the plugin but is no longer the enqueue target. Older Job rows may still show **`netbox_proxbox.sync`** in **Queue**; cancel/RQ lookup uses the stored name. Jobs call proxbox-api **SSE** via [`run_sync_stream`](./netbox_proxbox/services/backend_proxy.py) until a terminal `complete` event.
+- **RQ timeout vs HTTP stream:** NetBox’s default **`RQ_DEFAULT_TIMEOUT`** (often **300s** via `configuration.py`) applies to RQ jobs unless overridden. Long syncs were previously killed by RQ while `requests` was still reading the SSE body. The plugin sets a default **`job_timeout`** of **`PROXBOX_SYNC_JOB_TIMEOUT`** (7200s) in [`ProxboxSyncJob.enqueue`](./netbox_proxbox/jobs.py); pass a larger `job_timeout=` to `enqueue()` if needed. That is separate from the HTTP **between-chunk read** timeout (3600s) inside [`run_sync_stream`](./netbox_proxbox/services/backend_proxy.py).
+- **When a job looks “stuck”:** **pending** usually means **no RQ worker** is running (or it does not listen to **`default`**). **running** for a long time usually means proxbox-api is still syncing or the stream is slow/buffered; **errored** with **`JobTimeoutException`** means RQ’s wall-clock limit was hit—increase `job_timeout` or `PROXBOX_SYNC_JOB_TIMEOUT`. Inspect the job **log** and **error** fields before changing code.
+- **Cancel on Job detail:** For Proxbox Sync rows in **pending**, **scheduled**, or **running** state, the plugin adds **Cancel job** (POST to `proxbox-cancel`). It requires **delete** permission on the core **Job** model, cancels or stops the linked RQ job when possible, then marks the NetBox job **failed** with a “Cancelled by user.” message. Stopping a **running** job is best-effort (RQ stop + long HTTP reads may not abort instantly).
+- **Run now on Job detail:** Shown only when the job is in a **terminal** state (**completed**, **errored**, or **failed**), including after **Cancel** (failed). It is **not** shown for **pending**, **scheduled**, or **running**—use **Cancel** first if a queued run should be abandoned, then **Run now** on the finished row to queue a new sync with the same parameters.
+- **Full update (UI vs jobs):** The plugin home may still use non-streaming helpers such as [`sync_full_update_resource`](./netbox_proxbox/services/backend_proxy.py) for JSON/redirect flows. Scheduled or immediate **Proxbox Sync** jobs use **`full-update/stream`** on proxbox-api (devices then VMs in one stream; no backups in that stream). Run a **VM backups** sync type when backups are required.
+
 ## How To Navigate
 
 - Start with [`netbox_proxbox/CLAUDE.md`](./netbox_proxbox/CLAUDE.md) for the package-level map.