Skip to content

Releases: radlab-dev-group/llm-router-services

v0.0.3

19 Dec 12:15

Choose a tag to compare

Summary

Key changes:

  1. router.py – new central router that discovers which guard‑rail services are enabled via LLM_ROUTER_*_ENABLED environment variables and registers their routes dynamically.
  2. Guard‑rail modules (nask_pib_guard_app.py & sojka_guard_app.py) – converted into plug‑ins: each now provides a register_routes(app) function and a private _build_guardrail() helper. No more standalone if __name__ == '__main__' blocks.
  3. run_router.sh – a convenience Bash wrapper that sets sensible defaults, exports the required env‑vars, prints the effective configuration and launches the unified API with Gunicorn.
  4. README.md – updated to reflect the new unified API, the dynamic environment‑variable control, the new launch script and the simplified extensibility workflow.

Why

  • Simplify deployment – one process, one port, one set of env‑vars instead of maintaining separate Flask apps for each guard‑rail.
  • Make adding new services trivial – just implement register_routes(app) and add an entry to the router’s registry; no code changes to the core.
  • Consistent configuration – all services now honour the LLM_ROUTER_… prefix, which aligns with the rest of the LLM‑Router ecosystem.
  • Better developer experience – the new run_router.sh script mirrors the previous per‑service scripts but works for any combination of enabled guard‑rails.

How to test

  1. Enable both guard‑rails (or any subset) by exporting the appropriate LLM_ROUTER_*_ENABLED=1 flags.
  2. Run the script:
./run_router.sh
  1. Verify the output shows which services are enabled and that Gunicorn starts on the expected host/port.
  2. Send a request to each exposed endpoint, e.g.:
curl -X POST http://localhost:5000/api/guardrails/nask_guard \
        -H "Content-Type: application/json" \
        -d '{"payload":"test"}'

The response should contain a safe flag and a detailed list.
5. Turn off a service (unset its *_ENABLED flag) and repeat – the corresponding endpoint should return 404.

Backward compatibility

  • Existing per‑service scripts (run_nask_guardrail.sh, run_sojka_guardrail.sh, etc.) still work because the original Flask apps are unchanged – they are just imported by the router when enabled.
  • No public API changes; only the deployment entry point is consolidated.

v0.0.2

01 Dec 13:16

Choose a tag to compare

Merge branch 'features/sojka'

v0.0.1

28 Nov 20:22

Choose a tag to compare