Releases: radlab-dev-group/llm-router-services
Releases · radlab-dev-group/llm-router-services
v0.0.3
Summary
Key changes:
router.py– new central router that discovers which guard‑rail services are enabled viaLLM_ROUTER_*_ENABLEDenvironment variables and registers their routes dynamically.- Guard‑rail modules (
nask_pib_guard_app.py&sojka_guard_app.py) – converted into plug‑ins: each now provides aregister_routes(app)function and a private_build_guardrail()helper. No more standaloneif __name__ == '__main__'blocks. run_router.sh– a convenience Bash wrapper that sets sensible defaults, exports the required env‑vars, prints the effective configuration and launches the unified API with Gunicorn.README.md– updated to reflect the new unified API, the dynamic environment‑variable control, the new launch script and the simplified extensibility workflow.
Why
- Simplify deployment – one process, one port, one set of env‑vars instead of maintaining separate Flask apps for each guard‑rail.
- Make adding new services trivial – just implement
register_routes(app)and add an entry to the router’s registry; no code changes to the core. - Consistent configuration – all services now honour the
LLM_ROUTER_…prefix, which aligns with the rest of the LLM‑Router ecosystem. - Better developer experience – the new
run_router.shscript mirrors the previous per‑service scripts but works for any combination of enabled guard‑rails.
How to test
- Enable both guard‑rails (or any subset) by exporting the appropriate
LLM_ROUTER_*_ENABLED=1flags. - Run the script:
./run_router.sh- Verify the output shows which services are enabled and that Gunicorn starts on the expected host/port.
- Send a request to each exposed endpoint, e.g.:
curl -X POST http://localhost:5000/api/guardrails/nask_guard \
-H "Content-Type: application/json" \
-d '{"payload":"test"}'The response should contain a safe flag and a detailed list.
5. Turn off a service (unset its *_ENABLED flag) and repeat – the corresponding endpoint should return 404.
Backward compatibility
- Existing per‑service scripts (
run_nask_guardrail.sh,run_sojka_guardrail.sh, etc.) still work because the original Flask apps are unchanged – they are just imported by the router when enabled. - No public API changes; only the deployment entry point is consolidated.