Updated .env.example to include new settings for audit logging, artifact retention, and HTTPS enforcement.

- Added a link to HIPAA requirements in `README.md` for compliance guidance.
- Improved API reference documentation to clarify Phaxio backend behavior and artifact cleanup options.
- Documented E.164 phone number format in SIP setup and API reference for better user guidance.
- Added troubleshooting notes for HTTPS usage in production environments.

Author: DMontgomery40

.env.example

@@ -16,6 +16,7 @@ PHAXIO_API_SECRET=
 PUBLIC_API_URL=http://localhost:8080
 PHAXIO_STATUS_CALLBACK_URL=http://localhost:8080/phaxio-callback
 PHAXIO_VERIFY_SIGNATURE=true
+ENFORCE_PUBLIC_HTTPS=false
 
 # === SIP/ASTERISK BACKEND ===
 # Only needed if FAX_BACKEND=sip
@@ -43,3 +44,15 @@ TZ=UTC
 # - Use a reverse proxy for rate limiting and IP restrictions in production
 # - PDF token TTL controls how long the cloud fetch link is valid
 PDF_TOKEN_TTL_MINUTES=60
+
+# Retention (set >0 to enable automatic artifact cleanup)
+ARTIFACT_TTL_DAYS=0
+# Cleanup interval in minutes (default daily)
+CLEANUP_INTERVAL_MINUTES=1440
+
+# Audit logging (optional)
+AUDIT_LOG_ENABLED=false
+AUDIT_LOG_FORMAT=json
+# AUDIT_LOG_FILE=/var/log/faxbot_audit.log
+AUDIT_LOG_SYSLOG=false
+AUDIT_LOG_SYSLOG_ADDRESS=/dev/log

.github/workflows/ci.yml

@@ -0,0 +1,35 @@
+name: CI
+
+on:
+  push:
+    branches: [ main, master ]
+  pull_request:
+    branches: [ "**" ]
+
+jobs:
+  test-api:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - name: Set up Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: '3.11'
+      - name: Install system dependencies
+        run: |
+          sudo apt-get update
+          sudo apt-get install -y ghostscript
+      - name: Install Python deps
+        working-directory: api
+        run: |
+          python -m pip install --upgrade pip
+          pip install -r requirements.txt
+      - name: Run tests
+        env:
+          FAX_DISABLED: 'true'
+          FAX_DATA_DIR: './faxdata'
+          DATABASE_URL: 'sqlite:///./test_faxbot_ci.db'
+        working-directory: api
+        run: |
+          mkdir -p faxdata
+          pytest -q

HIPAA_REQUIREMENTS.md

@@ -0,0 +1,148 @@
+# HIPAA_REQUIREMENTS.md
+
+This document describes what is required to operate Faxbot in a HIPAA‑aligned manner. It is a technical guide and checklist for engineers and operators. It is not legal advice. Always consult your compliance team and counsel. You (the operator) are responsible for implementing and documenting the controls below and for executing a formal risk analysis and governance program.
+
+## Scope & Data Flows
+- Covered workflows: sending faxes that may contain PHI.
+- Not covered: receiving faxes (non‑goal), messaging, IVR, EHR integrations.
+
+Backends (choose one):
+- Phaxio (cloud): Client → Faxbot API → Phaxio → PSTN/Fax. Phaxio fetches the PDF from your `PUBLIC_API_URL` and posts status callbacks.
+- SIP/Asterisk (self‑hosted): Client → Faxbot API → Asterisk (T.38/UDPTL) → SIP trunk → PSTN/Fax.
+
+PHI touchpoints:
+- PDF/TXT upload to Faxbot API.
+- Stored job artifacts (original, PDF, TIFF for SIP).
+- Status updates (Phaxio callbacks or Asterisk AMI user events).
+- Application/Reverse proxy logs (must be PHI‑free).
+
+## Roles & Agreements
+- If you are a Covered Entity or Business Associate, you must:
+  - Execute a BAA with any cloud provider that may handle PHI (e.g., Phaxio). Contact provider sales to obtain a BAA; do not use without a BAA.
+  - Treat Faxbot and Asterisk operators as Business Associates if they are separate entities.
+- Self‑hosted SIP stack does not remove HIPAA obligations; it moves them to you.
+
+## Technical Safeguards (Security Rule)
+Implement the following as minimum controls:
+
+1) Transport security
+- Public API must be served over HTTPS. Use TLS certs from a reputable CA.
+- For Phaxio backend:
+  - `PUBLIC_API_URL` must be HTTPS in production.
+  - Enable callback signature verification (default on): `PHAXIO_VERIFY_SIGNATURE=true`. Server verifies `X-Phaxio-Signature` (HMAC‑SHA256 over raw body with `PHAXIO_API_SECRET`).
+- For SIP backend:
+  - SIP signaling should use TLS if supported by your provider; media (T.38 over UDPTL) is typically not encrypted. Mitigate with a site‑to‑site VPN/private interconnect to your SIP provider and strict firewalling.
+  - Never expose AMI (5038/tcp) to the public internet.
+
+2) Access control
+- Require API key on all /fax and /fax/{id} calls (`X-API-Key`). Do not run with blank `API_KEY` in production.
+- Restrict inbound traffic with a reverse proxy: IP allowlists and rate limiting.
+- Rotate credentials and set a strong AMI password. Do not use `changeme`.
+
+3) Data minimization & confidentiality
+- Do not log PHI. Ensure request bodies (PDF/TXT) and rendered content are never logged.
+- Faxbot redacts tokenized PDF URLs from logs.
+- Tokenized PDF access:
+  - The server issues a per‑job, random `pdf_token` with a short TTL (`PDF_TOKEN_TTL_MINUTES`, default 60). The `/fax/{job_id}/pdf` endpoint requires exact token equality and enforces expiry.
+  - Keep TTL as short as operationally feasible.
+
+4) Storage security (at rest)
+- Store database and artifacts on encrypted volumes or use a managed, encrypted database. SQLite is acceptable only if disk encryption and backups are in place.
+- Separate storage for development vs production. Limit admin access and use MFA on hosts.
+- Data retention policy: delete PDFs/TIFFs after transmission completes and your minimum retention requirement is satisfied.
+
+5) Integrity & auditing
+- Maintain audit logs of access to `/fax/{job_id}/pdf`, job creation, and status changes. No PHI in logs; use job IDs and metadata only.
+- Time synchronize servers (NTP) for accurate audit trails.
+
+6) Availability & recovery
+- Back up database (and optionally artifacts) on a secure, encrypted target with rotation.
+- Document restore procedures and test periodically.
+
+## Administrative Safeguards
+- Perform and document a HIPAA risk analysis for this system, covering threats to confidentiality, integrity, and availability.
+- Draft and adopt policies: access control, incident response, change management, data retention/secure destruction, vulnerability management.
+- Train workforce members on PHI handling and minimum necessary principles.
+- Maintain vendor due diligence (e.g., Phaxio BAA, SOC2 reports where applicable).
+
+## Physical Safeguards
+- Secure data center/hosting environment. For on‑prem: locked server rooms, visitor controls. For cloud: select providers with appropriate attestations.
+
+## Backend‑Specific Guidance
+
+### Phaxio (Cloud)
+- Required:
+  - BAA with Phaxio before sending PHI.
+  - HTTPS `PUBLIC_API_URL`, valid certificate.
+  - `PHAXIO_VERIFY_SIGNATURE=true`.
+  - Strong `API_KEY` and reverse proxy restrictions.
+- Recommended:
+  - Keep `PDF_TOKEN_TTL_MINUTES` small (e.g., 15–60 minutes).
+  - Immediately delete PDFs after successful transmission unless retention policy requires otherwise.
+  - Validate that `PHAXIO_STATUS_CALLBACK_URL` is reachable only over TLS.
+
+### SIP/Asterisk (Self‑Hosted)
+- T.38/UDPTL is not encrypted. Mitigations:
+  - Use a site‑to‑site VPN/private interconnect to your SIP provider, or run Asterisk in a private data center with dedicated connectivity.
+  - Strict firewall allows only necessary ports and only to/from provider IPs.
+  - Use SIP TLS for signaling if supported by your provider; still keep media protected by VPN.
+- Asterisk hardening:
+  - Do not expose AMI externally. Bind to private networks only.
+  - Use non‑default usernames, strong secrets, fail2ban/IDS.
+  - Rotate credentials periodically. Log and alert on failed auth.
+
+## MCP (AI Assistant) Considerations
+- MCP transmits base64 file content for the `send_fax` tool. Treat the MCP server with the same controls as the API (auth, network restrictions, audit logging).
+- Do not send PHI to LLMs or external services unless you have a signed BAA and an approved use case.
+- Keep MCP HTTP behind authentication and rate limiting.
+
+## Operational Checklist (Minimum)
+- [ ] Signed BAA with Phaxio (if using cloud backend).
+- [ ] TLS everywhere (HTTPS for public endpoints; VPN/private link for SIP media).
+- [ ] API auth enabled (`API_KEY` set). Reverse proxy with IP allowlist + rate limiting.
+- [ ] Callback signature verification enabled (`PHAXIO_VERIFY_SIGNATURE=true`).
+- [ ] Tokenized PDF access enabled with short TTL (`PDF_TOKEN_TTL_MINUTES`).
+- [ ] Logs do not contain PHI; tokens redacted; job IDs only.
+- [ ] Encrypted storage for DB and artifacts; backups configured.
+- [ ] Data retention policy implemented (delete artifacts after N days or on success).
+- [ ] Asterisk AMI not exposed; strong credentials; fail2ban.
+- [ ] Risk analysis, policies, and training documented.
+
+## Current Implementation Status (2025‑Q3)
+- Implemented:
+  - API key support, reverse proxy guidance.
+  - Tokenized PDF access with equality check and TTL expiry.
+  - Phaxio callback signature verification (HMAC‑SHA256).
+  - AMI concurrency/backoff improvements; SIP dialplan emits granular results.
+  - Docs for HTTPS, rate limiting, NAT/port‑forwarding.
+- Gaps (operator‑dependent):
+  - Encryption at rest (volume or DB) is operator‑managed.
+  - Automated retention cleanup (cron/job) recommended (see below).
+  - Centralized audit logging & alerting recommended.
+
+## Remediation Plan & Roadmap
+1) Automate artifact retention
+- Add `ARTIFACT_TTL_DAYS` env with a daily cleanup job to purge PDFs/TIFFs older than TTL when job status is final.
+
+2) Configurable audit logging
+- Structured logs with job lifecycle events; optional sink to SIEM.
+
+3) Optional hard fail on plain HTTP
+- Reject `PUBLIC_API_URL` with `http://` in non‑local environments unless `ALLOW_INSECURE_PUBLIC_URL=true`.
+
+4) Secrets management
+- Guidance and examples for loading secrets from a vault (AWS/GCP/Azure) instead of env files.
+
+5) Provider‑specific SIP hardening
+- Example configs for TLS signaling and site‑to‑site VPN topologies.
+
+## Example: Retention Cleanup (Operator)
+- Create a cron or systemd timer to delete artifacts after N days:
+```
+# delete PDFs/TIFFs older than 7 days
+find /path/to/faxdata -type f \( -name '*.pdf' -o -name '*.tiff' \) -mtime +7 -delete
+```
+- Ensure backups honor retention and secure destruction policies.
+
+## Legal Notice
+- This document does not constitute legal advice. HIPAA compliance depends on your specific implementation, vendor agreements, and organizational controls. Engage qualified counsel and security professionals.

README.md

@@ -28,6 +28,7 @@ Simple fax-sending API with AI integration. Choose your backend:
 ## Documentation
 - [API Reference](docs/API_REFERENCE.md) — Endpoints and examples
 - [Troubleshooting](docs/TROUBLESHOOTING.md) — Common issues
+- [HIPAA Requirements](HIPAA_REQUIREMENTS.md) — Security, BAAs, and compliance checklist
 
 ## Notes
 - Send-only. Receiving is out of scope.

TODO.md

@@ -14,9 +14,9 @@ This is a focused, prioritized backlog based on the latest audit.
 - [COMPLETED] AMI reconnect concurrency safety
   - Added connection lock; background reconnect from read loop
   - Keeps exponential backoff
-- Asterisk AMI username templating
-  - Use `${ASTERISK_AMI_USERNAME}` as the section name in `manager.conf.template`
-  - Keep credentials aligned between API env and Asterisk
+- [COMPLETED] Asterisk AMI username templating
+  - `manager.conf.template` now uses `${ASTERISK_AMI_USERNAME}` for the section name
+  - Docs updated to call out the alignment with API env
 - [COMPLETED] Redact sensitive tokens from logs
   - Do not log `pdf_url` tokens
   - Ensure all logs contain job IDs, not secrets
@@ -26,38 +26,42 @@ This is a focused, prioritized backlog based on the latest audit.
   - Docs updated; scripts and Makefile remain valid
 
 ## Medium Priority
-- Skip TIFF generation for Phaxio path
-  - Avoid `pdf_to_tiff` when backend is `phaxio` to save CPU/IO
-  - Page count can be filled by provider callback
-- Source of truth for pages
-  - Ensure provider callback updates overwrite any local estimates
-  - Document this behavior
-- Public URL defaults and docs
-  - Prefer HTTPS for `PUBLIC_API_URL` in docs
-  - Warn on startup when using localhost in non‑dev
-- Rate limiting and auth posture
-  - Warn if `API_KEY` is unset and `FAX_DISABLED=false`
-  - Add reverse proxy (nginx/Caddy) examples for IP allowlist + rate limits
+- [COMPLETED] Skip TIFF generation for Phaxio path
+  - Do not convert to TIFF when backend is `phaxio`; pages set by callback
+- [COMPLETED] Source of truth for pages
+  - Provider callback overwrites local estimates; documented
+- [COMPLETED] Public URL defaults and docs
+  - Startup warns when `PUBLIC_API_URL` is plain HTTP (non-local)
+  - Docs recommend HTTPS for production
+- [COMPLETED] Rate limiting and auth posture
+  - Startup warns when `API_KEY` is unset and faxing is enabled
+  - Added reverse proxy examples (Nginx/Caddy) to Troubleshooting
 
 ## Low Priority
-- Phone number normalization
-  - Document E.164 expectation; avoid guessing country codes
+- [COMPLETED] Phone number normalization
+  - Documented E.164 preference and best‑effort normalization note
 - MCP example configs
   - Treat `api/configs/*.json` as examples; rely on `setup-mcp.js` to generate paths
-- Ghostscript flags
-  - Clean up redundant options (e.g., lzw with tiffg4)
+- [COMPLETED] Ghostscript flags
+  - Removed redundant compression flag for tiffg4
 - File retention
   - Add TTL cleanup for `orig`, `pdf`, and `tiff` artifacts
   - Document storage footprint and retention controls
-- CI tests and scripts
-  - Add GH Actions to run `pytest -q` with `FAX_DISABLED=true`
+- [COMPLETED] CI tests and scripts
+  - GitHub Actions workflow added to run tests
 
 ## Documentation Improvements
 - Port forwarding appendix for novices (router examples/screenshots)
 - Asterisk template walkthrough with env examples per provider
 - “Data persistence” note: mount DB/file volumes for durability
+- [ADDED] HIPAA requirements document and operator checklist
+  - Status: Completed (HIPAA_REQUIREMENTS.md)
+- [COMPLETED] Audit logging docs
+  - Added optional SIEM sink configuration and events list
+- [COMPLETED] Provider TLS/VPN examples
+  - Added PJSIP TLS and WireGuard sketches to SIP guide
 
 ---
 
 # In Progress
-- Implement Asterisk AMI username templating
+- (none)

api/app/audit.py

@@ -0,0 +1,65 @@
+import logging
+import json
+from typing import Any, Dict, Optional
+from logging.handlers import SysLogHandler
+
+
+def init_audit_logger(
+    enabled: bool,
+    fmt: str = "json",
+    filepath: Optional[str] = None,
+    use_syslog: bool = False,
+    syslog_address: Optional[str] = None,
+) -> None:
+    logger = logging.getLogger("audit")
+    if not enabled:
+        logger.disabled = True
+        return
+    logger.setLevel(logging.INFO)
+    # Avoid duplicate handlers on reload
+    if logger.handlers:
+        return
+    if filepath:
+        handler = logging.FileHandler(filepath)
+    elif use_syslog:
+        address = syslog_address or "/dev/log"
+        handler = SysLogHandler(address=address)
+    else:
+        handler = logging.StreamHandler()
+    handler.setLevel(logging.INFO)
+
+    if fmt == "json":
+        handler.setFormatter(_JsonFormatter())
+    else:
+        handler.setFormatter(logging.Formatter("%(message)s"))
+    logger.addHandler(handler)
+
+
+def audit_event(event: str, **fields: Any) -> None:
+    logger = logging.getLogger("audit")
+    if logger.disabled:
+        return
+    payload: Dict[str, Any] = {"event": event}
+    payload.update(fields)
+    logger.info(payload)
+
+
+def mask_number(num: Optional[str]) -> Optional[str]:
+    if not num:
+        return num
+    digits = [c for c in num if c.isdigit()]
+    if len(digits) <= 4:
+        return "****"
+    masked = "*" * (len(digits) - 4) + "".join(digits[-4:])
+    return masked
+
+
+class _JsonFormatter(logging.Formatter):
+    def format(self, record: logging.LogRecord) -> str:
+        if isinstance(record.msg, dict):
+            return json.dumps(record.msg, separators=(",", ":"))
+        try:
+            return json.dumps({"message": str(record.getMessage())})
+        except Exception:
+            return str(record.getMessage())
+

api/app/config.py

@@ -36,6 +36,18 @@ class Settings(BaseModel):
 
     # Security
     pdf_token_ttl_minutes: int = Field(default_factory=lambda: int(os.getenv("PDF_TOKEN_TTL_MINUTES", "60")))
+    enforce_public_https: bool = Field(default_factory=lambda: os.getenv("ENFORCE_PUBLIC_HTTPS", "false").lower() in {"1", "true", "yes"})
+
+    # Retention / cleanup
+    artifact_ttl_days: int = Field(default_factory=lambda: int(os.getenv("ARTIFACT_TTL_DAYS", "0")))  # 0=disabled
+    cleanup_interval_minutes: int = Field(default_factory=lambda: int(os.getenv("CLEANUP_INTERVAL_MINUTES", "1440")))
+
+    # Audit logging
+    audit_log_enabled: bool = Field(default_factory=lambda: os.getenv("AUDIT_LOG_ENABLED", "false").lower() in {"1", "true", "yes"})
+    audit_log_format: str = Field(default_factory=lambda: os.getenv("AUDIT_LOG_FORMAT", "json"))
+    audit_log_file: str = Field(default_factory=lambda: os.getenv("AUDIT_LOG_FILE", ""))
+    audit_log_syslog: bool = Field(default_factory=lambda: os.getenv("AUDIT_LOG_SYSLOG", "false").lower() in {"1", "true", "yes"})
+    audit_log_syslog_address: str = Field(default_factory=lambda: os.getenv("AUDIT_LOG_SYSLOG_ADDRESS", "/dev/log"))
 
 
 settings = Settings()