docs: update README.md with AI-generated documentation

Author: WomB0ComB0

README.md

@@ -1,29 +1,7 @@
-<!--
-  Copyright (c) 2025 Foia Stream
-
-  Permission is hereby granted, free of charge, to any person obtaining a copy
-  of this software and associated documentation files (the "Software"), to deal
-  in the Software without restriction, including without limitation the rights
-  to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
-  copies of the Software, and to permit persons to whom the Software is
-  furnished to do so, subject to the following conditions:
-
-  The above copyright notice and this permission notice shall be included in all
-  copies or substantial portions of the Software.
-
-  THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
-  IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
-  FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
-  AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
-  LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
-  OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
-  SOFTWARE.
--->
-
 <!--
   Generated by AI-Powered README Generator
   Repository: https://github.com/WomB0ComB0/foia-stream
-  Generated: 2025-12-25T02:23:02.364Z
+  Generated: 2025-12-26T03:02:48.379Z
   Format: md
   Style: comprehensive
 -->
@@ -139,7 +117,27 @@ graph LR
     Services --> Storage
 ```
 
-### Technology Stack breakdown
+### Core Request Flow
+
+```mermaid
+sequenceDiagram
+    participant User
+    participant Frontend
+    participant API
+    participant DB
+    participant Mailer
+
+    User->>Frontend: Submit FOIA Request
+    Frontend->>API: POST /api/requests
+    API->>API: Validate Schema (Zod)
+    API->>DB: Persist Request (Status: Pending)
+    API->>Mailer: Queue Submission Email to Agency
+    Mailer-->>API: Email Sent Confirmation
+    API-->>Frontend: 201 Created
+    Frontend-->>User: Show Tracking Dashboard
+```
+
+### Technology Stack
 
 | Component | Technology | Description |
 | :--- | :--- | :--- |
@@ -173,8 +171,8 @@ graph LR
 2. **Environment Configuration:**
    Copy the example environment file and fill in your secrets.
    ```bash
-   cp .env.example .env
-   # Also check apps/api/.env and apps/astro/.env
+   cp apps/api/.env.example apps/api/.env
+   # Update variables in apps/api/.env and apps/astro/.env
    ```
 
 3. **Database Migration:**
@@ -194,58 +192,44 @@ graph LR
 
 | Command | Action |
 | :--- | :--- |
-| `bun run dev` | Starts both API and Astro frontend in parallel. |
+| `bun run dev` | Starts both API and Astro frontend in parallel via Turbo. |
 | `bun run build` | Compiles the entire monorepo for production. |
 | `bun run test` | Executes the Vitest suite across all packages. |
-| `bun run lint` | Runs Biome for formatting and linting. |
+| `bun run lint` | Runs Biome for formatting and linting check. |
 
 [Back to top ↑](#-table-of-contents)
 
 ---
 
 ## 📖 Usage & Workflows
 
-### The Request Lifecycle
-
-```mermaid
-sequenceDiagram
-    participant User
-    participant App as FOIA Stream
-    participant Agency as Government Agency
-
-    User->>App: Select Template (e.g. Body Cam)
-    User->>App: Select Agency (e.g. LAPD)
-    App->>App: Generate Legal Language
-    User->>App: Click "Submit Request"
-    App->>Agency: Dispatch Email/API Request
-    App->>App: Initialize Deadline Tracker (20 Days)
-    Note over App: Monitoring...
-    Agency-->>App: Responsive Records Provided
-    App-->>User: Notification: "Records Ready for Review"
+### 1. Creating a Request
+1. Navigate to the **Dashboard**.
+2. Click **New Request**.
+3. Search for an **Agency** (e.g., "Chicago PD").
+4. Select a **Template** (e.g., "Standard Records Request").
+5. Review the auto-generated legal text and click **Submit**.
+
+### 2. Managing Documents
+The Document Viewer supports inline PDF viewing and redaction verification.
+```typescript
+// Example: Fetching request details via the API
+const response = await fetch('/api/requests/req_12345', {
+  headers: { 'Authorization': `Bearer ${token}` }
+});
+const data = await response.json();
+console.log(data.status); // "Fulfillment_In_Progress"
 ```
 
-### Common Commands (API)
-The API is fully documented. Here are the primary interaction patterns:
-
+### 3. Redaction Workflow
 <details>
-<summary><b>View API Examples (cURL)</b></summary>
-
-**Create a Request:**
-```bash
-curl -X POST http://localhost:3000/api/v1/requests \
-  -H "Authorization: Bearer $TOKEN" \
-  -H "Content-Type: application/json" \
-  -d '{
-    "agencyId": "uuid-123",
-    "templateId": "body-cam-standard",
-    "description": "Footage from 12/25 incident at Main St."
-  }'
-```
+<summary><b>View Redaction Process</b></summary>
 
-**Check Agency Stats:**
-```bash
-curl http://localhost:3000/api/v1/agencies/stats
-```
+The system includes a `pdf-redaction.service.ts` that uses `pdf-lib` to overlay black boxes on sensitive coordinates before public disclosure.
+1. Upload Document.
+2. Select text/areas for redaction.
+3. Service generates a "Sanitized" version.
+4. Audit log records who performed the redaction and why.
 </details>
 
 [Back to top ↑](#-table-of-contents)
@@ -254,37 +238,37 @@ curl http://localhost:3000/api/v1/agencies/stats
 
 ## 🔐 Security & Compliance
 
-This repository includes a dedicated `/compliance` directory containing rigorous documentation for production environments.
+FOIA Stream is designed with strict data handling policies, as detailed in our `/compliance` directory.
 
-- **🔍 Audit Logging:** Every database mutation is tracked with a user ID and timestamp.
-- **🛡️ Rate Limiting:** Built-in middleware protects the API from brute-force and DDoS attempts.
-- **📄 Policies:** See `compliance/privacy/data-retention-policy.md` for details on how we handle PII.
-- **☣️ Security Monitoring:** Integrated with GitHub Actions and ZAP rules for vulnerability scanning.
+- **PII Protection:** All Personally Identifiable Information is encrypted at rest using AES-256-GCM.
+- **Rate Limiting:** Built-in Hono middleware prevents scraping or brute-force attacks on agency endpoints.
+- **Audit Logging:** Every interaction with a record is timestamped and attributed to a specific user ID.
+- **Compliance Documentation:** 
+  - [Data Retention Policy](./compliance/privacy/data-retention-policy.md)
+  - [Privacy Impact Assessment](./compliance/privacy/privacy-impact-assessment.md)
+  - [Incident Response Runbook](./compliance/runbooks/incident-response.md)
 
-| Feature | Implementation |
-| :--- | :--- |
-| **Auth** | JWT with short-lived tokens + Refresh tokens |
-| **Encryption** | AES-256-GCM for sensitive fields |
-| **MFA** | TOTP (Google Authenticator / Authy compatible) |
+⚠️ **Warning:** Ensure you rotate the `ENCRYPTION_SECRET` in your `.env` before deploying to a production environment.
 
 [Back to top ↑](#-table-of-contents)
 
 ---
 
 ## 🛠️ Development & Contributing
 
-We welcome contributions from developers, legal experts, and transparency advocates.
+We welcome contributions from developers, legal experts, and UI designers.
+
+### Project Structure
+- `apps/api`: Hono backend with Drizzle/SQLite.
+- `apps/astro`: Astro frontend with React islands.
+- `packages/shared`: Shared Zod schemas and utility functions.
+- `compliance/`: Legal frameworks and security policies.
 
 ### Contribution Rules
 1. **Branching:** Use `feature/` or `fix/` prefixes.
-2. **Code Quality:** All PRs must pass `bun run lint` (Biome) and `bun run test`.
-3. **Type Safety:** No `any` types. Use Zod schemas for all data boundaries.
-
-### Project Structure
-- `apps/api`: Hono backend, SQLite database, and business services.
-- `apps/astro`: Frontend UI components and pages.
-- `packages/shared`: Shared TypeScript types, Zod schemas, and utility functions.
-- `compliance/`: Policy documents and control catalogs.
+2. **Linting:** Run `bun run lint:fix` before committing.
+3. **Testing:** Ensure `bun run test` passes for all changed packages.
+4. **Documentation:** Update relevant `.md` files if adding new features.
 
 [Back to top ↑](#-table-of-contents)
 
@@ -293,14 +277,14 @@ We welcome contributions from developers, legal experts, and transparency advoca
 ## 🗺️ Roadmap & Limitations
 
 ### Current Limitations
-- ⚠️ **Jurisdictional Variance:** Currently optimized for US-based FOIA/Public Records laws.
-- ⚠️ **Manual Verification:** Some agency email addresses require manual verification before high-volume use.
+- ⚠️ **Email Integration:** Currently requires a manual SMTP setup for automated agency dispatching.
+- ⚠️ **File Size:** Uploads are limited to 50MB per document in the default configuration.
 
-### Planned Enhancements
-- [ ] **AI-Aided Redaction:** Auto-detect common redaction patterns to help users appeal denials.
-- [ ] **Direct Portal Integrations:** API-based submission for common portals like GovQA.
-- [ ] **Mobile App:** React Native application for recording incidents and filing on-the-spot.
-- [ ] **PDF Scraping:** Automated extraction of data from non-searchable government PDFs.
+### Future Enhancements
+- [ ] **AI-Powered Extraction:** Automatically extract entities (dates, names, dollar amounts) from disclosed PDFs.
+- [ ] **State-Specific Statutes:** Integration of all 50 US state FOIA response laws.
+- [ ] **Mobile App:** React Native companion app for on-the-go request tracking.
+- [ ] **Public API:** GraphQL interface for third-party transparency tools.
 
 [Back to top ↑](#-table-of-contents)
 
@@ -310,31 +294,26 @@ We welcome contributions from developers, legal experts, and transparency advoca
 
 Distributed under the **MIT License**. See `LICENSE` for more information.
 
-**Project Maintainer:** [WomB0ComB0](https://github.com/WomB0ComB0)
-**Security Issues:** Please report any security vulnerabilities directly to the maintainer via GitHub issues (marked private if possible) or via the contact info in `SECURITY.md`.
+**Project Lead:** [WomB0ComB0](https://github.com/WomB0ComB0)  
+**Security Contact:** security@foia-stream.org  
 
 ---
 
 ## ❓ FAQ & Troubleshooting
 
-### Troubleshooting
-
-<details>
-<summary><b>Bun install is failing</b></summary>
-Ensure you are using Bun version 1.0 or higher. Run `bun upgrade` to get the latest version.
-</details>
-
-<details>
-<summary><b>Database migration errors</b></summary>
-If the schema gets out of sync during development, you can reset the local database by deleting `apps/api/data/foia-stream.db` and re-running `bun run db:push`.
-</details>
-
 ### FAQ
+**Q: Does this work with federal FOIA.gov?**  
+A: Yes, FOIA Stream can be used to track federal requests, though direct API integration with FOIA.gov is planned for v1.2.
 
-**Q: Does this actually send the emails to agencies?**
-A: In development mode, emails are logged to the console. In production, it integrates with SendGrid or AWS SES to dispatch legal requests.
+**Q: Can I use PostgreSQL instead of SQLite?**  
+A: While Drizzle supports it, the current schema is optimized for SQLite. You would need to update `drizzle.config.ts` and the driver in `db/index.ts`.
 
-**Q: Can I use this for state-level requests?**
-A: Yes! FOIA Stream includes templates and logic for federal FOIA, state-level (e.g., CPRA in CA, OPRA in NJ), and local ordinances.
+### Troubleshooting
+| Issue | Solution |
+| :--- | :--- |
+| **Drizzle Migration Fails** | Delete the `./data/foia-stream.db` and re-run `db:push`. |
+| **Hydration Errors** | Ensure React components in Astro are marked with `client:load` or `client:visible`. |
+| **Bun Type Errors** | Run `bun add -d @types/bun` to sync global types. |
 
-[Back to top ↑](#-table-of-contents)
+---
+*Built with ❤️ for the Open Government community.*