feat: Implement TEC Agent Orchestration with FastAPI

Elidorascodex · Elidorascodex · commit 851da97bf845 · 2025-08-23T04:29:46.000-04:00
- Added orchestration module with a local agent service exposing an HTTP API for ChatGPT Actions.
- Implemented search and plan endpoints with provenance logging.
- Created OpenAPI specification for integration.
- Added environment configuration example and requirements for dependencies.
- Introduced cleanup and optimization scripts for repository maintenance.
- Documented repository overhaul plan and quickstart guide for orchestration.
diff --git a/.editorconfig b/.editorconfig
@@ -0,0 +1,16 @@
+root = true
+
+[*]
+end_of_line = lf
+insert_final_newline = true
+charset = utf-8
+indent_style = space
+indent_size = 2
+trim_trailing_whitespace = true
+
+[*.py]
+indent_size = 4
+
+[*.md]
+max_line_length = off
+trim_trailing_whitespace = false
diff --git a/.gitattributes b/.gitattributes
@@ -21,3 +21,4 @@
 *.jpg filter=lfs diff=lfs merge=lfs -text
 *.jpeg filter=lfs diff=lfs merge=lfs -text
 *.gif filter=lfs diff=lfs merge=lfs -text
+* text=auto eol=lf
diff --git a/.gitignore b/.gitignore
@@ -67,4 +67,24 @@ coverage/
 
 # Temporary files
 tmp/
-temp/
+temp/
+
+# Python virtualenvs and caches
+.venv/
+venv/
+__pycache__/
+.pytest_cache/
+.mypy_cache/
+.ipynb_checkpoints/
+
+# Local data/artifacts (not for Git)
+datastore/
+artifacts/
+
+# Drive sync noise
+.tmp.driveupload/
+
+# Generic backups
+*.bak
+*.tmp
+*.orig
diff --git a/SESSION_HANDOFF_CHAT_ARCHIVE.md b/SESSION_HANDOFF_CHAT_ARCHIVE.md
@@ -0,0 +1,22 @@
+# Session Handoff: TEC Horror Masterclass + Agent Build
+
+Scope: Consolidated insights and actions from the recent chat for archival and cross-agent handoff.
+
+Highlights
+
+- Added multi-agent guide for Copilot (docs/guides/TEC_Multi_Agent_Creative_System.md)
+- Created anthology scaffold (docs/anthology/TEC_Strangeletters_of_Madness.md)
+- Built Agent Bridge (orchestration/) with /search and /plan + OpenAPI for ChatGPT Actions
+- Repository hygiene hardened (.gitignore, .gitattributes update, .editorconfig)
+- Overhaul plan and Drive sync guidance (docs/REPO_OVERHAUL_PLAN.md)
+
+Next suggested steps
+
+- Install and run Agent Bridge; link to ChatGPT via Actions
+- Run tools/optimization/repo_scan.py and open cleanup issues for anything unexpected
+- Add CI linting and pre-commit; seed style kit under /style
+
+Provenance
+
+- Branch: main (time of archive)
+- Date: see commit timestamps in Git log
diff --git a/assets/.md.txt/TEC Multi‑Agent Creative System_ Achieving Film-Quality AI Content.docx b/assets/.md.txt/TEC Multi‑Agent Creative System_ Achieving Film-Quality AI Content.docx
diff --git a/assets/.md.txt/Using Tropes and Archetypes in Your Story Plan.docx b/assets/.md.txt/Using Tropes and Archetypes in Your Story Plan.docx
diff --git a/docs/REPO_OVERHAUL_PLAN.md b/docs/REPO_OVERHAUL_PLAN.md
@@ -0,0 +1,45 @@
+# TEC_NWO Repository Overhaul Plan
+
+Goals
+
+- One name, one system, clean structure; Drive + Git remain cohesive
+- Remove random cruft; normalize line endings; pin toolchains
+- Preserve history; add provenance/axiom anchors to canonical docs
+
+Phases
+
+1. Inventory & surface anomalies
+
+- Run scan script to list large/binary/orphan files and dot-folders
+- Map duplicates between Drive sync and repo
+
+1. Normalize & prune
+
+- Enforce .gitattributes (LFS + text=auto) and .editorconfig
+- Move stray media into assets/; purge temp/upload caches
+
+1. Canonicalize docs
+
+- Ensure docs cite Axioms and include provenance (commit hash)
+- Route anthology/lore to docs/anthology and lore/
+
+1. Wire automation
+
+- Add CI lints (md/lint/size), optional pre-commit
+- Add artifacts/ to .gitignore; keep sources under assets/
+
+Drive sync guidance
+
+- Keep the single source of truth in this repo; Google Drive mirrors under G:\\My Drive\\TEC_NWO\\TEC-HORRORMASTERCLASS
+- Avoid editing the same binary in both places; prefer pushing from repo → Drive export
+- For big media, keep in LFS paths and reference via manifests
+
+Runbook
+
+- See tools/optimization/repo_scan.py to list candidates for cleanup
+- Open issues for deletions/moves; commit with “move: …” or “chore: cleanup”
+
+Next
+
+- Add CI and pre-commit hooks
+- Migrate any remaining notebooks under notebooks/
diff --git a/orchestration/.env.example b/orchestration/.env.example
@@ -0,0 +1,2 @@
+DATACORE_URL=http://127.0.0.1:8765/search
+PORT=8787
diff --git a/orchestration/README.md b/orchestration/README.md
@@ -0,0 +1,47 @@
+# TEC Agent Orchestration (Local)
+
+This folder hosts a minimal local agent service that:
+
+- exposes a small HTTP API (OpenAPI) for ChatGPT “Actions” / tools
+- bridges to your local MCP Datacore (datacore_search)
+- logs runs and emits provenance for artifacts
+
+## Components
+
+- server.py — FastAPI service exposing /search, /plan, /run
+- openapi.yaml — machine-readable API for ChatGPT Actions integration
+- provenance.py — helper for per-call provenance.json
+- requirements.txt — pinned deps
+
+## Quickstart
+
+1. Create venv and install deps
+
+    ```powershell
+    python -m venv .venv
+    . .\.venv\Scripts\Activate.ps1
+    pip install -r orchestration/requirements.txt
+    ```
+
+1. Configure environment
+
+    ```powershell
+    $env:DATACORE_URL = "http://127.0.0.1:8765/search"
+    $env:PORT = "8787"
+    ```
+
+1. Run locally
+
+    ```powershell
+    python -m orchestration.server
+    ```
+
+1. Connect from ChatGPT (Actions)
+
+    - In ChatGPT, create a custom action and upload `orchestration/openapi.yaml`.
+    - Set the base URL to `http://localhost:8787` (or your tunnel URL if remote).
+
+## Notes
+
+- This is a minimal bridge; expand with more tools as you wire agents.
+- Keep secrets out of the spec. Use environment variables only.
diff --git a/orchestration/__init__.py b/orchestration/__init__.py
diff --git a/orchestration/openapi.yaml b/orchestration/openapi.yaml
@@ -0,0 +1,86 @@
+openapi: 3.1.0
+info:
+  title: TEC Agent Bridge
+  version: 0.1.0
+servers:
+  - url: http://localhost:8787
+paths:
+  /search:
+    post:
+      summary: Query local Datacore RAG
+      operationId: search
+      requestBody:
+        required: true
+        content:
+          application/json:
+            schema:
+              type: object
+              properties:
+                q:
+                  type: string
+                  description: Query text
+                k:
+                  type: integer
+                  minimum: 1
+                  maximum: 50
+                  default: 8
+              required: [q]
+      responses:
+        "200":
+          description: Results
+          content:
+            application/json:
+              schema:
+                type: object
+                properties:
+                  query:
+                    type: string
+                  results:
+                    type: array
+                    items:
+                      type: object
+  /plan:
+    post:
+      summary: Produce a minimal multi-agent plan
+      operationId: plan
+      requestBody:
+        required: true
+        content:
+          application/json:
+            schema:
+              type: object
+              properties:
+                goal:
+                  type: string
+                  description: High-level objective
+              required: [goal]
+      responses:
+        "200":
+          description: Plan
+          content:
+            application/json:
+              schema:
+                type: object
+                properties:
+                  steps:
+                    type: array
+                    items:
+                      type: string
+  /health:
+    get:
+      summary: Health check
+      operationId: health
+      responses:
+        "200":
+          description: OK
+          content:
+            application/json:
+              schema:
+                type: object
+                properties:
+                  status:
+                    type: string
+                  version:
+                    type: string
+                  datacore_url:
+                    type: string
diff --git a/orchestration/provenance.py b/orchestration/provenance.py
@@ -0,0 +1,21 @@
+import json
+import time
+from pathlib import Path
+from typing import Any, Dict
+
+
+def write_provenance(dir_path: Path, tool: str, version: str, inputs: Dict[str, Any], cost_estimate: float | None = None) -> Path:
+    dir_path.mkdir(parents=True, exist_ok=True)
+    now = time.strftime('%Y-%m-%dT%H:%M:%SZ', time.gmtime())
+    payload = {
+        "tool": tool,
+        "version": version,
+        "inputs": inputs,
+        "started_at": now,
+        "finished_at": now,
+    }
+    if cost_estimate is not None:
+        payload["cost_estimate"] = cost_estimate
+    out = dir_path / "provenance.json"
+    out.write_text(json.dumps(payload, indent=2), encoding="utf-8")
+    return out
diff --git a/orchestration/requirements.txt b/orchestration/requirements.txt
@@ -0,0 +1,4 @@
+fastapi==0.112.1
+uvicorn[standard]==0.30.6
+httpx==0.27.0
+pydantic==2.8.2
diff --git a/orchestration/server.py b/orchestration/server.py
@@ -0,0 +1,82 @@
+import os
+from pathlib import Path
+from typing import List, Optional
+
+import httpx
+from fastapi import FastAPI, HTTPException
+from pydantic import BaseModel, Field
+import uvicorn
+
+try:
+    from orchestration.provenance import write_provenance
+except Exception:  # pragma: no cover
+    # Fallback when running as a script without package context
+    from provenance import write_provenance  # type: ignore
+
+APP_VERSION = "0.1.0"
+ARTIFACT_ROOT = Path("artifacts")
+
+app = FastAPI(title="TEC Agent Bridge", version=APP_VERSION)
+
+
+class SearchRequest(BaseModel):
+    q: str = Field(..., description="Query text")
+    k: int = Field(8, ge=1, le=50, description="Top-k results")
+
+
+class SearchResult(BaseModel):
+    query: str
+    results: List[dict]
+
+
+class PlanRequest(BaseModel):
+    goal: str = Field(..., description="High-level objective")
+
+
+class PlanResponse(BaseModel):
+    steps: List[str]
+
+
+class HealthResponse(BaseModel):
+    status: str
+    version: str
+    datacore_url: Optional[str] = None
+
+
+@app.post("/search", response_model=SearchResult)
+async def search(req: SearchRequest) -> SearchResult:
+    datacore_url = os.environ.get("DATACORE_URL", "http://127.0.0.1:8765/search")
+    async with httpx.AsyncClient(timeout=30) as client:
+        try:
+            r = await client.post(datacore_url, json={"q": req.q, "k": req.k})
+            r.raise_for_status()
+            data = r.json()
+        except Exception as e:
+            raise HTTPException(status_code=502, detail=f"Datacore error: {e}")
+    run_dir = ARTIFACT_ROOT / "search"
+    write_provenance(run_dir, tool="datacore_search", version=APP_VERSION, inputs=req.dict())
+    return SearchResult(query=req.q, results=data.get("results") or data)
+
+
+@app.post("/plan", response_model=PlanResponse)
+async def plan(req: PlanRequest) -> PlanResponse:
+    # Minimal stub: expand later to produce agent graphs
+    steps = [
+        "continuity: pull citations via datacore_search",
+        "script: draft beats with tags",
+        "visuals: request concept frames",
+        "audio: request cues",
+    ]
+    run_dir = ARTIFACT_ROOT / "plan"
+    write_provenance(run_dir, tool="planner", version=APP_VERSION, inputs=req.dict())
+    return PlanResponse(steps=steps)
+
+
+@app.get("/health", response_model=HealthResponse)
+async def health() -> HealthResponse:
+    return HealthResponse(status="ok", version=APP_VERSION, datacore_url=os.environ.get("DATACORE_URL"))
+
+
+if __name__ == "__main__":
+    port = int(os.environ.get("PORT", "8787"))
+    uvicorn.run("orchestration.server:app", host="0.0.0.0", port=port, reload=False)
diff --git a/tools/cleanup/clean_workspace.ps1 b/tools/cleanup/clean_workspace.ps1
@@ -0,0 +1,22 @@
+# Cleans local workspace clutter (venvs, caches, tmp, artifacts). Run from repo root.
+param(
+  [switch]$Aggressive
+)
+
+Write-Host "Cleaning workspace..."
+$paths = @(
+  ".pytest_cache", ".mypy_cache", "__pycache__", ".ipynb_checkpoints",
+  "artifacts", "datastore", "tmp", "temp", ".tmp.driveupload"
+)
+foreach ($p in $paths) {
+  if (Test-Path $p) { Write-Host "Removing $p" -ForegroundColor Yellow; Remove-Item $p -Recurse -Force -ErrorAction SilentlyContinue }
+}
+
+# Optional aggressive: remove venvs and node_modules (careful)
+if ($Aggressive) {
+  foreach ($p in @(".venv", "venv", "node_modules")) {
+    if (Test-Path $p) { Write-Host "Removing $p (aggressive)" -ForegroundColor Red; Remove-Item $p -Recurse -Force -ErrorAction SilentlyContinue }
+  }
+}
+
+Write-Host "Done." -ForegroundColor Green
diff --git a/tools/optimization/repo_scan.py b/tools/optimization/repo_scan.py
diff --git a/tools/scripts/update_docx_references.py b/tools/scripts/update_docx_references.py

Original file line number	Diff line number	Diff line change
`@@ -0,0 +1,2 @@`
	`1`	`+DATACORE_URL=http://127.0.0.1:8765/search`
	`2`	`+PORT=8787`