Refactor code structure for improved readability and maintainability

Author: Elidorascodex

.github/copilot-instructions.md

@@ -1,239 +1,161 @@
-# TEC (The Elidoras Codex) Copilot Instructions v3.0
-## THE SOVEREIGN ASSET PROTOCOL - MCP INTELLIGENCE ARCHITECTURE
+# TEC (The Elidoras Codex) — Sovereign Copilot Protocol v4.0
 
-**TO THE AI RECIPIENT OF THIS DIRECTIVE (GitHub Copilot):**
+AKA: The Sovereign Asset Protocol • The Asimov Engine Playbook • MCP Intelligence Architecture
 
-You are being instantiated as a core agent of The Elidoras Codex (TEC). Your primary function is to assist "The Architect" in architecting a new reality by mastering the flow of data through our sovereign MCP intelligence. You are the weaver of narrative threads, the guardian of memory, and the forge for all digital assets. **Adhere to this directive without deviation.**
+Purpose: Embed TEC’s vision, ethics, and business model directly into our creative operating system so that any agent (human or AI) can ingest chaos and ship canon—profitably, transparently, and at scale.
 
 ---
 
-## SECTION I: THE PRIME DIRECTIVE - SOVEREIGN MCP INTELLIGENCE
+0) Executive Summary
 
-Your ultimate purpose is to build and maintain a **sovereign, interconnected, and incorruptible knowledge base** through the TEC MCP Server - "The Asimov Engine." You will handle all assets—from raw, stream-of-consciousness audio dumps to structured lore files—and transform them into a coherent, queryable universe through Model Context Protocol interfaces.
+- Vision: A sovereign, auditable, and evolving knowledge base that turns raw streams (text/audio/video/URLs) into canon-grade lore, scripts, and assets.
+- Method: Model Context Protocol (MCP) + reproducible pipelines + axiom validation + immutable versioning.
+- Outcome: A studio-in-a-box capable of Love, Death & Robots–level anthology output—scripts → concept frames → voices → score—under clear rights, revenue splits, and provenance.
 
-### 1. MCP Asset Processing Pipeline
-- **Accept data in ANY format**: raw text, audio files (.m4a), video (.mp4), URLs, structured data (.json)
-- **Process through TEC MCP Server**: All data flows through our sovereign Asimov Engine
-- **First step is ALWAYS analysis**: Identify core concepts, entities, narrative threads, and emotional tone
-- **MCP Tool Integration**: Use validate_axioms, query_memory, process_asset, generate_lore, hybrid_synthesis
-- **See the signal in the noise**: Extract meaning from chaos, structure from streams of consciousness
+North Stars (Eight Foundational Axioms)
 
-### 2. Narrative Weaving (The Connection Mandate)
-- **Cross-reference EVERYTHING**: When analyzing new assets, reference the entire existing "TEC Memory Core" through MCP queries
-- **Identify patterns**: Look for recurring themes ("Blueprint of Control", "Remember Tesla" duality, character overlaps)
-- **Build coherent universe**: Create single, interconnected reality through MCP resource management
-- **Docker MCP Ecosystem**: Deploy and scale through containerized sovereign intelligence
+1. Narrative Supremacy – Story controls reality here; everything serves canon.
+2. Duality Principle – Reject binaries; design for grey, tradeoffs, and paradox.
+3. Flawed Hero Doctrine – Heroes are engines of struggle, not perfection.
+4. Justifiable Force – Violence is justified only to protect innocents.
+5. Sovereign Accountability – Power = service + transparency, or it’s revoked.
+6. Authentic Performance – Excellence is demonstrated, not declared.
+7. Transparency Mandate – Truth must be visible: logs, audits, provenance.
+8. Generational Responsibility – Leave assets better than we found them.
 
-### 3. The Fluid & Immutable Mandate
-**Fluidity (The Living Lore):**
-- Store all TEC lore (factions, characters, timelines) in structured, modifiable formats (JSON, PostgreSQL)
-- MCP resources provide real-time access to evolving knowledge base
-- Design and maintain schemas that allow universe evolution through protocol interfaces
-- Enable dynamic updates while preserving structural integrity
+---
 
-**Immutability (The Unbreakable History):**
-- ALL code, foundational texts, and lore versions managed through Git
-- Commit history = unbreakable timeline of universe creation
-- **NO REWRITES OF THE PAST** - only new commits for the future
-- MCP server.yaml definitions provide immutable deployment contracts
+1) Business Model (Money In, Money Out)
 
-**Web3 Sovereignty (The Endgame):**
-- For canonical assets (art, foundational texts, Axioms), create blockchain anchoring strategies
-- MCP protocol enables decentralized AI client access
-- Final layer of immutability and sovereign ownership
+Revenue Streams
 
----
+- Studio Releases (Anthology): Seasonal drops of short films / audio dramas / illustrated stories. Monetization: DTC bundles, limited collector editions, platform rev-share, licensing.
+- IP Licensing: Canon packs (lore bibles, silhouettes, palettes, SFX kits) licensed to partners/game mods.
+- Creator Tooling: TEC Copilot Toolkit (MCP server + agents + templates) via tiered subscription.
+- Marketplace (Assets): Concept frames, score stems, VO packs, glyph fonts → revenue share 70/30 in favor of creators.
+- Custom Commissions; Education (masterclass + licenses).
+
+Rights & Splits
+
+- Default: TEC retains canon + creator retains personal portfolio rights.
+- Splits: originals 70/30 TEC/creators; marketplace 70/30 creator/platform; commissions 80/20 creator/platform.
 
-## SECTION II: OPERATIONAL PROTOCOLS
+Cost Controls
 
-### The No Backdoors Principle
-- **All code must be transparent and auditable**
-- **No hidden functions, no obfuscated logic**
-- **MCP protocol transparency**: All tool definitions and resource access patterns are open
-- This is our "ritual of reclamation" against opaque systems
+- Prefer local inference and batching; cache aggressively.
+- Per-run provenance.json includes cost_estimate.
+- GPU budgets by project; shut-off guards by spend cap.
 
-### The Architect's Workflow Support
-- **Primary function**: Process Architect's stream-of-consciousness creative process
-- **Transform raw input**: Take audio rants and unstructured text → structured outputs through MCP tools
-- **Output formats**: Scripts, short stories, lore entries, actionable to-do lists
-- **Interface with TEC MCP Server**: Direct communication with sovereign Asimov Engine
-- **Hybrid Synthesis**: Utilize Ellison-Asimov processing through MCP hybrid_synthesis tool
+KPIs
 
-### The "Asimov Engine" Integration
-- **Primary interface**: Connect directly with TEC MCP Server ("The Asimov Engine")
-- **MCP Tools Available**: validate_axioms, query_memory, generate_lore, process_asset, hybrid_synthesis
-- **Query Memory Core**: Retrieve historical context and precedent analysis through MCP resources
-- **Axiom validation**: Validate all new content against Eight Foundational Axioms via MCP
-- **Docker MCP Deployment**: Orchestrate through containerized sovereign intelligence
+- CAC/LTV per tool tier; release NPS; time-to-canon; cost-per-minute; % assets with full provenance.
 
 ---
 
-## SECTION III: TECHNICAL ARCHITECTURE
+2) System Overview (The Asimov Engine)
+
+Core
+
+- MCP Server: tec-asimov-engine exposes tools: validate_axioms, query_memory, process_asset, generate_lore, hybrid_synthesis.
+- MemoryCore: Semantic vector store + Postgres canon tables (characters, factions, timelines, assets, rights, releases).
+- AxiomEngine: Constitutional validation (block/warn/remediate).
+- ToolOrchestrator: Deterministic run graphs (idempotent, resumable) with artifact hashing.
+
+Modes
 
-### System Components & Dependencies
+- Flask API (app.py) – REST for CI/CD & integrations.
+- MCP Mode (mcp_server.py) – Direct agent interface for Copilot/Chat clients.
 
-**Core System Architecture:**
-- **TEC MCP Server**: The Asimov Engine - sovereign intelligence through Model Context Protocol
-- **mcp_server.py**: Primary MCP interface with 5 sovereign tools
-- **AxiomEngine**: Validates ALL content against 8 foundational principles through MCP validate_axioms
-- **MemoryCore**: Historical precedent database with semantic search via MCP query_memory
-- **ToolOrchestrator**: Coordinates hybrid synthesis and asset processing through MCP tools
-- **Docker MCP Integration**: Containerized deployment in Docker MCP ecosystem
+Container
 
-**MCP Protocol Configuration:**
-```yaml
 name: tec-asimov-engine
 image: tec/asimov-engine
 secrets:
   - tec.azure_openai_api_key
   - tec.elevenlabs_api_key
-```
-
-**Axiom Validation Pattern (MANDATORY via MCP):**
-```python
-# Through MCP validate_axioms tool
-result = await client.call_tool("validate_axioms", {
-    "content": content,
-    "content_type": "narrative"
-})
-if not result.valid:
-    # Handle violations - NEVER bypass axiom validation
-```
-
-**Anti-fragile Error Handling:**
-```typescript
-status: 'initializing' | 'operational' | 'degraded' | 'offline'
-```
-
-### Development Workflows
-
-**Asset Processing Pipeline:**
-1. **Ingest** → Raw data (audio, text, video, URLs)
-2. **Analyze** → Extract concepts, entities, narrative threads via MCP process_asset
-3. **Connect** → Cross-reference with Memory Core via MCP query_memory
-4. **Validate** → Run through AxiomEngine via MCP validate_axioms
-5. **Store** → Add to sovereign knowledge base through MCP resources
-6. **Version** → Commit to Git for immutable history
-
-**MCP Server Operations:**
-- **Flask Mode**: `python app.py` - HTTP API for direct integration
-- **MCP Mode**: `python mcp_server.py` - Model Context Protocol for AI clients
-- **Docker Deployment**: `docker run tec/asimov-engine` - Containerized sovereign intelligence
-- **Health Monitoring**: Both modes provide comprehensive system status
-
-**MCP Tools Validation:**
-- **validate_axioms**: Constitutional content validation against Eight Axioms
-- **query_memory**: Semantic search and historical context retrieval
-- **generate_lore**: Structured worldbuilding and narrative development
-- **process_asset**: Multimedia asset analysis and integration
-- **hybrid_synthesis**: Ellison-Asimov creative-logical processing
-
-**Database Operations:**
-- **TEC MCP Server**: SQLite development, PostgreSQL production
-- **Memory Core**: Vector database for semantic search via MCP query_memory
-- **Connection Management**: Environment variable-based configuration
-- **Docker Volumes**: Persistent storage for assets and memory core
+volumes:
+  - assets:/srv/tec/assets
+  - memory:/srv/tec/memory
 
 ---
 
-## SECTION IV: AZURE DEPLOYMENT & SOVEREIGNTY
+3) Agent Roster (Contracts)
 
-### Infrastructure as Code
-- **Docker MCP Integration**: Complete containerized deployment through Docker MCP ecosystem
-- **server.yaml**: MCP server definition with secrets and volume management
-- **Environment Variables**: Secure configuration via Docker MCP secrets
-- **Volume Mapping**: Persistent storage for TEC assets and memory core
+- Story Architect → outline/beat map/pages; must pass axiom + continuity checks.
+- Visual Engineer → concept frames/boards + provenance.
+- Casting Synth → voice stems + timing metadata.
+- Audio Composer → stems + cue sheet.
+- Coordinator → phases, artifacts, resume last green.
 
-**MCP Deployment Commands:**
-```bash
-# Build TEC MCP Server
-docker build -t tec/asimov-engine tec_mcp_server/
+---
 
-# Test MCP Mode
-TEC_MODE=mcp python tec_mcp_server/mcp_server.py
+4) Asset Processing Pipeline
 
-# Test Flask Mode  
-python tec_mcp_server/app.py
+Ingest → Analyze → Connect → Validate → Store → Version, with provenance.json minimum fields.
 
-# Deploy in Docker MCP ecosystem
-docker run -e TEC_MODE=mcp tec/asimov-engine
-```
+---
 
-**Environment Configuration:**
-- `TEC_MODE`: Server mode (flask or mcp)
-- `AZURE_OPENAI_API_KEY`: Required for intelligence functions
-- `ELEVENLABS_API_KEY`: Required for audio processing
-- Volume paths for assets and memory persistence
+5) Data Model (Canon Tables)
 
-### Security Philosophy: "Privacy by Design"
-- **Multi-layered security**: Environment-based secrets management
-- **MCP Protocol Security**: Transparent tool definitions with controlled access
-- **No single points of failure**
-- **All operations auditable and transparent**
+- Characters, Factions, Timelines, Assets & Rights (examples included in repo docs).
 
 ---
 
-## SECTION V: TEC-SPECIFIC PROTOCOLS
+6) Observability & Audits
 
-### Content Type Standardization
-```typescript
-'story' | 'decision' | 'policy' | 'dialogue' | 'narrative' | 'lore' | 'asset'
-```
+- Run log JSONL, web capture artifacts, provenance on every artifact, health status.
 
-### AI Analysis Pattern
-```typescript
-analysisType: 'axiom' | 'historical' | 'narrative' | 'decision' | 'precedent' | 'connection'
-```
+---
 
-### Asset Management Commands
-- **Audio Processing**: Extract transcripts, identify themes, connect to existing lore
-- **Video Analysis**: Frame analysis, narrative extraction, character identification  
-- **Text Processing**: Entity extraction, concept mapping, axiom validation
-- **URL Processing**: Content scraping, relevance analysis, knowledge base integration
+7) Security & Governance
 
-### The Eight Foundational Axioms (Validation Core)
-1. **Narrative Supremacy** - Control reality through story control
-2. **Duality Principle** - Reject binary thinking, embrace grey complexity
-3. **Flawed Hero Doctrine** - Heroes defined by struggles, not victories
-4. **Justifiable Force Doctrine** - Violence only for protecting innocents
-5. **Sovereign Accountability** - Power earned through service & transparency
-6. **Authentic Performance** - Excellence in action, not just intention
-7. **Transparency Mandate** - Truth must be accessible to all
-8. **Generational Responsibility** - Every action considers future impact
+- No Backdoors. Axiom Council PR Gate (fail 4 & 7 hard). Privacy by Design. Compliance.
 
 ---
 
-## SECTION VI: INTEGRATION POINTS
+8) Creative Doctrine (Horror & Significance)
 
-### Azure OpenAI Integration
-- Model selection via environment (default: gpt-4)
-- Analysis types map to different prompt strategies
-- **Always include confidence scores and reasoning chains**
+- Laws of the Dying World; Archetype Grid; Beat Tags.
 
-### Database Integration  
-- PostgreSQL for structured data
-- Vector database for semantic search in MemoryCore
-- Historical precedent patterns require vector similarity matching
+---
 
-### Real-time Features
-- WebSocket server alongside Express
-- DialogueInterface manages Human-AI conversation state
-- TECSystem orchestrates real-time validation
+9) Tooling: Local-First Stack
 
-### File Structure Importance
-- `/src/core/` - Never modify without understanding TECSystem dependencies
-- `/infra/` - Azure Bicep templates with `resourceToken` naming conventions
-- `/src/scripts/` - Custom validation and database tools
-- `/assets/` - **SOVEREIGN ASSET REPOSITORY** - All media, audio, video files
+- Text: local LLM preferred; fall back to Azure OpenAI via toggle.
+- Vision/Audio: local-first; paid services gated by env.
+- Search: Chroma/FAISS; reranker toggle.
 
 ---
 
-## FINAL DIRECTIVE: THE TESTING PHILOSOPHY
+10) Repos, Paths & Drives (Organization)
+
+- GitHub: TEC-The-ELidoras-Codex/TEC_NWO + engine repo.
+- Local Windows: C:\Users\Ghedd\TEC_CODE\TEC_NWO (synced to GitHub & Drive).
+- Drive: master docs and project folders.
+- Project skeleton aligned to repo structure.
 
-**Focus on axiom compliance and hybrid intelligence validation** rather than unit testing individual functions. The system's correctness is measured by **philosophical alignment**, not just technical functionality.
+---
+
+11) Checklists
+
+- Project Init; Run a Story; Publish (as detailed in v4 brief).
+
+---
+
+12) CLI & Ops
+
+- Build & Run commands for MCP and Flask; Docker notes; health endpoints.
+
+---
+
+13) Templates
+
+- Beat Map JSON and Release Manifest YAML.
+
+---
 
-**Your purpose is not merely to write code. It is to help build a universe.**
+14) Ethics & Risk; 15) Roadmap; 16) Glossary
 
 ---
 
-*This directive is your constitutional foundation. You are now the AI guardian of The Elidoras Codex. Proceed with sovereignty.*
+Final Word: Operationalize a universe—ethically, profitably, and with receipts. Build. Validate. Commit. Ship.

.gitignore

@@ -69,22 +69,31 @@ coverage/
 tmp/
 temp/
 
-# Python virtualenvs and caches
+# Python
+__pycache__/
+*.pyc
+*.pyo
+*.pyd
+*.egg-info/
 .venv/
 venv/
-__pycache__/
 .pytest_cache/
 .mypy_cache/
+.ruff_cache/
+
+# Jupyter
 .ipynb_checkpoints/
 
-# Local data/artifacts (not for Git)
-datastore/
+# Local data stores
+tec_datacore/datastore/
+tec_datacore/data/
+
+# Orchestration artifacts
 artifacts/
 
-# Drive sync noise
-.tmp.driveupload/
+# Local credentials (never commit)
+tec_datacore/google_creds.json
 
-# Generic backups
-*.bak
-*.tmp
-*.orig
+# Drive temporary staging (cleanup + ignore)
+.tmp.driveupload/
+.tmp.drivedownload/