adding docs

Author: HardMax71

.github/workflows/docs.yml

@@ -0,0 +1,145 @@
+name: docs
+
+on:
+  push:
+    branches: [ main , adding-docs ]
+    paths:
+      - 'docs/**'
+      - 'backend/docs/**'
+      - 'files_for_readme/ARCHITECTURE_IN_DETAILS.md'
+      - 'backend/tests/load/README.md'
+      - 'mkdocs.yml'
+      - '.github/workflows/docs.yml'
+  pull_request:
+    branches:
+      - main
+    paths:
+      - 'docs/**'
+      - 'backend/docs/**'
+      - 'files_for_readme/ARCHITECTURE_IN_DETAILS.md'
+      - 'backend/tests/load/README.md'
+      - 'mkdocs.yml'
+      - '.github/workflows/docs.yml'
+  workflow_dispatch:  # Allow manual trigger
+
+permissions:
+  contents: write
+  pages: write
+  id-token: write
+
+# Allow only one concurrent deployment
+concurrency:
+  group: pages
+  cancel-in-progress: false
+
+jobs:
+  build:
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v4
+        with:
+          fetch-depth: 0  # Full history for git info
+
+      - name: Set up Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: '3.12'
+
+      - name: Cache pip dependencies
+        uses: actions/cache@v4
+        with:
+          key: mkdocs-${{ hashFiles('mkdocs.yml') }}
+          path: ~/.cache/pip
+          restore-keys: |
+            mkdocs-
+
+      - name: Install MkDocs and dependencies
+        run: |
+          pip install --upgrade pip
+          pip install \
+            mkdocs-material \
+            mkdocs-mermaid2-plugin \
+            pymdown-extensions
+
+      - name: Prepare documentation structure
+        run: |
+          # Create directory structure for docs
+          mkdir -p docs/architecture
+          mkdir -p docs/reference
+          mkdir -p docs/components/sse
+          mkdir -p docs/components/workers
+          mkdir -p docs/operations
+          mkdir -p docs/security
+          mkdir -p docs/testing
+          mkdir -p docs/stylesheets
+          mkdir -p docs/assets/images
+
+          # Copy images and assets
+          cp files_for_readme/*.png docs/architecture/
+          cp files_for_readme/*.svg docs/architecture/ 2>/dev/null || true
+          cp files_for_readme/logo.png docs/assets/images/
+
+          # Copy architecture docs
+          cp files_for_readme/ARCHITECTURE_IN_DETAILS.md docs/architecture/overview.md
+          cp backend/docs/services-overview.md docs/architecture/
+          cp backend/docs/kafka-topic-architecture.md docs/architecture/
+          cp backend/docs/lifecycle.md docs/architecture/
+
+          # Copy API reference
+          cp backend/docs/api-reference.md docs/reference/
+
+          # Copy component docs - services overview
+          cp backend/docs/services-overview.md docs/components/
+
+          # Copy SSE docs
+          cp backend/docs/sse-partitioned-architecture.md docs/components/sse/
+          cp backend/docs/execution-sse-flow.md docs/components/sse/
+          cp backend/docs/workers/sse-architecture.md docs/components/sse/
+
+          # Copy worker docs
+          cp backend/docs/workers/pod_monitor.md docs/components/workers/
+          cp backend/docs/workers/result_processor.md docs/components/workers/
+
+          # Copy other component docs
+          cp backend/docs/dead-letter-queue.md docs/components/
+          cp backend/docs/schema-manager.md docs/components/
+
+          # Copy operations docs
+          cp backend/docs/tracing.md docs/operations/
+          cp backend/docs/metrics-contextvars.md docs/operations/
+          cp backend/docs/cpu-time-measurement.md docs/operations/
+          cp backend/docs/notification-types.md docs/operations/
+          cp backend/docs/troubleshooting-result-processor-di-crash.md docs/operations/
+
+          # Copy security docs
+          cp backend/docs/security/policies.md docs/security/
+
+          # Copy testing docs
+          cp backend/tests/load/README.md docs/testing/load-testing.md
+
+          # Create minimal extra.css if not exists
+          if [ ! -f docs/stylesheets/extra.css ]; then
+            echo "/* Custom styles for Integr8sCode docs */" > docs/stylesheets/extra.css
+          fi
+
+      - name: Build documentation
+        run: mkdocs build --strict
+
+      - name: Upload artifact
+        uses: actions/upload-pages-artifact@v3
+        with:
+          path: site/
+
+  deploy:
+    # Only deploy on push to main (not PRs)
+    if: github.event_name == 'push' && github.ref == 'refs/heads/main'
+    needs: build
+    runs-on: ubuntu-latest
+    environment:
+      name: github-pages
+      url: ${{ steps.deployment.outputs.page_url }}
+    steps:
+      - name: Deploy to GitHub Pages
+        id: deployment
+        uses: actions/deploy-pages@v4

.gitignore

@@ -0,0 +1,92 @@
+# MkDocs documentation build output
+site/
+
+# Python
+__pycache__/
+*.py[cod]
+*$py.class
+*.so
+.Python
+build/
+develop-eggs/
+dist/
+downloads/
+eggs/
+.eggs/
+lib/
+lib64/
+parts/
+sdist/
+var/
+wheels/
+*.egg-info/
+.installed.cfg
+*.egg
+
+# Virtual environments
+venv/
+ENV/
+env/
+.venv/
+
+# IDE
+.idea/
+.vscode/
+*.swp
+*.swo
+*~
+.project
+.pydevproject
+.settings/
+
+# Testing
+.coverage
+.pytest_cache/
+htmlcov/
+.tox/
+.nox/
+coverage.xml
+*.cover
+.hypothesis/
+
+# Linting/Type checking
+.mypy_cache/
+.ruff_cache/
+.dmypy.json
+dmypy.json
+
+# Jupyter
+.ipynb_checkpoints/
+
+# Environment files
+.env
+.env.local
+.env.*.local
+*.local.env
+
+# Certificates (generated locally)
+certs/
+*.pem
+*.key
+*.crt
+*.csr
+
+# Kubernetes
+kubeconfig.yaml
+kubeconfig.yml
+
+# Docker
+*.log
+
+# OS files
+.DS_Store
+Thumbs.db
+
+# Node (frontend)
+node_modules/
+npm-debug.log*
+yarn-debug.log*
+yarn-error.log*
+
+# Build outputs
+frontend/public/build/

.idea/vcs.xml

@@ -0,0 +1,144 @@
+# Integr8sCode
+
+**Run Python scripts online in isolated Kubernetes pods with resource limits and real-time feedback.**
+
+Integr8sCode is a platform where you can run Python scripts online with ease. Just paste your script, and the platform runs it in an isolated environment within its own Kubernetes pod, complete with resource limits to keep things safe and efficient. You'll get the results back in no time.
+
+## Quick Start
+
+### Deployment
+
+1. Clone the repository
+2. Ensure Docker is enabled, Kubernetes is running, and kubectl is installed
+3. Run `docker-compose up --build`
+
+**Access points:**
+
+| Service | URL |
+|---------|-----|
+| Frontend | `https://127.0.0.1:5001/` |
+| Backend API | `https://127.0.0.1:443/` |
+| Grafana | `http://127.0.0.1:3000` (admin/admin123) |
+
+### Verify Installation
+
+Check if the backend is running:
+
+```bash
+curl -k https://127.0.0.1/api/v1/k8s-limits
+```
+
+This should return JSON with current resource limits.
+
+### Enable Kubernetes Metrics
+
+If CPU and Memory metrics show as `null`, enable the metrics server:
+
+```bash
+kubectl create -f https://raw.githubusercontent.com/pythianarora/total-practice/master/sample-kubernetes-code/metrics-server.yaml
+```
+
+Verify with:
+
+```bash
+kubectl top node
+```
+
+## Core Features
+
+- **Isolated Execution**: Every script runs in its own Kubernetes pod
+- **Resource Limits**: CPU (1000m), Memory (128Mi), configurable timeouts
+- **Multi-version Python**: Support for Python 3.10+ and earlier versions
+- **Real-time Updates**: Server-Sent Events for live execution status
+- **Event Sourcing**: Complete audit trail via Kafka event streams
+- **Dead Letter Queue**: Automatic retry and recovery for failed events
+
+## Architecture Overview
+
+The platform is built on three main pillars:
+
+- **Frontend**: Svelte application for user interaction
+- **Backend**: FastAPI with MongoDB, Kafka, and Redis
+- **Kubernetes**: Isolated pod execution with Cilium network policies
+
+```
+User → Frontend → API → Coordinator → Saga → K8s Worker → Pod
+                                                  ↓
+                              Pod Monitor → Result Processor → SSE → User
+```
+
+For detailed architecture diagrams, see the [Architecture](architecture/overview.md) section.
+
+## Security
+
+- **Network Isolation**: Pods cannot make external network calls (Cilium deny-all policy)
+- **No Privileged Access**: Pods run as non-root with dropped capabilities
+- **Read-only Filesystem**: Containers use read-only root filesystem
+- **No Service Account**: Pods have no access to Kubernetes API
+
+## Documentation Sections
+
+<div class="grid cards" markdown>
+
+-   :material-architecture-outline: **[Architecture](architecture/overview.md)**
+
+    ---
+
+    System design, service interactions, and event flows
+
+-   :material-api: **[API Reference](reference/api-reference.md)**
+
+    ---
+
+    Complete REST and SSE endpoint documentation
+
+-   :material-cog: **[Components](components/services-overview.md)**
+
+    ---
+
+    SSE, Workers, DLQ, and Schema management
+
+-   :material-wrench: **[Operations](operations/tracing.md)**
+
+    ---
+
+    Tracing, metrics, monitoring, and troubleshooting
+
+</div>
+
+## Sample Test
+
+Verify your installation by running this Python 3.10+ code:
+
+```python
+from typing import TypeGuard
+
+def is_string(value: object) -> TypeGuard[str]:
+    return isinstance(value, str)
+
+def example_function(data: object):
+    match data:
+        case int() if data > 10:
+            print("An integer greater than 10")
+        case str() if is_string(data):
+            print(f"A string: {data}")
+        case _:
+            print("Something else")
+
+example_function(15)
+example_function("hello")
+example_function([1, 2, 3])
+```
+
+Expected output:
+
+```
+An integer greater than 10
+A string: hello
+Something else
+```
+
+## Links
+
+- [GitHub Repository](https://github.com/HardMax71/Integr8sCode)
+- [Live Demo](https://app.integr8scode.cc/)