feat: Comprehensive Permission Scheme with Tool-Based Access Control

.github/actions/create-release/action.yml

@@ -1,5 +1,5 @@
 name: 'Create Release'
-description: 'Build Python package, MCPB package, and create GitHub release'
+description: 'Build Python package, MCPB package, Docker image, and create GitHub release'
 inputs:
   package-version:
     description: 'Version from git tag (e.g., 0.5.9-dev-20250904075318)'
@@ -12,6 +12,10 @@ inputs:
     description: 'Skip existing packages during upload'
     required: false
     default: 'false'
+  build-docker:
+    description: 'Build and push Docker image (true/false)'
+    required: false
+    default: 'true'
 
 outputs:
   release-url:
@@ -49,6 +53,18 @@ runs:
       shell: bash
       run: make release-zip
 
+    - name: Build and push Docker image
+      if: ${{ inputs.build-docker == 'true' }}
+      shell: bash
+      env:
+        VERSION: ${{ inputs.package-version }}
+        ECR_REGISTRY: ${{ env.ECR_REGISTRY }}
+        AWS_ACCOUNT_ID: ${{ env.AWS_ACCOUNT_ID }}
+        AWS_DEFAULT_REGION: ${{ env.AWS_DEFAULT_REGION }}
+      run: |
+        echo "🐳 Building and pushing Docker image with scripts/docker.py"
+        uv run python scripts/docker.py push --version "$VERSION"
+
     - name: Create GitHub Release
       id: create-release
       uses: softprops/action-gh-release@v2

.github/workflows/pr.yml

@@ -6,6 +6,7 @@ on:
     branches: ['**']
   push:
     tags: ['v*-dev-*']
+
 jobs:
   test:
     runs-on: ubuntu-latest
@@ -41,6 +42,14 @@ jobs:
         QUILT_TEST_PACKAGE: ${{ secrets.QUILT_TEST_PACKAGE }}
         QUILT_TEST_ENTRY: ${{ secrets.QUILT_TEST_ENTRY }}
 
+    - name: Test Docker build (no push)
+      if: github.event_name == 'pull_request'
+      env:
+        VERSION: test-${{ github.sha }}
+      run: |
+        echo "🐳 Testing Docker build for PR (no push)..."
+        uv run python scripts/docker.py build --version "$VERSION"
+
     - name: Debug GitHub context for dev-release
       run: |
         echo "=== GitHub Context Debug ==="
@@ -75,6 +84,7 @@ jobs:
       with:
         python-version: '3.11'
         include-nodejs: 'true'
+
     - name: Debug dev-release context
       run: |
         echo "=== Dev-Release Context Debug ==="
@@ -94,9 +104,11 @@ jobs:
         TAG_VERSION=${GITHUB_REF#refs/tags/v}
         echo "tag_version=$TAG_VERSION" >> $GITHUB_OUTPUT
         echo "Dev Tag Version: $TAG_VERSION"
+
     - name: Create dev release
       uses: ./.github/actions/create-release
       with:
         package-version: ${{ steps.version.outputs.tag_version }}
         pypi-repository-url: https://test.pypi.org/legacy/
-        skip-existing: true
+        skip-existing: true
+        build-docker: 'false'  # Skip Docker for development releases

.github/workflows/push.yml

@@ -77,8 +77,25 @@ jobs:
         echo "tag_version=$TAG_VERSION" >> $GITHUB_OUTPUT
         echo "Production Tag Version: $TAG_VERSION"
 
+    - name: Configure AWS credentials for ECR
+      uses: aws-actions/configure-aws-credentials@v4
+      with:
+        aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
+        aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
+        aws-region: ${{ secrets.AWS_DEFAULT_REGION || 'us-east-1' }}
+
+    - name: Login to Amazon ECR
+      uses: aws-actions/amazon-ecr-login@v2
+
     - name: Create production release
       uses: ./.github/actions/create-release
       with:
         package-version: ${{ steps.version.outputs.tag_version }}
-        # pypi-repository-url defaults to '' for PyPI
+        build-docker: 'true'
+        # pypi-repository-url defaults to '' for PyPI
+      env:
+        # Docker registry configuration - these secrets are optional
+        # If ECR_REGISTRY is not set, falls back to constructing from AWS_ACCOUNT_ID
+        ECR_REGISTRY: ${{ secrets.ECR_REGISTRY || '' }}
+        AWS_ACCOUNT_ID: ${{ secrets.AWS_ACCOUNT_ID || '' }}
+        AWS_DEFAULT_REGION: ${{ secrets.AWS_DEFAULT_REGION || 'us-east-1' }}

AGENTS.md

@@ -1,3 +1,5 @@
+- Routing bucket tools through JWT helpers requires explicit client propagation. If `check_s3_authorization` returns without an `s3_client`, bubble an error rather than silently falling back to `get_s3_client()`.
+- Package tools only need the JWT claims today; when Quilt services demand credentials, extend `check_package_authorization` to hand back service instances instead of resurrecting legacy `_check_authorization`.
 <!-- markdownlint-disable MD013 -->
 # Development Guidelines for Claude
 
@@ -312,6 +314,12 @@ For this repository's specific commands and permissions, see this CLAUDE.md file
 - `make release` - Create and push release tag
 - `make release-dev` - Create and push development tag
 
+**Docker Operations (make.deploy):**
+
+- `make docker-build` - Build Docker image locally
+- `make docker-push` - Build and push Docker image to ECR (requires VERSION)
+- `make docker-push-dev` - Build and push development Docker image
+
 **Coordination & Utilities:**
 
 - `make help` - Show all available targets organized by category
@@ -412,9 +420,159 @@ The following permissions are granted for this repository:
 - `python-dist` builds local artifacts without credentials. `scripts/release.sh python-publish` (via `make python-publish`) requires either `UV_PUBLISH_TOKEN` or `UV_PUBLISH_USERNAME`/`UV_PUBLISH_PASSWORD`, defaults to TestPyPI (`PYPI_PUBLISH_URL`/`PYPI_REPOSITORY_URL` override), and respects `DIST_DIR`.
 - GitHub Actions builds dist artifacts via `python-dist`, publishes them with `pypa/gh-action-pypi-publish`, then runs `make mcpb`, `make mcpb-validate`, and `make release-zip` for complete packaging. Secrets supply the PyPI/TestPyPI token (`secrets.PYPI_TOKEN`).
 
+### 2025-09-23 Authentication & Role Assumption Architecture
+
+**Key Learnings from Implementing Automatic Role Assumption:**
+
+#### Middleware Architecture for Request Context
+- **Starlette Middleware Pattern**: Use `BaseHTTPMiddleware` for request-scoped operations
+- **Environment Variable Bridge**: Middleware extracts headers and sets environment variables for services to consume
+- **Order Matters**: Add custom middleware before CORS middleware to ensure proper header processing
+- **Error Handling**: Wrap middleware operations in try-catch to prevent request failures
+
+#### Authentication Service Design
+- **Multi-Method Authentication**: Support multiple auth methods with priority-based fallback
+- **Per-Request Role Assumption**: Enable automatic role assumption on each request when headers present
+- **Session Management**: Maintain assumed role sessions with expiration handling
+- **State Isolation**: Each request can trigger role assumption without affecting global state
+
+#### AWS IAM Role Assumption Patterns
+- **STS AssumeRole**: Use `sts:assume_role` with proper session naming and source identity
+- **Trust Policy Requirements**: Target roles must trust the ECS task role ARN
+- **Session Validation**: Always validate assumed roles with `get_caller_identity`
+- **Credential Management**: Store temporary credentials with proper expiration handling
+
+#### Quilt Integration Architecture
+- **Header-Based Communication**: Use `X-Quilt-User-Role` and `X-Quilt-User-Id` headers
+- **Automatic Role Switching**: No user intervention required - roles switch automatically
+- **Per-User Isolation**: Each user's MCP operations use their specific AWS role
+- **Fallback Mechanisms**: Support both header-based (production) and credential-based (development) auth
+
+#### Implementation Patterns
+```python
+# Middleware extracts headers and sets environment variables
+class QuiltRoleMiddleware(BaseHTTPMiddleware):
+    async def dispatch(self, request, call_next):
+        role_arn = request.headers.get("x-quilt-user-role")
+        if role_arn:
+            os.environ["QUILT_USER_ROLE_ARN"] = role_arn
+            auth_service.auto_attempt_role_assumption()
+        return await call_next(request)
+
+# Authentication service automatically assumes roles
+def auto_attempt_role_assumption(self) -> bool:
+    role_arn = os.environ.get("QUILT_USER_ROLE_ARN")
+    if role_arn and self._assumed_role_arn != role_arn:
+        return self.assume_quilt_user_role(role_arn)
+    return True
+```
+
+#### Key Architecture Decisions
+1. **Automatic vs Manual**: Role assumption is automatic when headers present, manual tools provided for override
+2. **Request-Scoped**: Each HTTP request can trigger role assumption independently
+3. **Environment Bridge**: Use environment variables to pass request context to services
+4. **Graceful Fallback**: Multiple authentication methods with priority-based selection
+5. **Security First**: Always validate assumed roles and handle credential expiration
+
+#### Deployment and Configuration Learnings
+- **Docker Image Tagging**: Use descriptive tags like `auto-role-assumption` for tracking deployments
+- **ECS Task Definition Updates**: Always update task definition with new image tags before deployment
+- **IAM Trust Policy Management**: Target roles must trust both ECS task role and execution role
+- **Role Name Validation**: Ensure frontend sends correct role names (e.g., `ReadWriteQuiltV2-sales-prod` not `ReadWriteQuiltBucket`)
+- **Platform Compatibility**: Always build Docker images with `--platform linux/amd64` for ECS compatibility
+
+#### Troubleshooting Patterns
+- **CloudWatch Log Analysis**: Monitor for "Automatic role assumption successful/failed" messages
+- **Header Validation**: Verify frontend sends full AWS ARNs, not just role names
+- **IAM Permission Checks**: Use `aws iam get-role` and `aws iam list-roles` to verify role existence and trust policies
+- **Environment Variable Debugging**: Check `QUILT_USER_ROLE_ARN` and `QUILT_USER_ID` in container logs
+
+### Docker container + release notes (2025-09-22)
+
+- `src/quilt_mcp/main.py` now honours a pre-set `FASTMCP_TRANSPORT`; container entrypoints should export `FASTMCP_TRANSPORT=http` and `FASTMCP_HOST=0.0.0.0` before invoking `quilt-mcp`.
+- The Dockerfile uses the `ghcr.io/astral-sh/uv:python3.11-bookworm-slim` base. Native deps (`build-essential`, `libcurl4(-openssl-dev)`, `zlib1g(-dev)`) are required for `pybigwig`; keep them in sync if the dependency list changes.
+- `make docker-build`, `make docker-run`, and `make docker-test` wrap common local workflows. `make docker-test` executes `tests/integration/test_docker_container.py`, which builds the image and probes `http://localhost:*/mcp` for a 30–60s readiness window.
+- Release automation logs into ECR via `aws-actions/amazon-ecr-login` and uses `scripts/docker_image.py` to generate version + `latest` tags. Configure either `secrets.ECR_REGISTRY` or fall back to `AWS_ACCOUNT_ID` + `AWS_DEFAULT_REGION`.
+- When running the integration test locally, Docker must be available and the build takes ~45s on warm caches. Expect the test to leave behind pulled base images but no running containers.
+- Claude Desktop still requires stdio transports; use a FastMCP proxy (`FastMCP.as_proxy(...).run(transport='stdio')`) and pass `--project /path/to/quilt-mcp-server` to `uv run` so `fastmcp` resolves correctly.
+- Remote deployments use the Terraform module in `deploy/terraform/modules/mcp_server`; it creates the ECS service, ALB target group, CloudWatch log group, and exposes `/healthz` for load balancer checks.
+- Follow existing Quilt production patterns when deploying remotely: reuse the `sales-prod` cluster and private subnets, publish linux/amd64 images, route traffic through a host/path rule that matches the ALB certificate (e.g., `demo.quiltdata.com/mcp/*`), and ensure security groups allow ALB↔ECS communication on port 8000.
+
+### Docker Build and Deployment Refactoring (2025-09-22)
+
+**Script-based Docker Operations:**
+
+- All Docker operations extracted to `scripts/docker.sh` for reusability and local testing
+- Script supports both `build` (local testing) and `push` (ECR deployment) commands
+- Integrates with existing `scripts/docker_image.py` for consistent tag generation
+- Supports dry-run mode via `--dry-run` for testing workflow changes
+
+**GitHub Actions Integration:**
+
+- Production releases (tags matching `v*` but not `v*-dev-*`) build and push Docker images automatically
+- Development releases skip Docker builds to reduce CI/CD time and resource usage
+- PR builds test Docker image building without pushing (build-only validation)
+- Docker operations moved from `push.yml` workflow into `create-release` action for better encapsulation
+
+**Makefile Integration:**
+
+- `make docker-build` - Build locally for development and testing
+- `make docker-push` - Build and push to ECR (requires VERSION environment variable)
+- `make docker-push-dev` - Build and push with timestamp-based development tags
+- All Docker targets include proper dependency checking for Docker daemon and required tools
+
+**GitHub Secrets Configuration:**
+
+Required secrets for Docker operations:
+- `ECR_REGISTRY` - ECR registry URL (preferred)
+- `AWS_ACCOUNT_ID` - AWS account ID (fallback for registry construction)
+- `AWS_DEFAULT_REGION` - AWS region (defaults to us-east-1)
+- Existing AWS credentials (`AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`) for ECR login
+
+**Environment Variable Support:**
+
+- `scripts/docker.sh` respects all environment variables from `env.example`
+- `ECR_REGISTRY`, `AWS_ACCOUNT_ID`, `AWS_DEFAULT_REGION` for registry configuration
+- `VERSION` for overriding image version tags
+- `DOCKER_IMAGE_NAME` for custom image naming (defaults to `quilt-mcp-server`)
+
+### Authentication fallback defaults
+
+- `AuthenticationService.initialize()` now always backfills `catalog_url` and `catalog_name` from environment defaults (falls back to `https://demo.quiltdata.com`) even when no authentication paths succeed, ensuring tools like `auth.catalog_url` have consistent context during unauthenticated startup.
+- `tests/unit/test_auth_service.py` contains two legacy integration cases invoking a non-existent `authenticate()` helper; leave them untouched—they document the original priority ordering but currently fail if executed wholesale.
+
+### Enhanced bearer-token validation
+
+- Enhanced JWTs now ship both compressed metadata (`b`, `p`, `r`) and expanded arrays (`buckets`, `permissions`, `roles`, `scope`, `level`). `BearerAuthService` trusts the expanded fields first, falling back to compression only when they are missing (`tests/unit/test_jwt_decompression.py#L164`).
+- `BearerAuthService.validate_bearer_token` enforces a 32-entry bucket list and validates that the expanded permission set exactly matches the role-derived AWS actions before authorising (`tests/unit/test_auth_service.py:305`).
+- Authorization logic now fails fast—and logs at error level—if either the bucket or permission lists are empty; no more permissive fallbacks for implicit S3 scopes.
+
+### Runtime environment detection
+
+- `quilt_mcp.runtime_context` isolates per-request environment + auth information via contextvars. `push_runtime_context` wraps HTTP requests while stdio sessions default to `desktop-stdio` (`tests/unit/test_utils.py:240`).
+- `QuiltAuthMiddleware` now sets the runtime environment to `web-jwt`, `web-bearer`, `web-role`, or `web-unauthenticated` based on headers, and restores prior state after each request; env vars remain only for backward compatibility (`tests/unit/test_utils.py:277`).
+- Tools and services (S3 buckets, bearer auth, GraphQL) prefer runtime context over environment variables, so desktop quilt3 flows and web JWT flows can coexist inside the same container without leaking credentials (`tests/unit/test_utils.py:319`).
+- No frontend changes are required—the middleware automatically infers context from the existing Authorization/role headers.
+
 ## important-instruction-reminders
 
 Do what has been asked; nothing more, nothing less.
 NEVER create files unless they're absolutely necessary for achieving your goal.
 ALWAYS prefer editing an existing file to creating a new one.
 NEVER proactively create documentation files (*.md) or README files. Only create documentation files if explicitly requested by the User.
+
+# important-instruction-reminders
+Do what has been asked; nothing more, nothing less.
+NEVER create files unless they're absolutely necessary for achieving your goal.
+ALWAYS prefer editing an existing file to creating a new one.
+NEVER proactively create documentation files (*.md) or README files. Only create documentation files if explicitly requested by the User.
+
+## 2025-09-25 JWT-only Auth Notes
+- The HTTP middleware now insists on `Authorization: Bearer` headers. `/healthz` remains unauthenticated for load balancers; all other routes return `401` if the header is missing or fails signature validation.
+- `quilt_mcp.services.bearer_auth_service.authenticate_header()` is the single entry point for JWT validation. It returns a normalized `JwtAuthResult`; stash it in `RuntimeAuthState.extras['jwt_auth_result']` to reuse boto3 sessions inside tools.
+- When testing with custom secrets, reset `quilt_mcp.services.bearer_auth_service._auth_service = None` so the singleton picks up new env vars; otherwise signature checks still use the previous secret.
+
+- Revised bucket and package tools now call JWT helpers for S3/quilt operations; only catalog/workflow tools remain on legacy pathways.
+- `BearerAuthService` now prefers `MCP_ENHANCED_JWT_SECRET`, then `MCP_ENHANCED_JWT_SECRET_SSM_PARAMETER` (with `AWS_REGION`/`AWS_DEFAULT_REGION`), caching SSM resolves per `(parameter, region)` and logging the source so mismatched secrets surface quickly.
+- `AuthenticationService._try_bearer_token_auth` trusts the runtime context's `jwt_auth_result` + `boto3_session` when available, avoiding extra signature checks and preventing IAM fallback between MCP tool invocations.
+- Bucket tools treat a missing `s3_client` from `check_s3_authorization` as a hard error—no more automatic fallback to `get_s3_client()`—so JWT plumbing must supply an explicit client for every bucket operation.