Tus: Introduce Automated Integration Tests for S3-Backed Uploads (e.g. with MinIO)

[andrewbrazzatti]

Summary

This issue tracks the introduction of automated integration tests for S3-backed attachment uploads, using a local S3-compatible service such as MinIO.

PR #3691 refactored attachment storage to support both file and s3 backends and modernised the TUS upload layer using @tus/server, @tus/file-store, and @tus/s3-store. While S3-backed uploads have been manually verified, there is currently no automated integration test coverage for the S3 storage path.

To ensure long-term reliability and prevent regressions, S3-backed uploads must be covered by automated integration tests in CI.

---

Background

PR: #3691

The PR delivered:

• A flexible attachment storage configuration model (file | s3)
• Deprecation of legacy stageDir
• Refactored TUS upload handling using @tus/server
• Dynamic selection of storage backend
• S3-backed upload support via @tus/s3-store
• Improved bootstrap validation and error handling
• Disk space checks limited to file storage
• Manual verification of S3 upload flow

Currently:

• File-backed uploads have test coverage and existing workflows
• S3-backed uploads have been manually verified
• No automated integration test suite exists for S3-backed uploads

This creates a regression risk as the storage layer evolves.

---

Problem Statement

S3-backed uploads introduce additional complexity:

• External storage service interaction
• Bucket configuration
• Object key generation
• URL and Location header normalization
• Event handling within the TUS server

Without automated integration tests:

• Refactors to upload handling may silently break S3 flows
• CI does not validate S3 behaviour
• Multi-environment deployments relying on S3 are higher risk

To support managed and multi-tenant deployments, S3-backed uploads must be validated automatically in CI.

---

Objectives

• Introduce automated integration tests for S3-backed uploads
• Use a local S3-compatible service such as MinIO for deterministic testing
• Validate end-to-end TUS upload flow using the S3 backend
• Ensure CI runs these tests reliably

---

Proposed Scope

1. Local S3 Test Environment

• Add MinIO as a test dependency (e.g. via Docker in CI)
• Configure test buckets automatically at test startup
• Use dedicated test credentials and isolated bucket namespace

CI considerations:

• Start MinIO as a service container in CircleCI
• Ensure test runs are fully self-contained
• Avoid reliance on external AWS infrastructure

2. Integration Test Coverage

Add tests covering:

• TUS upload initiation against S3 backend
• Chunked upload flow
• Successful upload completion
• Object existence in S3 after upload
• Correct object key structure
• Correct Location header handling
• Proper base URL handling
• Error handling for:
  • Invalid configuration
  • Missing bucket
  • Permission issues (where feasible)

3. Configuration Validation Tests

• Verify bootstrap validation rejects incomplete S3 configuration
• Verify file-only logic is not executed when S3 storage is active
• Confirm disk space checks do not run in S3 mode

4. Cleanup and Isolation

• Ensure uploaded test objects are cleaned up after test runs
• Avoid state leakage between tests
• Ensure test repeatability

---

Acceptance Criteria

• CI runs automated integration tests against an S3-compatible backend
• MinIO (or equivalent) is started and configured automatically during test execution
• End-to-end TUS upload flow to S3 is validated
• Tests confirm objects are stored correctly in the configured bucket
• Bootstrap validation for S3 configuration is covered by tests
• No external AWS dependencies are required for CI
• Existing file-backed tests remain unaffected
• CI builds and tests pass

---

Out of Scope

• Testing against real AWS S3 in CI
• Introducing new storage backends beyond file and s3
• Performance benchmarking of upload throughput

---

Notes

This work is critical for:

• Managed deployments using S3-backed storage
• Multi-environment SaaS deployments
• Preventing regressions in the modernised TUS upload layer

The testing strategy should remain backend-agnostic so that additional storage backends can be tested in the future with minimal structural changes to the test harness.