feat: Enhance session handling and observability improvements

- Refactored session management to improve clarity and efficiency, including the removal of self-referential parent-session-id annotations.
- Updated session workspace path handling to be relative to the content service's StateBaseDir, simplifying path management.
- Introduced graceful shutdown for the content service, enhancing reliability during server termination.
- Enhanced observability stack with new Grafana dashboard configurations and metrics for session lifecycle tracking.
- Cleaned up unused code and improved logging for better debugging and maintenance.

chore: Update .gitignore and remove obsolete deployment documentation

- Added build log and log file patterns to .gitignore to prevent accidental commits.
- Deleted outdated deployment documentation files: DEPLOYMENT_CHANGES.md, DIFF_IMPROVEMENTS.md, S3_MIGRATION_GAPS.md, and OPENSHIFT_SETUP.md, which are no longer relevant to the current architecture.
- Cleaned up observability-related files, including Grafana and Prometheus configurations, to streamline the observability stack.

feat: Enhance operator metrics and session handling

- Introduced Prometheus metrics for monitoring session lifecycle, including startup duration, phase transitions, and error tracking.
- Updated session handling to record metrics during reconciliation, including session creation and completion.
- Refactored session management logic to ensure consistent behavior across API and kubectl session creations.
- Increased QPS and Burst settings for Kubernetes client to improve performance under load.
- Added a new Service and ServiceMonitor for exposing operator metrics in the ambient-code namespace.

feat: Refactor AgenticSession handling to use Pods instead of Jobs

- Updated the operator to create and manage Pods directly for AgenticSessions, improving startup speed and reducing complexity.
- Changed environment variable references and logging to reflect the transition from Jobs to Pods.
- Adjusted cleanup logic to handle Pods appropriately, including service creation and monitoring.
- Modified deployment configurations to ensure compatibility with the new Pod-based architecture.

feat: Implement S3 storage configuration for session artifacts

- Added support for S3-compatible storage in the settings section, allowing users to configure S3 endpoint, bucket, region, access key, and secret key.
- Updated the operator to persist session state and artifacts in S3, replacing the previous temporary content pod mechanism.
- Removed deprecated references to temporary content pods and PVCs, transitioning to an EmptyDir storage model with S3 integration.
- Enhanced the operator's handling of S3 configuration, ensuring proper validation and logging for S3 settings.
- Updated Makefile to include new build targets for state-sync image and MinIO setup.

feat: Enhance operator deployment with controller-runtime features

- Added command-line arguments for metrics and health probe endpoints, enabling better observability.
- Implemented concurrent reconciliation with a configurable maximum, improving performance.
- Updated Dockerfile to use ENTRYPOINT for better argument handling.
- Enhanced health checks with HTTP probes for liveness and readiness.
- Updated README to reflect new configuration options and features.

feat: Enhance observability stack deployment and cleanup in Makefile

- Added new targets for deploying and cleaning up the observability stack, including OpenTelemetry and Grafana.
- Introduced commands for accessing Grafana and Prometheus dashboards.
- Updated .gitignore to include secrets template for MinIO credentials.
- Removed deprecated image-prepuller DaemonSet and associated metrics service from manifests.
- Updated Makefile to reflect changes in observability management and improve user experience.

refactor: Clean up observability stack and enhance session handling

- Removed obsolete observability stack deployment commands from Makefile.
- Updated session handling in the operator to improve clarity and efficiency.
- Introduced a new state sync image in deployment scripts and updated related configurations.
- Refactored metrics handling for session lifecycle, ensuring consistent error tracking and performance monitoring.
- Cleaned up unused code and improved readability across multiple files.

feat: Refactor S3 storage configuration in settings and operator

- Replaced S3_ENABLED with STORAGE_MODE to allow selection between shared and custom storage options.
- Updated settings section to include radio buttons for storage mode selection, enhancing user experience.
- Modified operator session handling to read and apply storage mode, ensuring proper configuration for S3 settings.
- Improved logging for storage mode usage, clarifying the configuration process for users.

Author: Gkrumbach07

.gitignore

@@ -136,3 +136,11 @@ e2e/langfuse/.env.langfuse-keys
 # Test Reporting
 logs/
 reports/
+
+# Secrets (should use .example templates)
+**/minio-credentials-secret.yaml
+
+# Build artifacts and logs
+build.log
+*.log
+!components/**/*.log

Makefile

@@ -1,10 +1,11 @@
-.PHONY: help setup build-all build-frontend build-backend build-operator build-runner deploy clean
+.PHONY: help setup build-all build-frontend build-backend build-operator build-runner build-state-sync deploy clean
 .PHONY: local-up local-down local-clean local-status local-rebuild local-reload-backend local-reload-frontend local-reload-operator local-sync-version
 .PHONY: local-dev-token
 .PHONY: local-logs local-logs-backend local-logs-frontend local-logs-operator local-shell local-shell-frontend
 .PHONY: local-test local-test-dev local-test-quick test-all local-url local-troubleshoot local-port-forward local-stop-port-forward
 .PHONY: push-all registry-login setup-hooks remove-hooks check-minikube check-kubectl
 .PHONY: e2e-test e2e-setup e2e-clean deploy-langfuse-openshift
+.PHONY: setup-minio minio-console minio-logs minio-status
 .PHONY: validate-makefile lint-makefile check-shell makefile-health
 .PHONY: _create-operator-config _auto-port-forward _show-access-info _build-and-load
 
@@ -36,6 +37,7 @@ FRONTEND_IMAGE ?= vteam_frontend:latest
 BACKEND_IMAGE ?= vteam_backend:latest
 OPERATOR_IMAGE ?= vteam_operator:latest
 RUNNER_IMAGE ?= vteam_claude_runner:latest
+STATE_SYNC_IMAGE ?= vteam_state_sync:latest
 
 # Build metadata (captured at build time)
 GIT_COMMIT := $(shell git rev-parse HEAD 2>/dev/null || echo "unknown")
@@ -91,7 +93,7 @@ help: ## Display this help message
 
 ##@ Building
 
-build-all: build-frontend build-backend build-operator build-runner ## Build all container images
+build-all: build-frontend build-backend build-operator build-runner build-state-sync ## Build all container images
 
 build-frontend: ## Build frontend image
 	@echo "$(COLOR_BLUE)▶$(COLOR_RESET) Building frontend with $(CONTAINER_ENGINE)..."
@@ -145,6 +147,13 @@ build-runner: ## Build Claude Code runner image
 		-t $(RUNNER_IMAGE) -f claude-code-runner/Dockerfile .
 	@echo "$(COLOR_GREEN)✓$(COLOR_RESET) Runner built: $(RUNNER_IMAGE)"
 
+build-state-sync: ## Build state-sync image for S3 persistence
+	@echo "$(COLOR_BLUE)▶$(COLOR_RESET) Building state-sync with $(CONTAINER_ENGINE)..."
+	@echo "  Git: $(GIT_BRANCH)@$(GIT_COMMIT_SHORT)$(GIT_DIRTY)"
+	@cd components/runners/state-sync && $(CONTAINER_ENGINE) build $(PLATFORM_FLAG) $(BUILD_FLAGS) \
+		-t vteam_state_sync:latest .
+	@echo "$(COLOR_GREEN)✓$(COLOR_RESET) State-sync built: vteam_state_sync:latest"
+
 ##@ Git Hooks
 
 setup-hooks: ## Install git hooks for branch protection
@@ -164,13 +173,59 @@ registry-login: ## Login to container registry
 
 push-all: registry-login ## Push all images to registry
 	@echo "$(COLOR_BLUE)▶$(COLOR_RESET) Pushing images to $(REGISTRY)..."
-	@for image in $(FRONTEND_IMAGE) $(BACKEND_IMAGE) $(OPERATOR_IMAGE) $(RUNNER_IMAGE); do \
+	@for image in $(FRONTEND_IMAGE) $(BACKEND_IMAGE) $(OPERATOR_IMAGE) $(RUNNER_IMAGE) $(STATE_SYNC_IMAGE); do \
 		echo "  Tagging and pushing $$image..."; \
 		$(CONTAINER_ENGINE) tag $$image $(REGISTRY)/$$image && \
 		$(CONTAINER_ENGINE) push $(REGISTRY)/$$image; \
 	done
 	@echo "$(COLOR_GREEN)✓$(COLOR_RESET) All images pushed"
 
+##@ MinIO S3 Storage
+
+setup-minio: ## Set up MinIO and create initial bucket
+	@echo "$(COLOR_BLUE)▶$(COLOR_RESET) Setting up MinIO for S3 state storage..."
+	@./scripts/setup-minio.sh
+	@echo "$(COLOR_GREEN)✓$(COLOR_RESET) MinIO setup complete"
+
+minio-console: ## Open MinIO console (port-forward to localhost:9001)
+	@echo "$(COLOR_BLUE)▶$(COLOR_RESET) Opening MinIO console at http://localhost:9001"
+	@echo "  Login: admin / changeme123 (or your configured credentials)"
+	@kubectl port-forward svc/minio 9001:9001 -n $(NAMESPACE)
+
+minio-logs: ## View MinIO logs
+	@kubectl logs -f deployment/minio -n $(NAMESPACE)
+
+minio-status: ## Check MinIO status
+	@echo "$(COLOR_BOLD)MinIO Status$(COLOR_RESET)"
+	@kubectl get deployment,pod,svc,pvc -l app=minio -n $(NAMESPACE)
+
+##@ Observability
+
+deploy-observability: ## Deploy observability (OTel + OpenShift Prometheus)
+	@echo "$(COLOR_BLUE)▶$(COLOR_RESET) Deploying observability stack..."
+	@kubectl apply -k components/manifests/observability/
+	@echo "$(COLOR_GREEN)✓$(COLOR_RESET) Observability deployed (OTel + ServiceMonitor)"
+	@echo "  View metrics: OpenShift Console → Observe → Metrics"
+	@echo "  Optional Grafana: make add-grafana"
+
+add-grafana: ## Add Grafana on top of observability stack
+	@echo "$(COLOR_BLUE)▶$(COLOR_RESET) Adding Grafana..."
+	@kubectl apply -k components/manifests/observability/overlays/with-grafana/
+	@echo "$(COLOR_GREEN)✓$(COLOR_RESET) Grafana deployed"
+	@echo "  Create route: oc create route edge grafana --service=grafana -n $(NAMESPACE)"
+
+clean-observability: ## Remove observability components
+	@echo "$(COLOR_BLUE)▶$(COLOR_RESET) Removing observability..."
+	@kubectl delete -k components/manifests/observability/overlays/with-grafana/ 2>/dev/null || true
+	@kubectl delete -k components/manifests/observability/ 2>/dev/null || true
+	@echo "$(COLOR_GREEN)✓$(COLOR_RESET) Observability removed"
+
+grafana-dashboard: ## Open Grafana (create route first)
+	@echo "$(COLOR_BLUE)▶$(COLOR_RESET) Opening Grafana..."
+	@oc create route edge grafana --service=grafana -n $(NAMESPACE) 2>/dev/null || echo "Route already exists"
+	@echo "  URL: https://$$(oc get route grafana -n $(NAMESPACE) -o jsonpath='{.spec.host}')"
+	@echo "  Login: admin/admin"
+
 ##@ Local Development (Minikube)
 
 local-up: check-minikube check-kubectl ## Start local development environment (minikube)