Implement Wasabi CDN synchronization workflow and integration with deployment pipeline

Author: mattmattox

.github/workflows/pipeline.yml

@@ -118,6 +118,9 @@ jobs:
       max-parallel: 1
       matrix:
         environment: [mst, dev, qas, tst, stg, prd]
+    outputs:
+      synccdn_needed: ${{ steps.check-sync-cdn.outputs.synccdn }}
+      environment: ${{ matrix.environment }}
 
     steps:
       - name: Checkout repository
@@ -170,4 +173,40 @@ jobs:
           if [ $COUNTER -eq $MAX_TRIES ]; then
             echo "Application did not become healthy in time."
             exit 1
-          fi
+          fi
+          
+      - name: Check if CDN Sync is needed
+        id: check-sync-cdn
+        run: |
+          if [[ "${{ matrix.environment }}" == "qas" || "${{ matrix.environment }}" == "tst" || "${{ matrix.environment }}" == "stg" || "${{ matrix.environment }}" == "prd" ]]; then
+            echo "synccdn=true" >> $GITHUB_OUTPUT
+          else
+            echo "synccdn=false" >> $GITHUB_OUTPUT
+          fi
+
+  Sync-CDN:
+    runs-on: ubuntu-latest
+    needs: Deploy
+    if: needs.Deploy.outputs.synccdn_needed == 'true'
+    
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v3
+        with:
+          fetch-depth: 0  # Fetch all history for proper change detection
+      
+      - name: Configure AWS credentials for Wasabi
+        uses: aws-actions/configure-aws-credentials@v2
+        with:
+          aws-access-key-id: ${{ secrets.WASABI_ACCESS_KEY }}
+          aws-secret-access-key: ${{ secrets.WASABI_SECRET_KEY }}
+          aws-region: us-central-1
+      
+      - name: Sync to Wasabi S3 bucket
+        run: |
+          echo "Starting sync of cdn.support.tools to Wasabi for environment ${{ needs.Deploy.outputs.environment }}"
+          aws s3 sync ./cdn.support.tools/ s3://cdn.support.tools/ \
+            --endpoint-url=https://s3.us-central-1.wasabisys.com \
+            --exclude ".git/*" \
+            --acl public-read
+          echo "Sync completed successfully"

.github/workflows/wasabi-sync.yml

@@ -0,0 +1,44 @@
+name: Sync CDN to Wasabi
+
+on:
+  # Trigger on changes to cdn.support.tools directory
+  push:
+    paths:
+      - 'cdn.support.tools/**'
+  
+  # Add manual trigger option
+  workflow_dispatch:
+  
+  # Optional: Add ability to call from other workflows
+  workflow_call:
+    inputs:
+      environment:
+        required: false
+        type: string
+        default: 'production'
+
+jobs:
+  sync-to-wasabi:
+    runs-on: ubuntu-latest
+    
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v3
+        with:
+          fetch-depth: 0  # Fetch all history for proper change detection
+      
+      - name: Configure AWS credentials for Wasabi
+        uses: aws-actions/configure-aws-credentials@v2
+        with:
+          aws-access-key-id: ${{ secrets.WASABI_ACCESS_KEY }}
+          aws-secret-access-key: ${{ secrets.WASABI_SECRET_KEY }}
+          aws-region: us-central-1
+      
+      - name: Sync to Wasabi S3 bucket
+        run: |
+          echo "Starting sync of cdn.support.tools to Wasabi"
+          aws s3 sync ./cdn.support.tools/ s3://cdn.support.tools/ \
+            --endpoint-url=https://s3.us-central-1.wasabisys.com \
+            --exclude ".git/*" \
+            --acl public-read
+          echo "Sync completed successfully"

memory-bank/activeContext.md

@@ -6,6 +6,7 @@
 - **Social Media Content**: Developed BlueSky and LinkedIn posts to promote the new guide
 - **Memory Bank Setup**: Established documentation structure for future work continuity
 - **GitHub CodeQL Fix**: Resolved code scanning configuration conflict
+- **CDN-Wasabi Integration**: Implemented automated CDN synchronization to Wasabi cloud storage
 
 ## Recent Changes
 1. Created comprehensive CKA Exam Preparation Guide:
@@ -39,6 +40,13 @@
    - Created documentation explaining the root cause and solution
    - Created `docs/github-codeql-fix.md` with detailed steps to resolve the issue
 
+7. Implemented CDN synchronization to Wasabi:
+   - Created dedicated GitHub Actions workflow for manual syncing (`wasabi-sync.yml`)
+   - Integrated automatic CDN sync with main deployment pipeline for production environments
+   - Configured sync to preserve existing files on the Wasabi bucket (no deletion)
+   - Used AWS S3 sync command with Wasabi endpoint for optimal compatibility
+   - Configured appropriate IAM credentials securely in GitHub Secrets
+
 ## Active Decisions and Considerations
 - **Training Series Structure**: Organized CKA content into logical sections following established blog training series pattern
 - **Kubernetes Best Practices**: Emphasized current Kubernetes best practices in all examples and solutions
@@ -47,6 +55,7 @@
 - **Technical Depth**: Emphasized advanced troubleshooting techniques over basic concepts to provide unique value
 - **Memory Bank Structure**: Implemented complete documentation structure for future reference
 - **CodeQL Configuration**: Prioritized custom workflow over default setup for better control and integration with CI/CD pipeline
+- **CDN Sync Strategy**: Implemented additive-only sync to Wasabi to prevent accidental file deletion
 
 ## Next Steps
 1. Test the new CKA Exam Preparation Guide with actual students to gather feedback
@@ -57,3 +66,5 @@
 6. Evaluate metrics to track engagement with the new content
 7. Explore additional social media platforms for content promotion
 8. Implement the recommended GitHub CodeQL fix to resolve the code scanning issue
+9. Test the new Wasabi CDN synchronization workflow with a manual trigger
+10. Monitor the first few automatic CDN syncs during deployment to ensure proper functionality

memory-bank/progress.md

@@ -10,6 +10,7 @@
 - **Deployment Pipeline**: ArgoCD configuration suggests functioning GitOps deployment
 - **CDN Configuration**: External asset hosting appears to be configured correctly
 - **Technical Documentation**: Created documentation for resolving GitHub CodeQL configuration issues
+- **CDN-Wasabi Integration**: Implemented automated sync of CDN assets to Wasabi cloud storage
 
 ## What's Left to Build
 - **Training Diagrams**: Consider adding visual diagrams to enhance the CKA training series
@@ -19,6 +20,7 @@
 - **Content Series**: Develop related content that builds on the Cilium troubleshooting guide and CKA training series
 - **Analytics Integration**: Ensure proper tracking of content engagement
 - **Additional Platforms**: Evaluate expanding social media presence beyond current platforms
+- **CDN Sync Monitoring**: Add monitoring and notification for CDN sync failures
 
 ## Current Status
 - **CKA Training Series**: Complete - 10-part comprehensive guide created for Kubernetes certification exam preparation
@@ -30,10 +32,12 @@
 - **CDN Resources**: Pending - need to evaluate if additional images are needed
 - **Deployment**: Pending - new content ready for deployment process
 - **GitHub CodeQL**: Documented solution - created guidance on fixing the configuration conflict
+- **CDN Sync Automation**: Complete - implemented GitHub Actions workflows for both manual and automatic synchronization to Wasabi
 
 ## Known Issues
 - GitHub CodeQL scanning reporting configuration conflict - solution documented in `docs/github-codeql-fix.md`
 - Need to verify how images are referenced in blog posts to ensure CDN paths are correct
 - CKA training series would benefit from accompanying diagrams to enhance understanding
 - May need to test if the social media post formatting displays correctly on actual platforms
 - Should confirm that 2025 publication date functions correctly with the Hugo platform
+- Wasabi sync configuration uses additive-only approach which may accumulate stale files over time

memory-bank/techContext.md

@@ -13,6 +13,7 @@
 - **Kubernetes**: Container orchestration (indicated by ArgoCD configs)
 - **ArgoCD**: GitOps-based deployment to Kubernetes
 - **Nginx**: Web server configuration present
+- **Wasabi**: Cloud storage provider used for CDN asset hosting
 
 ## Development Setup
 - Source code stored in GitHub repository
@@ -25,25 +26,40 @@
 - Image optimization for CDN delivery
 - Kubernetes deployment considerations
 - Container size and performance optimization
+- CDN synchronization timing with deployments
 
 ## Dependencies
 - Go dependencies managed through go.mod
 - Hugo theme dependencies (using theme m10c)
 - Container base images
 - Kubernetes infrastructure dependencies
+- AWS CLI for Wasabi S3 compatibility
 
 ## Infrastructure Setup
 ### Hosting
 - Appears to be using Kubernetes (ArgoCD configurations present)
 - Multiple environments defined: dev, tst, qas, stg, prd
 - CDN setup for static assets at cdn.support.tools
+- Wasabi cloud storage (s3.us-central-1.wasabisys.com) hosts the CDN content
 
 ### CI/CD
-- Deployment appears to use ArgoCD for GitOps workflows
+- Deployment uses GitHub Actions workflow (`pipeline.yml`)
+- ArgoCD for GitOps deployments to Kubernetes clusters
 - `deploy.sh` script suggests automated deployment process
 - Build process includes Docker image creation
 - Multiple Kubernetes environments with dedicated configurations
+- CDN synchronization with Wasabi via GitHub Actions:
+  - Standalone workflow (`wasabi-sync.yml`) for manual syncs
+  - Integrated CDN sync job in main pipeline for production environments
+  - AWS S3 sync command used with Wasabi endpoint for compatibility
 
 ## Domain Structure
 - Main site: support.tools
 - CDN: cdn.support.tools
+
+## Cloud Services
+- Wasabi S3-compatible storage:
+  - Bucket name: cdn.support.tools
+  - Endpoint: s3.us-central-1.wasabisys.com
+  - Authentication: IAM credentials stored in GitHub Secrets
+  - Sync strategy: Additive-only (no file deletion)