Add automated dependency lock file management system

- Create automated weekly lock file update workflow
- Add scripts for manual lock file generation
- Document automated dependency management approach
- Update source files with conservative upper bounds
- Combine automated lock files with upper bounds for robust dependency management

Co-authored-by: pancetta <7158893+pancetta@users.noreply.github.com>

Author: Copilot

.github/workflows/update_lockfiles.yml

@@ -0,0 +1,149 @@
+---
+name: Update Dependency Lock Files
+
+on:
+  # Run weekly on Sunday (before Monday CI run)
+  schedule:
+    - cron: '0 2 * * 0'
+  # Allow manual triggering
+  workflow_dispatch:
+  # Run when environment files change
+  push:
+    paths:
+      - 'etc/environment-*.yml'
+      - 'pySDC/projects/*/environment.yml'
+      - 'pyproject.toml'
+
+jobs:
+  update-lockfiles:
+    runs-on: ubuntu-latest
+    permissions:
+      contents: write
+      pull-requests: write
+    
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v4
+        with:
+          fetch-depth: 0
+      
+      - name: Set up Micromamba
+        uses: mamba-org/setup-micromamba@v1
+        with:
+          environment-name: lockfile-env
+          create-args: >-
+            python=3.11
+            conda-lock
+            pip-tools
+      
+      - name: Generate lock files for etc environments
+        shell: bash -l {0}
+        run: |
+          mkdir -p etc/lockfiles
+          
+          for env_file in etc/environment-*.yml; do
+            if [[ -f "$env_file" ]]; then
+              base_name=$(basename "$env_file" .yml)
+              echo "Generating lock file for $env_file..."
+              
+              # Generate unified lock file (works across platforms)
+              conda-lock lock --file "$env_file" \
+                --platform linux-64 \
+                --lockfile "etc/lockfiles/${base_name}-lock.yml" \
+                || echo "Warning: Failed to generate lock file for $env_file"
+            fi
+          done
+      
+      - name: Generate lock files for project environments
+        shell: bash -l {0}
+        run: |
+          for env_file in pySDC/projects/*/environment.yml; do
+            if [[ -f "$env_file" ]]; then
+              project_dir=$(dirname "$env_file")
+              project_name=$(basename "$project_dir")
+              
+              echo "Generating lock file for $project_name..."
+              
+              # Create lockfiles directory in project
+              mkdir -p "$project_dir/lockfiles"
+              
+              # Generate lock file
+              conda-lock lock --file "$env_file" \
+                --platform linux-64 \
+                --lockfile "$project_dir/lockfiles/environment-lock.yml" \
+                || echo "Warning: Failed to generate lock file for $project_name"
+            fi
+          done
+      
+      - name: Generate pip lock file from pyproject.toml
+        shell: bash -l {0}
+        run: |
+          # Generate pinned requirements from pyproject.toml
+          pip-compile pyproject.toml --resolver=backtracking -o requirements-lock.txt \
+            || echo "Warning: Failed to generate pip lock file"
+      
+      - name: Check for changes
+        id: check_changes
+        run: |
+          git add -A
+          if git diff --staged --quiet; then
+            echo "has_changes=false" >> $GITHUB_OUTPUT
+            echo "No lock file changes detected"
+          else
+            echo "has_changes=true" >> $GITHUB_OUTPUT
+            echo "Lock files have been updated"
+          fi
+      
+      - name: Create Pull Request
+        if: steps.check_changes.outputs.has_changes == 'true'
+        uses: peter-evans/create-pull-request@v5
+        with:
+          token: ${{ secrets.GITHUB_TOKEN }}
+          commit-message: 'chore: Update dependency lock files'
+          title: '🔒 Automated dependency lock file update'
+          body: |
+            ## Automated Lock File Update
+            
+            This PR updates the dependency lock files with the latest compatible versions.
+            
+            ### What changed
+            
+            Lock files have been regenerated from the source environment files and pyproject.toml.
+            This ensures we're using the latest compatible versions of all dependencies while
+            maintaining reproducibility.
+            
+            ### Testing
+            
+            The CI pipeline will test these lock files to ensure all tests pass with the updated dependencies.
+            
+            ### Action Required
+            
+            - [ ] Review the lock file changes
+            - [ ] Check CI test results
+            - [ ] Merge if all tests pass, or investigate failures
+            
+            ### Related Documentation
+            
+            See [Dependency Management Guide](./docs/contrib/08_dependency_management.md) for details on our dependency strategy.
+            
+            ---
+            
+            This PR was automatically created by the `update_lockfiles.yml` workflow.
+            
+            **Trigger**: ${{ github.event_name }}
+            **Workflow Run**: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
+          branch: automated/update-lockfiles
+          delete-branch: true
+          labels: |
+            automated
+            dependencies
+            maintenance
+          draft: false
+      
+      - name: Summary
+        run: |
+          if [[ "${{ steps.check_changes.outputs.has_changes }}" == "true" ]]; then
+            echo "✅ Lock files updated and PR created"
+          else
+            echo "ℹ️ No changes to lock files"
+          fi

CONTRIBUTING.md

@@ -34,5 +34,7 @@ for more details. Additional guideline are also given on how to
 5. [Custom Implementations](./docs/contrib/04_custom_implementations.md)
 6. [Documenting Code](./docs/contrib/05_documenting_code.md)
 7. [Adding a project](./docs/contrib/06_new_project.md)
+8. [Release Guide](./docs/contrib/07_release_guide.md)
+9. [Dependency Management](./docs/contrib/08_dependency_management.md)
 
 :arrow_left: [Back to main page](./README.md)

docs/contrib/02_continuous_integration.md

@@ -36,13 +36,14 @@ When you receive an automated failure PR:
 1. Review the `failure_analysis.md` file attached to the PR
 2. Check the linked workflow run and job logs for full details
 3. Investigate the root cause (dependency issues, flaky tests, code bugs, etc.)
-4. Push fixes directly to the PR branch or close if it's a transient failure
-5. Test fixes locally or wait for CI to verify
-6. Merge when the issue is confirmed resolved
+4. **For dependency-related failures**: See the [dependency management guide](./08_dependency_management.md) for strategies to handle version conflicts
+5. Push fixes directly to the PR branch or close if it's a transient failure
+6. Test fixes locally or wait for CI to verify
+7. Merge when the issue is confirmed resolved
 
 For more details, see the [automated failure handling documentation](../../.github/scripts/README.md).
 
-> :bell: **Note:** These automated PRs are informational and require manual review. They help centralize failure information but don't automatically fix issues. If you can identify and fix the problem, push commits to the auto-generated branch.
+> :bell: **Note:** These automated PRs are informational and require manual review. They help centralize failure information but don't automatically fix issues. If you can identify and fix the problem, push commits to the auto-generated branch. Common causes of weekly failures include dependency updates—see the [dependency management guide](./08_dependency_management.md) for how to handle these.
 
 ## Code linting
 

docs/contrib/06_new_project.md

@@ -23,7 +23,10 @@ name: pySDC
 channels:
   - conda-forge
 dependencies:
-  - numpy
+  - numpy>=1.15.4,<3.0
+  - scipy>=0.17.1,<2.0
+  - matplotlib>=3.0,<4.0
+  - dill>=0.2.6
   - pip
   - pip:
     - qmat>=0.1.8
@@ -34,6 +37,11 @@ The name should stay `pySDC`. The channels cover most of the usual packages.
 If a package is needed that cannot be found in those channels by conda (or mamba),
 please add the correct channel to the list.
 
+**Important**: Follow the [dependency management guidelines](./08_dependency_management.md) when specifying version constraints:
+- Include both **lower** and **upper** bounds for major dependencies (e.g., `numpy>=1.15.4,<3.0`)
+- This prevents unexpected breaking changes from major version updates while allowing minor updates
+- See the [dependency management guide](./08_dependency_management.md) for detailed recommendations
+
 ## Add tests to the project
 
 In order to automatically find the tests of your project, please add the tests to a subdirectory called `tests` in the directory of your project.