Skip to content

Commit 4968f8b

Browse files
committed
Add nf-co2footprint plugin for emissions tracking
Track CO2 emissions during nf-test runs and upload reports to S3 for analysis with DuckDB. Refs: #9291
1 parent 17369d8 commit 4968f8b

File tree

2 files changed

+29
-0
lines changed

2 files changed

+29
-0
lines changed

.github/workflows/nf-test.yml

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -171,6 +171,31 @@ jobs:
171171
total_shards: ${{ env.TOTAL_SHARDS }}
172172
paths: "${{ join(fromJson(steps.filter.outputs.filtered_paths), ' ') }}"
173173

174+
- name: Upload CO2 footprint reports to S3
175+
if: always() && steps.filter.outputs.filtered_paths != '[]'
176+
shell: bash
177+
run: |
178+
# Find all CO2 footprint trace files
179+
CO2_FILES=$(find . -name "co2footprint_trace*.txt" -type f)
180+
181+
if [ -n "$CO2_FILES" ]; then
182+
# Get current date and sanitize branch name
183+
DATE=$(date +%Y-%m-%d)
184+
BRANCH_NAME=$(echo "${{ github.head_ref || github.ref_name }}" | sed 's/[^a-zA-Z0-9._-]/_/g')
185+
186+
# S3 destination path
187+
S3_BASE="s3://nf-core-resource-optimization/co2reports/modules/${DATE}/${BRANCH_NAME}/${{ matrix.profile }}/${{ matrix.shard }}"
188+
189+
# Upload each CO2 trace file
190+
for file in $CO2_FILES; do
191+
filename=$(basename "$file")
192+
aws s3 cp "$file" "${S3_BASE}/${filename}" --no-progress
193+
echo "Uploaded: $file -> ${S3_BASE}/${filename}"
194+
done
195+
else
196+
echo "No CO2 footprint trace files found"
197+
fi
198+
174199
confirm-pass-nf-test:
175200
runs-on:
176201
- runs-on=${{ github.run_id }}-confirm-pass-nf-test

tests/config/nf-test.config

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,7 @@
1+
plugins {
2+
3+
}
4+
15
params {
26
publish_dir_mode = "copy"
37
singularity_pull_docker_container = false

0 commit comments

Comments
 (0)