Overall Protocol for Testing

Before a new release can occur:

• Unit tests must all pass.
• Regression tests must be run using the benchmark dataset.
• Set output directory to '/tdata/CPAC/vversion number with no periods/output/benchmark.
• Set working directory to '/tdata/CPAC/vversion number with no periods/working/benchmark.
• Set crash directory to '/tdata/CPAC/vversion number with no periods/crash/benchmark.
• Set logs directory to '/tdata/CPAC/vversion number with no periods/logs/benchmark.
• Save the results for the last regression test for use in the benchmark package.
• Group analysis is part of regression test.
• Question: Could we run this in Docker using a modified Dockerfile for the C-PAC BIDS app?'
• Retention : Max 2 datasets.
• Platforms:
• AWS : Perform a run on a freshly-formed AMI that tests the AWS functions (i.e., uses data stored in S3 for a participant list, puts data into S3 for outputs).
• CentOS
• Ubuntu
• OS X
• Particular features to test:
• Resource allocation
• C-PAC must be run on novel datasets to determine if any anomalies occur with non-benchmark data (small runs):
• Perform a run on:
• Rhyme Judgement
• Serial Scanning
• Brain Genomics Superstruct Project
• Pipelines should be configured to use the following directories:
• Output directory : '/tdata/CPAC/vversion number with no periods/output/dataset name.
• Working directory : '/tdata/CPAC/vversion number with no periods/working/dataset name.
• Crash directory : '/tdata/CPAC/vversion number with no periods/crash/dataset name.
• Logs directory : '/tdata/CPAC/vversion number with no periods/logs/dataset name.
• GUI testing must be finished with no failures.
  • Subject list builder
  • Pipeline configuration (individual-level)
  • Group analysis
  • Tool tips and config comments - make sure they are current
• All issues in the tracker must be iterated through and tested to ensure that they have been fixed.
• Perform a fresh installation.
• Platforms:
• Ubuntu
• CentOS
• OS X