ReFrame 3.3-dev1

v3.3-dev1

ReFrame 3.3-dev1

ReFrame 3.3-dev0

v3.3-dev0

ReFrame 3.3-dev0

ReFrame 3.2

ReFrame v3.2 Release Notes

New features and enhancements

• Delay the setup of performance logging, so that perf_patterns can be defined after the test construction (#1509)
• Redesign job polling mechanism in the framework (#1402)
• Add configuration parameter for automatically capturing command errors and exit code in job scripts (#1490)
• Add utility function to look for available environment modules (#1417)
• Drop support of scheduler-specific command-line options (#1483)
• Change default JSON report file pattern (#1480)
• Drop tuple syntax for timer fields (#1470)
• Allow disabling pipeline hooks from the command line (#1462)

Bug fixes

• Fix typo in modules system unit tests (#1517)
• Fix error message when a sane Tmod installation cannot be found (#1510)
• Treat correctly forced local jobs and other small fixes (#1513)
• Do not require a configuration to be resolved always at system level (#1508)
• Print current environment and partition correctly even if a test fails before the setup (#1506)
• Add importlib_metadata to requirements (#1503)
• Load configuration even if the username cannot be retrieved (#1499)
• Restore environment correctly in case of module load failures (#1501)
• Fix bootstrap script for Ubuntu/Debian installation (#1493)
• Remove unnecessary error message from failure report (#1491)
• Fix JSON report generation for deferred expressions (#1489)
• Fix module import for files under site-packages (#1488)
• Fix formatting of summary listing of checks (#1475)
• Fix crash when git is not available (#1472)
• Fix OrderedSet implementation (#1468)
• Use new the 2020 pip dependency resolver in the bootstrap script (#1461)
• Add always units in the default reference tuple (#1460)
• Fix parsing of multiline exec_host in qstat output (#1457)
• Fix glob pattern to search for both txt and yaml (#1439)
• Fix JSON schema $id parameter (#1438)

Other

• Remove the unit tests from the coverage report (#1518)
• Set save location for JSON report files for CSCS (#1471)

ReFrame 3.1

ReFrame 3.1 Release Notes

New features and enhancements

• Support multiple tags in extract* sanity functions (#1433)
• Produce detailed JSON report for a regression testing session (#1377)
• Increase default concurrency of asynchronous execution policy to 8 jobs (#1428)
• Allow Syslog's address to be set from an environment variable (#1427)
• Add utility function for retrieving Cray Linux Environment (CLE) information (#1419)
• Add utility function for retrieving the current Cray CDT version (#1416)
• Add option to disable test stage directory cleanup (#1404)
• Allow flexible node allocation to select nodes in any state (#1374)
• Add bootstrap script to install dependencies of ReFrame (#1371)

Bug fixes

• Capture all pending test tasks in the exit loop of the asynchronous execution policy (#1434)
• Convert Syslog handler port parameter into integer (#1424)
• Use '&' to combine Slurm constraints (#1423)
• Fix unit test failure of asynchronous execution policy (#1411)
• Copy scheduler options when initializing job instances (#1413)
• Normalize ReFrame path prefixes (#1400)
• Process execution modes before configuring logging (#1399)
• Add missing launchers upcrun and upcxx-run in configuration schema (#1393)
• Combine access and job options for slurm '--constraint' (#1363)
• Fix AttributeError when pipeline hooks are defined in a mixin class (#1373)

Regression tests

• TrilinosTest_static build workaround for cdt/20.06 (#1403)
• Workaround for NetCDF tests for CDT20.06 (#1426)
• Enhance workaround for PGI in CDT 20.06 (#1421)
• Add account in executable options of RequiredConstraintCheck (#1407)
• Update configuration and tests for MCH system (#1384)
• PrgEnv-pgi workaround for CDT 20.06 (#1389)
• Fix deprecation warning in IPCMagicCheck (#1387)
• Remove PrgEnv-cray_classic from Dom and add it to Daint (#1388)
• Update P2P test on Dom (#1385)
• Add check for IPCMagic (#1360)
• Exclude Slurm memory overconsumption test from Tsa (#1364)
• New Slurm setup for Cp2kGpuCheck (#1304)

Other

• Revert dom config to TMod modules system (#1435)
• Change modules system for Dom to TMod4 (#1431)
• Dynamically set the account in CSCS config (#1418)
• Set the account dynamically in the configuration file for unit tests (#1409)
• Use account option in CSCS CI (#1405)
• Use eb 3.1-dev0 recipe as template for deployment (#1396)
• Document that sanity functions use 're.MULTILINE' (#1425)
• Renew the tutorials (#1397)
• Expand documentation of @require_deps decorator (#1401)
• Create man pages in proper man subdirectories (#1386)

ReFrame 3.1-dev1

v3.1-dev1

ReFrame 3.1-dev1

ReFrame 3.1-dev0

v3.1-dev0

ReFrame 3.1-dev0

ReFrame 3.0

ReFrame v3.0 Release Notes

New features and enhancements

• Add configuration option for always emitting --nodes in Slurm job scripts (#1356)
• Add execution time profiling for the framework's pipeline (#1325)
• Drop option --flex-alloc-tasks and internal unused sanity function utilities (#1357)
• Add node list information for jobs submitted with the Torque backend (#1347)
• Add configuration option to disable the ReqNodeNotAvail check (#1342)
• Replace scheduler-specific command-line options (#1338)
• Unify the attribute names of pre-/post-run and pre-/post-build commands (#1336)
• Add configuration option for using a login shell for jobs (#1343)
• Add test description to failure report (#1332)
• Add support for UPC language and launchers (#1333)
• Print the config file loaded by ReFrame (#1334)
• Print ReFrame environment variables in debug output (#1326)
• Insert 'noqa' annotations in long lines generated by the configuration conversion tool (#1324)
• Add JSON support to the configuration conversion tool (#1320)
• Allow users to set the output file in the configuration conversion tool (#1302)
• Introduce a new powerful configuration mechanism (#1242)
• Deprecate test syntax that overrides RegressionTest methods (#1228)
• Support *:<partition name> name pattern in the list of valid systems (#1253)
• Include progress report in output (#1240)
• Print test dependencies in output of --list-detailed (#1237)
• Add a tool to convert from the old to the new configuration syntax (#1186)
• Print failure statistics table as well as the run options for failures (#1176)
• Impose units in performance reference tuple (#1223)
• Allow tests to timeout if their associated job is pending for too long (#1099)
• Drop support for Python 3.5 (#1225)
• Add command-line option to suppress deprecation warnings (#1206)
• Include commit hash in ReFrame version when running (#1205)
• Add support for the Torque scheduler (#1172)
• Set default value of sourcesdir to 'src' only if such a directory exists, otherwise set to None (#1185)
• Change time_limit attribute to a string-based format (#1142)
• Allow Sarus backend to load local images (#1170)
• Print the current working directory in "ReFrame paths" output section (#1184)
• Support RFC3339 compliant time zone formatting when logging time fields (#1150)
• Introduce new configuration syntax and validation schema (#1128)
• Make the asynchronous execution policy default (#1127)
• Evaluate extra_resources just before job submission (#1129)

Bug fixes

• Fix parsing of completion time of Slurm jobs (#1359)
• Unset all ReFrame environment variables before running the unit tests (#1355)
• Fix the setup.py package installation script (#1341)
• Fix early abort when compile step fails with async policy (#1331)
• Fix configuration option value for non-existent name-addressable configuration objects (#1340)
• Fix handling of perflogdir in the configuration conversion tool (#1321)
• Disable recursive search of checks by default (#1309)
• Fix behaviour of verbosity option (#1307)
• Fix formatting of search path info line in output (#1311)
• Fix possible deadlock in PBS-based scheduler backends when a job is cancelled immediately after submission (#1301)
• Use absolute path as stage and output directory prefixes (#1303)
• Fix deprecation warnings in unit tests (#1287)
• Fix logging crash when increasing verbosity level (#1275)
• More robust completion check for the Torque scheduler (#1279)
• Fix string alignment in failure statistics (#1278)
• Improve the message of the deprecation warning about pipeline methods overrides (#1268)
• Fix AttributeError in flexible node allocation from the Slurm backend (#1259)
• Do not crash in case of broken pipe errors (#1257)
• Do not update progress count when a test fails with status ERROR (#1247)
• Fix formatting of the error message when an OS command fails (#1243)
• Fix Slurm backend deadlock when ReFrame runs through midnight (#1231)
• Make unit test for TERM signal handling more robust (#1227)
• Execute setup hooks for CompileOnlyRegressionTest and make the run hooks for the RunOnlyRegressionTest execute at the right place (#1163)
• Improve unit tests for the handling of the TERM signal (#1220)
• Fix crash when creating a regression test interactively (#1214)
• Fix random 'test_sigterm' unit test failure (#1217)
• Fix long description in setup script (#1209)
• Do not leave stale test jobs when ReFrame receives a TERM signal (#1193)
• Fix TypeError crash in the performance checking phase when the extracted performance variable value is not a number (#1183)
• Fix failure when check search path contains duplicates (#1152)
• Fix infinite loop bug when cleanup phase fails (#1124)
• Fix incomplete printout of the performance report in case of test failures (#1117)
• Fix dependency bug in async policy (#1109)
• Remove type key from the environment configuration (#1110)

Regression tests

• Convert JacobiNoToolHybrid to pipeline hook format (#1344)
• Update QuantumESPRESSO GPU test (#1345)
• Add new MultiDeviceOpenaccTest on Kesch and Tsa (#1314)
• Fix Slurm constraint check for cabinets (#1296)
• Add specific failure message in HPCG test when number of processes cannot be factorized (#1300)
• Fix the import of getlauncher() function in Greasy test (#1293)
• Update GridTools tests (#1291)
• Fix DGEMM test for PrgEnv-intel (#1289)
• Update tests for QuantumESPRESSO 6.5 (#1270)
• Enable units in StreamTest when running with an unknown programming environment (#1277)
• Update LAMMPS test (#1281)
• Update GREASY test (#1276)
• Update G2G latency check (#1272)
• Update GPU burn test (#1271)
• Implement Cray linker workaround for CLE7.0.UP01.PS09 (#1264)
• Update OSU Allreduce check (#1262)
• Add GREASY test (#1210)
• Update GROMACS test (#1230)
• Add MPI Tool Information Interfaces (MPI_T) check (#1251)
• Modernize the default programming environment check (#1255)
• Set max_pending_time for the Jupyter tests (#1252)
• Update QE CPU checks (#1250)
• Update CP2K CPU small check (#1249)
• Update CP2K GPU checks (#1248)
• Update nvprof test and add maintenance tag (#1246)
• Remove unnecessary tags in the HaloExchangeTest (#1239)
• Remove kesch:pn from supported systems of FlexAlltoallTest (#1241)
• Move MpiP and VTune checks to the eth-cscs/hpctools repo (#1229)
• Update P2P latency tests (#1232)
• Move patrun checks to the 'eth-cscs/hpctools' repo (#1233)
• Update Vc and nsimd tests to Cray PE/19.10 (#1235)
• Use builtin programming environment for tests loading EB modules (#1224)
• Update NAMD test (#1218)
• Fix deprecated time_limit syntax in checks (#1199)
• More robust treatment of unknown references in GpuBandwidthCheck (#1208)
• Correct the test for number of nodes in DGEMM (#1211)
• Adapt PrgEnv-cray flags for Trilinos check (#1148)
• Migrate tool checks to hpctools.git (#1156)
• Temporarily fix HPCG reference version (#1191)
• Implement workaround for PETSc test when using PrgEnv-intel and static linking (#1207)
• Fix misleading variant description in the affinity check (#1200)
• Update tests with the new syntax using decorators (#1160)
• Fix GridTools test failures on Dom (#1181)
• Update GROMACS maintenance reference values (#1182)
• Fix CDO checks (#1179)
• Adapt CudaStressTest to Dom (#1180)
• Move Intel Inspector test to the external eth-cscs/hpctools repo (#1178)
• Adjust num_cpus_per_task on Arolla and Tsa (#1174)
• Fix cuda module for GpuBurn on Kesch (#1173)
• Port ReFrame checks on Arolla and Tsa with PE 19.04 (#1154)
• Introduce workaround for static linking tests on Dom (#1151)
• Remove stale FIXME and NOTE tags from tests (#1138)
• Improvements and fixes for static linking tests (#1140)
• Make SimpleMpiCheck with CUDA more robust (#1135)
• Add native Spark check (#1112)

Other

• Change the EB template version to 3.0-dev6 when deploying (#1337)
• Create deployment Jenkinsfile (#1315)
• Implement XALT workaround for unit tests on Dom (#1266)
• Remove 'Public Test' stage from CSCS CI (#1215)
• Fix CI for Tsa (#1166)
• Use colon separated path convenction in CI (#1141)
• Add Python 3.8 testing in Travis CI (#1118)
• Update deployment script to recognize the new version output (#1108)
• Add documentation about the cleanup stage for tests with dependencies (#1358)
• Fine tune documentation (#1346)
• Update help message (#1335)
• Add 'jsonschema' to documentation requirements (#1328)
• Update the required Sphinx version in conf.py (#1327)
• Refine and enhance documentation (#1323)
• Fix tutorial configuration (#1306)
• Update "Use Cases" page of the documentation (#1256)
• Add ReFrame logo to the documentation (#1244)
• Add ReFrame logo to README (#1195)
• Adjust copyright dates in documentation (#1190)

ReFrame 3.0-dev6

v3.0-dev6

ReFrame 3.0-dev6

ReFrame 3.0-dev5

v3.0-dev5

ReFrame 3.0-dev5

ReFrame 3.0-dev4

v3.0-dev4

ReFrame 3.0-dev4