Skip to content

Commit 2831af0

Browse files
author
Vasileios Karakasis
committed
Document the --ci-generate option
1 parent 57f2318 commit 2831af0

File tree

3 files changed

+67
-0
lines changed

3 files changed

+67
-0
lines changed

docs/_static/img/gitlab-ci.png

76.2 KB
Loading

docs/manpage.rst

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -147,6 +147,14 @@ There are currently two actions that can be performed on tests: (a) list the tes
147147
An action must always be specified.
148148

149149

150+
.. option:: --ci-generate=FILE
151+
152+
Do not run the tests, but generate a Gitlab `child pipeline <https://docs.gitlab.com/ee/ci/parent_child_pipelines.html>`__ specification in ``FILE``.
153+
You can set up your Gitlab CI to use the generated file to run every test as a separate Gitlab job respecting test dependencies.
154+
For more information, have a look in :ref:`generate-ci-pipeline`.
155+
156+
.. versionadded:: 3.5
157+
150158
.. option:: -l, --list
151159

152160
List selected tests.

docs/tutorial_tips_tricks.rst

Lines changed: 59 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -367,6 +367,8 @@ This option is useful when you combine it with the various test filtering option
367367
For example, you might want to rerun only the failed tests or just a specific test in a dependency chain.
368368
Let's see an artificial example that uses the following test dependency graph.
369369

370+
.. _fig-deps-complex:
371+
370372
.. figure:: _static/img/deps-complex.svg
371373
:align: center
372374

@@ -477,3 +479,60 @@ If we tried to run :class:`T6` without restoring the session, we would have to r
477479
478480
[ PASSED ] Ran 5 test case(s) from 5 check(s) (0 failure(s))
479481
[==========] Finished on Thu Jan 21 14:32:09 2021
482+
483+
484+
.. _generate-ci-pipeline:
485+
486+
Integrating into a CI pipeline
487+
------------------------------
488+
489+
.. versionadded:: 3.5
490+
491+
Instead of running your tests, you can ask ReFrame to generate a `child pipeline <https://docs.gitlab.com/ee/ci/parent_child_pipelines.html>`__ specification for the Gitlab CI.
492+
This will spawn a CI job for each ReFrame test respecting test dependencies.
493+
You could run your tests in a single job of your Gitlab pipeline, but you would not take advantage of the parallelism across different CI jobs.
494+
Having a separate CI job per test makes it also easier to spot the failing tests.
495+
496+
As soon as you have set up a `runner <https://docs.gitlab.com/ee/ci/quick_start/>`__ for your repository, it is fairly straightforward to use ReFrame to automatically generate the necessary CI steps.
497+
The following is an example of ``.gitlab-ci.yml`` file that does exactly that:
498+
499+
.. code-block:: yaml
500+
501+
stages:
502+
- generate
503+
- test
504+
505+
generate-pipeline:
506+
stage: generate
507+
script:
508+
- reframe --ci-generate=${CI_PROJECT_DIR}/pipeline.yml -c ${CI_PROJECT_DIR}/path/to/tests
509+
artifacts:
510+
paths:
511+
- ${CI_PROJECT_DIR}/pipeline.yml
512+
513+
test-jobs:
514+
stage: test
515+
trigger:
516+
include:
517+
- artifact: pipeline.yml
518+
job: generate-pipeline
519+
strategy: depend
520+
521+
522+
It defines two stages.
523+
The first one, called ``generate``, will call ReFrame to generate the pipeline specification for the desired tests.
524+
All the usual `test selection options <manpage.html#test-filtering>`__ can be used to select specific tests.
525+
ReFrame will process them as usual, but instead of running them, it will generate the correct steps for running them in Gitlab.
526+
We then pass the generated CI pipeline file to second phase as an artifact and we are done!
527+
528+
The following figure shows one part of the automatically generated pipeline for the test graph depicted `above <#fig-deps-complex>`__.
529+
530+
.. figure:: _static/img/gitlab-ci.png
531+
:align: center
532+
533+
:sub:`Snapshot of a Gitlab pipeline generated automatically by ReFrame.`
534+
535+
536+
.. note::
537+
538+
The ReFrame executable must be available in the Gitlab runner that will run the CI jobs.

0 commit comments

Comments
 (0)