Skip to content

Commit 90893ee

Browse files
committed
Address review comments.
1 parent 390d7ff commit 90893ee

File tree

5 files changed

+91
-89
lines changed

5 files changed

+91
-89
lines changed

src/7-to-8/major-changes/compatibility-mode.rst

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -34,8 +34,9 @@ The ``suite.rc`` filename triggers a backward compatibility mode in which:
3434
branching)
3535

3636
- ``succeeded`` task outputs are :ref:`*required* <User Guide Required Outputs>`,
37-
so the scheduler will retain all other tasks with :term:`final status`
38-
in the :term:`n=0 window <n-window>`.
37+
so in the absence of suicide triggers the scheduler will retain other
38+
:term:`final status` tasks in the :term:`n=0 window <n-window>` to stall the
39+
workflow.
3940

4041
- (in Cylc 8, **all** outputs are *required* unless marked as
4142
:ref:`*optional* <User Guide Optional Outputs>` by the new ``?`` syntax)

src/glossary.rst

Lines changed: 53 additions & 45 deletions
Original file line numberDiff line numberDiff line change
@@ -70,7 +70,6 @@ Glossary
7070

7171
active
7272
active task
73-
active tasks
7473
An active task is a task which is near ready to run, in the process of
7574
running, or which requires user intervention. These are all the tasks
7675
being actively managed by the scheduler at this point in the run.
@@ -98,22 +97,19 @@ Glossary
9897

9998

10099
active cycle
101-
active cycle point
102-
A cycle is active if it contains any :term:`active tasks <active task>`.
100+
A cycle point is active if it contains any :term:`active tasks <active task>`.
103101

104102
Active cycles are counted towards the :term:`runahead limit`.
105103

106104

107-
active window
108-
The active window is the ``n=0`` core of the :term:`n-window`.
109-
It contains the current :term:`active tasks`.
110-
111-
112105
window
113106
n-window
114-
This is a :term:`graph`-based window or view of the workflow at runtime,
115-
including tasks out to ``n`` graph edges from current
116-
:term:`active tasks<active task>`. The active tasks are in ``n=0``.
107+
active window
108+
The UI provides a :term:`graph`-based window or view of the workflow at
109+
runtime, including tasks out to ``n`` graph edges from current
110+
:term:`active tasks <active task>`.
111+
112+
Active tasks form the ``n=0`` core of the :term:`n-window`.
117113

118114
.. seealso::
119115

@@ -572,7 +568,7 @@ Glossary
572568

573569
Expired is a :term:`final status` - an expired task will not run
574570
even if its prerequisites get satisfied.
575-
571+
576572
The associated ``:expire`` :term:`output <task output>` can be used to
577573
trigger other tasks. It must be marked as an :term:`optional output`,
578574
i.e. expiry cannot be :term:`required <required output>`.
@@ -1386,10 +1382,23 @@ Glossary
13861382
- ``:succeeded``, or ``:failed``
13871383

13881384

1385+
prerequisite
1386+
task prerequisite
1387+
A task's prerequisites are the :term:`outputs <task outputs>` of
1388+
parent tasks upstream in the graph that must be completed before
1389+
it can run. Tasks are said to "depend on" their prerequisites, hence
1390+
the term :term:`dependency graph <graph>`.
1391+
1392+
Tasks can depend on :term:`external triggers <xtrigger>` as well as
1393+
the outputs of other, upstream, tasks.
1394+
1395+
1396+
output
13891397
task output
13901398
Task outputs mark the progression of a :term:`task` from waiting (for
1391-
prerequisites to be satisfied) through to success or failure at run
1392-
time. Tasks can trigger off upstream task outputs in the :term:`graph`.
1399+
:term:`prerequisites <prerequisite>` to be satisfied) through to success
1400+
or failure at run time. Tasks can trigger off of upstream task outputs in
1401+
the :term:`graph`.
13931402

13941403
Outputs are written as ``task-name:output`` in the :term:`graph`, and can
13951404
be :term:`required <required output>` or :term:`optional <optional output>`.
@@ -1411,8 +1420,6 @@ Glossary
14111420
foo => bar # means foo:succeeded => bar
14121421
14131422
1414-
complete
1415-
incomplete
14161423
output completion
14171424
output completion condition
14181425
A task's outputs are *complete* if its *output completion condition*
@@ -1428,14 +1435,12 @@ Glossary
14281435
submit-fails.
14291436
- Or, if expiry is optional, then the outputs are complete if it expires.
14301437

1431-
Tasks that achieve a :term:`final status` with
1432-
:term:`complete outputs <output completion>` have done their job in the
1433-
workflow, allowing the workflow to move on.
1434-
1435-
Tasks that achieve a final status with
1436-
:term:`incomplete outputs <output completion>`
1437-
are retained in :term:`n=0 <n-window>` pending user
1438-
intervention, and will :term:`stall` the workflow.
1438+
Tasks that achieve a :term:`final status` with complete outputs have done
1439+
their job, allowing the workflow to move on.
1440+
1441+
Tasks that achieve a final status with incomplete outputs are retained in
1442+
:term:`n=0 <n-window>` pending user intervention, and will :term:`stall`
1443+
the workflow.
14391444

14401445

14411446
dependence
@@ -1579,15 +1584,14 @@ Glossary
15791584

15801585
stall
15811586
stalled workflow
1582-
stalled state
1583-
In a stalled workflow, there is nothing more the scheduler can do,
1584-
but it stays up for a while awaiting manual intervention because
1585-
the presence of :term:`incomplete <output completion>` tasks, or
1586-
partially satisfied tasks, in the :term:`n=0 window <n-window>`
1587-
implies that the workflow has not successfully
1588-
run to :term:`completion <workflow completion>`.
1587+
A stalled workflow has not run to :term:`completion <workflow completion>`
1588+
but cannot continue without manual intervention.
15891589

1590-
Stalls are usually, but not always, caused by unexpected task failures:
1590+
A stalled scheduler stays alive for a configurable timeout period
1591+
pending manual intervention. If it shuts down (on the stall timeout
1592+
or otherwise) it will remain in the stalled state on restart.
1593+
1594+
Stalls are usually caused by unexpected task failures:
15911595

15921596
.. digraph:: Example
15931597
:align: center
@@ -1607,7 +1611,7 @@ Glossary
16071611

16081612
suicide trigger
16091613
Suicide triggers remove :term:`tasks <task>` from the
1610-
:term:`n=0 window <n-window>` at runtime.
1614+
:term:`active window <n-window>` at runtime.
16111615

16121616
They are denoted by exclamation marks, and are triggered like normal
16131617
dependencies. For instance, the following suicide trigger will remove the
@@ -1695,7 +1699,7 @@ Glossary
16951699

16961700

16971701
flow front
1698-
:term:`Active tasks <active tasks>`, i.e. tasks in the
1702+
:term:`Active tasks <active task>`, i.e. tasks in the
16991703
:term:`n=0 window <n-window>`, with a common :term:`flow number`
17001704
comprise the active front of the flow.
17011705

@@ -1741,11 +1745,11 @@ Glossary
17411745
runahead limit
17421746
runahead
17431747
In a :term:`cycling workflow`, the runahead limit determines how far
1744-
ahead of the oldest :term:`active cycle point` the workflow is permitted
1748+
ahead of the oldest :term:`active cycle` the workflow is permitted
17451749
to run.
17461750

17471751
The "oldest active cycle point" is the first cycle in the workflow to contain
1748-
any :term:`active tasks` (e.g. running tasks).
1752+
any :term:`active tasks <active task>` (e.g. running tasks).
17491753

17501754
.. seealso::
17511755

@@ -1755,12 +1759,16 @@ Glossary
17551759

17561760

17571761
workflow completion
1758-
A workflow is complete if there are no more tasks to run according
1759-
to the graph, and there are no :term:`incomplete <output completion>`
1760-
tasks present in the :term:`n=0 window <n-window>`.
1761-
1762-
If the workflow is complete, the scheduler will automatically
1763-
:term:`shut down <shutdown>`.
1764-
1765-
If there are no more tasks to run, but there are incomplete tasks
1766-
present, the scheduler will :term:`stall` rather than shut down.
1762+
A workflow is complete, and the scheduler will automatically
1763+
:term:`shut down <shutdown>`, if no tasks remain in the
1764+
:term:`n=0 <n-window>`.
1765+
1766+
That is, all active tasks have finished, and no tasks remain waiting on
1767+
:term:`prerequisites <prerequisite>` or "external" constraints (such as
1768+
:term:`xtriggers <xtrigger>` or task :term:`hold`).
1769+
1770+
If no active tasks remain and all external constraints are satisfied,
1771+
but the n=0 window contains tasks waiting with partially satisfied
1772+
:term:`prerequisites`, or tasks with :term:`final status` and
1773+
:term:`incomplete outputs <output completion>`, then the workflow is
1774+
not complete and the scheduler will :term:`stall` pending manual intervention.

src/user-guide/running-workflows/reflow.rst

Lines changed: 6 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -128,9 +128,9 @@ Special Case: Triggering ``n=0`` Tasks
128128

129129
- Triggering a task with a submitted or running job has no effect
130130
(it is already triggered).
131-
- Triggering other :term:`active tasks <active task>`, including
132-
:term:`incomplete <output completion>` tasks with a :term:`final status`,
133-
queues them to run in the same flow.
131+
- Triggering other :term:`n=0 tasks <n-window>`, including tasks with
132+
:term:`incomplete outputs <output completion>` queues them to run in
133+
the flow that they already belong to.
134134

135135

136136
.. _flow-merging:
@@ -145,8 +145,9 @@ of itself (same task ID) already there, the two will merge and carry both
145145
Flow merging is necessary because :term:`active <active task>` task IDs
146146
must be unique.
147147

148-
If original task instance has a :term:`final status` but is
149-
:term:`incomplete <output completion>` the merged task will be reset to
148+
If the original task instance has a :term:`final status` (and has been retained
149+
in the :term:`n=0 window <n-window>` with
150+
:term:`incomplete outputs <output completion>`) the merged task will be reset to
150151
run again without manual intervention.
151152

152153

src/user-guide/running-workflows/workflow-completion.rst

Lines changed: 22 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -3,39 +3,37 @@
33
Workflow Completion
44
===================
55

6-
If there is nothing more to run (according to the graph) and there are no
7-
:term:`incomplete <output completion>` tasks present in :term:`n=0 <n-window>`
8-
the workflow is complete and the scheduler will shut down automatically.
6+
A workflow is complete, and the scheduler will automatically
7+
:term:`shut down <shutdown>`, if no tasks remain in the
8+
:term:`n=0 <n-window>`.
9+
10+
That is, all active tasks have finished, and no tasks remain waiting on
11+
:term:`prerequisites <prerequisite>` or "external" constraints (such as
12+
:term:`xtriggers <xtrigger>` or task :term:`hold`).
13+
14+
If no active tasks remain and all external constraints are satisfied,
15+
but the n=0 window contains tasks waiting with partially satisfied
16+
:term:`prerequisites`, or tasks with :term:`final status` and
17+
:term:`incomplete outputs <output completion>`, then the workflow is
18+
not complete and the scheduler will :term:`stall` pending manual intervention.
919

1020

1121
.. _scheduler stall:
1222

1323
Scheduler Stall
1424
===============
1525

16-
If there is nothing more to run, but there are
17-
:term:`incomplete <output completion>` tasks present in the
18-
:term:`n=0 window <n-window>` the workflow did not run to
19-
completion, so the scheduler will :term:`stall` and stay
20-
alive for 1 hour (by default) awaiting user intervention
21-
to allow the workflow to continue.
22-
23-
A stall can be caused by tasks with partially satisfied
24-
prerequisites or tasks that achieved a :term:`final status`
25-
with incomplete outputs. Partially satisfied prerequisites
26-
normally result from incomplete tasks with final status
27-
upstream in the graph.
28-
29-
Restarting a stalled workflow resets the stall timer.
30-
31-
32-
.. note::
26+
A stalled workflow has not run to :term:`completion <workflow completion>`
27+
but cannot continue without manual intervention.
3328

34-
Partially satisfied prerequisites can also cause a stall. If ``a & b => c``,
35-
and ``a`` succeeds but ``b`` never even runs, the scheduler will take
36-
partial completion of ``c``'s prerequisites as a sign that the workflow did
37-
not run to completion as expected.
29+
A stalled scheduler stays alive for a configurable timeout period
30+
pending manual intervention. If it shuts down (on the stall timeout
31+
or otherwise) it will remain in the stalled state on restart.
3832

33+
Stalls are often caused by unexpected task failures, either directly (tasks
34+
with :term:`final status` and :term:`incomplete outputs <output completion>`)
35+
or indirectly (tasks with partially satisfed prerequisites, downstream of an
36+
unexpected failure).
3937

4038
.. warning::
4139

src/user-guide/writing-workflows/scheduling.rst

Lines changed: 7 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -1506,11 +1506,9 @@ Family triggers are also provided for task expiry:
15061506

15071507
.. code-block:: cylc-graph
15081508
1509-
R1 = """
1510-
foo:expire? => bar
1511-
FAM:expire-all? => baz
1512-
FAM:expire-any? => qux
1513-
"""
1509+
foo:expire? => bar
1510+
FAM:expire-all? => baz
1511+
FAM:expire-any? => qux
15141512
15151513
Task expiration is configured with
15161514
:cylc:conf:`[scheduling][special tasks]clock-expire` using a syntax like
@@ -1707,16 +1705,12 @@ is optional" and that a non-optional version of the trigger makes sense.
17071705
.. code-block:: cylc-graph
17081706
17091707
# Good:
1710-
R1 = """
1711-
foo:finish => bar
1712-
foo? => baz
1713-
"""
1708+
foo:finish => bar
1709+
foo? => baz
17141710
17151711
# Error:
1716-
R1 = """
1717-
foo:finish => bar
1718-
foo => baz # ERROR : foo:succeed must be optional here!
1719-
"""
1712+
foo:finish => bar
1713+
foo => baz # ERROR : foo:succeed must be optional here!
17201714
17211715
.. _optional outputs.family triggers:
17221716

0 commit comments

Comments
 (0)