Skip to content

Commit 1147b83

Browse files
authored
Merge pull request #824 from hjoliver/disambiguate-active
Disambiguate the word "active".
2 parents 0209f7f + 2ee3159 commit 1147b83

File tree

13 files changed

+136
-164
lines changed

13 files changed

+136
-164
lines changed

src/7-to-8/major-changes/scheduling.rst

Lines changed: 7 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -23,21 +23,22 @@ Cylc can manage infinite workflows of repeating tasks:
2323

2424
Cylc 8 has a new scheduling algorithm that:
2525

26-
- Is much more efficient because it only has to manage active tasks
26+
- Is much more efficient because it only has to manage
27+
:term:`active tasks <active task>`.
2728

28-
- waiting tasks are not pre-spawned before they are needed
29-
- succeeded tasks are not kept across the active task window
30-
- no costly indiscriminate dependency matching is done
29+
- Tasks are not loaded into memory before they are needed.
30+
- Tasks are not retained in memory once they complete.
31+
- There is no costly indiscriminate dependency matching.
3132
- Distinguishes between :term:`optional <optional output>` and
3233
:term:`required <required output>` task outputs, to support:
3334

3435
- :term:`graph branching` without :term:`suicide triggers <suicide trigger>`
35-
- correct diagnosis of :term:`workflow completion`
36+
- correct diagnosis of :ref:`workflow completion`
3637
- Causes no implicit dependence on previous-instance job submit
3738

3839
- instances of same task can run out of cycle point order
3940
- the workflow will not unnecessarily stall downstream of failed tasks
40-
- Provides a sensible active-task based window on the evolving workflow
41+
- Provides a sensible activity-based window on the evolving workflow
4142

4243
- (to fully understand which tasks appeared in the Cylc 7 GUI you had to
4344
understand the scheduling algorithm)

src/7-to-8/summary.rst

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -134,7 +134,7 @@ intervention, which will :term:`stall` the workflow.
134134
Alternatively, outputs can be marked as :term:`optional <optional output>`,
135135
which allows :term:`optional graph branching <graph branching>`.
136136

137-
This allows the scheduler to correctly diagnose :term:`workflow completion`.
137+
This allows the scheduler to correctly diagnose :ref:`workflow completion`.
138138

139139

140140
Platform Awareness

src/glossary.rst

Lines changed: 37 additions & 67 deletions
Original file line numberDiff line numberDiff line change
@@ -68,50 +68,36 @@ Glossary
6868
Submit number also appears in the job log path so that job log files
6969
don't get overwritten.
7070

71-
72-
active
7371
active task
74-
An active task is a task which is near ready to run, in the process of
75-
running, or which requires user intervention. These are all the tasks
76-
being actively managed by the scheduler at this point in the run.
72+
Active tasks are those tasks held in memory to feed
73+
the scheduling algorithm. They form the ``n=0`` :term:`n-window`,
74+
so are always visible in the GUI.
7775

78-
Active tasks are:
76+
Active tasks include:
7977

80-
- Tasks which have some, but not all of their
81-
:term:`prerequisites <prerequisite>` satisfied.
82-
- ``waiting`` tasks, that are actively waiting on:
78+
- ``submitted`` and ``running`` tasks (i.e, tasks with active jobs)
79+
- ``preparing`` tasks in the job submission pipeline
80+
- ``waiting`` tasks that are nearly ready to run but:
8381

84-
- :term:`xtriggers <xtrigger>`.
85-
- :ref:`internal queues <InternalQueues>`
86-
- :ref:`runahead limit <RunaheadLimit>`
82+
- have partially satisfied :term:`prerequisites <prerequisite>`
83+
- or are waiting on :term:`xtriggers <xtrigger>`,
84+
:ref:`internal queues <InternalQueues>`, or the :ref:`runahead limit <RunaheadLimit>`
8785

88-
- ``preparing`` tasks - i.e. tasks in the process of submitting jobs
89-
- ``submitted`` and ``running`` tasks - i.e. those with active jobs
9086
- tasks that reached a :term:`final status` without completing their
9187
:term:`required outputs <required output>`
9288
(e.g. a task failed where success was required).
9389

94-
Active tasks are in the ``n=0`` :term:`window <n-window>` which means they
95-
will always be displayed in the GUI and Tui.
96-
97-
The distinction between active and non-active tasks is important for
98-
the computing of the :term:`runahead limit`.
9990

91+
n-window
92+
The GUI provides a view of the workflow extending ``n`` graph edges out
93+
from :term:`active tasks <active task>` - which form the ``n=0``
94+
window. The default n-window extent is ``n=1``.
10095

10196
active cycle
102-
A cycle point is active if it contains any :term:`active tasks <active task>`.
103-
104-
Active cycles are counted towards the :term:`runahead limit`.
105-
106-
107-
window
108-
n-window
109-
active window
110-
The GUI provides a :term:`graph`-based window or view of the workflow at
111-
runtime, including tasks out to ``n`` graph edges from current
97+
A cycle point is considered to be active if it contains any
11298
:term:`active tasks <active task>`.
11399

114-
Active tasks form the ``n=0`` window.
100+
Active cycles count toward the :term:`runahead limit`.
115101

116102
.. seealso::
117103

@@ -358,7 +344,6 @@ Glossary
358344
"1/bar" -> "2/bar" -> "3/bar"
359345

360346

361-
362347
.. seealso::
363348

364349
* :ref:`tutorial-integer-cycling`
@@ -376,12 +361,15 @@ Glossary
376361

377362

378363
cycle point
379-
The unique label given to tasks that belong to a particular :term:`cycle`.
364+
The label given to tasks that belong to a particular :term:`cycle`.
380365
For :term:`integer cycling` these will be integers, e.g. ``1``, ``2``,
381366
``3``, etc.
382367
For :term:`datetime cycling` they will be :term:`ISO 8601` datetimes,
383368
e.g. ``2000-01-01T00:00Z``.
384369

370+
Cylc can run multiple cycles at once, dependencies allowing, so each
371+
task instance has its own cycle point label.
372+
385373
.. seealso::
386374

387375
* :term:`initial cycle point`
@@ -993,7 +981,8 @@ Glossary
993981
This refers to starting a new instance of the Cylc :term:`scheduler`
994982
program to manage a particular :term:`workflow`. This can be from
995983
scratch, for installed workflows that haven't run previously, or to
996-
restart one that shut down prior to :term:`completion <workflow completion>`.
984+
restart one that shut down prior to
985+
:ref:`completion <workflow completion>`.
997986

998987
.. seealso::
999988

@@ -1170,7 +1159,7 @@ Glossary
11701159
stop
11711160
shutdown
11721161
A :term:`scheduler` can shut down on request, or automatically on
1173-
:term:`workflow completion`. The :term:`workflow` is then stopped and no
1162+
:ref:`workflow completion`. The :term:`workflow` is then stopped and no
11741163
further :term:`jobs <job>` will be submitted.
11751164

11761165
By default, the scheduler waits for any submitted or running task
@@ -1216,9 +1205,10 @@ Glossary
12161205
workflow :term:`source directory` before reload, rather than made by
12171206
editing the installed files directly.
12181207

1219-
:ref:`RemoteInit` will be redone for each job platform, when the first job is submitted there after a reload.
1208+
:ref:`RemoteInit` will be redone for each job platform, when the first
1209+
job is submitted there after a reload.
12201210

1221-
Any :term:`task` that is :term:`active <active task>` at reload
1211+
Any task that is :term:`active <active task>` at reload
12221212
will continue with its pre-reload configuration.
12231213
The next instance of the task (at the next cycle point)
12241214
will adopt the new configuration.
@@ -1433,7 +1423,7 @@ Glossary
14331423
- Or, if expiry is optional, then the outputs are complete if it expires.
14341424

14351425
Tasks that achieve a :term:`final status` with complete outputs have done
1436-
their job, allowing the workflow to move on.
1426+
their job in the workflow, allowing the scheduler to move on.
14371427

14381428
Tasks that achieve a final status with incomplete outputs are retained in
14391429
:term:`n=0 <n-window>` pending user intervention, and will :term:`stall`
@@ -1587,7 +1577,7 @@ Glossary
15871577

15881578
stall
15891579
stalled workflow
1590-
A stalled workflow has not run to :term:`completion <workflow completion>`
1580+
A stalled workflow has not :ref:`run to completion <workflow completion>`
15911581
but cannot continue without manual intervention.
15921582

15931583
A stalled scheduler stays alive for a configurable timeout period
@@ -1613,8 +1603,7 @@ Glossary
16131603

16141604

16151605
suicide trigger
1616-
Suicide triggers remove :term:`tasks <task>` from the
1617-
:term:`active window <n-window>` at runtime.
1606+
Suicide triggers remove tasks from the :term:`n=0 window <n-window>`.
16181607

16191608
They are denoted by exclamation marks, and are triggered like normal
16201609
dependencies. For instance, the following suicide trigger will remove the
@@ -1702,16 +1691,14 @@ Glossary
17021691

17031692

17041693
flow front
1705-
:term:`Active tasks <active task>`, i.e. tasks in the
1706-
:term:`n=0 window <n-window>`, with a common :term:`flow number`
1694+
:term:`Active tasks <active task>` with a common :term:`flow number`
17071695
comprise the active front of the flow.
17081696

17091697

17101698
flow merge
1711-
When a :term:`flow` tries to spawn a child task and finds an instance
1712-
with the same task ID already exists in the ``n=0`` :term:`active
1713-
window`, one merged task will carry both flow numbers forward.
1714-
1699+
If a spawned task encounters another :term:`active task` with the same
1700+
task ID, the two instances will merge and carry both :term:`flow`
1701+
numbers forward.
17151702

17161703
event
17171704
An event is a milestone in the lifecycle of a :term:`workflow` or
@@ -1747,12 +1734,9 @@ Glossary
17471734

17481735
runahead limit
17491736
runahead
1750-
In a :term:`cycling workflow`, the runahead limit determines how far
1751-
ahead of the oldest :term:`active cycle` the workflow is permitted
1752-
to run.
1753-
1754-
The "oldest active cycle point" is the first cycle in the workflow to contain
1755-
any :term:`active tasks <active task>` (e.g. running tasks).
1737+
In a :term:`cycling workflow` the runahead limit determines how
1738+
far ahead, in :term:`cycle points <cycle point>`, activity can
1739+
extend beyond the earliest submitted or running tasks.
17561740

17571741
.. seealso::
17581742

@@ -1761,26 +1745,12 @@ Glossary
17611745
* :term:`active cycle`
17621746

17631747

1764-
workflow completion
1765-
A workflow is complete, and the scheduler will automatically
1766-
:term:`shut down <shutdown>`, if no tasks remain in the
1767-
:term:`n=0 <n-window>`.
1768-
1769-
That is, all active tasks have finished, and no tasks remain waiting on
1770-
:term:`prerequisites <prerequisite>` or "external" constraints (such as
1771-
:term:`xtriggers <xtrigger>` or task :term:`hold`).
1772-
1773-
If no active tasks remain and all external constraints are satisfied,
1774-
but the n=0 window contains tasks waiting with partially satisfied
1775-
:term:`prerequisites <prerequisite>`, or tasks with :term:`final status` and
1776-
:term:`incomplete outputs <output completion>`, then the workflow is
1777-
not complete and the scheduler will :term:`stall` pending manual intervention.
1778-
17791748
dummy task
17801749
A task which runs a trivially simple script such as ``sleep 1``,
17811750
``exit 0`` or ``true``, or which uses :ref:`task-run-modes.skip`
17821751
to avoid running a script at all.
17831752

1753+
17841754
dummy mode
17851755
A workflow run mode that replaces all tasks with :term:`dummy tasks <dummy task>`.
17861756
See :ref:`workflow-run-modes.dummy`.

src/reference/changes.rst

Lines changed: 7 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -245,17 +245,14 @@ N-Window selector in the GUI
245245

246246
The :term:`n-window` determines how much of a workflow is visible in the GUI / Tui.
247247

248-
The ``n=0`` window contains only the active tasks
249-
(i.e. queued, preparing, submitted or running tasks).
248+
You can change the n-window extent in the GUI with a toolbar button, to display
249+
more or less of the graph around current :term:`active tasks <active task>`.
250+
This affects all GUI views equally, not just the graph view.
250251

251-
The ``n=1`` window, also contains tasks one "edge" out from active tasks
252-
(i.e. the tasks immediately upstream or downstream of active tasks).
252+
The ``n=0`` window contains only the active tasks.
253253

254-
The ``n=2`` window, also contains tasks two "edges" out from active tasks,
255-
and so on.
256-
257-
It is now possible to change the window extent in the GUI via a button in the
258-
toolbar allowing you to see tasks further back in the workflow's history.
254+
The ``n=1`` window displays tasks out to one graph edge around the active
255+
tasks; ``n=2`` out to two graph edges; and so on.
259256

260257
.. image:: changes/gui-n-window-selector.gif
261258
:width: 100%
@@ -306,7 +303,7 @@ When a task achieves a final status, its outputs are validated against a "comple
306303
expression" to ensure that it has produced all of its
307304
:term:`required outputs <required output>`.
308305
If a task fails this validation check it is said to have "incomplete outputs"
309-
and will be retained in the :term:`active window` pending user intervention.
306+
and will be retained in the :term:`n=0 window <n-window>` pending user intervention.
310307

311308
This completion expression is generated automatically from the graph.
312309
By default, tasks are expected to succeed, if you register any additional

src/user-guide/interventions/index.rst

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -558,7 +558,8 @@ Remove Tasks
558558

559559
The removed task will be greyed out but it might not
560560
disappear from view because the GUI displays all tasks
561-
in a graph-based window around current active tasks.
561+
in a graph-based :term:`n-window` surrounding current
562+
:term:`active tasks <active task>`.
562563

563564

564565
.. tab-item:: CLI

src/user-guide/running-workflows/reflow.rst

Lines changed: 20 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -26,8 +26,8 @@ See :ref:`below<flow-trigger-use-cases>` for uses, and an
2626
:ref:`example<new-flow-example>`, of concurrent flows.
2727

2828
.. note::
29-
Flows :ref:`merge<flow-merging>` where (and if) tasks collide in the ``n=0``
30-
:term:`active window`. Downstream of a merge, tasks are considered to belong
29+
Flows :ref:`merge<flow-merging>` where (and if) tasks collide in the
30+
:term:`n=0 window <n-window>`. Downstream of a merge, tasks are considered to belong
3131
to all of their constituent flows.
3232

3333

@@ -92,7 +92,7 @@ Triggering in Specific Flows
9292

9393
The result is like the default above, except that tasks in the new front
9494
belong only to the specified flow(s), regardless of which flows are
95-
:term:`active` at triggering time.
95+
active at triggering time.
9696

9797
Triggering a New Flow
9898
``cylc trigger --flow=new ID``
@@ -117,20 +117,21 @@ Triggering with No Active Flows
117117
``cylc trigger [--wait] ID``
118118

119119
By default, triggered tasks will be given the flow numbers of the most
120-
recent active task. This can happen, for instance, if you restart a
121-
completed workflow and then trigger a task in it. The result will be the
122-
same as if you had triggered the task just before the workflow completed.
120+
recent :term:`active tasks <active task>`. This can happen, for instance,
121+
if you restart an already-completed workflow and then manually trigger a
122+
task in it. The task's flow number will be the same as if you
123+
had triggered it just before the workflow completed.
123124

124-
Special Case: Triggering ``n=0`` Tasks
125-
Tasks in the ``n=0`` window are :term:`active tasks <active task>`.
126-
Their flow membership is already determined - that of
127-
the parent tasks that spawned them.
125+
Special Case: Triggering ``n=0`` (:term:`active tasks <active task>`)
126+
Active tasks already have flow membership assigned.
127+
Flow numbers are inherited, on entering the active wndow, from parent
128+
(upstream) tasks in the graph.
128129

129130
- Triggering a task with a submitted or running job has no effect
130131
(it is already triggered).
131-
- Triggering other :term:`n=0 tasks <n-window>`, including tasks with
132-
:term:`incomplete outputs <output completion>` queues them to run in
133-
the flow that they already belong to.
132+
- Triggering other :term:`active tasks <active task>`, including those with
133+
:term:`incomplete outputs <output completion>`, queues them to run with
134+
their existing flow numbers.
134135

135136

136137
.. _flow-merging:
@@ -142,8 +143,9 @@ If a task spawning into the :term:`n=0 window <n-window>` finds another instance
142143
of itself (same task ID) already there, the two will merge and carry both
143144
(sets of) flow numbers forward. Downstream tasks will belong to both flows.
144145

145-
Flow merging is necessary because :term:`active <active task>` task IDs
146-
must be unique.
146+
Flows merge because every :term:`active task` must have a unique ID. However,
147+
merging is also useful: it allows a simpler single flow to continue downstream of
148+
multi-flow interventions.
147149

148150
If the original task instance has a :term:`final status` (and has been retained
149151
in the :term:`n=0 window <n-window>` with
@@ -157,9 +159,9 @@ Stopping Flows
157159
By default, ``cylc stop`` halts the workflow and shuts the scheduler down.
158160

159161
It can also stop specific flows: ``cylc stop --flow=N`` removes the flow number
160-
``N`` from :term:`active tasks <active task>`. Tasks that have no flow
161-
numbers left as a result do not spawn children at all. If there are no active
162-
flows left, the scheduler shuts down.
162+
``N`` from :term:`active tasks <active task>`. Tasks with no remaining flow
163+
numbers will not spawn downstream activity. If there are
164+
no active flows left, the scheduler shuts down.
163165

164166
.. TODO update this section post https://github.com/cylc/cylc-flow/issues/4741
165167

0 commit comments

Comments
 (0)