Some doc updates for 8.3.0.dev changes.

Author: hjoliver

src/7-to-8/major-changes/compatibility-mode.rst

@@ -34,7 +34,8 @@ The ``suite.rc`` filename triggers a backward compatibility mode in which:
     branching)
 
 - only ``succeeded`` task outputs are :ref:`*required* <User Guide Required Outputs>`,
-  meaning the scheduler will retain tasks that do not succeed as incomplete
+  so the scheduler will retain all non-succeeded :term:`finished tasks <finished task>`
+  in the :term:`n=0 window <n-window>`.
 
   - (in Cylc 8, **all** outputs are *required* unless marked as
     :ref:`*optional* <User Guide Optional Outputs>` by the new ``?`` syntax)

src/7-to-8/summary.rst

@@ -121,14 +121,16 @@ Optional and Required Task Outputs
    * User Guide::ref:`User Guide Optional Outputs`
 
 By default, all Cylc 8 tasks are required to succeed - i.e., success is
-a :term:`required output`. Otherwise they will be marked
-as :term:`incomplete tasks<incomplete task>` needing user intervention.
-In a workflow with incomplete tasks, if there is nothing left to do, the
-scheduler will :term:`stall` rather than shut down.
-
-Alternatively, task outputs can be marked as :term:`optional <optional output>`.
-This supports :term:`graph branching` and it allows the scheduler to
-correctly diagnose :term:`workflow completion`.
+a :term:`required output`. Tasks that :term:`finish <finished task>`
+without completing their required outputs get retained in the
+:term:`n=0 window <n-window>` pending user intervention, which
+will :term:`stall` the workflow.
+
+Alternatively, outputs can be marked as :term:`optional <optional output>`,
+which allows :term:`optional graph branching <graph branching>`.
+
+This allows the scheduler to correctly diagnose :term:`workflow completion`.
+
 
 Platform Awareness
 ------------------

src/glossary.rst

@@ -70,8 +70,10 @@ Glossary
 
    active
    active task
+   active tasks
       An active task is a task which is near ready to run, in the process of
-      running, or which requires user intervention.
+      running, or which requires user intervention. These are all the tasks
+      being actively managed by the scheduler at this point in the run. 
 
       Active tasks are:
 
@@ -84,9 +86,9 @@ Glossary
 
       - ``preparing`` tasks - i.e. tasks in the process of submitting jobs
       - ``submitted`` and ``running`` tasks - i.e. those with active jobs
-      - :term: `incomplete` tasks which have completed without satisfying all of their
-        :term:`required outputs <required output>`, (e.g. a failed task
-        where success was required).
+      - tasks that finished without completing their
+        :term:`required outputs <required output>`
+        (e.g. a task failed where success was required).
 
       Active tasks are in the ``n=0`` :term:`window <n-window>` which means they
       will always be displayed in the GUI and Tui.
@@ -96,18 +98,22 @@ Glossary
 
 
    active cycle
+   active cycle point
       A cycle is active if it contains any :term:`active tasks <active task>`.
 
       Active cycles are counted towards the :term:`runahead limit`.
 
 
+   active window
+      The active window is the ``n=0`` core of the :term:`n-window`.
+      It contains the current :term:`active tasks`.
+      
+
    window
    n-window
-   active window
-   active task pool
       This is a :term:`graph`-based window or view of the workflow at runtime,
-      including tasks out to ``n`` graph edges from current :term:`active
-      tasks<active task>`. The *active window* is ``n=0``.
+      including tasks out to ``n`` graph edges from current 
+      :term:`active tasks<active task>`. The active tasks are in ``n=0``.
 
       .. seealso::
 
@@ -210,8 +216,8 @@ Glossary
    queue
    internal queue
       Internal queues (so called to distinguish them from external batch
-      queueing systems) allow you to limit how many :term:`tasks <task>` can be
-      active (submitted or running) at once, across defined groups of tasks.
+      queueing systems) limit how many :term:`jobs <job>` can be
+      active at once, across defined groups of tasks.
 
       Use queues prevent large or busy workflows from swamping their
       :term:`job platforms <job platform>` with too many jobs at once.
@@ -559,6 +565,23 @@ Glossary
          * :term:`exact datetime unit`
 
 
+   clock expire
+   expired task
+      Tasks in :term:`datetime cycling` workflows can be configured to *expire*
+      if the :term:`wallclock time` exceeds some offset from the cycle point.
+
+      Expired is a :term:`final task status` - an expired task will not run
+      even if its prerequisites get satisfied.
+      
+      The associated ``:expire`` :term:`output <task output>` can be used to
+      trigger other tasks. It must be marked as an :term:`optional output`,
+      i.e. expiry cannot be :term:`required <required output>`.
+
+      .. seealso::
+
+         * :ref:`Cylc User Guide <ClockExpireTasks>`
+
+
    clock trigger
       Clock triggers connect cycle points to the :term:`wallclock time`, in
       :term:`datetime cycling` workflows. Tasks that depend on a clock trigger
@@ -839,8 +862,9 @@ Glossary
 
 
    job
-      Jobs are real processes that perform :term:`tasks <task>` in a
-      :term:`workflow`. In Cylc, they are implemented by :term:`job scripts
+      Jobs are the real processes that run on a computer to realise
+      :term:`tasks <task>` in a :term:`workflow`. In Cylc, they are
+      implemented by :term:`job scripts
       <job script>` prepared by the :term:`scheduler`.
 
 
@@ -1195,8 +1219,9 @@ Glossary
 
       :ref:`RemoteInit` will be redone for each job platform, when the first job is submitted there after a reload.
 
-      Any :term:`task` that is active at reload will continue with its
-      pre-reload configuration. It's next instance (at the next cycle point)
+      Any :term:`task` that is :term:`active <active task>` at reload
+      will continue with its pre-reload configuration.
+      The next instance of the task (at the next cycle point)
       will adopt the new configuration.
 
       Reloading changes is safe providing they don't affect the
@@ -1346,10 +1371,18 @@ Glossary
          * :ref:`Cylc User Guide <FamilyTriggers>`
 
 
+   final task status
+   finished task
+      A finished task has achieved a final task status, i.e., expired,
+      submit-failed, succeeded, or failed. Finished tasks cannot change
+      state again except by manual intervention (e.g. by retriggering).
+
+
    standard output
      Every :term:`task` has a set of standard :term:`outputs <task output>`
      that trigger :term:`task state` changes:
 
+      - ``:expired```
       - ``:submitted``, or ``:submit-failed``
       - ``:started``
       - ``:succeeded``, or ``:failed``
@@ -1358,8 +1391,7 @@ Glossary
    task output
       Task outputs mark the progression of a :term:`task` from waiting (for
       prerequisites to be satisfied) through to success or failure at run
-      time. Downstream tasks can trigger off of the outputs of other tasks, as
-      determined by the :term:`graph`.
+      time. Tasks can trigger off upstream task outputs in the :term:`graph`.
 
       Outputs are written as ``task-name:output`` in the :term:`graph`, and can
       be :term:`required <required output>` or :term:`optional <optional output>`.
@@ -1381,6 +1413,30 @@ Glossary
          foo => bar  # means foo:succeeded => bar
 
 
+   output completion
+   output completion condition
+      A task's outputs are *complete* if its *output completion condition* 
+      is satisfied.
+
+      The completion condition can be defined by the user, or otherwise
+      it is automatically generated according to the rules:
+
+      - All :term:`required outputs <required output>` (referenced in the graph)
+        must be completed.
+      - Or, if success is optional, then the outputs are complete if it fails.
+      - Or, if submission is optional, then the outputs are complete if it
+        submit-fails.
+      - Or, if expiry is optional, then the outputs are complete if it expires.
+
+      Tasks that :term:`finish <finished task>` with
+      :term:`complete outputs <output completion>`
+      have done their job, allowing the workflow to move on.
+      
+      Tasks that finish with :term:`incomplete outputs <output completion>`
+      are retained in :term:`n=0 <n-window>` pending user
+      intervention, and will :term:`stall` the workflow.
+
+
    dependence
    dependency
       Dependencies in the :term:`graph` show how :term:`tasks <task>` depend on
@@ -1510,38 +1566,26 @@ Glossary
    required output
    expected output
       Task outputs that are not marked as :term:`optional <optional output>`
-      in the :term:`graph` are required to be completed at runtime. If not, the
-      :term:`scheduler` retains the task as :term:`incomplete` pending user
-      intervention.
+      in the :term:`graph` must be completed at runtime. If a task
+      :term:`finishes <finished task>` without completing its required
+      outputs, the :term:`scheduler` will keep it in the
+      :term:`n=0 window <n-window>` pending user intervention.
 
       .. seealso::
 
          * :ref:`Cylc User Guide <required outputs>`
 
 
-   incomplete
-   incomplete task
-      Incomplete tasks are :term:`tasks <task>` that finish (succeed or fail)
-      without completing all :term:`required outputs <required output>`. They
-      are retained by the :term:`scheduler` in the :term:`n=0 window
-      <n-window>` pending user intervention, and will cause a :term:`stall`
-      if there are no more tasks to run.
-
-      .. seealso::
-
-         * :term:`optional output`
-         * :ref:`Cylc User Guide <incomplete tasks>`
-
-
    stall
    stalled workflow
    stalled state
-      If there are no more tasks to run according to the :term:`graph`, but
-      :term:`incomplete tasks <incomplete task>` are present, the
-      :term:`scheduler` will stall and stay up for a time instead of
-      shutting down with the workflow :term:`complete <workflow completion>`.
+      In a stalled workflow, there is nothing more the scheduler can do,
+      but it stays up for a while awaiting manual intervention because
+      the presence of :term:`incomplete <output completion>` tasks in
+      the :term:`n=0 window <n-window>` implies that the workflow has not successfully
+      run to :term:`completion <workflow completion>`.
 
-      Stalls are usually caused by unexpected task failures:
+      Stalls are usually, but not always, caused by unexpected task failures:
 
       .. digraph:: Example
          :align: center
@@ -1556,16 +1600,16 @@ Glossary
       run, but ``bar:fail`` was not marked as an :term:`optional output`.
 
       User intervention is required to fix a stall, e.g. by retriggering
-      incomplete tasks after fixing the problems that caused them to fail.
+      tasks after fixing the problems that caused them to fail.
 
 
    suicide trigger
-      Suicide triggers remove :term:`tasks <task>` from the :term:`scheduler's
-      <scheduler>` active (:term:`n=0 <n-window>`) window at runtime.
+      Suicide triggers remove :term:`tasks <task>` from the 
+      :term:`n=0 window <n-window>` at runtime.
 
       They are denoted by exclamation marks, and are triggered like normal
       dependencies. For instance, the following suicide trigger will remove the
-      task ``bar`` from the active window if ``foo`` succeeds:
+      task ``bar`` if ``foo`` succeeds:
 
       .. code-block:: cylc-graph
 
@@ -1649,15 +1693,15 @@ Glossary
 
 
    flow front
-      Active (submitted or running) tasks, i.e. tasks in the ``n=0``
-      :term:`active window`, with a common :term:`flow number` comprise the
-      active front of that flow.
+      :term:`Active tasks <active tasks>`, i.e. tasks in the
+      :term:`n=0 window <n-window>`, with a common :term:`flow number`
+      comprise the active front of the flow.
 
 
    flow merge
       When a :term:`flow` tries to spawn a child task and finds an instance
       with the same task ID already exists in the ``n=0`` :term:`active
-      window`, the one active task will carry both flow numbers forward.
+      window`, one merged task will carry both flow numbers forward.
 
 
    event
@@ -1695,11 +1739,11 @@ Glossary
    runahead limit
    runahead
       In a :term:`cycling workflow`, the runahead limit determines how far
-      ahead of the oldest :ref:`active cycle point` the workflow is permitted
+      ahead of the oldest :term:`active cycle point` the workflow is permitted
       to run.
 
       The "oldest active cycle point" is the first cycle in the workflow to contain
-      any :term:`active tasks <active task>` (e.g. running tasks).
+      any :term:`active tasks` (e.g. running tasks).
 
       .. seealso::
 
@@ -1709,12 +1753,12 @@ Glossary
 
 
    workflow completion
-      A workflow is deemed complete if there are no more tasks to run,
-      according to the graph, and there are no :term:`incomplete task
-      <incomplete task>` present.
+      A workflow is complete if there are no more tasks to run according
+      to the graph, and there are no :term:`incomplete <output completion>`
+      tasks present in the :term:`n=0 window <n-window>`.
 
-      If the workflow is complete, the scheduler will automatically :term:`shut
-      down <shutdown>`.
+      If the workflow is complete, the scheduler will automatically
+      :term:`shut down <shutdown>`.
 
       If there are no more tasks to run, but there are incomplete tasks
       present, the scheduler will :term:`stall` rather than shut down.

src/user-guide/running-workflows/reflow.rst

@@ -128,38 +128,26 @@ Special Case: Triggering ``n=0`` Tasks
 
    - Triggering a task with a submitted or running job has no effect
      (it is already triggered).
-   - Triggering other :term:`active tasks <active task>` e.g. (a waiting
-     task which is held) queues it to run in the same flow.
-   - Triggering an :term:`incomplete task` queues it to re-run in the same flow.
+   - Triggering other :term:`active tasks <active task>` (including
+     :term:`incomplete <output completion>`
+     :term:`finished <finished task>` tasks) queues them to run in the same flow.
 
 
 .. _flow-merging:
 
 Flow Merging in ``n=0``
 -----------------------
 
-If a task spawning into the ``n=0`` :term:`window` finds another instance
-of itself already there (i.e., same name and cycle point, different flow
-number) a single instance will carry both (sets of) flow numbers forward from
-that point. Downstream tasks belong to both flows.
+If a task spawning into the :term:`n=0 window <n-window>` finds another instance
+of itself (same task ID) already there, the two will merge and carry both
+(sets of) flow numbers forward. Downstream tasks will belong to both flows.
 
-Flow merging in ``n=0`` means flows are not completely independent. One flow
-might not be able to entirely overtake another because one or more of its tasks
-might merge in ``n=0``. Merging is necessary while task IDs - and associated
-log directory paths etc. - do not incorporate flow numbers, because task IDs
-must be unique in the :term:`active task pool`.
+Flow merging is necessary because :term:`active <active task>` task IDs
+must be unique.
 
-Merging with Incomplete tasks
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-:term:`Incomplete<incomplete>` tasks are retained in the active window in
-expectation of retriggering to complete :term:`required outputs<required
-output>` and continue their flow.
-
-If another flow encounters an incomplete task (i.e. if another instance of the
-same task collides with it in the ``n=0`` :term:`active window`) the task will
-run again and carry both flow numbers forward if it successfully completes its
-required outputs.
+If original task instance is :term:`finished <finished task>` but
+:term:`incomplete <output completion>` the merged task will be reset to
+run again without manual intervention.
 
 
 Stopping Flows
@@ -168,7 +156,7 @@ Stopping Flows
 By default, ``cylc stop`` halts the workflow and shuts the scheduler down.
 
 It can also stop specific flows: ``cylc stop --flow=N`` removes the flow number
-``N`` from tasks in the :term:`active task pool`. Tasks that have no flow
+``N`` from :term:`active tasks <active task>`. Tasks that have no flow
 numbers left as a result do not spawn children at all. If there are no active
 flows left, the scheduler shuts down.