Merge pull request #708 from MetRonnie/fcm-make

Add workaround for mirror/fcm make tasks conflicting with symlink dirs (cherry-pick from master)

Author: oliver-sanders

.github/workflows/test.yml

@@ -63,14 +63,10 @@ jobs:
           metomi_rose: true
           metomi_rose_opts: ''
 
-      - name: install eslint
-        run: |
-          npm install -g eslint
-
       - name: lint
         run: |
           flake8
-          eslint .
+          npx eslint@v8 .
 
       - name: (debug only) list language dictionaries
         if: runner.debug

src/7-to-8/major-changes/cylc-install.rst

@@ -114,6 +114,34 @@ use :cylc:conf:`flow.cylc[scheduler]install`.
 
 See :ref:`the user guide <RemoteInit>` for more details.
 
+.. warning::
+
+   If you have tasks that mirror/copy the run directory to a remote platform
+   (such as `FCM make <FCM_>`__ tasks), this can cause conflicts with
+   :ref:`symlink directory setup <SymlinkDirs>`.
+
+   You can find out if symlink directories are configured for the platform by
+   running::
+
+      cylc config -i '[install][symlink dirs][<platform-name>]'
+
+   The recommended workaround is to use a "dummy" task that runs on the
+   particular platform before any such mirror tasks in order to setup symlink
+   directories, but without running anything.
+
+   For example:
+
+   .. code-block:: cylc
+
+      [scheduling]
+          [[graph]]
+              R1 = hpc_init => fcm_make
+
+      [runtime]
+          [[hpc_init]]
+              platform = <platform-name>
+              script = true
+
 
 Migrating From ``rose suite-run``
 ---------------------------------

src/reference/config/writing-platform-configs.rst

@@ -112,7 +112,7 @@ require remote initialization.
            install target = localhost
 
 Our Cylc scheduler does not have a job runner defined. Any job submitted to
-this ``localhost`` platform will run as a background job. Users can now set 
+this ``localhost`` platform will run as a background job. Users can now set
 :cylc:conf:`flow.cylc[runtime][<namespace>]platform` = ``pbs_cluster`` to run
 pbs jobs.
 
@@ -280,15 +280,17 @@ the login nodes, without the concern of selecting a specific platform.
 
 Symlinking Directories
 ----------------------
+
 To minimize the disk space used by ``~/cylc-run``, set
-:cylc:conf:`global.cylc[install][symlink dirs]`.
-The entire workflow directory can be symlinked, using the config item ``run`` 
-The following sub-directories  are also available for configuration:
+:cylc:conf:`global.cylc[install][symlink dirs]` to offload files onto other
+locations. The entire :term:`run directory` can be symlinked, as well as
+certain sub-directories.
 
-   * log
-   * share
-   * share/cycle
-   * work
+* ``run`` - the :term:`run directory` itself
+* ``log``
+* ``share`` (see :term:`share directory`)
+* ``share/cycle`` (typically used by `Rose`_ tasks)
+* ``work`` (see :term:`work directory`)
 
 These should be configured per :term:`install target`.
 
@@ -303,7 +305,7 @@ you could write the following in ``global.cylc``:
            [[[localhost]]]
                log = /somewhere/else
 
-This would result in the following file structure on the Cylc Scheduler:
+This would result in the following file structure on the Cylc scheduler host:
 
 .. code-block:: none
 

src/user-guide/installing-workflows.rst

@@ -157,14 +157,6 @@ sub-directories as symlinks to other locations. This is a useful way of
 offloading data onto other drives to limit the disk space taken up by
 ``~/cylc-run``.
 
-Directories that can be individually symlinked are:
-
-* ``log``
-* ``share``
-* ``share/cycle``
-* ``work``
-* the :term:`run directory` itself
-
 The symlink targets are configured per install target in
 :cylc:conf:`global.cylc[install][symlink dirs]`. For more information see
 :ref:`SymlinkDirsSetup`