Skip to content

Commit 314293f

Browse files
authored
Add workaround for mirror/fcm make tasks conflicting with symlink dirs (#692)
Add workaround for mirror/fcm make tasks conflicting with symlink dirs * Tidy symlink dirs docs * Add to spellcheck dictionary
1 parent 6b189e9 commit 314293f

File tree

4 files changed

+43
-19
lines changed

4 files changed

+43
-19
lines changed

src/7-to-8/major-changes/cylc-install.rst

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -114,6 +114,36 @@ use :cylc:conf:`flow.cylc[scheduler]install`.
114114
115115
See :ref:`the user guide <RemoteInit>` for more details.
116116

117+
.. warning::
118+
119+
If you have tasks that mirror to a remote platform (such as
120+
`FCM make <FCM_>`__ tasks), this can cause conflicts with
121+
:ref:`symlink directory setup <SymlinkDirs>`.
122+
123+
You can find out if symlink directories are configured for the platform by
124+
running::
125+
126+
cylc config -i '[install][symlink dirs][<platform-name>]'
127+
128+
The recommended workaround is to use a "dummy" task that runs on the
129+
particular platform before any such mirror tasks in order to setup symlink
130+
directories, but without running anything.
131+
132+
For example:
133+
134+
.. code-block:: cylc
135+
136+
[scheduling]
137+
[[graph]]
138+
R1 = hpc_init => fcm_make
139+
140+
[runtime]
141+
[[hpc_init]]
142+
platform = <platform-name>
143+
script = true
144+
145+
146+
117147
118148
Migrating From ``rose suite-run``
119149
---------------------------------

src/dictionaries/words

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -173,6 +173,7 @@ subcommands
173173
subdir
174174
subdirectories
175175
subdirectory
176+
subprocess
176177
subprocesses
177178
subshell
178179
subshelled

src/reference/config/writing-platform-configs.rst

Lines changed: 12 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -112,7 +112,7 @@ require remote initialization.
112112
install target = localhost
113113
114114
Our Cylc scheduler does not have a job runner defined. Any job submitted to
115-
this ``localhost`` platform will run as a background job. Users can now set
115+
this ``localhost`` platform will run as a background job. Users can now set
116116
:cylc:conf:`flow.cylc[runtime][<namespace>]platform` = ``pbs_cluster`` to run
117117
pbs jobs.
118118

@@ -280,15 +280,17 @@ the login nodes, without the concern of selecting a specific platform.
280280

281281
Symlinking Directories
282282
----------------------
283+
283284
To minimize the disk space used by ``~/cylc-run``, set
284-
:cylc:conf:`global.cylc[install][symlink dirs]`.
285-
The entire workflow directory can be symlinked, using the config item ``run``
286-
The following sub-directories are also available for configuration:
285+
:cylc:conf:`global.cylc[install][symlink dirs]` to offload files onto other
286+
locations. The entire :term:`run directory` can be symlinked, as well as
287+
certain sub-directories.
287288

288-
* log
289-
* share
290-
* share/cycle
291-
* work
289+
* ``run`` - the :term:`run directory` itself
290+
* ``log``
291+
* ``share`` (see :term:`share directory`)
292+
* ``share/cycle`` (typically used by `Rose`_ tasks)
293+
* ``work`` (see :term:`work directory`)
292294

293295
These should be configured per :term:`install target`.
294296

@@ -303,7 +305,7 @@ you could write the following in ``global.cylc``:
303305
[[[localhost]]]
304306
log = /somewhere/else
305307
306-
This would result in the following file structure on the Cylc Scheduler:
308+
This would result in the following file structure on the Cylc scheduler host:
307309

308310
.. code-block:: none
309311
@@ -421,7 +423,7 @@ will store the workflow under ``/g/data/foo`` and submit jobs under project
421423

422424
You can also forward variables from the server to other platforms. You should
423425
first ensure the variable is available on the server, e.g. by also forwarding
424-
the variable to ``[[localhost]]``.
426+
the variable to ``[[localhost]]``.
425427

426428
This setting only affects the task submission (e.g. ``qsub``) which may use
427429
environment variables to set default directives. To set a variable once the
@@ -439,4 +441,3 @@ task has started see
439441
# Here qsub has been configured to read from $PROJECT
440442
ssh forward environment variables = PROJECT
441443
submit method = pbs
442-

src/user-guide/installing-workflows.rst

Lines changed: 0 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -157,14 +157,6 @@ sub-directories as symlinks to other locations. This is a useful way of
157157
offloading data onto other drives to limit the disk space taken up by
158158
``~/cylc-run``.
159159

160-
Directories that can be individually symlinked are:
161-
162-
* ``log``
163-
* ``share``
164-
* ``share/cycle``
165-
* ``work``
166-
* the :term:`run directory` itself
167-
168160
The symlink targets are configured per install target in
169161
:cylc:conf:`global.cylc[install][symlink dirs]`. For more information see
170162
:ref:`SymlinkDirsSetup`

0 commit comments

Comments
 (0)