install incrementally

[oliver-sanders]

Because cylc install uses rsync --delete any files installed by metomi-rose file install (i.e. [file:...]) are deleted by cylc install, then re-created by cylc-rose via the post-install hook.

This is functionally ok but really inefficient. We should get fileinstall working in an incremental way (as it used to in Rose 2019).

This would involve getting cylc-rose to send back a list of the files it manages to cylc install before installation (i.e via a pre-install hook?) which cylc install would then add to .cylcignore (but without modifying the copy in the source dir please!).

Here's an example of how this could look:

# psudopython
# ask the pre_install plugin for a list of excluded files
ret = pre_install(src_dir, run_dir)

# add these to the .cylcignore in a temporary file
with TemporaryFile as temp_file:
    temp_file.write(cylc_ignore.read())
    temp_file.write('\n' + '\n'.join(ret['excluded_paths']))

# pass this temporary file to rsync
install_command = ['rsync', '--exclude-file', temp_file]

If Rose does/could support a dry-mode then this could also close #149

Pull requests welcome!