Change ActiveLinkStyle to return the folder instead of link

Soft links are not like hard links or folders. Sometimes, they are treated
differently by the operating system (and Julia). As a result, for some
operations, an extra step is required to work with the output produced by the
ActiveStyle in OutputPathGenerator.

This PR changes the returned path to be the newly created folder instead of the
link. The link is still being created and managed, which means that users can
still assume that their output can always be found in `output_active`, but the
path that is presented to Julia code that uses `OutputPathGenerator` is now the
path of the newly created folder. This has the advantage that the path is a
normal folder, possibly simplifying certain operations.

Author: Sbozzolo

NEWS.md

@@ -1,6 +1,13 @@
 ClimaUtilities.jl Release Notes
 ===============================
 
+v0.1.8
+------
+
+- `generate_output_path(..., ::ActiveLinkStyle)` now returns the folder instead
+  of the link. Links are still being created and managed. PR
+  [#63](https://github.com/CliMA/ClimaUtilities.jl/pull/63)
+
 v0.1.7
 ------
 
@@ -24,3 +31,5 @@ v0.1.3
 -------
 - Add `DataStructures` module containing `LRUCache` object. PR [#35](https://github.com/CliMA/ClimaUtilities.jl/pull/35)
 - Add `OutputPathGenerator`. PR [#28](https://github.com/CliMA/ClimaLand.jl/pull/28)
+
+[badge-💥breaking]: https://img.shields.io/badge/💥BREAKING-red.svg

docs/src/outputpathgenerator.md

@@ -42,12 +42,12 @@ results.
 
 Let's assume your `output_path` is set to `data`.
 
-* If `data` doesn't exist, the module creates it and returns
-  `data/output_active`. This link points to the newly created subfolder
-  `data/output_0000`.
-* If `data` exists and contains an `output_active` link pointing to
-  `data/output_0005`, the module creates a new subfolder `data/output_0006` and
-  updates `output_active` to point to it.
+* If `data` doesn't exist, the module creates it and returns `data/output_0000`.
+  In doing this, a link `data/output_active` to `data/output_0000` is created so
+  that you can always access your data in `data/output_active`.
+*  If `data` exists and contains an `output_active` link pointing to
+  `data/output_0005`, `output_active` is updated to point to a new subfolder
+  called `data/output_0006`
 * If `data` exists with or without an `output_active` link, the module checks
   for existing subfolders named `data/output_XXXX` (with `XXXX` a number). If
   none are found, it creates `data/output_0000` and a link `data/output_active`
@@ -60,7 +60,6 @@ users, so some details about links might be slightly different depending on your
 system. If you are using Windows, please have a look at docstring on the
 `ActiveLinkStyle` to learn more about possible differences.
 
-
 ## API
 
 ```@docs

src/OutputPathGenerator.jl

@@ -69,16 +69,21 @@ This style is designed to:
 - provide a deterministic and fixed path for the latest available data,
 - and have nearly zero runtime overhead.
 
+`generate_output_path` returns path to the newly created folder with the next available
+increment (of the form `output_1234`), and ensures that a valid `output_active` link points
+to that folder.
+
 # Examples:
 
 Let us assume that `output_path = dormouse`.
 
 - `dormouse` does not exist in the current working directory: `ActiveLinkStyle` will create
-  it and return `dormouse/output_active`. `dormouse/output_active` is a symlink that points
-  to the newly created `dormouse/output_0000` directory.
+  it and return `dormouse/output_0000`. In the process, a symlink `dormouse/output_active`
+  is also created. This symlink points to `dormouse/output_0000`.
 - `dormouse` exists and contains a `output_active` link that points to
-  `dormouse/output_0005`, `ActiveLinkStyle` will a new directory `dormouse/output_0006` and
-  change the `output_active` to point to this directory.
+  `dormouse/output_0005`, `ActiveLinkStyle` will a create new directory
+  `dormouse/output_0006`, return this path, and change the `output_active` to point to this
+  directory.
 - `dormouse` exists and does not contain a `output_active`, `ActiveLinkStyle` will check if
   any `dormouse/output_XXXX` exists. If not, it creates `dormouse/output_0000` and a link
   `dormouse/output_active` that points to this directory.
@@ -112,13 +117,13 @@ How the output should be structured (in terms of directory tree) is determined b
 - `RemovePreexistingStyle`: the `output_path` provided is the actual output path. If a directory
   already exists there, remove it without asking for confirmation.
 
-- `ActiveLinkStyle`: the `output_path` provided is `output_path/output_active`, a link to a
-  subdirectory within `output_path`. This style looks at the content of `output_path` and
-  adds new subdirectories. The added directories are named with a counter, with the latest
-  always accessible via the `output_path/output_active` symlink. This is style is
-  non-destructive.
+- `ActiveLinkStyle`: the `output_path` returned is a new folder of the form
+  `output_path/output_1234`, where the number is incremented every time this function is
+  called. `ActiveLinkStyle` also creates a link `output_path/output_active` that ensures
+  that the most recent output is always accessible at the `output_path/output_active` path.
+  This is style is non-destructive.
 
-(Note, "styles" have nothing to do with traits.)
+(Note, "styles" have nothing to do with Julia traits.)
 """
 function generate_output_path(
     output_path;
@@ -244,7 +249,7 @@ function generate_output_path(::ActiveLinkStyle, output_path; context = nothing)
         end
     end
     maybe_wait_filesystem(context, new_output_folder)
-    return active_link
+    return new_output_folder
 end
 
 end

test/output_path_generator.jl

@@ -68,29 +68,33 @@ end
     # Folder does not yet exist
     output_path = joinpath(base_output_path, "dormouse")
 
-    expected_output = joinpath(output_path, "output_active")
+    output_link = joinpath(output_path, "output_active")
+
+    expected_output = joinpath(output_path, "output_0000")
 
     @test expected_output ==
           generate_output_path(output_path, context = context)
 
     # Check that it exists now
     @test isdir(output_path)
 
-    # Check folder output_0000 was created
-    @test isdir(joinpath(output_path, "output_0000"))
+    # Check link output_active was created
+    @test islink(output_link)
 
-    # Check link points to folder
-    @test readlink(expected_output) == "output_0000"
+    # # Check link points to folder
+    @test readlink(output_link) == "output_0000"
 
     # Now the folder exists, let us see if the rotation works
+    expected_output = joinpath(output_path, "output_0001")
+
     @test expected_output ==
           generate_output_path(output_path, context = context)
 
-    # Check folder output_0001 was created
+    # Check folder was created
     @test isdir(joinpath(output_path, "output_0001"))
 
     # Check link points to new folder
-    @test readlink(expected_output) == "output_0001"
+    @test readlink(output_link) == "output_0001"
 
     # Now let us check something wrong
 
@@ -101,7 +105,7 @@ end
     ClimaComms.barrier(context)
     let_filesystem_catch_up()
 
-    output_link = generate_output_path(output_path, context = context)
+    generate_output_path(output_path, context = context)
     @test readlink(output_link) == "output_0002"
 
     ClimaComms.barrier(context)
@@ -119,7 +123,7 @@ end
     if ClimaComms.iamroot(context)
         wrong_dir = joinpath(output_path, "wrong")
         mkdir(wrong_dir)
-        symlink(wrong_dir, expected_output, dir_target = true)
+        symlink(wrong_dir, output_link, dir_target = true)
     end
     ClimaComms.barrier(context)
     @test_throws ErrorException generate_output_path(