Support symlinks for filesystems that don't support symlinks

[ap--]

This issue starts collecting ideas how symlink support could be added to universal-pathlib and filesystems that don't support symlinks.

Ideas

For all cases below, it holds that the read only case is fine, but a read write case will require careful locking. And avoiding race conditions here might be complicated / impossible.

(1) storing all symlink targets per directory in file with a specialized name

So we could make UPath special case a name, let's say .upath.symlinks that contains a path to another resource and the name of the symlink as well as its type.

# .upath.symlinks
name=abc,efg
type=file,dir
target=foo/bar/baz,foo/bar

The problem with this is of course that we'd have to always read this file, and there would have to be logic built around the case where the name collides with an actual file.

(2) storing the symlink target in a file with its actual symlink name

# bucket/abc
type=file
target=foo/bar/baz

This is the worst. Because for every single file we'd have to read it. And on object store we wouldn't be able to make directories anyways.

(3) having a global setting for each universal-pathlib implementation.

We could provide a {symlink: target} mapping that would be class scoped for each UPath implementation.
There would have to be some mechanism that would allow the user to load the symlinks they want for their UPath class, and a way to serialize it back to disk.

from upath.symlinks import set_protocol_symlinks

set_protocol_symlinks("s3", {"bucket/foo": "bucket/abc"})

UPath("s3://bucket/foo")

This mapping could of course be stored on the remote filesystem serialized to json too.

(4) implementing a chainable fsspec filesystem for symlink support

This would work similar to (3), just that we write a new fsspec filesystem, that basically does pass-through, but does the translation for each requested path, and returns the symlinks as needed when ls or glob are run.

UPath("symlinks::s3://bucket/foo", symlinks={"mapping": {"bucket/foo": "bucket/abc"}, }, s3={"anon": True})

This would remove any of these details from UPath, and we'd move symlink support basically to the fsspec layer.

(5) other options ???

@sdvillal I remember discussing this with you years ago, did you have any other idea how remote filesystem symlink support could be implemented?