Merge pull request #9369 from mr-raj12/docs-platformdirs-7332

docs: document platformdirs change and platform-specific directory paths, fixes #7332

Author: ThomasWaldmann

docs/changes.rst

@@ -126,7 +126,19 @@ Compatibility notes:
 
   - XDG_*_HOME is not honoured on macOS and on Windows.
   - BORG_BASE_DIR can still be used to enforce some base dir + .config/ or .cache/.
-  - the default macOS config and cache dir will now be in ~/Library/Application Support/borg/.
+  - on macOS, the default directories move to native locations:
+    config/data: ``~/Library/Application Support/borg/``,
+    cache: ``~/Library/Caches/borg/``,
+    runtime: ``~/Library/Caches/TemporaryItems/borg/``.
+  - on Windows, the default directories are:
+    config: ``C:\Users\<user>\AppData\Roaming\borg``,
+    cache: ``C:\Users\<user>\AppData\Local\borg\Cache``,
+    data: ``C:\Users\<user>\AppData\Local\borg``.
+  - **keyfile users on macOS (and Windows)**: borg 2 will look for key files in the
+    new platform-specific config directory instead of ``~/.config/borg/keys/`` where
+    borg 1.x stored them. You can set ``BORG_KEYS_DIR`` to point to the old location,
+    or copy the key file to the new location. See :ref:`env_vars` and the ``borg transfer``
+    documentation for details.
 - create: different included/excluded status chars, #7321
 
   - dry-run: now uses "+" (was: "-") and "-" (was: "x") for included/excluded status

docs/faq.rst

@@ -375,18 +375,21 @@ Security
 
 .. _home_config_borg:
 
-How important is the $HOME/.config/borg directory?
---------------------------------------------------
+How important is the borg config directory?
+-------------------------------------------
 
-The Borg config directory has content that you should take care of:
+The borg config directory (``~/.config/borg`` on Linux,
+``~/Library/Application Support/borg`` on macOS,
+``C:\Users\<user>\AppData\Roaming\borg`` on Windows -- see :ref:`env_vars`)
+has content that you should take care of:
 
 ``keys`` subdirectory
   All your borg keyfile keys are stored in this directory. Please note that borg
   repokey keys are stored inside the repository. In any case, you MUST make sure
   to have an independent backup of the borg keys, see :ref:`borg_key_export` for
   more details.
 
-Make sure that only you have access to the Borg config directory.
+Make sure that only you have access to the borg config directory.
 
 
 Note about creating multiple keyfile repositories at the same path
@@ -398,7 +401,7 @@ was moved away or unmounted), Borg will not overwrite or reuse the existing
 key file in your keys directory. Instead, it creates a new key file by
 appending a numeric suffix to the base name (e.g., .2, .3, ...).
 
-This means you may see multiple key files like:
+This means you may see multiple key files like (example paths for Linux):
 
 - ~/.config/borg/keys/home_user_backup
 - ~/.config/borg/keys/home_user_backup.2
@@ -410,10 +413,13 @@ overwrite.
 
 .. _home_data_borg:
 
-How important is the $HOME/.local/share/borg directory?
--------------------------------------------------------
+How important is the borg data directory?
+-----------------------------------------
 
-The Borg data directory has content that you should take care of:
+The borg data directory (``~/.local/share/borg`` on Linux,
+``~/Library/Application Support/borg`` on macOS,
+``C:\Users\<user>\AppData\Local\borg`` on Windows -- see :ref:`env_vars`)
+has content that you should take care of:
 
 ``security`` subdirectory
   Each directory here represents one Borg repository by its ID and contains the last known status.

docs/internals/data-structures.rst

@@ -736,7 +736,8 @@ Key files
 .. seealso:: The :ref:`key_encryption` section for an in-depth review of the key encryption.
 
 When initializing a repository with one of the "keyfile" encryption modes,
-Borg creates an associated key file in ``$HOME/.config/borg/keys``.
+Borg creates an associated key file in the keys subdirectory of the borg config
+directory (see :ref:`env_vars` for platform-specific default paths).
 
 The same key is also used in the "repokey" modes, which store it in the repository.
 

docs/usage/general/environment.rst.inc

@@ -161,30 +161,86 @@ Some automatic "answerers" (if set, they automatically answer confirmation quest
 .. _XDG env var: https://specifications.freedesktop.org/basedir-spec/0.6/ar01s03.html
 
 Directories and files:
+
+    .. note::
+
+        Borg 2 uses the `platformdirs <https://pypi.org/project/platformdirs/>`_ library to determine
+        default directory locations. This means that default paths are **platform-specific**:
+
+        - **Linux**: Uses XDG Base Directory Specification paths (e.g., ``~/.config/borg``,
+          ``~/.cache/borg``, ``~/.local/share/borg``). `XDG env var`_ variables are honoured.
+        - **macOS**: Uses native macOS directories (e.g., ``~/Library/Application Support/borg``,
+          ``~/Library/Caches/borg``). `XDG env var`_ variables are **not** honoured.
+        - **Windows**: Uses Windows AppData directories (e.g., ``C:\Users\<user>\AppData\Roaming\borg``,
+          ``C:\Users\<user>\AppData\Local\borg``). `XDG env var`_ variables are **not** honoured.
+
+        On all platforms, you can override each directory individually using the specific environment
+        variables described below. You can also set ``BORG_BASE_DIR`` to force borg to use
+        ``BORG_BASE_DIR/.config/borg``, ``BORG_BASE_DIR/.cache/borg``, etc., regardless of the platform.
+
+    Default directory locations by platform (when no ``BORG_*`` environment variables are set):
+
+    .. list-table::
+       :header-rows: 1
+       :widths: 15 25 30 30
+
+       * - Directory
+         - Linux
+         - macOS
+         - Windows
+       * - Config
+         - ``~/.config/borg``
+         - ``~/Library/Application Support/borg``
+         - ``%APPDATA%\borg``
+       * - Cache
+         - ``~/.cache/borg``
+         - ``~/Library/Caches/borg``
+         - ``%LOCALAPPDATA%\borg\Cache``
+       * - Data
+         - ``~/.local/share/borg``
+         - ``~/Library/Application Support/borg``
+         - ``%LOCALAPPDATA%\borg``
+       * - Runtime
+         - ``/run/user/<uid>/borg``
+         - ``~/Library/Caches/TemporaryItems/borg``
+         - ``%TEMP%\borg``
+       * - Keys
+         - ``<config_dir>/keys``
+         - ``<config_dir>/keys``
+         - ``<config_dir>\keys``
+       * - Security
+         - ``<data_dir>/security``
+         - ``<data_dir>/security``
+         - ``<data_dir>\security``
+
     BORG_BASE_DIR
         Defaults to ``$HOME`` or ``~$USER`` or ``~`` (in that order).
         If you want to move all borg-specific folders to a custom path at once, all you need to do is
         to modify ``BORG_BASE_DIR``: the other paths for cache, config etc. will adapt accordingly
         (assuming you didn't set them to a different custom value).
     BORG_CACHE_DIR
-        Defaults to ``$BORG_BASE_DIR/.cache/borg``. If ``BORG_BASE_DIR`` is not explicitly set while
-        `XDG env var`_ ``XDG_CACHE_HOME`` is set, then ``$XDG_CACHE_HOME/borg`` is being used instead.
+        Defaults to the platform-specific cache directory (see table above).
+        If ``BORG_BASE_DIR`` is set, defaults to ``$BORG_BASE_DIR/.cache/borg``.
+        On Linux, `XDG env var`_ ``XDG_CACHE_HOME`` is also honoured if ``BORG_BASE_DIR`` is not set.
         This directory contains the local cache and might need a lot
         of space for dealing with big repositories. Make sure you're aware of the associated
         security aspects of the cache location: :ref:`cache_security`
     BORG_CONFIG_DIR
-        Defaults to ``$BORG_BASE_DIR/.config/borg``. If ``BORG_BASE_DIR`` is not explicitly set while
-        `XDG env var`_ ``XDG_CONFIG_HOME`` is set, then ``$XDG_CONFIG_HOME/borg`` is being used instead.
+        Defaults to the platform-specific config directory (see table above).
+        If ``BORG_BASE_DIR`` is set, defaults to ``$BORG_BASE_DIR/.config/borg``.
+        On Linux, `XDG env var`_ ``XDG_CONFIG_HOME`` is also honoured if ``BORG_BASE_DIR`` is not set.
         This directory contains all borg configuration directories, see the FAQ
         for a security advisory about the data in this directory: :ref:`home_config_borg`
     BORG_DATA_DIR
-        Defaults to ``$BORG_BASE_DIR/.local/share/borg``. If ``BORG_BASE_DIR`` is not explicitly set while
-        `XDG env var`_ ``XDG_DATA_HOME`` is set, then ``$XDG_DATA_HOME/borg`` is being used instead.
+        Defaults to the platform-specific data directory (see table above).
+        If ``BORG_BASE_DIR`` is set, defaults to ``$BORG_BASE_DIR/.local/share/borg``.
+        On Linux, `XDG env var`_ ``XDG_DATA_HOME`` is also honoured if ``BORG_BASE_DIR`` is not set.
         This directory contains all borg data directories, see the FAQ
         for a security advisory about the data in this directory: :ref:`home_data_borg`
     BORG_RUNTIME_DIR
-        Defaults to ``$BORG_BASE_DIR/.cache/borg``. If ``BORG_BASE_DIR`` is not explicitly set while
-        `XDG env var`_ ``XDG_RUNTIME_DIR`` is set, then ``$XDG_RUNTIME_DIR/borg`` is being used instead.
+        Defaults to the platform-specific runtime directory (see table above).
+        If ``BORG_BASE_DIR`` is set, defaults to ``$BORG_BASE_DIR/.cache/borg``.
+        On Linux, `XDG env var`_ ``XDG_RUNTIME_DIR`` is also honoured if ``BORG_BASE_DIR`` is not set.
         This directory contains borg runtime files, like e.g. the socket file.
     BORG_SECURITY_DIR
         Defaults to ``$BORG_DATA_DIR/security``.

docs/usage/key.rst

@@ -28,6 +28,15 @@ Examples
     Remember your passphrase. Your data will be inaccessible without it.
     Key updated
 
+.. note::
+
+    The key file paths shown above are the defaults for Linux (``~/.config/borg/keys/``).
+    On macOS, key files are stored in ``~/Library/Application Support/borg/keys/``.
+    On Windows, they are stored in ``C:\Users\<user>\AppData\Roaming\borg\keys\``.
+    See :ref:`env_vars` for details.
+
+::
+
     # Import a previously-exported key into the specified
     # key file (creating or overwriting the output key)
     # (keyfile repositories only)

docs/usage/repo-create.rst

@@ -22,6 +22,7 @@ Examples
     $ export BORG_REPO=ssh://user@hostname/~/backup
     # repokey: stores the encrypted key in <REPO_DIR>/config
     $ borg repo-create --encryption=repokey-aes-ocb
-    # keyfile: stores the encrypted key in ~/.config/borg/keys/
+    # keyfile: stores the encrypted key in the config dir's keys/ subdir
+    # (e.g. ~/.config/borg/keys/ on Linux, ~/Library/Application Support/borg/keys/ on macOS)
     $ borg repo-create --encryption=keyfile-aes-ocb
 

docs/usage/repo-create.rst.inc

@@ -122,7 +122,9 @@ The easiest way to find out what is fastest is to run ``borg benchmark cpu``.
 the key will be stored in the repository (in ``repo_dir/config``).
 
 `keyfile` modes: if you want "passphrase and having-the-key" security -
-the key will be stored in your home directory (in ``~/.config/borg/keys``).
+the key will be stored in the keys subdirectory of the borg config directory
+(e.g., ``~/.config/borg/keys`` on Linux, ``~/Library/Application Support/borg/keys``
+on macOS -- see :ref:`env_vars` for platform-specific default paths).
 
 The following table is roughly sorted in order of preference, the better ones are
 in the upper part of the table, in the lower part is the old and/or unsafe(r) stuff:

docs/usage/transfer.rst

@@ -30,3 +30,41 @@ Examples
     $ borg --repo       ssh://borg2@borgbackup/./tests/b20 transfer --upgrader=From12To20 \
            --other-repo ssh://borg2@borgbackup/./tests/b12 --dry-run
 
+Keyfile considerations when upgrading from borg 1.x
+++++++++++++++++++++++++++++++++++++++++++++++++++++
+
+If you are using a ``keyfile`` encryption mode (not ``repokey``), borg 2
+may not automatically find your borg 1.x key file, because the default
+key file directory has changed on some platforms due to the switch to
+the `platformdirs <https://pypi.org/project/platformdirs/>`_ library.
+
+On **Linux**, there is typically no change -- both borg 1.x and borg 2
+use ``~/.config/borg/keys/``.
+
+On **macOS**, borg 1.x stored key files in ``~/.config/borg/keys/``,
+but borg 2 defaults to ``~/Library/Application Support/borg/keys/``.
+
+On **Windows**, borg 1.x used XDG-style paths (e.g. ``~/.config/borg/keys/``),
+while borg 2 defaults to ``C:\Users\<user>\AppData\Roaming\borg\keys\``.
+
+If borg 2 cannot find your key file, you have several options:
+
+1. **Copy the key file** from the old location to the new one.
+2. **Set BORG_KEYS_DIR** to point to the old key file directory::
+
+       export BORG_KEYS_DIR=~/.config/borg/keys
+
+3. **Set BORG_KEY_FILE** to point directly to the specific key file::
+
+       export BORG_KEY_FILE=~/.config/borg/keys/your_key_file
+
+4. **Set BORG_BASE_DIR** to force borg 2 to use the same base directory
+   as borg 1.x::
+
+       export BORG_BASE_DIR=$HOME
+
+   This makes borg 2 use ``$HOME/.config/borg``, ``$HOME/.cache/borg``,
+   etc., matching borg 1.x behaviour on all platforms.
+
+See :ref:`env_vars` for more details on directory environment variables.
+