Skip to content

Commit 2e0f793

Browse files
committed
Update documentation.
1 parent e90077a commit 2e0f793

File tree

2 files changed

+71
-20
lines changed

2 files changed

+71
-20
lines changed

.github/workflows/code_test_and_deploy.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -111,7 +111,7 @@ jobs:
111111
AWS_REGION: ${{ secrets.AWS_REGION }}
112112
AWS_BUCKET_NAME: ${{ secrets.AWS_BUCKET_NAME }}
113113
run: |
114-
pytest tests/tests_transfers/aws -v
114+
pytest tests/tests_transfers/aws
115115
116116
- name: All Other Tests
117117
run: |

docs/source/pages/get_started/set-up-a-project.md

Lines changed: 70 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -547,35 +547,86 @@ project.setup_aws_connection()
547547
Running [](setup_aws_connection()) will require entering your
548548
`AWS Secret Access Key` and the setup will be completed.
549549

550+
::::
551+
550552

551553
(password-protection)=
552-
### Password protecting your connection credentials
554+
# Password protecting your connection credentials
553555

554-
+ Add links to this page in the TUI / Api
556+
Datashuttle uses [RClone](https://rclone.org/) for all data transfers by default.
557+
RClone stores connection credentials (such as SSH keys or API tokens) in a local configuration file that, by default, is not encrypted.
555558

556-
Datashuttle uses the software `RClone` for all data transfers by default.
557-
RClone stores the credentials for connection by default in an unencrypted configuration file.
558-
This includes:
559+
This file can include:
559560

560-
- ssh connection: the private SSH key
561-
- Google Drive: some API thing
562-
- Amazon S3: Access key ID and XXX
561+
- SSH connections: your private SSH key
562+
- Google Drive connections: your OAuth access token and client secret
563+
- Amazon S3 connections: your AWS Access Key ID and Secret Access Key
563564

564565
By default, these are stored in your home directory which should be secure. However, for an
565-
additional layer of security, it is possible to encrpy the Rclone config file.
566+
additional layer of security, it is possible to encrypt the Rclone config file using the
567+
system credential manager of your operating system. This file will then be
568+
non-readable for anyone who does not have access to your machine user account. Note that
569+
anyone with access to the machine user account will be able to decrypt the Rclone file.
570+
571+
Despite this layer of security, it is not recommended to use datashuttle for remote connectivity on
572+
a machine to which you do not have secure access, even with user account encryption of the RClone config.
566573

567-
When setting up the connection, datashuttle will offer the option to set the RClone configuration.
568-
This automatically uses the system credential manager:
574+
For details on setting up encryption, see the section below. On Windows, you will
575+
need to be running in PowerShell, and on Linux you will need `pass` package installed.
569576

570-
Windows : (requires powershell)
571-
macOS : set up
572-
Linux : requires pass
577+
::::{tab-set}
573578

574-
This means the file is only uncryptable on your local machine or user) CHECK USER.
579+
:::{tab-item} Windows
575580

576-
TODO: think more about the credentials file... its' stupid to have this itself plain text in datashuttle?
581+
On Windows, Datashuttle uses the PowerShell `PSCredential` system to encrypt the RClone config file.
577582

578-
Despite this layer of security, it is not recommended to use datashuttle for remote connectivity on
579-
a machine to which you do not have secure access, even with password protection of the RClone config.
583+
- A random password is generated and stored as a `.clixml` credential file under a `credentials` folder in the project config location.
584+
- The password can only be decrypted by the same Windows user account that created it.
585+
- The encryption and decryption process uses PowerShell, so PowerShell must be available (it will not work from `cmd.exe`).
586+
587+
When encryption is enabled, RClone automatically retrieves the password from the PSCredential store whenever it runs.
588+
589+
:::
590+
591+
:::{tab-item} macOS
592+
593+
On macOS, Datashuttle uses the built-in Keychain via the `security` command-line tool.
594+
595+
- A random password is generated using `openssl rand -base64 40`.
596+
- The password is securely stored in your login Keychain under the service name corresponding to your RClone config.
597+
- Only your macOS user account can access this key.
598+
599+
When you first set up encryption, macOS may prompt you to authorize access to the Keychain.
600+
Once approved, RClone will automatically retrieve the key when needed.
601+
602+
:::
603+
604+
:::{tab-item} Linux
605+
606+
1. Install `pass`:
607+
```bash
608+
sudo apt install pass
609+
```
610+
2. Initialize the password store with your GPG key:
611+
```bash
612+
pass init <your-gpg-id>
613+
```
614+
615+
Once initialized, Datashuttle will:
616+
- Generate a random password with `openssl rand -base64 40`
617+
- Store it securely in the GPG-encrypted password store
618+
- Configure RClone to retrieve it automatically with:
619+
```bash
620+
/usr/bin/pass <rclone_config_name>
621+
```
622+
623+
:::
624+
625+
::::
626+
627+
## Removing encryption
628+
629+
Encryption of the rclone config used for the central connection (either SSH, Google Drive or AWS)
630+
can be removed with the following command:
580631

581-
TODO: test if `pass` is not installed on linux that the error is propagated to the TUI properly
632+
[](remove_rclone_encryption())

0 commit comments

Comments
 (0)