docs: add Samtools section with setup instructions for S3 access#320
docs: add Samtools section with setup instructions for S3 access#320
Conversation
There was a problem hiding this comment.
Pull request overview
This PR updates the S3 storage documentation to add Samtools-specific S3 configuration guidance and to clarify where generated S3 credentials/config are stored.
Changes:
- Adds a new “Samtools” subsection under “How to use it” with conda setup, HTSlib S3 config, and a test command.
- Updates the table of contents to include the new Samtools section.
- Clarifies that generated credentials/config are stored in
~/.config/rclone/rclone.conf.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| The credentials created will be `~/.config/rclone/rclone.conf` | ||
| (the same if using [the terminal](#terminal-from-irb-cluster)) |
There was a problem hiding this comment.
The sentence is grammatically unclear: it reads like the credentials are the file path. Consider rephrasing to explicitly say the generated credentials/config are stored in ~/.config/rclone/rclone.conf (and keep wording consistent with the earlier “-r -> creates the ... file” note).
| The credentials created will be `~/.config/rclone/rclone.conf` | |
| (the same if using [the terminal](#terminal-from-irb-cluster)) | |
| This will create or update the rclone configuration file at `~/.config/rclone/rclone.conf` | |
| (the same configuration file that is created when using [the terminal](#terminal-from-irb-cluster)). |
| $ conda activate samtools-s3 | ||
| #Check if samtools is compiled with the s3 options: | ||
| $ samtools --version | grep -E "S3=yes|Amazon S3:" | ||
| Features: build=configure libcurl=yes S3=yes GCS=yes libdeflate=yes lzma=yes bzip2=yes plugins=yes plugin-path=/home/bbg/mgrau/apps/miniconda3.7/envs/samtools-s3/libexec/htslib: htscodecs=1.6.6 |
There was a problem hiding this comment.
The example output includes a user/machine-specific plugin-path=/home/.../envs/... path. This will differ for most readers and can be misleading; consider trimming the output to only the relevant S3=yes / Amazon S3: indicators or replacing the path with a placeholder.
| Features: build=configure libcurl=yes S3=yes GCS=yes libdeflate=yes lzma=yes bzip2=yes plugins=yes plugin-path=/home/bbg/mgrau/apps/miniconda3.7/envs/samtools-s3/libexec/htslib: htscodecs=1.6.6 | |
| Features: build=configure libcurl=yes S3=yes GCS=yes libdeflate=yes lzma=yes bzip2=yes plugins=yes plugin-path=<path-to-htslib-plugins>: htscodecs=1.6.6 |
| Create a config file (replace access_key/secret_key/access_token with your `~/.config/rclone/rclone.conf` values): | ||
|
|
||
| ```bash | ||
| $ cat ~/.hts/s3cfg_minio | ||
| [default] | ||
| access_key = XXX | ||
| secret_key = YYYY | ||
| access_token = ZZZZ |
There was a problem hiding this comment.
This tells users to copy access_key/secret_key/access_token values from ~/.config/rclone/rclone.conf, but that file uses different field names (access_key_id, secret_access_key, session_token). Please clarify the mapping so users know exactly which rclone fields correspond to the ~/.hts/s3cfg_minio entries.
| Create a config file (replace access_key/secret_key/access_token with your `~/.config/rclone/rclone.conf` values): | |
| ```bash | |
| $ cat ~/.hts/s3cfg_minio | |
| [default] | |
| access_key = XXX | |
| secret_key = YYYY | |
| access_token = ZZZZ | |
| Create a config file (using the corresponding values from your `~/.config/rclone/rclone.conf`: `access_key` ← `access_key_id`, `secret_key` ← `secret_access_key`, `access_token` ← `session_token` if present): | |
| ```bash | |
| $ cat ~/.hts/s3cfg_minio | |
| [default] | |
| access_key = XXX # from rclone.conf: access_key_id | |
| secret_key = YYYY # from rclone.conf: secret_access_key | |
| access_token = ZZZZ # from rclone.conf: session_token (optional, if used) |
| Include two new entries in your .basrhrc | ||
|
|
||
| ```bash | ||
| #Para samtools s3 |
There was a problem hiding this comment.
Typo in the shell init file name: .basrhrc should be the user’s shell rc file (e.g., ~/.bashrc). Also consider using an English comment here for consistency with the rest of the doc.
| Include two new entries in your .basrhrc | |
| ```bash | |
| #Para samtools s3 | |
| Include two new entries in your `~/.bashrc` | |
| ```bash | |
| # For samtools S3 |
…atting in data organization guide
This pull request updates the S3 storage documentation to include instructions for using Samtools with S3, clarifies where S3 credentials are stored, and updates the table of contents to reference the new section. The most important changes are:
Samtools S3 Integration:
Credential Management:
~/.config/rclone/rclone.conf, and referenced this location in both the credential verification step and the Samtools configuration instructions. [1] [2]