diff --git a/docs/storage/object.md b/docs/storage/object.md index 604b2d2f..ba720c41 100644 --- a/docs/storage/object.md +++ b/docs/storage/object.md @@ -22,8 +22,8 @@ CSCS offers a public cloud object storage service, based on the Ceph Object Gate The first step is to configure the profile: -```bash -> aws configure --profile naret-testuser +```console +$ aws configure --profile naret-testuser AWS Access Key ID [None]: [REDACTED] AWS Secret Access Key [None]: [REDACTED] Default region name [None]: cscs-zonegroup @@ -42,8 +42,8 @@ s3 = #### Creating a pre-signed URL -```bash -> aws --profile=naret-testuser s3 presign s3://test-bucket/file.txt --expires-in 300 +```console +$ aws --profile=naret-testuser s3 presign s3://test-bucket/file.txt --expires-in 300 https://rgw.cscs.ch/test-bucket/file.txt?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=IA6AOCNMKPDXQ0YNA3DP%2F20241209%2Fcscs-zonegroup%2Fs3%2Faws4_request&X-Amz-Date=20241209T080748Z&X-Amz-Expires=300&X-Amz-SignedHeaders=host&X-Amz-Signature=f2e2adb457f6fd43401124e4ea2650fba528e614ab661f9c05e2fa2e77691b5d ``` @@ -56,8 +56,7 @@ A more thorough explanation can be found in the [RGW documentation](https://docs First, a bucket policy needs to be written: -```json -> cat test-public-bucket-anon-from-internet.json +```json title="test-public-bucket-anon-from-internet.json" { "Version": "2012-10-17", "Statement": [ @@ -76,16 +75,16 @@ First, a bucket policy needs to be written: Then, it can be applied to the bucket: -``` -> aws --profile=naret-testuser s3api put-bucket-policy \ +```console +$ aws --profile=naret-testuser s3api put-bucket-policy \ --bucket test-public-bucket --policy \ file://test-public-bucket-anon-from-internet.json ``` At this point, the objects in test-public-bucket are accessible via direct links: -``` -> s3cmd --configure +```console +$ s3cmd --configure Enter new values or accept defaults in brackets with Enter. Refer to user manual for detailed description of all options. @@ -133,15 +132,15 @@ And then confirm. __IMPORTANT__: The configuration is not complete yet. -``` -> s3cmd ls s3://test-bucket +```console +$ s3cmd ls s3://test-bucket ERROR: S3 error: 403 (SignatureDoesNotMatch) ``` To fix this, it is necessary to edit the `.s3cfg` file, normally located in the user's home directory, and change the `signature_v2` setting to true. ```console -~ $ cat .s3cfg | grep signature_v2 +$ cat .s3cfg | grep signature_v2 signature_v2 = True $ s3cmd ls s3://test-bucket @@ -152,6 +151,33 @@ $ s3cmd ls s3://test-bucket #### Configuration -In order to be able to connect to the S3 endpoint using Cyberduck, a profile supporting path-style requests must be downloaded from here. +In order to be able to connect to the S3 endpoint using Cyberduck, a profile supporting path-style requests must be downloaded from [here](https://profiles.cyberduck.io/S3%20(Deprecated%20path%20style%20requests).cyberduckprofile) or copied from below. + +```xml title="S3 (Deprecated path style requests).cyberduckprofile" + + + + + Protocol + s3 + Vendor + s3-path-style + Scheme + https + Description + S3 (Deprecated path style requests) + Hostname Configurable + + Port Configurable + + Username Configurable + + Properties + + s3.bucket.virtualhost.disable=true + + + +``` ![cyberduck](../images/storage/cyberduck.png) diff --git a/docs/storage/transfer.md b/docs/storage/transfer.md index 802b33c5..7f141c9c 100644 --- a/docs/storage/transfer.md +++ b/docs/storage/transfer.md @@ -26,15 +26,9 @@ Currently Globus provide the following mount points at CSCS: | Mount Point | Description | |---|---| -| `/scratch/snx3000` | old Daint scratch area | -| `/store` | old Daint store area | -| `/project` | old Daint project area | -| `/users` | old Daint home directory | -| `/scratch/shared` | old Scratch-Shared area ( old meteoswiss clusters ) | | `/iopsstor/scratch/cscs` | Mounted on Clariden | | `/capstor/scratch/cscs` | New Alps Daint scratch area | | `/capstor/store/cscs` | New Alps Daint store area | -| `/capstor/users/cscs` | Home directory for Bristen/Scopi/Errigal | | `/vast/users/cscs` | New Alps vclusters home directory ( Alps Daint and others ) | [](){#ref-data-xfer-internal}