Skip to content

Commit 3f0bdb9

Browse files
authored
Storage doc updates and fixes (#134)
1 parent 13b87eb commit 3f0bdb9

File tree

2 files changed

+40
-20
lines changed

2 files changed

+40
-20
lines changed

docs/storage/object.md

Lines changed: 40 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -22,8 +22,8 @@ CSCS offers a public cloud object storage service, based on the Ceph Object Gate
2222

2323
The first step is to configure the profile:
2424

25-
```bash
26-
> aws configure --profile naret-testuser
25+
```console
26+
$ aws configure --profile naret-testuser
2727
AWS Access Key ID [None]: [REDACTED]
2828
AWS Secret Access Key [None]: [REDACTED]
2929
Default region name [None]: cscs-zonegroup
@@ -42,8 +42,8 @@ s3 =
4242

4343
#### Creating a pre-signed URL
4444

45-
```bash
46-
> aws --profile=naret-testuser s3 presign s3://test-bucket/file.txt --expires-in 300
45+
```console
46+
$ aws --profile=naret-testuser s3 presign s3://test-bucket/file.txt --expires-in 300
4747
4848
https://rgw.cscs.ch/test-bucket/file.txt?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=IA6AOCNMKPDXQ0YNA3DP%2F20241209%2Fcscs-zonegroup%2Fs3%2Faws4_request&X-Amz-Date=20241209T080748Z&X-Amz-Expires=300&X-Amz-SignedHeaders=host&X-Amz-Signature=f2e2adb457f6fd43401124e4ea2650fba528e614ab661f9c05e2fa2e77691b5d
4949
```
@@ -56,8 +56,7 @@ A more thorough explanation can be found in the [RGW documentation](https://docs
5656

5757
First, a bucket policy needs to be written:
5858

59-
```json
60-
> cat test-public-bucket-anon-from-internet.json
59+
```json title="test-public-bucket-anon-from-internet.json"
6160
{
6261
"Version": "2012-10-17",
6362
"Statement": [
@@ -76,16 +75,16 @@ First, a bucket policy needs to be written:
7675

7776
Then, it can be applied to the bucket:
7877

79-
```
80-
> aws --profile=naret-testuser s3api put-bucket-policy \
78+
```console
79+
$ aws --profile=naret-testuser s3api put-bucket-policy \
8180
--bucket test-public-bucket --policy \
8281
file://test-public-bucket-anon-from-internet.json
8382
```
8483

8584
At this point, the objects in test-public-bucket are accessible via direct links:
8685

87-
```
88-
> s3cmd --configure
86+
```console
87+
$ s3cmd --configure
8988
9089
Enter new values or accept defaults in brackets with Enter.
9190
Refer to user manual for detailed description of all options.
@@ -133,15 +132,15 @@ And then confirm.
133132

134133
__IMPORTANT__: The configuration is not complete yet.
135134

136-
```
137-
> s3cmd ls s3://test-bucket
135+
```console
136+
$ s3cmd ls s3://test-bucket
138137
ERROR: S3 error: 403 (SignatureDoesNotMatch)
139138
```
140139

141140
To fix this, it is necessary to edit the `.s3cfg` file, normally located in the user's home directory, and change the `signature_v2` setting to true.
142141

143142
```console
144-
~ $ cat .s3cfg | grep signature_v2
143+
$ cat .s3cfg | grep signature_v2
145144
signature_v2 = True
146145
147146
$ s3cmd ls s3://test-bucket
@@ -152,6 +151,33 @@ $ s3cmd ls s3://test-bucket
152151

153152
#### Configuration
154153

155-
In order to be able to connect to the S3 endpoint using Cyberduck, a profile supporting path-style requests must be downloaded from here.
154+
In order to be able to connect to the S3 endpoint using Cyberduck, a profile supporting path-style requests must be downloaded from [here](https://profiles.cyberduck.io/S3%20(Deprecated%20path%20style%20requests).cyberduckprofile) or copied from below.
155+
156+
```xml title="S3 (Deprecated path style requests).cyberduckprofile"
157+
<?xml version="1.0" encoding="UTF-8"?>
158+
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
159+
<plist version="1.0">
160+
<dict>
161+
<key>Protocol</key>
162+
<string>s3</string>
163+
<key>Vendor</key>
164+
<string>s3-path-style</string>
165+
<key>Scheme</key>
166+
<string>https</string>
167+
<key>Description</key>
168+
<string>S3 (Deprecated path style requests)</string>
169+
<key>Hostname Configurable</key>
170+
<true/>
171+
<key>Port Configurable</key>
172+
<true/>
173+
<key>Username Configurable</key>
174+
<true/>
175+
<key>Properties</key>
176+
<array>
177+
<string>s3.bucket.virtualhost.disable=true</string>
178+
</array>
179+
</dict>
180+
</plist>
181+
```
156182

157183
![cyberduck](../images/storage/cyberduck.png)

docs/storage/transfer.md

Lines changed: 0 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -26,15 +26,9 @@ Currently Globus provide the following mount points at CSCS:
2626

2727
| Mount Point | Description |
2828
|---|---|
29-
| `/scratch/snx3000` | old Daint scratch area |
30-
| `/store` | old Daint store area |
31-
| `/project` | old Daint project area |
32-
| `/users` | old Daint home directory |
33-
| `/scratch/shared` | old Scratch-Shared area ( old meteoswiss clusters ) |
3429
| `/iopsstor/scratch/cscs` | Mounted on Clariden |
3530
| `/capstor/scratch/cscs` | New Alps Daint scratch area |
3631
| `/capstor/store/cscs` | New Alps Daint store area |
37-
| `/capstor/users/cscs` | Home directory for Bristen/Scopi/Errigal |
3832
| `/vast/users/cscs` | New Alps vclusters home directory ( Alps Daint and others ) |
3933

4034
[](){#ref-data-xfer-internal}

0 commit comments

Comments
 (0)