Skip to content

Commit 087d56b

Browse files
authored
Add internet access page (#39)
1 parent 4af065c commit 087d56b

File tree

4 files changed

+67
-0
lines changed

4 files changed

+67
-0
lines changed

docs/alps/hardware.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,7 @@ This approach to cooling provides greater efficiency for the rack-level cooling,
2121
* Maximum of 64 quad-blade compute blades
2222
* Maximum of 64 Slingshot switch blades
2323

24+
[](){#ref-alps-hsn}
2425
## Alps High Speed Network
2526

2627
!!! todo

docs/guides/internet-access.md

Lines changed: 59 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,59 @@
1+
[](){#ref-guides-internet-access}
2+
# Internet Access on Alps
3+
4+
The [Alps network][ref-alps-hsn] is mostly configured with private IP addresses (`172.28.0.0/16`).
5+
Login nodes have public IP addresses which means that they can directly access the internet, while a proxy server provides internet access for compute nodes.
6+
7+
??? info "Compute node proxy configuration"
8+
9+
Compute nodes are configured with the following environment variables to use the proxy server:
10+
11+
```bash
12+
export https_proxy=http://proxy.cscs.ch:8080
13+
export http_proxy=http://proxy.cscs.ch:8080
14+
export no_proxy=.local, .cscs.ch, localhost, 148.187.0.0/16, 10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16
15+
export HTTPS_PROXY=http://proxy.cscs.ch:8080
16+
export HTTP_PROXY=http://proxy.cscs.ch:8080
17+
export NO_PROXY=.local, .cscs.ch, localhost, 148.187.0.0/16, 10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16
18+
```
19+
20+
!!! warning "Public IPs are a shared resource"
21+
Be aware that public IPs, whether on login nodes or through the proxy, are essentially a shared resource.
22+
Many services will rate limit or block usage based on the IP address if abused.
23+
An example is pulling container images from Docker Hub.
24+
[Authenticating with Docker Hub][ref-ce-third-party-private-registries] makes their rate limit apply per user instead.
25+
26+
## Using SSH through the proxy server
27+
28+
While use of the proxy server is transparent for most use cases, others need additional configuration for compute nodes.
29+
An example is cloning git repositories from GitHub over SSH.
30+
Cloning over https works without additional configuration.
31+
To make SSH use the proxy server, add the following to your `~/.ssh/config` file:
32+
33+
``` title="~/.ssh/config"
34+
Match Host *,!148.187.0.0/16,!192.168.0.0/16,!172.16.0.0/12,!10.0.0.0/8exec "hostname -I | grep -vqF 148.187."
35+
ProxyCommand nc -X connect -x proxy.cscs.ch:8080 %h %p
36+
```
37+
38+
This configuration takes into account that login and compute nodes require a different setup.
39+
40+
??? warning "Error message when cloning without the proxy set up for SSH"
41+
When cloning a git repository without the correct SSH configuration, cloning will time out as follows:
42+
```bash
43+
[daint][<user>@daint-ln001 ~]$ git clone [email protected]:open-mpi/ompi.git
44+
Cloning into 'ompi'...
45+
ssh: connect to host github.com port 22: Connection timed out
46+
fatal: Could not read from remote repository.
47+
48+
Please make sure you have the correct access rights
49+
and the repository exists.
50+
```
51+
52+
## Accessing the public IP of a node
53+
54+
When on a login node configured with a public IP address, you can retrieve the public IP address for example as follows:
55+
56+
```bash
57+
[daint][<user>@daint-ln001 ~]$ curl api.ipify.org
58+
148.187.6.19
59+
```

docs/software/container-engine.md

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -268,10 +268,16 @@ image = "/capstor/scratch/cscs/<username>/nvidia+cuda+11.8.0-cudnn8-devel-ubuntu
268268
!!! note
269269
It is recommended to save images in `/capstor/scratch/cscs/<username>` or its subdirectories before using them with the CE.
270270

271+
[](){#ref-ce-third-party-private-registries}
271272
### Third-party and private registries
272273

273274
[Docker Hub](https://hub.docker.com/) is the default registry from which remote images are imported.
274275

276+
!!! warning "Registry rate limits"
277+
Some registries will rate limit image pulls by IP address.
278+
Since [public IPs are a shared resource][ref-guides-internet-access] we recommend authenticating even for publicly available images.
279+
For example, [Docker Hub applies its rate limits per user when authenticated](https://docs.docker.com/docker-hub/usage/).
280+
275281
To use an image from a different registry, the corresponding registry URL has to be prepended to the image reference, using a hash character (#) as a separator. For example:
276282

277283
```bash

mkdocs.yml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -83,6 +83,7 @@ nav:
8383
- 'Object Storage': storage/object.md
8484
- 'Guides':
8585
- guides/index.md
86+
- 'Internet Access on Alps': guides/internet-access.md
8687
- 'Storage': guides/storage.md
8788
- 'Policies':
8889
- policies/index.md

0 commit comments

Comments
 (0)