Skip to content

Commit 1d56fdf

Browse files
committed
further clean up
1 parent b112aba commit 1d56fdf

File tree

1 file changed

+7
-8
lines changed

1 file changed

+7
-8
lines changed

docs/guides/internet-access.md

Lines changed: 7 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -23,25 +23,24 @@ $ curl api.ipify.org
2323
[](){#ref-guides-internet-access-ext}
2424
## Guidelines on communicating with external services (web scraping, bulk downloads,…)
2525

26-
Alps is a an excellent machine to simulate, evaluate and analyze data, and communication within Alps is optimized. Communication with external services is often needed to set up a calculation or communicate results to others.
26+
Communication with external services from Alps is provided by a high-capacity 400 GBit/s connection to [SWITCH](https://www.switch.ch/en/network/ip-access).
27+
SWITCH provides internet services to the research and education infrastructure in Switzerland.
2728

28-
To enable this CSCS has excellent connection (400 GBit/s) to SWITCH.ch, that provides internet services to the research and education infrastructure in Switzerland.
29-
30-
Still communication with external services is not the focus of CSCS, it is rather seen as a way to enable the use of our resources, so for example as explained below from Alps **do not** put load on services that do not expect it, for example through **scraping**.
29+
However, communication with external services is not the focus of CSCS, it is rather seen as a way to enable the use of our resources, so for example as explained below from Alps **do not** put load on services that do not expect it, for example through **scraping**.
3130

3231
### Shared resources
3332

3433
If you need to heavily interact with external systems there are some caveats that you have to keep in mind, in general some resources are shared resources, and a single user should not monopolize their use.
3534

3635
To avoid abuse there are measures in place at CSCS, on the transit networks, and on the remote systems, but these measures are often very blunt and would affect the CSCS as whole, so care should be taken to avoid triggering them.
37-
We have a good relationship with Switch, so if we trigger some of their fail-safes (for example their anti-DDoS tools), they will contact us. Other might take action without telling us anything.
36+
We have a good relationship with SWITCH, so if we trigger some of their fail-safes (for example their anti-DDoS tools), they will contact us. Other might take action without telling us anything.
3837

3938
For example a website might blacklist IPs, or whole subnets from CSCS, which would make the service unavailable for all other CSCS users too.
4039
Many sites use content delivery networks (CDN), like Cloudflare, Akamai, or similar, and if those blacklist the CSCS many users will be affected.
4140
In addition, once we are blacklisted, it's extremely difficult and long be able to get out of these blacklists.
4241

4342
!!! info
44-
Sites do not publish the number of requests/queries per second that trigger blacklisting, for some obvious reason that bad-intentioned people would stay just below this limit.
43+
Sites do not publish the number of requests/queries per second that trigger blacklisting, for some obvious reason that bad-intentioned people would stay just below this limit.
4544

4645
So you should be mindful of your usage, in particular of the number of requests to the DNS and the network bandwidth.
4746
Every access to a different domain will trigger a DNS request, using multiple nodes does not solve the problem, because they will still be hitting the same DNS resolver.
@@ -50,8 +49,8 @@ CSCS has protection in place for our public DNS server, but other DNS servers mi
5049
Alps uses an internal DNS, which is also used to resolve the different nodes in alps, and does not have special protections against abuse.
5150
For this reason **avoid scraping from Alps**, as it could lead to it being blacklisted.
5251

53-
!!! info
54-
Given the excellent connection of the CSCS network with SWITCH a sustained use of it can saturate the connection of a large provider like Google, which would affect all Swiss Google users.
52+
!!! warning
53+
The high-capacity of the CSCS-SWITCH connection can saturate the connection of a large provider like Google, which would affect all Swiss Google users.
5554

5655
### Conclusions
5756

0 commit comments

Comments
 (0)