You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/guide/why.md
+17-22Lines changed: 17 additions & 22 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -2,11 +2,15 @@
2
2
3
3
## Problems
4
4
5
-
Before Roselab servers were available, students run their machine learning projects on either the Nautilus cluster or CSE shared servers. The two options were at extreme ends of the virtualization spectrum. Nautilus provided complete virtualization in an ephemeral application container while the shared servers offered no virtualization. The actual machine learning development falls between the two. Student developers aimed to protect their computing resources and data from disruption by others. Also, they needed a persistent development environment that would not disappear after every run.
5
+
Before Roselab servers were available, students ran their machine learning projects on either the Nautilus cluster or CSE shared servers. These two options were at extreme ends of the virtualization spectrum. Nautilus provided complete virtualization in ephemeral application containers, while the shared servers offered no virtualization. The actual machine learning development needs fell between these two extremes. Student developers aimed to protect their computing resources and data from disruption by others. Additionally, they needed a persistent development environment that would not disappear after every run.
6
6
7
-
Attempts to resolve these issues through the use of Linux user permission and [persistent storage](https://kubernetes.io/docs/concepts/storage/persistent-volumes/) were made, but these solutions still had their limitations. Linux user permission provided isolation for each user's environment. However, it lacks network isolation, requires a significant amount of conventional rules, and makes the installation of dependencies difficult. Although persistent storage allows for a persistent environment in the container, the storage was network mounted and spatially distant from the running server, resulting in slower read and write speeds compared to ephemeral storage.
7
+
::: tip Note
8
+
The problems for CSE shared servers no longer apply as the servers have undergone migration to Linux containers (similar system to Roselab servers) and are now managed by Roselab administrators.
9
+
:::
8
10
9
-
Roselab servers aims to address these issues by leveraging advanced technology such as [ZFS](https://en.wikipedia.org/wiki/ZFS) and [Linux Containers](https://linuxcontainers.org/). These servers offer a sufficient level of storage, CPU, memory, and network isolation, while maintaining minimal performance costs. Each user has root and desktop access to the server as if it were their personal workstation.
11
+
Attempts to resolve these issues through the use of Linux user permissions and [persistent storage](https://kubernetes.io/docs/concepts/storage/persistent-volumes/) were made, but these solutions still had their limitations. Linux user permissions provided isolation for each user's environment. However, it lacked network isolation, required a significant number of conventional rules, and made the installation of dependencies difficult. Although persistent storage allowed for a persistent environment in the container, the storage was network-mounted and spatially distant from the running server, resulting in slower read and write speeds compared to ephemeral storage.
12
+
13
+
Roselab servers aim to address these issues by leveraging advanced technology such as [ZFS](https://en.wikipedia.org/wiki/ZFS) and [Linux Containers](https://linuxcontainers.org/). These servers offer sufficient levels of storage, CPU, memory, and network isolation while maintaining minimal performance costs. Each user has root and desktop access to the server as if it were their personal workstation.
10
14
11
15
## Security and Privacy
12
16
@@ -19,31 +23,31 @@ The Nautilus cluster manages user credentials on a namespace basis, which means
19
23
- Inspect anyone's stored secrets
20
24
- Delete anyone's persistent volumes
21
25
22
-
Additionally, if a developer carelessly stores a Github SSH key with write access in the Nautilus secrets, others could potentially delete all their Github repositories.
26
+
Additionally, if a developer carelessly stores a GitHub SSH key with write access in the Nautilus secrets, others could potentially delete all their GitHub repositories.
23
27
24
-
The CSE shared servers offer many users sudo access, without designating an administrator. This gives many accounts the permission to perform any actions mentioned above. Additionally, many users use short, explicit passwords instead of SSH to log in, further increasing the security risk. Although the CSE shared servers are currently secured under the UCSD intranet, if a device with lesser protection in the CSE server room is compromised, the CSE shared servers is vulnerable.
28
+
The CSE shared servers offer many users sudo access, without designating an administrator. This gives many accounts the permission to perform any actions mentioned above. Additionally, many users use short, explicit passwords instead of SSH to log in, further increasing the security risk. Although the CSE shared servers are currently secured under the UCSD intranet, if a device with lesser protection in the CSE server room is compromised, the CSE shared servers are vulnerable.
25
29
26
-
Roselab servers have a single adminstrator. Only this administrator has full access to all containers. By default, all containers require an SSH private key for login. Thanks to the isolation between the containers, every user cannot access others' resources. Even if one container were to become compromised due to leaking SSH key, the others would remain invulnerable.
30
+
Roselab servers have a single administrator. Only this administrator has full access to all containers. By default, all containers require an SSH private key for login. Thanks to the isolation between the containers, users cannot access each other's resources. Even if one container were to become compromised due to a leaked SSH key, the others would remain secure.
27
31
28
32
## Usability
29
33
30
-
We list a few use cases where Roselab servers would make your life much easier.
34
+
Here are a few use cases where Roselab servers can make your life much easier.
31
35
32
36
### Big Data Transfer
33
37
34
-
The Nautilus system has gained a reputation for its sluggish dataset transfer speeds, due to its network-mounted persistent storage. The CSE shared servers have faster speeds but limited storage. Each server has up to 4 TB disk space with no virtualization technology, leaving them vulnerable to disk failure.
38
+
The Nautilus system is known for its slow dataset transfer speeds due to its network-mounted persistent storage. The CSE shared servers have faster speeds but limited storage. Each server has up to 4 TB of disk space with no virtualization technology, making them vulnerable to disk failure.
35
39
36
-
On the other hand, the RoseLab servers offer a massive 120 TB data server with the reliable [RAID-Z](https://www.raidz-calculator.com/raidz-types-reference.aspx) technology that can continue functioning after one disk failure. Because RoseLab is located in the CSE building, uploading data is quick and efficient. Students can also directly plug in an external hard drive to transfer data by contacting the administrator.
40
+
In contrast, the RoseLab servers offer a massive 120 TB data server with reliable [RAID-Z](https://www.raidz-calculator.com/raidz-types-reference.aspx) technology that can continue functioning even after a disk failure. The same shared storage server is accessible from all machines with dual 100 Gbps connections, avoiding the need to copy dataset between servers.
37
41
38
42
### Remote GUI Desktop
39
43
40
-
The ability to use a remote desktop is essential for some machine learning tasks, such as OpenAI Gym, and for quickly inspecting generated videos or images. However, setting up a remote desktop on the CSE shared servers often requires elevated permissions and can interfere with other services due to a lack of network isolation. Using a remote desktop on Nautilus can also result in high latency if the assigned server is far away, and requires manual creation of a pod with a limited lifespan.
44
+
The ability to use a remote desktop is essential for some machine learning tasks, such as OpenAI Gym, and for quickly inspecting generated videos or images. However, setting up a remote desktop on the CSE shared servers often requires elevated permissions and can interfere with other services due to a lack of network isolation. Using a remote desktop on Nautilus can also result in high latency if the assigned server is far away, and it requires manual creation of a pod with a limited lifespan.
41
45
42
-
Fortunately, the RoseLab servers offer seamless remote desktop support for all users, without interfering with each other. The remote desktop connection is always available, providing a reliable and convenient solution for monitoring machine learning tasks.
46
+
Fortunately, the RoseLab servers offer seamless remote desktop support for all users without interference. The remote desktop connection is always available, providing a reliable and convenient solution for monitoring machine learning tasks.
43
47
44
48
### Network Access
45
49
46
-
Nautilus may sometimes have stability issues because of the large volume of nodes and requests. For instance, it could encounter DNS resolution failures.
50
+
Nautilus may sometimes have stability issues due to the large volume of nodes and requests. For instance, it could encounter DNS resolution failures.
@@ -52,14 +56,5 @@ Failed to download (trying next):
52
56
<urlopen error [Errno -3] Temporary failure in name resolution>
53
57
```
54
58
55
-
While the CSE shared servers have a more reliable internet connection, they also have many outbound firewall rules in place. These rules can limit access to commonly used development websites, like github.com.
56
-
57
-
```bash
58
-
$ ping github.com
59
-
PING github.com (140.82.114.4) 56(84) bytes of data.
60
-
^C
61
-
--- github.com ping statistics ---
62
-
5 packets transmitted, 0 received, 100% packet loss, time 4103ms
63
-
```
59
+
While the CSE shared servers have a more reliable internet connection, they also have many outbound firewall rules in place. These rules limit access to commonly used development websites like `github.com`. In contrast, RoseLab does not enforce outbound firewall rules, leaving network security responsibilities to individual users. Nevertheless, the servers still have a comprehensive list of inbound rules that are strictly enforced, providing strong protection against malware and cyber attacks.
64
60
65
-
In contrast, RoseLab does not enforce outbound firewall rules, leaving network security responsibilities in the hands of individual users. Nevertheless, the servers still have a comprehensive list of inbound rules that are strictly enforced, providing strong protection against malware and cyber attacks.
0 commit comments