Skip to content

Commit 45f605f

Browse files
rochaportorochaporto
authored
Merge pull request #25 from cern-eos/docs/how-to-use
docs: update `how-to-use.md`
2 parents aa04275 + de07ba2 commit 45f605f

File tree

1 file changed

+216
-1
lines changed

1 file changed

+216
-1
lines changed

‎docs/how-to-use.md‎

Lines changed: 216 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,218 @@
11
# How to use EOSxd CSI driver in Kubernetes
22

3-
(TODO)
3+
`eosxd-csi` supports mounting EOS instances and exposing them inside your Pods as a single PVC.
4+
5+
## Getting Started
6+
7+
Ensure that you have the appropriate `StorageClass` configured in your cluster this can be done either be enabling `commonStorageClass.enabled` on installation or by running the below.
8+
9+
```bash
10+
cat << EOF | kubectl apply -f -
11+
apiVersion: storage.k8s.io/v1
12+
kind: StorageClass
13+
metadata:
14+
name: eos
15+
provisioner: eosxd.csi.cern.ch
16+
EOF
17+
```
18+
19+
Provision a `PVC` using this `StorageClass` and mount this to the `Pod`'s which require access to `EOS`.
20+
21+
```bash
22+
cat << EOF | kubectl apply -f -
23+
apiVersion: v1
24+
kind: PersistentVolumeClaim
25+
metadata:
26+
name: eos
27+
spec:
28+
accessModes:
29+
- ReadWriteMany
30+
resources:
31+
requests:
32+
# Volume size value has no effect and is ignored
33+
# by the driver, but must be non-zero.
34+
storage: 1
35+
storageClassName: eos
36+
37+
EOF
38+
```
39+
40+
Please note that when mounting the PVC, the Pod’s volumeMounts spec of the volume must set `mountPropagation: HostToContainer`.
41+
42+
```yaml
43+
...
44+
volumeMounts:
45+
- name: eos
46+
mountPath: /eos
47+
mountPropagation: HostToContainer
48+
...
49+
```
50+
51+
`eosxd-csi` exposes the autofs-EOS root in `/var/eos` on the host node, whilst mounting via `PVC` is preferred you can use this `hostPath`.
52+
53+
```yaml
54+
...
55+
containers:
56+
- volumeMounts:
57+
- name: eos
58+
mountPath: /eos
59+
mountPropagation: HostToContainer
60+
...
61+
volumes:
62+
- name: eos
63+
hostPath:
64+
path: /var/eos
65+
type: Directory
66+
...
67+
```
68+
69+
## Authentication
70+
71+
Access to EOS requires authentication. Popular options are:
72+
73+
* Kerberos tickets
74+
* OAuth2 tokens
75+
76+
### Kerberos
77+
78+
Authentication using Kerberos tickets requires that your Pod to come with Kerberos tools installed.
79+
80+
Steps:
81+
82+
1. Create a Pod mounting the EOS PVC.
83+
2. Inside the Pod, use `kinit <CERN login name>@CERN.CH` to create a Kerberos ticket.
84+
Example:
85+
86+
```bash
87+
cat <<EOF | kubectl apply -f -
88+
apiVersion: v1
89+
kind: Pod
90+
metadata:
91+
name: eos-kerberos
92+
spec:
93+
containers:
94+
- name: my-container
95+
image: registry.cern.ch/docker.io/cern/alma9-base:latest
96+
imagePullPolicy: IfNotPresent
97+
command: ["sleep", "inf"]
98+
volumeMounts:
99+
- name: eos
100+
mountPath: /eos
101+
mountPropagation: HostToContainer
102+
volumes:
103+
- name: eos
104+
persistentVolumeClaim:
105+
claimName: eos
106+
EOF
107+
108+
# This example exec's into the eos-kerberos Pod, creates
109+
# a Kerberos ticket for user "rvasek" using the kinit command,
110+
# and lists the home directory.
111+
$ kubectl exec -it eos-kerberos -- bash
112+
[root@eos-kerberos /]# kinit [email protected]
113+
Password for [email protected]: <CERN login password>
114+
[root@eos-kerberos /]# cd /eos/home-r/rvasek
115+
[root@eos-kerberos rvasek]# ls -l
116+
total 24
117+
drwxr-xr-x. 2 110701 2763 4096 Sep 9 2022 Documents
118+
drwxr-xr-x. 2 110701 2763 4096 Feb 2 2020 Music
119+
drwxr-xr-x. 2 110701 2763 4096 Feb 2 2020 Pictures
120+
drwxr-xr-x. 2 110701 2763 4096 May 30 14:59 SWAN_projects
121+
drwxr-xr-x. 2 110701 2763 4096 Feb 2 2020 Videos
122+
...
123+
```
124+
125+
### OAuth2
126+
127+
At CERN, we maintain a `oauth2-refresh-controller` that allows the use of an OIDC application for authentication.
128+
129+
For more details please see the [gitlab repo](https://gitlab.cern.ch/kubernetes/security/oauth2-refresh-controller).
130+
131+
We detail the setup process for using this controller at CERN as an illustrative end-to-end example, but naturally this will differ according to your organisation.
132+
133+
To ensure you have a valid access token at CERN, it must satisfy the following:
134+
135+
* Be created with the openid scope (at least) and an EOS-enabled client ID.
136+
* Stored in `/tmp/oauthtk_<Container's UID>`*.
137+
* Group and ownership set to match the container’s GID and UID*.
138+
* Have file permissions set to 0400.
139+
* Have it's file contents in the format of oauth2:<OAuth2 access token>:auth.cern.ch/auth/realms/cern/protocol/openid-connect/userinfo.
140+
141+
*: or whatever the effective UID and GID are set to in the process accessing the EOS storage)
142+
143+
The example below then creates a `Secret` with an `Oauth2` JWT and a `ConfigMap` with the EOS-specific token format. The `oauth2-refresh-controller` annotation on the pod is then used to inject and continually refresh your EOS token.
144+
```bash
145+
cat << EOF | kubectl apply -f -
146+
apiVersion: v1
147+
kind: Secret
148+
metadata:
149+
name: my-oauth2-token
150+
annotations:
151+
oauth2-refresh-controller.cern.ch/is-token: "true"
152+
stringData:
153+
oauth2: "<OAuth2 JWT>"
154+
clientID: "<OIDC Client ID>"
155+
clientSecret: "<OIDC Client password>"
156+
157+
---
158+
apiVersion: v1
159+
kind: ConfigMap
160+
metadata:
161+
name: oauth2-token-eos-template
162+
data:
163+
# This defines what the token file will contain.
164+
# The "$(ACCESS_TOKEN)" variable is expanded into
165+
# the actual value by oauth2-refresh-controller.
166+
template: "oauth2:$(ACCESS_TOKEN):auth.cern.ch/auth/realms/cern/protocol/openid-connect/userinfo"
167+
168+
---
169+
apiVersion: v1
170+
kind: Pod
171+
metadata:
172+
name: eos-oauth2
173+
annotations:
174+
oauth2-refresh-controller.cern.ch/to-inject: |
175+
[
176+
{
177+
"secretName": "my-oauth2-token",
178+
"container": "my-container",
179+
"templateConfigMapName": "oauth2-token-eos-template",
180+
"owner": 0
181+
}
182+
]
183+
spec:
184+
containers:
185+
- name: my-container
186+
image: registry.cern.ch/docker.io/cern/alma9-base:latest
187+
imagePullPolicy: IfNotPresent
188+
command: ["sleep", "inf"]
189+
volumeMounts:
190+
- name: eos
191+
mountPath: /eos
192+
mountPropagation: HostToContainer
193+
volumes:
194+
- name: eos
195+
persistentVolumeClaim:
196+
claimName: eos
197+
EOF
198+
199+
# This example exec's into the eos-oauth2 Pod and lists
200+
# the home directory of the user "rvasek". The OAuth2 access
201+
# token is injected into the container and renewed automatically.
202+
203+
$ kubectl exec -it eos-oauth2 -- bash
204+
Defaulted container "my-container" out of: my-container, oauth2-refresh-bootstrap-0 (init)
205+
[root@eos-oauth2 rvasek]# cd /eos/home-r/rvasek
206+
[root@eos-oauth2 rvasek]# ls -l
207+
total 24
208+
drwxr-xr-x. 2 110701 2763 4096 Sep 9 2022 Documents
209+
drwxr-xr-x. 2 110701 2763 4096 Feb 2 2020 Music
210+
drwxr-xr-x. 2 110701 2763 4096 Feb 2 2020 Pictures
211+
drwxr-xr-x. 2 110701 2763 4096 May 30 14:59 SWAN_projects
212+
drwxr-xr-x. 2 110701 2763 4096 Feb 2 2020 Videos
213+
...
214+
```
215+
216+
217+
For more information on getting started please refer to examples in [../example/](../example/) or CERN's public documentation for [eosxd-csi](https://kubernetes.docs.cern.ch/docs/storage/eos/).
218+

0 commit comments

Comments
 (0)