Skip to content

Commit 770dcaf

Browse files
AlldaAllda
committed
Create custom SCC with privilege escalation
The OpenShift pipelines 1.9 no longer allow running containers with privilege escalation. Since a subset of pipeline tasks requires the privilege escalation we need to use a custom SCC that is build on top of pipeline SCC and add the escalation. This commit adds the custom SCC, links it with the pipeline service account and refactors ansible to be compatible with the new setup. JIRA: ISV-3280
1 parent ea9e673 commit 770dcaf

15 files changed

+264
-127
lines changed

‎README.md‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -72,6 +72,16 @@ tkn pipeline start operator-ci-pipeline \
7272
--showlog
7373
```
7474

75+
A subset of tasks in the pipeline requires privilege escalation which is no longer
76+
supported with OpenShift Pipelines 1.9. Thus a new `SCC` needs to be created and linked
77+
with `pipeline` service account. Creating [SCC](https://docs.openshift.com/container-platform/4.11/authentication/managing-security-context-constraints.html#security-context-constraints-creating_configuring-internal-oauth)
78+
requires user with cluster-admin privileges.
79+
```bash
80+
# Create a new SCC
81+
oc apply -f ansible/roles/operator-pipeline/templates/openshift/openshift-pipelines-custom-scc.yml
82+
# Add SCC to a pipeline service account
83+
oc adm policy add-scc-to-user pipelines-custom-scc -z pipeline
84+
```
7585

7686

7787
To enable opening the PR and uploading the pipeline logs (visible to the certification project

‎ansible/init.sh‎

Lines changed: 4 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -25,19 +25,17 @@ get_environments() {
2525

2626
# execute playbook for given environment
2727
execute_playbook() {
28-
local secret=$(dirname "$0")/vaults/$env/secret-vars.yml
28+
local secret=$(dirname "$0")/vaults/$env/ocp-token.yml
2929
if [ ! -f $secret ]; then
3030
touch $secret
3131
echo "File $secret was not found, empty one was created"
3232
fi
3333

34-
ansible-playbook -i inventory/operator-pipeline playbooks/deploy.yml \
34+
ansible-playbook -i inventory/operator-pipeline-$1 playbooks/operator-pipeline-env-setup.yml \
3535
--vault-password-file=$2 \
3636
-e "env=$1" \
3737
-e "ocp_host=`oc whoami --show-server`" \
38-
-e "ocp_token=`oc whoami -t`" \
39-
--tags init \
40-
-vvvv
38+
-e "ocp_token=`oc whoami -t`"
4139
}
4240

4341
# update token for given environment
@@ -49,7 +47,7 @@ update_token() {
4947
| awk '$2=="kubernetes.io/service-account-token" && $1~/^operator-pipeline-admin-token/ {print $3; exit}' \
5048
| base64 -d)
5149

52-
local secret=$(dirname "$0")/vaults/$env/secret-vars.yml
50+
local secret=$(dirname "$0")/vaults/$env/ocp-token.yml
5351

5452
echo "ocp_token: $token" > $secret
5553
ansible-vault encrypt $secret --vault-password-file $2 > /dev/null

‎ansible/inventory/group_vars/operator-pipeline.yml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,7 @@
33
ansible_connection: local
44
env: unset
55
oc_namespace: "operator-pipeline-{{ env }}"
6+
service_account_name: operator-pipeline-admin
67
branch: "{{ env }}"
78
preflight_min_version: 0.0.0
89
preflight_trigger_environment: preprod

‎ansible/playbooks/operator-pipeline-env-setup.yml‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
---
2+
- name: Setup operator-pipeline namespace
3+
hosts: "operator-pipeline-{{ env }}"
4+
5+
tasks:
6+
- name: Configure namespace + service account
7+
include_role:
8+
name: operator-pipeline
9+
tasks_from: operator-pipeline-service-account

‎ansible/roles/operator-pipeline/tasks/main.yml‎

Lines changed: 12 additions & 68 deletions
Original file line numberDiff line numberDiff line change
@@ -15,77 +15,21 @@
1515
- name: Deploy namespace resources
1616
when: namespace_state == 'present'
1717
block:
18-
- include_tasks: tasks/pipeline-secrets.yml
1918

20-
# We can't use the k8s module here because it always tries to GET the resource
21-
# prior to creation or patching. The ImageStreamImport API only supports POST.
22-
- name: Import certified-operator-index imagestream
23-
tags:
24-
- import-index-images
25-
no_log: yes
26-
uri:
27-
url: "{{ ocp_host }}/apis/image.openshift.io/v1/namespaces/{{ oc_namespace }}/imagestreamimports"
28-
method: POST
29-
# The 'or' condition is needed to support Ansible versions < 2.13
30-
validate_certs: "{{ lookup('env', 'K8S_AUTH_VERIFY_SSL', default='yes') or 'yes' }}"
31-
status_code: 201
32-
body_format: json
33-
headers:
34-
Authorization: "Bearer {{ ocp_token }}"
35-
body:
36-
apiVersion: image.openshift.io/v1
37-
kind: ImageStreamImport
38-
metadata:
39-
name: certified-operator-index
40-
labels:
41-
app: operator-pipeline
42-
suffix: "{{ suffix }}"
43-
env: "{{ env }}"
44-
spec:
45-
import: true
46-
repository:
47-
from:
48-
kind: DockerImage
49-
name: "{{ certified_operator_index }}"
50-
importPolicy:
51-
insecure: "{{ insecure_index_import | default(false) }}"
52-
scheduled: true
53-
referencePolicy:
54-
type: Local
19+
- include_tasks: tasks/pipeline-secrets.yml
20+
- include_tasks: tasks/operator-pipeline-import-index-images.yml
5521

56-
- name: Import redhat-marketplace-index imagestream
22+
- name: Add custom pipeline SCC and link it with pipeline SA
5723
tags:
58-
- import-index-images
59-
no_log: true
60-
ansible.builtin.uri:
61-
url: "{{ ocp_host }}/apis/image.openshift.io/v1/namespaces/{{ oc_namespace }}/imagestreamimports"
62-
method: POST
63-
# The 'or' condition is needed to support Ansible versions < 2.13
64-
validate_certs: "{{ lookup('env', 'K8S_AUTH_VERIFY_SSL', default='yes') or 'yes' }}"
65-
status_code: 201
66-
body_format: json
67-
headers:
68-
Authorization: "Bearer {{ ocp_token }}"
69-
body:
70-
apiVersion: image.openshift.io/v1
71-
kind: ImageStreamImport
72-
metadata:
73-
name: redhat-marketplace-index
74-
labels:
75-
app: operator-pipeline
76-
suffix: "{{ suffix }}"
77-
env: "{{ env }}"
78-
spec:
79-
import: true
80-
repository:
81-
from:
82-
kind: DockerImage
83-
name: "{{ redhat_marketplace_index }}"
84-
importPolicy:
85-
insecure: "{{ insecure_index_import | default(false) }}"
86-
scheduled: true
87-
referencePolicy:
88-
type: Local
24+
- pipeline-scc
25+
k8s:
26+
state: present
27+
apply: true
28+
definition: "{{ lookup('template', '{{ item }}') }}"
29+
with_items:
30+
- ../templates/openshift/openshift-pipelines-custom-scc.yml
31+
- ../templates/openshift/openshift-pipeline-sa-scc-role.yml
32+
- ../templates/openshift/openshift-pipeline-sa-scc-role-bindings.yml
8933

9034
- name: Deploy pipeline tasks
9135
tags:

‎ansible/roles/operator-pipeline/tasks/operator-pipeline-import-index-images.yml‎

Lines changed: 70 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,70 @@
1+
---
2+
# We can't use the k8s module here because it always tries to GET the resource
3+
# prior to creation or patching. The ImageStreamImport API only supports POST.
4+
- name: Import certified-operator-index imagestream
5+
tags:
6+
- import-index-images
7+
no_log: yes
8+
uri:
9+
url: "{{ ocp_host }}/apis/image.openshift.io/v1/namespaces/{{ oc_namespace }}/imagestreamimports"
10+
method: POST
11+
# The 'or' condition is needed to support Ansible versions < 2.13
12+
validate_certs: "{{ lookup('env', 'K8S_AUTH_VERIFY_SSL', default='yes') or 'yes' }}"
13+
status_code: 201
14+
body_format: json
15+
headers:
16+
Authorization: "Bearer {{ ocp_token }}"
17+
body:
18+
apiVersion: image.openshift.io/v1
19+
kind: ImageStreamImport
20+
metadata:
21+
name: certified-operator-index
22+
labels:
23+
app: operator-pipeline
24+
suffix: "{{ suffix }}"
25+
env: "{{ env }}"
26+
spec:
27+
import: true
28+
repository:
29+
from:
30+
kind: DockerImage
31+
name: "{{ certified_operator_index }}"
32+
importPolicy:
33+
insecure: "{{ insecure_index_import | default(false) }}"
34+
scheduled: true
35+
referencePolicy:
36+
type: Local
37+
38+
- name: Import redhat-marketplace-index imagestream
39+
tags:
40+
- import-index-images
41+
no_log: true
42+
ansible.builtin.uri:
43+
url: "{{ ocp_host }}/apis/image.openshift.io/v1/namespaces/{{ oc_namespace }}/imagestreamimports"
44+
method: POST
45+
# The 'or' condition is needed to support Ansible versions < 2.13
46+
validate_certs: "{{ lookup('env', 'K8S_AUTH_VERIFY_SSL', default='yes') or 'yes' }}"
47+
status_code: 201
48+
body_format: json
49+
headers:
50+
Authorization: "Bearer {{ ocp_token }}"
51+
body:
52+
apiVersion: image.openshift.io/v1
53+
kind: ImageStreamImport
54+
metadata:
55+
name: redhat-marketplace-index
56+
labels:
57+
app: operator-pipeline
58+
suffix: "{{ suffix }}"
59+
env: "{{ env }}"
60+
spec:
61+
import: true
62+
repository:
63+
from:
64+
kind: DockerImage
65+
name: "{{ redhat_marketplace_index }}"
66+
importPolicy:
67+
insecure: "{{ insecure_index_import | default(false) }}"
68+
scheduled: true
69+
referencePolicy:
70+
type: Local

‎ansible/roles/operator-pipeline/tasks/operator-pipeline-service-account.yml‎

Lines changed: 47 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,47 @@
1+
---
2+
- name: Configure Namespace
3+
tags:
4+
- namespace
5+
- init
6+
k8s:
7+
state: "present"
8+
definition:
9+
kind: Namespace
10+
apiVersion: v1
11+
metadata:
12+
name: "{{ oc_namespace }}"
13+
14+
- name: Create service account
15+
tags:
16+
- service-account
17+
- init
18+
k8s:
19+
state: present
20+
namespace: "{{ oc_namespace }}"
21+
definition:
22+
apiVersion: v1
23+
kind: ServiceAccount
24+
metadata:
25+
name: "{{ service_account_name }}"
26+
27+
- name: Grant SA "{{ service_account_name }}" role
28+
tags:
29+
- service-account
30+
- init
31+
k8s:
32+
state: present
33+
namespace: "{{ oc_namespace }}"
34+
definition:
35+
apiVersion: rbac.authorization.k8s.io/v1
36+
kind: ClusterRoleBinding
37+
metadata:
38+
name: "{{ service_account_name }}-{{ item }}"
39+
roleRef:
40+
kind: ClusterRole
41+
name: "{{ item }}"
42+
subjects:
43+
- kind: ServiceAccount
44+
name: "{{ service_account_name }}"
45+
namespace: "{{ oc_namespace }}"
46+
with_items:
47+
- cluster-admin

‎ansible/roles/operator-pipeline/templates/openshift/openshift-pipeline-sa-scc-role-bindings.yml‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
---
2+
kind: ClusterRoleBinding
3+
apiVersion: rbac.authorization.k8s.io/v1
4+
metadata:
5+
name: "system:openshift:scc:pipelines-custom-scc"
6+
subjects:
7+
- kind: ServiceAccount
8+
name: pipeline
9+
namespace: "{{ oc_namespace }}"
10+
roleRef:
11+
apiGroup: rbac.authorization.k8s.io
12+
kind: ClusterRole
13+
name: "system:openshift:scc:pipelines-custom-scc"

‎ansible/roles/operator-pipeline/templates/openshift/openshift-pipeline-sa-scc-role.yml‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
kind: ClusterRole
2+
apiVersion: rbac.authorization.k8s.io/v1
3+
metadata:
4+
name: 'system:openshift:scc:pipelines-custom-scc'
5+
rules:
6+
- verbs:
7+
- use
8+
apiGroups:
9+
- security.openshift.io
10+
resources:
11+
- securitycontextconstraints
12+
resourceNames:
13+
- pipelines-custom-scc

‎ansible/roles/operator-pipeline/templates/openshift/openshift-pipelines-custom-scc.yml‎

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,42 @@
1+
---
2+
apiVersion: security.openshift.io/v1
3+
kind: SecurityContextConstraints
4+
metadata:
5+
annotations:
6+
include.release.openshift.io/ibm-cloud-managed: "true"
7+
include.release.openshift.io/self-managed-high-availability: "true"
8+
include.release.openshift.io/single-node-developer: "true"
9+
kubernetes.io/description: pipelines-custom-scc is a close replica of pipeline scc with privilege escalation.
10+
release.openshift.io/create-only: "true"
11+
name: pipelines-custom-scc
12+
allowHostDirVolumePlugin: false
13+
allowHostIPC: false
14+
allowHostNetwork: false
15+
allowHostPID: false
16+
allowHostPorts: false
17+
allowPrivilegeEscalation: true
18+
allowPrivilegedContainer: true
19+
allowedCapabilities:
20+
- SETFCAP
21+
defaultAddCapabilities: null
22+
fsGroup:
23+
type: MustRunAs
24+
groups:
25+
- system:cluster-admins
26+
priority: 10
27+
readOnlyRootFilesystem: false
28+
requiredDropCapabilities:
29+
- MKNOD
30+
runAsUser:
31+
type: RunAsAny
32+
seLinuxContext:
33+
type: MustRunAs
34+
supplementalGroups:
35+
type: RunAsAny
36+
volumes:
37+
- configMap
38+
- downwardAPI
39+
- emptyDir
40+
- persistentVolumeClaim
41+
- projected
42+
- secret

0 commit comments

Comments
 (0)