Skip to content

Commit bed5500

Browse files
johnmarcojohnmarco
committed
Initial draft done
1 parent 336a37b commit bed5500

File tree

1 file changed

+307
-19
lines changed

1 file changed

+307
-19
lines changed

‎articles/openshift/confidential-containers-deploy.md

Lines changed: 307 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -141,16 +141,12 @@ This article describes the steps required to deploy Confidential Containers for
141141
142142
`$ watch oc get csv -n openshift-sandboxed-containers-operator`
143143
144-
Example Output:
144+
**Example Output**
145145
```
146146
NAME DISPLAY VERSION REPLACES PHASE
147147
openshift-sandboxed-containers openshift-sandboxed-containers-operator 1.7.0 1.6.0 Succeeded
148148
```
149149
150-
151-
152-
153-
154150
### Create the peer pods secret
155151
156152
You must create the peer pods secret for OpenShift sandboxed containers. The secret stores credentials for creating the pod virtual machine (VM) image and peer pod instances.
@@ -166,18 +162,21 @@ By default, the OpenShift sandboxed containers Operator creates the secret based
166162
```
167163
168164
1. Generate the RBAC content by running the following command:
169-
165+
170166
```
171167
$ az ad sp create-for-rbac --role Contributor --scopes /subscriptions/$AZURE_SUBSCRIPTION_ID \
172-
--query "{ client_id: appId, client_secret: password, tenant_id: tenant }"
173-
Example output
168+
--query "{ client_id: appId, client_secret: password, tenant_id: tenant }"
169+
```
170+
171+
**Example output**
172+
```
174173
{
175-
"client_id": `AZURE_CLIENT_ID`,
176-
"client_secret": `AZURE_CLIENT_SECRET`,
177-
"tenant_id": `AZURE_TENANT_ID`
178-
}
174+
"client_id": `AZURE_CLIENT_ID`,
175+
"client_secret": `AZURE_CLIENT_SECRET`,
176+
"tenant_id": `AZURE_TENANT_ID`
177+
}
179178
```
180-
179+
181180
1. Record the RBAC output to use in the secret object.
182181
183182
1. Create a peer-pods-secret.yaml manifest file according to the following example:
@@ -360,7 +359,7 @@ metadata:
360359
361360
`$ watch oc get csv -n trustee-operator-system`
362361
363-
Example output:
362+
**Example output**
364363
```
365364
NAME DISPLAY PHASE
366365
trustee-operator.v0.1.0 Trustee Operator 0.1.0 Succeeded
@@ -393,7 +392,7 @@ Create a secure route with edge TLS termination for Trustee. External ingress tr
393392
394393
`$ echo $TRUSTEE_HOST`
395394
396-
Example output:
395+
**Example output**
397396
`kbs-service-trustee-operator-system.apps.memvjias.eastus.aroapp.io`
398397
399398
Record this value for the peer pods config map.
@@ -402,7 +401,7 @@ Create a secure route with edge TLS termination for Trustee. External ingress tr
402401
403402
1. Create a cc-feature-gate.yaml manifest file:
404403
405-
```azurecli
404+
```
406405
apiVersion: v1
407406
kind: ConfigMap
408407
metadata:
@@ -533,8 +532,7 @@ Create a secure route with edge TLS termination for Trustee. External ingress tr
533532
534533
$ oc get runtimeclass
535534
536-
Example output:
537-
535+
**Example output**
538536
```
539537
NAME HANDLER AGE
540538
kata-remote kata-remote 152m
@@ -561,7 +559,7 @@ Create a secure route with edge TLS termination for Trustee. External ingress tr
561559
562560
### Create the Trustee config map
563561
564-
1. Create a kbs-config-cm.yaml manifest file:
562+
1. Create a `kbs-config-cm.yaml` manifest file:
565563
566564
```
567565
apiVersion: v1
@@ -608,8 +606,298 @@ Create a secure route with edge TLS termination for Trustee. External ingress tr
608606
609607
610608
609+
### Configure attestation policies
610+
611+
You can configure the following attestation policy settings:
612+
613+
**Reference values**
614+
615+
You can configure reference values for the Reference Value Provider Service (RVPS) by specifying the trusted digests of your hardware platform.
616+
617+
The client collects measurements from the running software, the Trusted Execution Environment (TEE) hardware and firmware and it submits a quote with the claims to the Attestation Server. These measurements must match the trusted digests registered to the Trustee. This process ensures that the confidential VM (CVM) is running the expected software stack and has not been tampered with.
618+
619+
**Secrets for clients**
620+
621+
You must create one or more secrets to share with attested clients.
622+
623+
**Resource access policy**
624+
625+
You must configure a policy for the Trustee policy engine to determine which resources to access.
626+
627+
Do not confuse the Trustee policy engine with the Attestation Service policy engine, which determines the validity of TEE evidence.
628+
629+
**Attestation policy**
630+
631+
Optional: You can overwrite the default attestation policy by creating your own attestation policy.
632+
Provisioning Certificate Caching Service for TDX
633+
634+
If your TEE is Intel Trust Domain Extensions (TDX), you must configure the Provisioning Certificate Caching Service (PCCS). The PCCS retrieves Provisioning Certification Key (PCK) certificates and caches them in a local database.
635+
636+
1. Create an `rvps-configmap.yaml` manifest file:
637+
638+
```
639+
apiVersion: v1
640+
kind: ConfigMap
641+
metadata:
642+
name: rvps-reference-values
643+
namespace: trustee-operator-system
644+
data:
645+
reference-values.json: |
646+
[
647+
]
648+
```
649+
650+
**Notes:**
651+
652+
For `reference-values.json` specify the trusted digests for your hardware platform if required. Otherwise, leave it empty.
653+
654+
1. Create the RVPS config map by running the following command:
655+
656+
`$ oc apply -f rvps-configmap.yaml`
657+
658+
1. Create one or more secrets to share with attested clients according to the following example:
659+
660+
```
661+
$ oc create secret generic kbsres1 --from-literal key1=<res1val1> \
662+
--from-literal key2=<res1val2> -n trustee-operator-system
663+
```
664+
665+
In this example, the `kbsres1` secret has two entries (key1, key2), which the Trustee clients retrieve. You can add more secrets according to your requirements.
666+
667+
1. Create a resourcepolicy-configmap.yaml manifest file:
668+
669+
```
670+
apiVersion: v1
671+
kind: ConfigMap
672+
metadata:
673+
name: resource-policy
674+
namespace: trustee-operator-system
675+
data:
676+
policy.rego:
677+
package policy
678+
default allow = false
679+
allow {
680+
input["tee"] != "sample"
681+
}
682+
```
683+
**Notes:**
684+
685+
- The name of the resource policy, `policy.rego`, must match the resource policy defined in the Trustee config map.
686+
- The resource package policy follows the Open Policy Agent specification. This example allows the retrieval of all resources when the TEE is not the sample attester.
687+
688+
1. Create the resource policy config map by running the following command:
689+
690+
`$ oc apply -f resourcepolicy-configmap.yaml`
691+
692+
1. Optional: Create an attestation-policy.yaml manifest file according to the following example:
693+
694+
```
695+
apiVersion: v1
696+
kind: ConfigMap
697+
metadata:
698+
name: attestation-policy
699+
namespace: trustee-operator-system
700+
data:
701+
default.rego: |
702+
package policy
703+
import future.keywords.every
704+
705+
default allow = false
706+
707+
allow {
708+
every k, v in input {
709+
judge_field(k, v)
710+
}
711+
}
712+
713+
judge_field(input_key, input_value) {
714+
has_key(data.reference, input_key)
715+
reference_value := data.reference[input_key]
716+
match_value(reference_value, input_value)
717+
}
718+
719+
judge_field(input_key, input_value) {
720+
not has_key(data.reference, input_key)
721+
}
722+
723+
match_value(reference_value, input_value) {
724+
not is_array(reference_value)
725+
input_value == reference_value
726+
}
727+
728+
match_value(reference_value, input_value) {
729+
is_array(reference_value)
730+
array_include(reference_value, input_value)
731+
}
732+
733+
array_include(reference_value_array, input_value) {
734+
reference_value_array == []
735+
}
736+
737+
array_include(reference_value_array, input_value) {
738+
reference_value_array != []
739+
some i
740+
reference_value_array[i] == input_value
741+
}
742+
743+
has_key(m, k) {
744+
_ = m[k]
745+
}
746+
```
747+
**Notes:**
748+
749+
For `package policy`, The attestation policy follows the Open Policy Agent specification. In this example, the attestation policy compares the claims provided in the attestation report to the reference values registered in the RVPS database. The attestation process is successful only if all the values match.
750+
751+
1. Create the attestation policy config map by running the following command:
752+
753+
`$ oc apply -f attestation-policy.yaml`
754+
755+
1. If your TEE is Intel TDX, create a tdx-config.yaml manifest file:
756+
757+
```
758+
apiVersion: v1
759+
kind: ConfigMap
760+
metadata:
761+
name: tdx-config
762+
namespace: trustee-operator-system
763+
data:
764+
sgx_default_qcnl.conf: | \
765+
{
766+
"collateral_service": "https://api.trustedservices.intel.com/sgx/certification/v4/",
767+
"pccs_url": "<pccs_url>"
768+
}
769+
```
770+
**Notes:**
771+
772+
For `pccs_url`, specify the PCCS URL, for example, https://localhost:8081/sgx/certification/v4/.
773+
774+
1. Create the TDX config map by running the following command:
775+
776+
`$ oc apply -f tdx-config.yaml`
777+
778+
779+
### Create the KbsConfig custom resource
780+
781+
You must create the KbsConfig custom resource (CR) to launch Trustee. Then, you check the Trustee pods and pod logs to verify the configuration.
782+
783+
1. Create a `kbsconfig-cr.yaml` manifest file:
784+
785+
```
786+
apiVersion: confidentialcontainers.org/v1alpha1
787+
kind: KbsConfig
788+
metadata:
789+
labels:
790+
app.kubernetes.io/name: kbsconfig
791+
app.kubernetes.io/instance: kbsconfig
792+
app.kubernetes.io/part-of: trustee-operator
793+
app.kubernetes.io/managed-by: kustomize
794+
app.kubernetes.io/created-by: trustee-operator
795+
name: kbsconfig
796+
namespace: trustee-operator-system
797+
spec:
798+
kbsConfigMapName: kbs-config-cm
799+
kbsAuthSecretName: kbs-auth-public-key
800+
kbsDeploymentType: AllInOneDeployment
801+
kbsRvpsRefValuesConfigMapName: rvps-reference-values
802+
kbsSecretResources: ["kbsres1"]
803+
kbsResourcePolicyConfigMapName: resource-policy
804+
```
805+
806+
1. Create the KbsConfig CR by running the following command:
807+
808+
`$ oc apply -f kbsconfig-cr.yaml`
809+
810+
#### Verification
611811
812+
1. Set the default project by running the following command:
612813
814+
`$ oc project trustee-operator-system`
815+
816+
1. Check the pods by running the following command:
817+
818+
`$ oc get pods -n trustee-operator-system`
819+
820+
**Example output**
821+
```
822+
NAME READY STATUS RESTARTS AGE
823+
trustee-deployment-8585f98449-9bbgl 1/1 Running 0 22m
824+
trustee-operator-controller-manager-5fbd44cd97-55dlh 2/2 Running 0 59m
825+
```
826+
827+
1. Set the POD_NAME environmental variable by running the following command:
828+
829+
`$ POD_NAME=$(oc get pods -l app=kbs -o jsonpath='{.items[0].metadata.name}' -n trustee-operator-system)`
830+
831+
1. Check the pod logs by running the following command:
832+
833+
`$ oc logs -n trustee-operator-system $POD_NAME`
834+
835+
**Example output**
836+
```
837+
[2024-05-30T13:44:24Z INFO kbs] Using config file /etc/kbs-config/kbs-config.json
838+
[2024-05-30T13:44:24Z WARN attestation_service::rvps] No RVPS address provided and will launch a built-in rvps
839+
[2024-05-30T13:44:24Z INFO attestation_service::token::simple] No Token Signer key in config file, create an ephemeral key and without CA pubkey cert
840+
[2024-05-30T13:44:24Z INFO api_server] Starting HTTPS server at [0.0.0.0:8080]
841+
[2024-05-30T13:44:24Z INFO actix_server::builder] starting 12 workers
842+
[2024-05-30T13:44:24Z INFO actix_server::server] Tokio runtime found; starting in existing Tokio runtime
843+
```
844+
845+
846+
### Verify the attestation process
847+
848+
You can verify the attestation process by creating a test pod and retrieving its secret.
849+
850+
Important: This procedure is an example to verify that attestation is working. Do not write sensitive data to standard I/O because the data can be captured by using a memory dump. Only data written to memory is encrypted.
851+
852+
By default, an agent side policy embedded in the pod VM image disables the exec and log APIs for a Confidential Containers pod. This policy ensures that sensitive data is not written to standard I/O.
853+
854+
In a test scenario, you can override the restriction at runtime by adding a policy annotation to the pod. For Technology Preview, runtime policy annotations are not verified by remote attestation.
855+
856+
1. Create a verification-pod.yaml manifest file:
857+
858+
```
859+
apiVersion: v1
860+
kind: Pod
861+
metadata:
862+
name: ocp-cc-pod
863+
labels:
864+
app: ocp-cc-pod
865+
annotations:
866+
io.katacontainers.config.agent.policy: cGFja2FnZSBhZ2VudF9wb2xpY3kKCmRlZmF1bHQgQWRkQVJQTmVpZ2hib3JzUmVxdWVzdCA6PSB0cnVlCmRlZmF1bHQgQWRkU3dhcFJlcXVlc3QgOj0gdHJ1ZQpkZWZhdWx0IENsb3NlU3RkaW5SZXF1ZXN0IDo9IHRydWUKZGVmYXVsdCBDb3B5RmlsZVJlcXVlc3QgOj0gdHJ1ZQpkZWZhdWx0IENyZWF0ZUNvbnRhaW5lclJlcXVlc3QgOj0gdHJ1ZQpkZWZhdWx0IENyZWF0ZVNhbmRib3hSZXF1ZXN0IDo9IHRydWUKZGVmYXVsdCBEZXN0cm95U2FuZGJveFJlcXVlc3QgOj0gdHJ1ZQpkZWZhdWx0IEV4ZWNQcm9jZXNzUmVxdWVzdCA6PSB0cnVlCmRlZmF1bHQgR2V0TWV0cmljc1JlcXVlc3QgOj0gdHJ1ZQpkZWZhdWx0IEdldE9PTUV2ZW50UmVxdWVzdCA6PSB0cnVlCmRlZmF1bHQgR3Vlc3REZXRhaWxzUmVxdWVzdCA6PSB0cnVlCmRlZmF1bHQgTGlzdEludGVyZmFjZXNSZXF1ZXN0IDo9IHRydWUKZGVmYXVsdCBMaXN0Um91dGVzUmVxdWVzdCA6PSB0cnVlCmRlZmF1bHQgTWVtSG90cGx1Z0J5UHJvYmVSZXF1ZXN0IDo9IHRydWUKZGVmYXVsdCBPbmxpbmVDUFVNZW1SZXF1ZXN0IDo9IHRydWUKZGVmYXVsdCBQYXVzZUNvbnRhaW5lclJlcXVlc3QgOj0gdHJ1ZQpkZWZhdWx0IFB1bGxJbWFnZVJlcXVlc3QgOj0gdHJ1ZQpkZWZhdWx0IFJlYWRTdHJlYW1SZXF1ZXN0IDo9IHRydWUKZGVmYXVsdCBSZW1vdmVDb250YWluZXJSZXF1ZXN0IDo9IHRydWUKZGVmYXVsdCBSZW1vdmVTdGFsZVZpcnRpb2ZzU2hhcmVNb3VudHNSZXF1ZXN0IDo9IHRydWUKZGVmYXVsdCBSZXNlZWRSYW5kb21EZXZSZXF1ZXN0IDo9IHRydWUKZGVmYXVsdCBSZXN1bWVDb250YWluZXJSZXF1ZXN0IDo9IHRydWUKZGVmYXVsdCBTZXRHdWVzdERhdGVUaW1lUmVxdWVzdCA6PSB0cnVlCmRlZmF1bHQgU2V0UG9saWN5UmVxdWVzdCA6PSB0cnVlCmRlZmF1bHQgU2lnbmFsUHJvY2Vzc1JlcXVlc3QgOj0gdHJ1ZQpkZWZhdWx0IFN0YXJ0Q29udGFpbmVyUmVxdWVzdCA6PSB0cnVlCmRlZmF1bHQgU3RhcnRUcmFjaW5nUmVxdWVzdCA6PSB0cnVlCmRlZmF1bHQgU3RhdHNDb250YWluZXJSZXF1ZXN0IDo9IHRydWUKZGVmYXVsdCBTdG9wVHJhY2luZ1JlcXVlc3QgOj0gdHJ1ZQpkZWZhdWx0IFR0eVdpblJlc2l6ZVJlcXVlc3QgOj0gdHJ1ZQpkZWZhdWx0IFVwZGF0ZUNvbnRhaW5lclJlcXVlc3QgOj0gdHJ1ZQpkZWZhdWx0IFVwZGF0ZUVwaGVtZXJhbE1vdW50c1JlcXVlc3QgOj0gdHJ1ZQpkZWZhdWx0IFVwZGF0ZUludGVyZmFjZVJlcXVlc3QgOj0gdHJ1ZQpkZWZhdWx0IFVwZGF0ZVJvdXRlc1JlcXVlc3QgOj0gdHJ1ZQpkZWZhdWx0IFdhaXRQcm9jZXNzUmVxdWVzdCA6PSB0cnVlCmRlZmF1bHQgV3JpdGVTdHJlYW1SZXF1ZXN0IDo9IHRydWUK
867+
spec:
868+
runtimeClassName: kata-remote
869+
containers:
870+
- name: skr-openshift
871+
image: registry.access.redhat.com/ubi9/ubi:9.3
872+
command:
873+
- sleep
874+
- "36000"
875+
securityContext:
876+
privileged: false
877+
seccompProfile:
878+
type: RuntimeDefault
879+
```
880+
881+
Notes:
882+
883+
The pod metada `annotations` overrides the policy that prevents sensitive data from being written to standard I/O.
884+
885+
1. Create the pod by running the following command:
886+
887+
`$ oc create -f verification-pod.yaml`
888+
889+
1. Connect to the Bash shell of the ocp-cc-pod pod by running the following command:
890+
891+
`$ oc exec -it ocp-cc-pod -- bash`
892+
893+
1. Fetch the pod secret by running the following command:
894+
895+
`$ curl http://127.0.0.1:8006/cdh/resource/default/kbsres1/key1`
896+
897+
**Example output**
898+
`res1val1`
899+
900+
The Trustee server returns the secret only if the attestation is successful.
613901
614902
615903

0 commit comments

Comments
 (0)