Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
70 changes: 70 additions & 0 deletions content/patterns/coco-pattern/_index.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
---
title: Confidential Containers pattern
date: 2024-10-03
tier: sandbox
summary: This pattern helps you get started with deploying confidential containers in OpenShift Container Platform.
rh_products:
- Red Hat OpenShift Container Platform
- Red Hat Advanced Cluster Management
- Red Hat OpenShift Sandbox Containers
industries:
- General
aliases: /coco-pattern/
pattern_logo: coco-logo.png
links:
install: coco-getting-started
help: https://groups.google.com/g/validatedpatterns
bugs: https://github.com/validatedpatterns/coco-pattern/issues
---
:toc:
:imagesdir: /images
:_content-type: ASSEMBLY
include::modules/comm-attributes.adoc[]

= About coco-pattern

Confidential computing is a technology for securing data in use. It uses a https://en.wikipedia.org/wiki/Trusted_execution_environment[Trusted Execution Environment] provided within the hardware of the processor to prevent access from others who have access to the system.
https://confidentialcontainers.org/[Confidential containers] is a project to standardize the consumption of confidential computing by making the security boundary for confidential computing to be a Kubernetes pod. [Kata containers](https://katacontainers.io/) is used to establish the boundary via a shim VM.

A core goal of confidential computing is to use this technology to isolate the workload from both Kubernetes and hypervisor administrators.

image::coco-pattern/isolation.png[Schematic describing the isolation of confidential contains from the hosting system]


This pattern uses https://docs.redhat.com/en/documentation/openshift_sandboxed_containers/1.7/html/user_guide/deploying-on-azure#deploying-cc_azure-cc[Red Hat OpenShift sandbox containers] to deploy and configure confidential containers on Microsoft Azure.

It deploys three copies of 'Hello OpenShift' to demonstrate some of the security boundaries that enforced with confidential containers.

== Requirements

- An an azure account with the link:./coco-pattern-azure-requirements/[required access rights]
- An OpenShift cluster, within the Azure environment updated beyond 4.16.10


== Security considerations

**This pattern is a demonstration only and contains configuration that is not best practice**

- The default configuration deploys everything in a single cluster for testing purposes. The https://www.ietf.org/archive/id/draft-ietf-rats-architecture-22.html[RATS] architecture mandates that the Key Broker Service (e.g. https://github.com/confidential-containers/trustee[Trustee]) is in a trusted security zone.
- The https://github.com/confidential-containers/trustee/tree/main/attestation-service[Attestation Service] has wide open security policies.

== Future work

* Deploying the environment the 'Trusted' environment including the KBS on a separate cluster to the secured workloads
* Deploying to alternative environments supporting confidential computing including bare metal x86 clusters; IBM Cloud; IBM Z
* Finishing the sample AI application

== Architecture

Confidential Containers typically has two environments. A trusted zone, and an untrusted zone. In these zones, Trustee, and the sandbox container operator are deployed, respectively.

** For demonstration purposes the pattern currently is converged on one cluster**

image::coco-pattern/overview-schematic.png[Schematic describing the high level architecture of confidential containers]




== References
- https://docs.redhat.com/en/documentation/openshift_sandboxed_containers/1.7/html/user_guide/about-osc#about-confidential-containers_about-osc[OpenShift sandboxed containers documentation]
- https://www.redhat.com/en/blog/exploring-openshift-confidential-containers-solution[OpenShift confidential containers solution blog]
70 changes: 70 additions & 0 deletions content/patterns/coco-pattern/coco-pattern-azure-requirements.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
---
title: Azure requirements
weight: 10
aliases: /coco-pattern/coco-pattern-azure-requirements/
---

:toc:
:_content-type: ASSEMBLY
include::modules/comm-attributes.adoc[]

:imagesdir: ../../../images
= Azure requirements

This demo currently has been tested only on azure.
The configuration tested used the `openshift-install`.
https://docs.openshift.com/container-platform/4.16/installing/installing_azure/installing-azure-default.html[OpenShift documentation] contains details on how to do this.

The documentation outlines https://docs.openshift.com/container-platform/4.16/installing/installing_azure/installing-azure-account.html[minimum required configuration] for an azure account.

== Changes required

Do not accept default sizes for OpenShift install. It is recommended to up the workers to at least `Standard_D8s_v5`.
This can be done by using `openshift-install create install-config` first and adjusting the workers under platform e.g.:

[source,yaml]
----
- architecture: amd64
hyperthreading: Enabled
name: worker
platform:
azure:
type: Standard_D8s_v5
replicas: 3
----

On a cloud provider the virtual machines for the kata containers use "peer pods" which are running directly on the cloud provider's hypervisor (see the diagram below).
This means that access is required to the "confidential computing" virtual machine class. On Azure the `Standard_DCas_v5` class of virtual machines are used.
These virtual machines are *NOT* available in all regions. Users will also need to up the specific limits for `Standard_DC2as_v5` virtual machines.

image::coco-pattern/peer_pods.png[Schematic diagram of peer pods vs standard kata containers]

DNS for the openshift cluster also *MUST* be provided by azure DNS.

== Azure configuration required for the validated pattern

The validated pattern requires access to azure apis to provision peer-pod VMs and to obtain certificates from let's encrypt.

Azure configuration information must be provided in two places:

- The a secret must be loaded using a ../../../learn/secrets-management-in-the-validated-patterns-framework/[values-secret] file.
The https://github.com/validatedpatterns/coco-pattern/blob/main/values-secret.yaml.template[`values-secret.yaml.template`] file provides the appropriate structure

- A broader set of information about the cluster is required in https://github.com/validatedpatterns/coco-pattern/blob/main/values-global.yaml[`values-global.yaml`] (see below).


[source,yaml]
----
global:
azure:
clientID: '' # Service principle ID
subscriptionID: ''
tenantID: '' # Tenant ID
DNSResGroup: '' # Resource group for the azure DNS hosted zone
hostedZoneName: '' # the hosted zone name
clusterResGroup: '' # Resource group of the cluster
clusterSubnet: '' # subnet of the cluster
clusterNSG: '' # network security group of the worker nodes in the cluster
clusterRegion: ''
----

86 changes: 86 additions & 0 deletions content/patterns/coco-pattern/coco-pattern-getting-started.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
---
title: Getting started
weight: 10
aliases: /coco-pattern/coco-pattern-getting-started/
---

:toc:

:_content-type: ASSEMBLY
include::modules/comm-attributes.adoc[]


== Deploying

1. Install an link:../coco-pattern-azure-requirements/[OpenShift Cluster on Azure]

2. Update the link:../coco-pattern-azure-requirements/#_azure_configuration_required_for_the_validated_pattern[required Azure configuration and secrets]

3. `./pattern.sh make install`

4. Wait: The cluster needs to reboot all nodes at least once, and reprovision the ingress to use the let's encrypt certificates.

5. If the services do not come up use the ArgoCD UI to triage potential timeouts.

== Simple Confidential container tests

The pattern deploys some simple tests of CoCo with this pattern.
A "Hello Openshift" (e.g. `curl` to return "Hello Openshift!") application has been deployed in three form factor.

1. A vanilla kubernetes pod at `oc get pods -n hello-openshift standard`
2. A confidential container `oc get pods -n hello-openshift secure`
3. A confidential container with a relaxed policy at `oc get pods -n hello-openshift insecure-policy`

In this case the insecure policy is designed to allow a user to be able to exec into the confidential container.
Typically this is disabled by an immutable policy established at pod creation time.


Doing a `oc get pods` for either of the pods running a confidential container should show the `runtimeClassName: kata-remote` for the pod.

// Add a azure portal image grab next boot
Logging into azure once the pods have been provisioned will show that each of these two pods has been provisioned with it's own `Standard_DC2as_v5` virtual machine.

=== `oc exec` testing

In a OpenShift cluster without confidential containers, Role Based Access Control (RBAC), may be used to prevent users from execing into a container to mutate it.
However:

1. Cluster admins can always circumvent this capability
2. Anyone logged into the node directly can also circumvent this capability.

Confidential containers can prevent this. Running: `oc exec -n hello-openshift -it secure -- bash` will result in a denial of access, irrespective of the user undertaking the action, including `kubeadmin`.
For running this with either the standard pod `oc exec -n hello-openshift -it standard -- bash`, or the CoCo pod with the policy disabled `oc exec -n hello-openshift -it insecure-policy -- bash` will allow shell access.



=== Confidential Data Hub testing

Part of the CoCo VM is a component called the Confidential Data Hub (CDH), which simplifies access to the Trustee Key Broker service for end applications.
Find out more about how the CDH and Trustee work together https://www.redhat.com/en/blog/introducing-confidential-containers-trustee-attestation-services-solution-overview-and-use-cases[here].

image::coco-pattern/trustee.png[]

The CDH presents to containers within the pod (only), via a localhost URL. The CoCo container with an insecure policy can be used for testing the behaviour.


- `oc exec -n hello-openshift -it insecure-policy -- bash` to get a shell into a confidential container

- https://github.com/validatedpatterns/coco-pattern/blob/main/charts/hub/trustee/templates/kbs.yaml[Trustee's configuration] specifies the list of secrets which the KBS can access with the `kbsSecretResources` attribute.

- Secrets within the CDH can be accessed (by default) at `http://127.0.0.1:8006/cdh/resource/default/$K8S_SECRET/$K8S_SECRET_KEY`.

- In this case `http://127.0.0.1:8006/cdh/resource/default/passphrase/passphrase` by default will return a string which was randomly generated when the pattern was deployed.

- This should be the same as result as `oc get secrets -n trustee-operator-system passphrase -o yaml | yq '.data.passphrase'` | base64 -d`

- Tailing the logs for the kbs container e.g. `oc logs -n trustee-operator-system kbs-deployment-5b574bccd6-twjxh -f` shows the evidence which is flowing to the KBS from the CDH.










Binary file added static/images/coco-pattern/isolation.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added static/images/coco-pattern/overview-schematic.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added static/images/coco-pattern/peer_pods.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added static/images/coco-pattern/trustee.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added static/images/logos/coco-logo.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.