feat(docs): adds docs for hardened mode and addresses feedback

Author: adhityaravi

.github/workflows/sphinx-python-dependency-build-checks.yml

@@ -51,5 +51,5 @@ jobs:
       run: |
         set -ex
         make install \
-          PIPOPTS="--no-binary :all:" \
+          PIPOPTS="--no-binary :all: --only-binary html5lib" \
           || ( cat .sphinx/venv/pip_install.log && exit 1 )

docs/.custom_wordlist.txt

@@ -1,6 +1,8 @@
 # Leave a blank line at the end of this file to support concatenation
+ambient's
 backend
 backends
+CIDR
 Charmcraft
 config
 Consol
@@ -20,6 +22,7 @@ http
 https
 html
 instantiation
+IPs
 Istio
 Istio's
 Intersphinx
@@ -63,8 +66,11 @@ toctree
 txt
 uncommenting
 ui
+ztunnel
 utils
 VMs
+waypoint
+waypoints
 Waypoint
 WCAG
 Wordpress

docs/assets/images/charmed-bookinfo-architecture.drawio.png

@@ -25,12 +25,12 @@
 #
 # TODO: Update with the official name of your project or product
 
-project = "Service Mesh"
+project = "Canonical Service Mesh"
 author = "Canonical Ltd."
 
 
 # Sidebar documentation title; best kept reasonably short
-html_title = project + " Documentation"
+html_title = "Documentation"
 
 
 # Copyright string; shown at the bottom of the page
@@ -184,16 +184,20 @@
 
 linkcheck_ignore = [
     "http://127.0.0.1:8000",
-    "https://github.com/canonical/ACME/*"
+    "https://github.com/canonical/ACME/*",
+    r"https://matrix\.to/.*",
     ]
 
 
 # A regex list of URLs where anchors are ignored by 'make linkcheck'
 
-linkcheck_anchors_ignore_for_url = [r"https://github\.com/.*"]
+linkcheck_anchors_ignore_for_url = [
+    r"https://github\.com/.*",
+    r"https://istio\.io/.*",
+    ]
 
 # give linkcheck multiple tries on failure
-# linkcheck_timeout = 30
+linkcheck_timeout = 60
 linkcheck_retries = 3
 
 ########################
@@ -236,6 +240,7 @@
     "sphinx_last_updated_by_git",
     "sphinx.ext.intersphinx",
     "sphinx_sitemap",
+    "sphinxcontrib.mermaid",
 ]
 
 # Excludes files or directories from processing

docs/conf.py

@@ -0,0 +1,17 @@
+# Compatibility with Cilium CNI
+
+`Cilium` is an `eBPF` based CNI which includes some intelligent kernel level traffic re-routing using `eBPF` programs. This traffic re-routing can potentially interfere with `Istio ambient's` mesh features.
+
+```{note}
+The complete compatibility documentation between Cilium and Istio can be found [here](https://docs.cilium.io/en/stable/network/servicemesh/istio/). This documentation only covers certain gotchas that is not explicitly clear from the mentioned doc.
+```
+
+By default `Cilium` is designed to be the exclusive CNI which wont allow the Istio CNI plugin to be successfully deployed. Hence it is important to make sure `Cilium` is configured to allow external CNI plugins.
+
+Assuming `Cilium` allows `Istio's` CNI plugin, the major compatibility issue between them is caused by the fact that `Cilium` load balances traffic at the kernel level. This means traffic directed at Kubernetes Services are re-routed by Cilium directly to the destination Pod.
+
+`Istio ambient`, in order to apply policies and encrypt traffic, routes the traffic via its `ztunnel` component either using `iptables` or higher level `eBPF` programs. Since the `Cilium's` re-routing applies at the kernel level, the traffic is never routed through the `ztunnel` at the source. This prevents `Istio ambient` from successfully encrypting the traffic or applying the `AuthorizationPolicies`. This issue can be solved by the `socketLB.hostNamespaceOnly: true` setting in `Cilium` which basically instructs `Cilium` to limit load balancing to the host network namespace. Hence the K8s internal traffic flows through the normal network stack allowing `Istio ambient` to function normally.
+
+An important gotcha that is not mentioned in the `Cilium` documentation is the fact that, even when `socketLB.hostNamespaceOnly` is set to be `false`, the destination `ztunnel` will successfully capture the traffic to the destination pod and might apply L4 policies directed at the pod. This might give the impression that service mesh is working successfully and will mask the fact the traffic between the source pod and destination pod is not encrypted by Istio ambient (if `mTLS` mode is set to be `PERMISSIVE`).
+
+The recommended configuration to successfully use Charmed Istio with Canonical Kubernetes with the `Cilium` CNI can be found in [this](../how-to/use-charmed-istio-with-canonical-kubernetes.md) how-to documentation.

docs/explanation/cilium-cni-compatibility.md

@@ -1,8 +1,91 @@
 # Hardened Mode
 
-```{note}
-This page is planned but not yet written
+In a zero-trust network, no service is trusted by default and every request must be explicitly authorized before it is allowed through.  Hardened mode brings this principle to the service mesh by enforcing a deny-by-default security posture: all inbound traffic to services on the mesh is denied unless an authorization policy explicitly allows it.
+
+## Why hardened mode is needed
+
+By default, Istio ambient only enforces authorization on workloads that are explicitly targeted by an `AuthorizationPolicy`.  Any workload without a policy applied to it will accept all inbound traffic.  This means:
+
+* if a service on the mesh has no policy targeting it, the **ztunnel** will allow any traffic through to that service
+* when [`auto-allow-waypoint-policy`](https://charmhub.io/istio-k8s/configurations#auto-allow-waypoint-policy) is enabled (the default), if no policy targets a service at the **waypoint**, any service can reach it
+
+In practice, this leaves services that haven't been explicitly locked down wide open.  Hardened mode closes this gap by ensuring that all traffic is denied by default across the entire mesh.
+
+## How it works
+
+Enabling hardened mode on the [`istio-k8s`](https://charmhub.io/istio-k8s) charm creates two global **allow-nothing** `AuthorizationPolicies`.
+
+### Why allow-nothing instead of deny-all?
+
+The distinction matters because of how Istio ambient [evaluates authorization policies](https://istio.io/latest/docs/concepts/security/#authorization-policy-precedence).  Istio ambient processes policies in this order: `CUSTOM` then `DENY` then `ALLOW`.  A `DENY` policy always takes precedence.  Once traffic matches a `DENY` rule, it is rejected regardless of any `ALLOW` policies.  This means a global `DENY`-all policy would lock down the cluster permanently with no way to override it.
+
+An allow-nothing policy works differently.  It is an `ALLOW` policy with an empty rule set, meaning it matches no traffic.  However, its presence activates Istio ambient's [implicit deny behavior](https://istio.io/latest/docs/concepts/security/#allow-nothing-deny-all-and-allow-all-policy): once at least one `ALLOW` policy exists for a workload, any traffic that does not match an `ALLOW` rule is denied.  Other `ALLOW` policies can then selectively open up the specific traffic that should be permitted.  This gives us a secure default that can still be overridden by explicit allow rules.
+
+### ztunnel allow-nothing policy
+
+An `AuthorizationPolicy` with an empty spec acts as a global allow-nothing policy.  This ensures that any traffic not matched by an explicit `ALLOW` policy is denied at the ztunnel layer:
+
+```yaml
+apiVersion: security.istio.io/v1
+kind: AuthorizationPolicy
+metadata:
+  name: istio-k8s-istio-system-policy-global-allow-nothing-ztunnel
+  namespace: istio-system
+spec: {}
+```
+
+### Waypoint allow-nothing policy
+
+A similar policy targets the `istio-waypoint` `GatewayClass`, locking down all traffic that passes through the waypoint:
+
+```yaml
+apiVersion: security.istio.io/v1
+kind: AuthorizationPolicy
+metadata:
+  name: istio-k8s-istio-system-policy-global-allow-nothing-waypoint
+  namespace: istio-system
+spec:
+  targetRefs:
+  - kind: GatewayClass
+    group: gateway.networking.k8s.io
+    name: istio-waypoint
+```
+
+Together, these two policies ensure that no service-to-service communication is allowed anywhere on the mesh unless an explicit `ALLOW` policy exists for it.  See [Traffic authorization](./traffic-authorization.md) for how these allow policies are created by the beacon charm.
+
+## Effect on ingress traffic
+
+Hardened mode also blocks external traffic from reaching the Istio ingress gateway, since the global deny applies to all inbound traffic including traffic arriving from a `LoadBalancer`.
+
+To handle this, the [`istio-ingress-k8s`](https://charmhub.io/istio-ingress-k8s) charm automatically creates an `ALLOW` policy that permits external traffic to reach the gateway:
+
+```yaml
+apiVersion: security.istio.io/v1
+kind: AuthorizationPolicy
+metadata:
+  name: istio-ingress-k8s-istio-system-external-traffic
+  namespace: istio-system
+spec:
+  action: ALLOW
+  targetRefs:
+  - kind: Gateway
+    group: gateway.networking.k8s.io
+    name: istio-ingress-k8s
+  rules:
+  - from:
+    - source:
+        ipBlocks:
+        - "0.0.0.0/0"
 ```
-```{note}
-This should be written for general service mesh charms, not just istio
-```
+
+By default, this allows traffic from any source IP (`0.0.0.0/0`) because an ingressed application is typically meant to be publicly accessible.  To restrict access to specific source IPs or CIDR ranges, use the `external-traffic-policy-cidrs` configuration option on [`istio-ingress-k8s`](https://charmhub.io/istio-ingress-k8s/configurations).
+
+## Enabling hardened mode
+
+To enable hardened mode, set the `hardened-mode` configuration option on the `istio-k8s` charm:
+
+```bash
+juju config istio-k8s hardened-mode=true
+```
+
+For all available configuration options, see the [istio-k8s](https://charmhub.io/istio-k8s/configurations) and [istio-ingress-k8s](https://charmhub.io/istio-ingress-k8s/configurations) charm configuration pages.

docs/explanation/hardened-mode.md

@@ -3,13 +3,10 @@
 These documents provide an explanation of key topics and concepts of the Canonical Service Mesh.
 
 ```{toctree}
-:maxdepth: 1
+:maxdepth: 2
 
 service-mesh
 istio
-hardened-mode
-managed-mode
-traffic-authorization
 service-mesh-in-coordinated-worker-charms
 ```
 

docs/explanation/index.md

@@ -1,4 +1,4 @@
-# Istio
+# Charmed Istio Ambient
 
 [Istio](https://istio.io) is an implementation of a [service mesh](./service-mesh.md).  It helps you:
 
@@ -8,39 +8,23 @@
 
 Although Kubernetes natively provides facilities to do some of this, Istio implements richer solutions.  For example, Istio's [AuthorizationPolicy](https://istio.io/latest/docs/reference/config/security/authorization-policy/) object implements fine-grained authorization controls, and Istio can automate mutual TLS between all applications on the mesh.
 
-## Charmed Istio
-
-Charmed Istio is an opinionated deployment of Istio using [Juju](http://juju.is/).  The goals of Charmed Istio are to:
+Charmed Istio ambient is an opinionated deployment of Istio's [Ambient Mode](https://istio.io/latest/docs/ambient/overview/) using [Juju](http://juju.is/).  The goals of Charmed Istio ambient are to:
 
 * provide a simple-to-deploy, easy-to-manage Istio experience, giving most of Istio's benefits without a need for advanced Istio experience
 * be customizable for power users, so users can build advanced use cases on top of the standard Charmed Istio base
 
-Charmed Istio uses Istio's [Ambient Mode](https://istio.io/latest/docs/ambient/overview/) and is implemented through the following charms:
+It is implemented through the following charms:
 
 * [istio-k8s](https://charmhub.io/istio-k8s): for deploying and managing the Istio control panel, such as the Istio daemon and its resources
 * [istio-beacon-k8s](https://charmhub.io/istio-beacon-k8s/): for integrating a Juju model and its applications to Charmed Istio, as well as deploying an Istio Waypoint for those applications
 * [istio-ingress-k8s](https://charmhub.io/istio-ingress-k8s): for deploying and managing an Istio ingress gateway
 
-Core elements of Charmed Istio include:
-
-* automatic mTLS communication between all applications on the service mesh
-* default security features through [hardened mode](./hardened-mode.md)
-* easy, fine-grained, app-to-app authorization control through [managed mode](./managed-mode.md)
-
-## Using Istio with Cilium CNI
+```{toctree}
+:maxdepth: 1
+:hidden:
 
-`Cilium` is an `eBPF` based CNI which includes some intelligent kernel level traffic re-routing using `eBPF` programs. This traffic re-routing can potentially interfere with `Istio's` mesh features.
-
-```{note}
-The complete compatibility documentation between Cilium and Istio can be found [here](https://docs.cilium.io/en/stable/network/servicemesh/istio/). This documentation only covers certain gotchas that is not explicitly clear from the mentioned doc.
+hardened-mode
+managed-mode
+traffic-authorization
+cilium-cni-compatibility
 ```
-
-By default `Cilium` is designed to be the exclusive CNI which wont allow the Istio CNI plugin to be successfully deployed. Hence it is important to make sure `Cilium` is configured to allow external CNI plugins.
-
-Assuming `Cilium` allows `Istio's` CNI plugin, the major compatibility issue between them is caused by the fact that `Cilium` load balances traffic at the kernel level. This means traffic directed at Kubernetes Services are re-routed by Cilium directly to the destination Pod. 
-
-`Istio`, in order to apply policies and encrypt traffic, routes the traffic via its `ztunnel` component either using `iptables` or higher level `eBPF` programs. Since the `Cilium's` re-routing applies at the kernel level, the traffic is never routed through the `ztunnel` at the source. This prevents `Istio` from successfully encrypting the traffic or applying the `AuthorizationPolicies`. This issue can be solved by the `socketLB.hostNamespaceOnly: true` setting in `Cilium` which basically instructs `Cilium` to limit load balancing to the host network namespace. Hence the K8s internal traffic flows through the normal network stack allowing `Istio` to function normally.
-
-An important gotcha that is not mentioned in the `Cilium` documentation is the fact that, even when `socketLB.hostNamespaceOnly` is set to be `false`, the destination `ztunnel` will successfully capture the traffic to the destination pod and might apply L4 policies directed at the pod. This might give the impression that service mesh is working successfully and will mask the fact the traffic between the source pod and destination pod is not encrypted by Istio (if `mTLS` mode is set to be `PERMISSIVE`).
-
-The recommended configuration to successfully use Charmed Istio with Canonical Kubernetes with the `Cilium` CNI can be found in [this](../how-to/use-charmed-istio-with-canonical-kubernetes.md) how-to documentation.

docs/explanation/istio.md

@@ -1,5 +1,59 @@
 # Managed Mode
 
-All Charmed Service Mesh Beacon charms (for example, [istio-beacon-k8s](https://charmhub.io/istio-beacon-k8s/)) include optional [management of authorization policies between applications](../explanation/traffic-authorization.md).  When enabled, the beacon charms will automatically generate policies that allow related applications to communicate with each other [as specified by the charm authors via the `ServiceMeshConsumer` library](../how-to/add-mesh-support-to-your-charm.md).  
+Managed mode refers to a set of configuration options that together control how authorization policies are automatically created and enforced on the mesh.  When fully enabled, the beacon charm automatically generates `AuthorizationPolicies` based on what charm authors define via the [`ServiceMeshConsumer` library](../how-to/add-mesh-support-to-your-charm.md), so administrators do not need to create policies manually.
 
-If managed mode is disabled, policy creation is left up to the administrator to do manually.
+## manage-authorization-policies
+
+The [`manage-authorization-policies`](https://charmhub.io/istio-beacon-k8s/configure#manage-authorization-policies) option on the beacon charm is the core of managed mode.  When set to `true` (the default), the beacon charm reads the policies defined by each charm via the `ServiceMeshConsumer` library and creates the corresponding Istio `AuthorizationPolicies` automatically.
+
+For example, using the [Get started with Charmed Istio ambient](../tutorial/get-started-with-the-charmed-istio-mesh.md) tutorial, the `bookinfo-details-k8s` charm defines a policy allowing `GET` requests to `/health` and `/details/*` on port `9080`.  When `bookinfo-productpage-k8s` is related to `bookinfo-details-k8s` and both are on the mesh, the beacon charm creates an `AuthorizationPolicy` like the following:
+
+```yaml
+apiVersion: security.istio.io/v1
+kind: AuthorizationPolicy
+metadata:
+  name: istio-beacon-k8s-bookinfo-policy-bookinfo-productpage-k8s-bookinfo-bookinfo-details-k8s-ad9cfa91
+  namespace: bookinfo
+  labels:
+    app.kubernetes.io/instance: istio-beacon-k8s-bookinfo
+    kubernetes-resource-handler-scope: istio-authorization-policy
+spec:
+  rules:
+  - from:
+    - source:
+        principals:
+        - cluster.local/ns/bookinfo/sa/bookinfo-productpage-k8s
+    to:
+    - operation:
+        methods:
+        - GET
+        paths:
+        - /health
+        - /details/*
+        ports:
+        - '9080'
+  targetRefs:
+  - group: ''
+    kind: Service
+    name: bookinfo-details-k8s
+```
+
+This policy allows `bookinfo-productpage-k8s` (identified by its service account) to make `GET` requests to the specified paths and port on `bookinfo-details-k8s`, and nothing else.
+
+When `manage-authorization-policies` is set to `false`, the beacon charm will not create any authorization policies but will still perform other functions like providing a waypoint.  In this case, policy creation is left to the administrator.
+
+## Related configuration options
+
+Managed mode works alongside two configuration options on the [`istio-k8s`](https://charmhub.io/istio-k8s) charm that affect how the automatically created policies behave:
+
+### hardened-mode
+
+When [hardened mode](./hardened-mode.md) is enabled, global allow-nothing policies ensure that all traffic is denied unless an explicit `ALLOW` policy exists.  Without hardened mode, the policies created by managed mode only restrict traffic to workloads that are explicitly targeted.  Workloads without any policy still accept all traffic.  Enabling both managed mode and hardened mode together provides full zero-trust enforcement: managed mode creates the allow rules, and hardened mode ensures everything else is denied.
+
+### auto-allow-waypoint-policy
+
+The [`auto-allow-waypoint-policy`](https://charmhub.io/istio-k8s/configure#auto-allow-waypoint-policy) option (enabled by default) tells Istio ambient to automatically create synthetic L4 authorization policies that allow waypoints to forward traffic to their workloads.  Without this, the policies created by managed mode would be evaluated at the waypoint but the traffic from the waypoint to the destination workload would be blocked at the ztunnel layer unless a separate L4 policy exists.  Keeping this option enabled means administrators only need to think about the application level policies that managed mode handles.
+
+## Disabling managed mode
+
+All of the policy related configuration options described above can be disabled.  When they are, Charmed Istio ambient hands off all authorization control to the administrator.  No policies are automatically created or enforced, and all traffic management must be done manually by creating and maintaining `AuthorizationPolicies` directly.

docs/explanation/managed-mode.md

@@ -126,6 +126,6 @@ juju relate my-db-provider:database my-db-consumer:database
 juju relate my-db-provider:provide-cmr-support my-db-consumer:require-cmr-support
 ```
 
-For a more detailed tutorial using cross-model integrations, follow the [Use the Istio Mesh across different Juju models](../tutorial/use-the-istio-mesh-across-different-juju-models.md) tutorial.
+For a more detailed tutorial using cross-model integrations, follow the [Use Istio ambient across different Juju models](../tutorial/use-the-istio-mesh-across-different-juju-models.md) tutorial.
 
 [^1]: For a detailed explanation of exactly what is generated automatically, see [Authorization Policy Creation in Istio](../explanation/traffic-authorization.md)