diff --git a/source/operations_and_monitoring.rst b/source/operations_and_monitoring.rst index f265d36..9190b4f 100644 --- a/source/operations_and_monitoring.rst +++ b/source/operations_and_monitoring.rst @@ -269,6 +269,158 @@ The images stored locally in the seed host can be seen using ``docker image ls`` Old and redundant images can be identified from their names and tags, and removed using ``docker image rm``. +Octavia ++++++++ + +Octavia uses mutual TLS to secure communication between the amphorae and +Octavia services. It uses a private CA to sign both client and server +certificates. We use the kolla-ansible built-in support for generating these +certificates. + +We ensure that duplicate subject names are allowed, by setting +``unique_subject=no`` in +``etc/kayobe/kolla/octavia-certificates/client_ca/index.txt.attr``. This should +already be set. + +Checking certificate expiry +~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. code-block:: console + + kayobe# ansible-vault decrypt client.cert-and-key.pem --vault-password-file ~/vault + Decryption successful + kayobe# openssl x509 -enddate -noout -in client.cert-and-key.pem + notAfter=Aug 12 10:45:35 2022 GMT + +There is also support in Kolla-Ansible to check if certs will expire within a +given number of days: + +.. code-block:: console + + kayobe# kayobe kolla ansible run "octavia-certificates + +Rotating client certificates +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Octavia client certificates expire after a year, and should be rotated before +they expire. + +.. note:: + + Only the client certificate needs to be rotated annually. The server CA and + client CA both have 10 year expiries, and should not be rotated before this + since it would break the trust chain. + +Decrypt all certificate files: + +.. code-block:: console + + git grep -l VAULT etc/kayobe/kolla/octavia-certificates | xargs ansible-vault decrypt --vault-password-file + +Remove the existing client certificate and keys to ensure they are regenerated: + +.. code-block:: console + + rm etc/kayobe/kolla/octavia-certificates/client_ca/client.* + +Generate new certificates: + +.. code-block:: console + + kayobe kolla ansible run octavia-certificates -ke node_custom_config=$KAYOBE_CONFIG_PATH/kolla/config -ke node_config=$KAYOBE_CONFIG_PATH/kolla + +This is likely to modify several files, and possibly add some new ones. Add the +new files: + +.. code-block:: console + + git add etc/kayobe/kolla/octavia-certificates/client_ca/ + +Encrypt the changed files that we are interested in: + +.. code-block:: console + + git grep -L VAULT etc/kayobe/kolla/octavia-certificates/client_ca | xargs ansible-vault encrypt --vault-password-file + git grep -L VAULT etc/kayobe/kolla/config/octavia/*.pem | xargs ansible-vault encrypt --vault-password-file + +Add and commit: + +.. code-block:: console + + git add etc/kayobe/kolla/octavia-certificates/client_ca/ + git add etc/kayobe/kolla/config/octavia/client.cert-and-key.pem etc/kayobe/kolla/config/octavia/client_ca.cert.pem + git commit -m "Octavia certs: rotate client certs" + +Discard other unnecessary changes: + +.. code-block:: console + + git status # check for anything unexpected! + git checkout etc/kayobe/kolla/octavia-certificates/ etc/kayobe/kolla/config/octavia + +Deploy the new client certs: + +.. code-block:: console + + kayobe overcloud service deploy --kolla-tags octavia + +Rotating the CAs +~~~~~~~~~~~~~~~~ + +The CAs have a 10 year lifetime. When rotating these, you will also need to +generate new certificates. + +Decrypt all certificate files: + +.. code-block:: console + + git grep -l VAULT etc/kayobe/kolla/octavia-certificates | xargs ansible-vault decrypt --vault-password-file + +Remove the existing CAs where appropriate: + +.. code-block:: console + + rm -rf etc/kayobe/kolla/octavia-certificates/client_ca/ + rm -rf etc/kayobe/kolla/octavia-certificates/server_ca/ + +Generate new CAs and certificates: + +.. code-block:: console + + kayobe kolla ansible run octavia-certificates -ke node_custom_config=$KAYOBE_CONFIG_PATH/kolla/config -ke node_config=$KAYOBE_CONFIG_PATH/kolla + +Encrypt the changed files that we are interested in: + +.. code-block:: console + + git grep -L VAULT etc/kayobe/kolla/octavia-certificates/client_ca | xargs ansible-vault encrypt --vault-password-file + git grep -L VAULT etc/kayobe/kolla/config/octavia/*.pem | xargs ansible-vault encrypt --vault-password-file + +Add and commit: + +.. code-block:: console + + git add etc/kayobe/kolla/octavia-certificates/client_ca/ + git add etc/kayobe/kolla/octavia-certificates/server_ca/ + git add etc/kayobe/kolla/config/octavia/client.cert-and-key.pem etc/kayobe/kolla/config/octavia/client_ca.cert.pem + git commit -m "Octavia certs: rotate CAs" + +Discard other unnecessary changes: + +.. code-block:: console + + git status # check for anything unexpected! + git checkout etc/kayobe/kolla/octavia-certificates/ etc/kayobe/kolla/config/octavia + +Deploy the new client certs: + +.. code-block:: console + + kayobe overcloud service deploy --kolla-tags octavia + +Follow any instructions in the `upstream docs +`_. + Backup of the OpenStack Control Plane =====================================