updated docs

Author: wtripp180901

README.md

@@ -63,7 +63,7 @@ Use the `cookiecutter` template to create a new environment to hold your configu
     cd environments
     cookiecutter ../cookiecutter
 
-and follow the prompts to complete the environment name and description.
+and follow the prompts to complete the environment name and description, leaving `is_site_env` and `parent_site_env` as their defaults.
 
 **NB:** In subsequent sections this new environment is referred to as `$ENV`.
 

docs/production.md

@@ -15,19 +15,20 @@ production-ready deployments.
   A `dev` environment should also be created if considered required, or this
   can be left until later.
 
-  These can all be produced using the cookicutter instructions, but the
-  `production` and `staging` environments will need their
-  `environments/$ENV/ansible.cfg` file modifying so that they point to the
-  `site` environment:
-
-    ```ini
-    inventory = ../common/inventory,../site/inventory,inventory
-    ```
+  These can all be produced with cookiecutter by running
+  ```sh
+  cd environments
+  cookiecutter ../cookiecutter
+  ```
+  You should ensure that you set the `is_site_env` prompt to true for your `site` environment
+  and set the `parent_site_env` to `site` for your `production` and `staging` environments
 
 - To avoid divergence of configuration all possible overrides for group/role
 vars should be placed in `environments/site/inventory/group_vars/all/*.yml`
 unless the value really is environment-specific (e.g. DNS names for
-`openondemand_servername`).
+`openondemand_servername`). It is therefore recommended that you delete the initial
+`environments/{production/staging}/inventory/group_vars/all/*.yml` files in your `production` and
+`staging` environments.
 
 - Where possible hooks should also be placed in `environments/site/hooks/`
 and referenced from the `site` and `production` environments, e.g.:
@@ -38,38 +39,10 @@ and referenced from the `site` and `production` environments, e.g.:
       import_playbook: "{{ lookup('env', 'APPLIANCES_ENVIRONMENT_ROOT') }}/../site/hooks/pre.yml"
     ```
 
-- OpenTofu configurations should be defined in the `site` environment and used
-  as a module from the other environments. This can be done with the
-  cookie-cutter generated configurations:
-  - Delete the *contents* of the cookie-cutter generated `tofu/` directories
-    from the `production` and `staging` environments.
-  - Create a `main.tf` in those directories which uses `site/tofu/` as a
-    [module](https://opentofu.org/docs/language/modules/), e.g. :
-
-    ```
-    ...
-    variable "environment_root" {
-      type = string
-      description = "Path to environment root, automatically set by activate script"
-    }
-
-    module "cluster" {
-        source = "../../site/tofu/"
-        environment_root = var.environment_root
-
-        cluster_name = "foo"
-        ...
-    }
-    ```
-
-    Note that:
-    
-    - Environment-specific variables (`cluster_name`) should be hardcoded
-      into the cluster module block.
+- Define OpenTofu configurations
+    - Environment-specific variables (e.g `cluster_name`) should be set in `environments/$ENV/tofu/{$ENV}.tfvars`
     - Environment-independent variables (e.g. maybe `cluster_net` if the
-      same is used for staging and production) should be set as *defaults*
-      in `environments/site/tofu/variables.tf`, and then don't need to
-      be passed in to the module.
+      same is used for staging and production) should be set in `environments/site/tofu/site.auto.tfvars`
 
 - Vault-encrypt secrets. Running the `generate-passwords.yml` playbook creates
   a secrets file at `environments/$ENV/inventory/group_vars/all/secrets.yml`.
@@ -103,8 +76,8 @@ and referenced from the `site` and `production` environments, e.g.:
       state_volume_provisioning = "attach"
 
   either for a specific environment within the cluster module block in
-  `environments/$ENV/tofu/main.tf`, or as the site default by changing the
-  default in `environments/site/tofu/variables.tf`.
+  `environments/$ENV/tofu/{$ENV}.tfvars`, or as the site default by changing the
+  default in `environments/site/tofu/site.auto.tfvars`.
 
   For a development environment allowing OpenTofu to manage the volumes using
   the default value of `"manage"` for those varibles is usually appropriate, as

docs/upgrades.md

@@ -62,7 +62,8 @@ All other commands should be run on the Ansible deploy host.
 
 1. If required, build an "extra" image with local modifications, see [docs/image-build.md](./image-build.md).
 
-1. Modify your site-specific environment to use this image, e.g. via `cluster_image_id` in `environments/$SITE_ENV/tofu/variables.tf`.
+1. Modify your site-specific environment to use this image, e.g. via `cluster_image_id` in `environments/site/tofu/site.auto.tfvars` if all environments use the same image
+   or `environments/$SUB_ENV/tofu/{$SUB_ENV}.tfvars` if the image is environment-specific.
 
 1. Test this in your staging cluster.
 
@@ -101,3 +102,46 @@ playbook to reconfigure the cluster, e.g. as described in the main [README.md](.
 
 1. Tell users the cluster is available again.
 
+## Upgrading OpenTofu
+
+As of v2.3, environments now import the appliance's latest Tofu as a module, ensuring that your Tofu infrastructure is up to date
+with the configuration expected upstream. Environment defined before v2.3 must therefore be manually migrated to the new model.
+
+### Upgrading Site Environments
+
+1. Identify any custom defaults you have set in your `variables.tf` file with `diff environments/site/tofu/variables.tf tofu/variables.tf`
+
+1. Create a new `environments/site/tofu/site.auto.tfvars` file and assign any variables you previously set custom defaults for in `variables.tf` with their default value. For
+   example,
+   ```sh
+   variable "key_pair" {
+       type = string
+       description = "Name of an existing keypair in OpenStack"
+       default = "my-key"
+   }
+   ```
+   in `variables.tf` becomes
+   ```sh
+   key_pair = "my-key"
+   ```
+   in `site.auto.tfvars`
+
+1. Delete the contents of the `environments/site/tofu` except for the `site.auto.tfvars` file
+
+### Upgrading Production/Staging Environments
+
+1. In the `environments/$ENV_NAME/tofu/main.tf` file of your environment, identify any variables (other than `environment_root`) you have hardcoded as arguments to your site module
+
+1. Move these variable assignments to a new `environments/$ENV_NAME/tofu/$ENV_NAME.tfvars` file
+
+1. Delete the contents of the `environments/$ENV_NAME/tofu` directory, except for `.terraform`, `terraform.tfstate`, `.terraform.lock.hcl` and the new tfvars file
+
+1. Create a symlink from `tofu/layouts/main.tf` to `environments/$ENV_NAME/tofu/main.tf`
+
+1. Create a symlink from `tofu/variables.tf` to `environments/$ENV_NAME/tofu/main.tf`
+
+1. Create a symlink from `environments/site/tofu/site.auto.tfvars` to `environments/$ENV_NAME/tofu/site.auto.tfvars`
+
+1. Import the new module with `tofu init`
+
+1. Verify no destructive changes were made to your existing infrastructure with `tofu plan -var-file=$YOUR-TFVARS-FILE`

environments/README.md

@@ -42,7 +42,8 @@ To define an environment using cookiecutter:
     cd environments
     cookiecutter ../cookiecutter
 
-This will present you with a series of questions which you must answer.
+This will present you with a series of questions which you must answer. For guidance on setting
+`is_site_env` and  `parent_site_env`, see [production docs](../docs/production.md).
 Once you have answered all questions, a new environment directory will
 be created. The directory will be named according to the answer you gave
 for `environment`.