Make CaaS specific role: persist_openhpc_secrets idempotent

[bertiethorpe]

CaaS env now inherits from both common and site envs

As a result of this we hit a problem where alertmanager is now turned on by default but there’s no secret for it. Also secret is added to persist_openhpc_secrets.

Aligning CaaS to site also re-raises the issue of CaaS secrets not being idempotent. This is resolved by:

• Fixing upgrade by changing the persist_openhpc_secrets role to read the facts file if it exists
• template it out again, with the template being changed to be idempotent for values a la the approach in https://github.com/stackhpc/ansible-slurm-appliance/blob/main/ansible/roles/passwords/defaults/main.yml
• then read it again (getting any new passwords)

[bertiethorpe]

tested here: https://github.com/azimuth-cloud/ansible-collection-azimuth-ops/actions/runs/17547446720/job/49853940167?pr=947

[JohnGarbutt]

This seems like the least invasive fix, to get Azimuth working again.

But I would like to see what Steve thinks before we merge this.

[sjpb]

@bertiethorpe have you checked this is idempotent too? I.e. if you create a caas cluster, check the secrets, then patch it, do secrets definitely not get changed??

[bertiethorpe]

Tested fully now so going to go ahead and merge