Skip to content

Commit 38d5709

Browse files
authored
DPE-4235, DPE-4315, DPE-3263 Common UX for replication (#421)
* port common ux * fix async test and bring in lib fixes * wait for initialization, bump black to match system and update secret label * add action to final test * password propagation support * bump workflows version * updated CI to 3.4.3 * asyncio_mode=auto is breaking pytest_operator_groups * sync dpw everywhere
1 parent d310ba9 commit 38d5709

File tree

14 files changed

+1404
-1094
lines changed

14 files changed

+1404
-1094
lines changed

.github/workflows/ci.yaml

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,7 @@ on:
1616
jobs:
1717
lint:
1818
name: Lint
19-
uses: canonical/data-platform-workflows/.github/workflows/lint.yaml@v13.1.0
19+
uses: canonical/data-platform-workflows/.github/workflows/lint.yaml@v13.3.0
2020

2121
unit-test:
2222
name: Unit test charm
@@ -56,7 +56,7 @@ jobs:
5656

5757
build:
5858
name: Build charm
59-
uses: canonical/data-platform-workflows/.github/workflows/build_charm.yaml@v13.1.0
59+
uses: canonical/data-platform-workflows/.github/workflows/build_charm.yaml@v13.3.0
6060
with:
6161
cache: true
6262

@@ -65,17 +65,17 @@ jobs:
6565
fail-fast: false
6666
matrix:
6767
juju:
68-
- agent: 2.9.46
68+
- agent: 2.9.49
6969
libjuju: ^2
7070
allure: false
71-
- agent: 3.1.7
71+
- agent: 3.4.3
7272
allure: true
7373
name: Integration test charm | ${{ matrix.juju.agent }}
7474
needs:
7575
- lint
7676
- unit-test
7777
- build
78-
uses: canonical/data-platform-workflows/.github/workflows/integration_test_charm.yaml@v13.1.0
78+
uses: canonical/data-platform-workflows/.github/workflows/integration_test_charm.yaml@v13.3.0
7979
with:
8080
artifact-prefix: ${{ needs.build.outputs.artifact-prefix }}
8181
cloud: microk8s

.github/workflows/release.yaml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -36,15 +36,15 @@ jobs:
3636

3737
build:
3838
name: Build charm
39-
uses: canonical/data-platform-workflows/.github/workflows/build_charm.yaml@v13.1.0
39+
uses: canonical/data-platform-workflows/.github/workflows/build_charm.yaml@v13.3.0
4040

4141
release:
4242
name: Release charm
4343
needs:
4444
- lib-check
4545
- ci-tests
4646
- build
47-
uses: canonical/data-platform-workflows/.github/workflows/release_charm.yaml@v13.1.0
47+
uses: canonical/data-platform-workflows/.github/workflows/release_charm.yaml@v13.3.0
4848
with:
4949
channel: 8.0/edge
5050
artifact-prefix: ${{ needs.build.outputs.artifact-prefix }}

.github/workflows/sync_issue_to_jira.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@ on:
99
jobs:
1010
sync:
1111
name: Sync GitHub issue to Jira
12-
uses: canonical/data-platform-workflows/.github/workflows/sync_issue_to_jira.yaml@v13.1.0
12+
uses: canonical/data-platform-workflows/.github/workflows/sync_issue_to_jira.yaml@v13.3.0
1313
with:
1414
jira-base-url: https://warthogs.atlassian.net
1515
jira-project-key: DPE

actions.yaml

Lines changed: 13 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -62,10 +62,20 @@ pre-upgrade-check:
6262
resume-upgrade:
6363
description: Resume a rolling upgrade after asserting successful upgrade of a new revision.
6464

65-
promote-standby-cluster:
65+
create-replication:
6666
description: |
67-
Promotes this cluster to become the leader in the cluster-set. Used for safe switchover or failover.
68-
Must be run against the charm leader unit of a standby cluster.
67+
Create replication between two related clusters.
68+
This action is must be run on the offer side of the relation.
69+
params:
70+
name:
71+
type: string
72+
description: A (optional) name for this replication.
73+
default: default
74+
75+
promote-to-primary:
76+
description: |
77+
Promotes this cluster to become the primary in the cluster-set. Used for safe switchover or failover.
78+
Can only be run against the charm leader unit of a standby cluster.
6979
params:
7080
cluster-set-name:
7181
type: string
@@ -86,24 +96,6 @@ recreate-cluster:
8696
each unit will be kept in blocked status. Recreating the cluster allows to rejoin the async replication
8797
relation, or usage as a standalone cluster.
8898
89-
fence-writes:
90-
description: |
91-
Stops write traffic to a primary cluster of a ClusterSet.
92-
params:
93-
cluster-set-name:
94-
type: string
95-
description: |
96-
The name of the cluster-set. Mandatory option, used for confirmation.
97-
98-
unfence-writes:
99-
description: |
100-
Resumes write traffic to a primary cluster of a ClusterSet.
101-
params:
102-
cluster-set-name:
103-
type: string
104-
description: |
105-
The name of the cluster-set. Mandatory option, used for confirmation.
106-
10799
rejoin-cluster:
108100
description: |
109101
Rejoins an invalidated cluster to the cluster-set, after a previous failover or switchover.
Lines changed: 144 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,144 @@
1+
"""Secrets related helper classes/functions."""
2+
3+
# Copyright 2023 Canonical Ltd.
4+
# See LICENSE file for licensing details.
5+
6+
from typing import Dict, Literal, Optional
7+
8+
from ops import Secret, SecretInfo
9+
from ops.charm import CharmBase
10+
from ops.model import SecretNotFoundError
11+
12+
# The unique Charmhub library identifier, never change it
13+
LIBID = "d77fb3d01aba41ed88e837d0beab6be5"
14+
15+
# Increment this major API version when introducing breaking changes
16+
LIBAPI = 0
17+
18+
# Increment this PATCH version before using `charmcraft publish-lib` or reset
19+
# to 0 if you are raising the major API version
20+
LIBPATCH = 2
21+
22+
23+
APP_SCOPE = "app"
24+
UNIT_SCOPE = "unit"
25+
Scopes = Literal["app", "unit"]
26+
27+
28+
class DataSecretsError(Exception):
29+
"""A secret that we want to create already exists."""
30+
31+
32+
class SecretAlreadyExistsError(DataSecretsError):
33+
"""A secret that we want to create already exists."""
34+
35+
36+
def generate_secret_label(charm: CharmBase, scope: Scopes) -> str:
37+
"""Generate unique group_mappings for secrets within a relation context.
38+
39+
Defined as a standalone function, as the choice on secret labels definition belongs to the
40+
Application Logic. To be kept separate from classes below, which are simply to provide a
41+
(smart) abstraction layer above Juju Secrets.
42+
"""
43+
members = [charm.app.name, scope]
44+
return f"{'.'.join(members)}"
45+
46+
47+
# Secret cache
48+
49+
50+
class CachedSecret:
51+
"""Abstraction layer above direct Juju access with caching.
52+
53+
The data structure is precisely re-using/simulating Juju Secrets behavior, while
54+
also making sure not to fetch a secret multiple times within the same event scope.
55+
"""
56+
57+
def __init__(self, charm: CharmBase, label: str, secret_uri: Optional[str] = None):
58+
self._secret_meta = None
59+
self._secret_content = {}
60+
self._secret_uri = secret_uri
61+
self.label = label
62+
self.charm = charm
63+
64+
def add_secret(self, content: Dict[str, str], scope: Scopes) -> Secret:
65+
"""Create a new secret."""
66+
if self._secret_uri:
67+
raise SecretAlreadyExistsError(
68+
"Secret is already defined with uri %s", self._secret_uri
69+
)
70+
71+
if scope == APP_SCOPE:
72+
secret = self.charm.app.add_secret(content, label=self.label)
73+
else:
74+
secret = self.charm.unit.add_secret(content, label=self.label)
75+
self._secret_uri = secret.id
76+
self._secret_meta = secret
77+
return self._secret_meta
78+
79+
@property
80+
def meta(self) -> Optional[Secret]:
81+
"""Getting cached secret meta-information."""
82+
if self._secret_meta:
83+
return self._secret_meta
84+
85+
if not (self._secret_uri or self.label):
86+
return
87+
88+
try:
89+
self._secret_meta = self.charm.model.get_secret(label=self.label)
90+
except SecretNotFoundError:
91+
if self._secret_uri:
92+
self._secret_meta = self.charm.model.get_secret(
93+
id=self._secret_uri, label=self.label
94+
)
95+
return self._secret_meta
96+
97+
def get_content(self) -> Dict[str, str]:
98+
"""Getting cached secret content."""
99+
if not self._secret_content:
100+
if self.meta:
101+
self._secret_content = self.meta.get_content()
102+
return self._secret_content
103+
104+
def set_content(self, content: Dict[str, str]) -> None:
105+
"""Setting cached secret content."""
106+
if self.meta:
107+
self.meta.set_content(content)
108+
self._secret_content = content
109+
110+
def get_info(self) -> Optional[SecretInfo]:
111+
"""Wrapper function for get the corresponding call on the Secret object if any."""
112+
if self.meta:
113+
return self.meta.get_info()
114+
115+
116+
class SecretCache:
117+
"""A data structure storing CachedSecret objects."""
118+
119+
def __init__(self, charm):
120+
self.charm = charm
121+
self._secrets: Dict[str, CachedSecret] = {}
122+
123+
def get(self, label: str, uri: Optional[str] = None) -> Optional[CachedSecret]:
124+
"""Getting a secret from Juju Secret store or cache."""
125+
if not self._secrets.get(label):
126+
secret = CachedSecret(self.charm, label, uri)
127+
128+
# Checking if the secret exists, otherwise we don't register it in the cache
129+
if secret.meta:
130+
self._secrets[label] = secret
131+
return self._secrets.get(label)
132+
133+
def add(self, label: str, content: Dict[str, str], scope: Scopes) -> CachedSecret:
134+
"""Adding a secret to Juju Secret."""
135+
if self._secrets.get(label):
136+
raise SecretAlreadyExistsError(f"Secret {label} already exists")
137+
138+
secret = CachedSecret(self.charm, label)
139+
secret.add_secret(content, scope)
140+
self._secrets[label] = secret
141+
return self._secrets[label]
142+
143+
144+
# END: Secret cache

0 commit comments

Comments
 (0)