Skip to content

Commit f8f6041

Browse files
odyody
authored
Merge pull request #286 from ody/documenting_add_actions
Provide documentation on expanding a deployment
2 parents 9d9cc99 + f02d272 commit f8f6041

File tree

2 files changed

+218
-3
lines changed

2 files changed

+218
-3
lines changed

‎README.md‎

Lines changed: 9 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -60,12 +60,18 @@ Follow the links below to usage instructions for each peadm plan.
6060

6161
## Reference
6262

63-
Additional documentation and information pertaining to various aspects or elements of peadm.
63+
Information from the Puppet documentation site that will help you understand which architecture is right for you.
6464

65-
* [DR Component Recovery](https://github.com/puppetlabs/puppetlabs-peadm/blob/main/documentation/recovery.md)
6665
* [PE Architecture Documentation](https://puppet.com/docs/pe/latest/choosing_an_architecture.html)
67-
* [Classification](https://github.com/puppetlabs/puppetlabs-peadm/blob/main/documentation/classification.md)
66+
* [PE Multi-region reference architecture](https://puppet.com/docs/patterns-and-tactics/latest/reference-architectures/pe-multi-region-reference-architectures.html)
67+
68+
69+
Documentation pertaining to additional uses of peadm.
70+
71+
* [DR Component Recovery](https://github.com/puppetlabs/puppetlabs-peadm/blob/main/documentation/recovery.md)
6872
* [Architectures](https://github.com/puppetlabs/puppetlabs-peadm/blob/main/documentation/architectures.md)
73+
* [Expanding Deployment](https://github.com/puppetlabs/puppetlabs-peadm/blob/main/documentation/expanding.md)
74+
* [Classification](https://github.com/puppetlabs/puppetlabs-peadm/blob/main/documentation/classification.md)
6975
* [Testing](https://github.com/puppetlabs/puppetlabs-peadm/blob/main/documentation/pre_post_checks.md)
7076
* [Docker Based Examples](https://github.com/puppetlabs/puppetlabs-peadm/blob/main/documentation/docker_examples.md)
7177

‎documentation/expanding.md‎

Lines changed: 209 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,209 @@
1+
# Expanding your deployment
2+
3+
Documentation which provides instructions for expanding existing PEADM based deployments of Puppet Enterprise with compilers, disaster recovery replicas, and external databases.
4+
5+
* [Adding External Databases with peadm::add_database](#adding-external-databases-with-peadmadd_database)
6+
* [Enable Disaster Recovery and Add a Replica with peadm::add_replica](#enable-disaster-recovery-and-add-a-replica-with-peadmadd_replica)
7+
* [Adding Compilers with peadm::add_compiler](#adding-compilers-with-peadmadd_compiler)
8+
9+
### Notes
10+
11+
* CLI options for `add_replica`, `add_compiler`, and `add_database` are unfortunately inconsistent
12+
* This is the result of a history of organic development
13+
* There is an inconsistency in the output of the task `peadm::get_peadm_config` and the naming of related parameters
14+
* The documentation and CLI refer to availability groups but the output from the task will refer to associated data as role letters
15+
* The term host and server are interchangeable throughout the documentation
16+
* When ever possible documentation will prefer the term server but plan parameters and `peadm::get_peadm_config` often uses the term host
17+
18+
### Key
19+
20+
* _\<primary-server-fqdn\>_ - The FQDN and certname of the Primary Puppet server
21+
* _\<new-postgres-server-fqdn\>_ - The FQDN and certname of the new PE-PostgreSQL server to initialize
22+
* _\<new-replica-server-fqdn\>_ - The FQDN and certname of the new Replica Puppet server to initialize
23+
* _\<replica-postgres-server-fqdn\>_ - The FQDN and certname of the Replica PE-PostgreSQL server
24+
* _\<new-compiler-fqdn\>_ - The FQDN and certname of the new Compiler to initialize
25+
* _\<new-compiler-target-group\>_ - The target availability group letter to assign to the new Compiler
26+
* _\<target-group-server-fqdn\>_ - The FQDN and certname of the Primary Puppet server that is assigned to the new Compiler's target availability group letter
27+
* _\<target-group-postgresql-fqdn\>_ - The FQDN and certname of the PE-PostgreSQL server that is assigned to the new Compiler's target availability group letter
28+
29+
## Adding External Databases with peadm::add_database
30+
31+
An external PE-PostgreSQL server is the component which separates the Extra Large and Large deployment architectures. These external database servers are PuppetDB specific and do not serve databases for other components of Puppet Enterprise. When the Extra Large deployment architecture is being utilized and disaster recovery (DR) is enabled, two external PE-PostgreSQL servers must be provisioned and it is required that both PE-PostgreSQL servers exist prior to provisioning a Replica Puppet server because it is immediately dependent upon this second external database server. In both situations, with **no DR** plans and when preparing to enable it for the first time, the command is exactly the same. This specific plan is intelligent enough to determine the most appropriate values for your second external PE-PostgreSQL server by examining the values which were used for the first.
32+
33+
### Add an external PE-PostgreSQL server in all scenarios
34+
35+
bolt plan run peadm::add_database -t <new-postgres-server-fqdn> primary_host=<primary-server-fqdn>:w
36+
37+
## Enable Disaster Recovery and Add a Replica with peadm::add_replica
38+
39+
All three deployment architectures have two variations, disaster recovery (DR) enabled or disabled. The deployment architecture a Puppet Enterprise deployment adopts is not changed by the addition of a Replica Puppet server. If you adopt the Standard deployment architecture and enable DR by provisioning a Replica Puppet server, the adopted deployment architecture remains as Standard. The basic process is also the same when adding a Replica Puppet server to any existing PEADM based deployment of Puppet Enterprise but there are differences in CLI arguments when operating on the Extra Large deployment architecture.
40+
41+
PEADM creates availability groups to logically group failure domains within a Puppet Enterprise deployment. These availability group designations control how various components connect together with backend services like database servers. Availability group **A** is created by PEADM in all scenarios but when you enable DR using PEADM, a second availability group will be created, **B**.
42+
43+
The initial Primary will be assigned availability group **A** and the initial Replica is assigned **B**. The availability group assignments stay with the server and importantly not with the role of the server. If you choose to promote the Replica Puppet server assigned to **B** to a Primary than availability group **B** is now simply the group containing the Primary Puppet server. When adding a Replica Puppet server you do not need to know the availability group of the Primary it is being paired with, the `peadm::add_replica` plan will determine the appropriate group by looking up to which group the Primary Puppet server is assigned.
44+
45+
### Adding a Replica to Standard and Large deployments
46+
47+
bolt plan run peadm::add_replica primary_host=<primary-server-fqdn> replica_host=<new-replica-server-fqdn>
48+
49+
### Adding a Replica to Extra Large deployments
50+
51+
In deployments which adopted the Extra Large deployment architecture you must provide the `replica_postgresql_host` parameter set to the PE-PostgreSQL server which will be collocated within the same availability group as the new Replica Puppet server. The `peadm::get_peadm_config` task will help you determine the most appropriate value. In the **Example** section below, the task has figured out which PE-PostgreSQL server is the Replica PE-PostgreSQL database server. You'll find the value at `params.replica_postgresql_host`, which is equal to `pe-psql-6251cd-1.us-west1-b.c.slice-cody.internal`. Reminder, the Replica PE-PostgreSQL server **MUST** be provisioned and deployed prior to initializing a Replica Puppet server.
52+
53+
bolt plan run peadm::get_peadm_config --targets <primary-server-fqdn>
54+
bolt plan run peadm::add_replica primary_host=<primary-server-fqdn> replica_host=<new-replica-server-fqdn> replica_postgresql_host=<replica-postgres-server-fqdn>
55+
56+
**Example**
57+
58+
% bolt task run peadm::get_peadm_config --targets pe-server-6251cd-0.us-west1-a.c.slice-cody.internal -- INSERT --
59+
Started on pe-server-6251cd-0.us-west1-a.c.slice-cody.internal...
60+
Finished on pe-server-6251cd-0.us-west1-a.c.slice-cody.internal:
61+
{
62+
"params": {
63+
"primary_host": "pe-server-6251cd-0.us-west1-a.c.slice-cody.internal",
64+
"replica_host": null,
65+
"primary_postgresql_host": "pe-psql-6251cd-0.us-west1-a.c.slice-cody.internal",
66+
"replica_postgresql_host": "pe-psql-6251cd-1.us-west1-b.c.slice-cody.internal",
67+
"compilers": [
68+
"pe-compiler-6251cd-0.us-west1-a.c.slice-cody.internal",
69+
"pe-compiler-6251cd-1.us-west1-b.c.slice-cody.internal"
70+
],
71+
"compiler_pool_address": "puppet.pe-compiler-lb-6251cd.il4.us-west1.lb.slice-cody.internal",
72+
"internal_compiler_a_pool_address": "pe-server-6251cd-0.us-west1-a.c.slice-cody.internal",
73+
"internal_compiler_b_pool_address": null
74+
},
75+
"role-letter": {
76+
"server": {
77+
"A": "pe-server-6251cd-0.us-west1-a.c.slice-cody.internal",
78+
"B": null
79+
},
80+
"postgresql": {
81+
"A": "pe-psql-6251cd-0.us-west1-a.c.slice-cody.internal",
82+
"B": "pe-psql-6251cd-1.us-west1-b.c.slice-cody.internal"
83+
},
84+
"compilers": {
85+
"A": [
86+
"pe-compiler-6251cd-0.us-west1-a.c.slice-cody.internal",
87+
"pe-compiler-6251cd-1.us-west1-b.c.slice-cody.internal"
88+
],
89+
"B": [
90+
91+
]
92+
}
93+
}
94+
}
95+
Successful on 1 target: pe-server-6251cd-0.us-west1-a.c.slice-cody.internal
96+
Ran on 1 target in 2.56 sec
97+
98+
## Adding Compilers with peadm::add_compiler
99+
100+
The Standard deployment architecture is the only deployment architecture of the three which does not include Compilers, the lack of them is what differentiates the Standard from Large deployment architecture. Deployment architecture has no effect on the process for adding Compilers to a deployment. The [peadm::add_compiler](https://github.com/puppetlabs/puppetlabs-peadm/blob/main/plans/add_compiler.pp) plan functions identical in all three deployment architectures, whether you are adding the 1st or the 100th but some options do change slightly depending.
101+
102+
### Adding Compilers to Standard and Large without disaster recovery
103+
104+
The command invocation is identical when adding Compilers to a Standard or Large deployment architecture if disaster recovery (DR) is not enabled and a Replica Puppet server has not been provisioned. Take note of the values for `avail_group_letter` and `primary_postgresql_host`, in this **no DR** scenario, the value of these parameters will always be set to **A** and the FQDN of the Primary Puppet server.
105+
106+
bolt plan run peadm::add_compiler primary_host=<primary-server-fqdn> compiler_host=<new-compiler-fqdn> avail_group_letter=A primary_postgresql_host=<primary-server-fqdn>
107+
108+
### Adding Compilers to Extra Large without disaster recovery
109+
110+
When adding a Compiler to a deployment which has adopted the Extra Large deployment architecture in a **no DR** scenario, the only difference is that the `primary_postgresql_host` changes to be the value of the Primary PE-PostgreSQL server as opposed to the Primary Puppet server.
111+
112+
bolt plan run peadm::add_compiler primary_host=<primary-server-fqdn> compiler_host=<new-compiler-fqdn> avail_group_letter=A primary_postgresql_host=<primary-postgresql-server-fqdn>
113+
114+
### Adding Compilers to Standard and Large when disaster recovery has been enabled
115+
116+
As was described in the section documenting [peadm::add_replica](#enable-disaster-recovery-and-add-a-replica-with-peadmadd_replica), when disaster recovery (DR) is enabled and a Replica provisioned, PEADM creates a second availability group, **B**. You must take this second availability group into consideration when adding new compilers and ensure you are assigning appropriate values for the group the Compiler is targeted for. It is a good idea to keep these two availability groups populated with an equal quantity of Compilers. Besides the value of `avail_group_letter` being dependent on which group the new Compiler is targeted towards, the value of `primary_postgresql_host` will also vary.
117+
118+
The name of the `primary_postgresql_host` parameter can be confusing, it is **NOT** always equal to the Primary Puppet server or Primary PE-PostgreSQL server, it can also be equal to the Replica Puppet server or Replica PE-PostgreSQL server. It should be set to the server which is a member of the Compiler's target availability group. The easiest way to determine this value is to first run the `peadm::get_peadm_config` task and source the value from its output. In the **Example** section the value to use when targeting the **B** group is `pe-server-59ab63-1.us-west1-b.c.slice-cody.internal`. You'll find the value at `role-letter.server.B`.
119+
120+
bolt plan run peadm::get_peadm_config --targets <primary-server-fqdn>
121+
bolt plan run peadm::add_compiler primary_host=<primary-server-fqdn> compiler_host=<new-compiler-fqdn> avail_group_letter=<new-compiler-target-group> primary_postgresql_host=<target-group-server-fqdn>
122+
123+
**Example**
124+
125+
% bolt task run peadm::get_peadm_config --targets pe-server-59ab63-0.us-west1-a.c.slice-cody.internal
126+
Started on pe-server-59ab63-0.us-west1-a.c.slice-cody.internal...
127+
Finished on pe-server-59ab63-0.us-west1-a.c.slice-cody.internal:
128+
{
129+
"params": {
130+
"primary_host": "pe-server-59ab63-0.us-west1-a.c.slice-cody.internal",
131+
"replica_host": "pe-server-59ab63-1.us-west1-b.c.slice-cody.internal",
132+
"primary_postgresql_host": null,
133+
"replica_postgresql_host": null,
134+
"compilers": [
135+
"pe-compiler-59ab63-0.us-west1-a.c.slice-cody.internal",
136+
"pe-compiler-59ab63-1.us-west1-b.c.slice-cody.internal"
137+
],
138+
"compiler_pool_address": "puppet.pe-compiler-lb-59ab63.il4.us-west1.lb.slice-cody.internal",
139+
"internal_compiler_a_pool_address": "pe-server-59ab63-0.us-west1-a.c.slice-cody.internal",
140+
"internal_compiler_b_pool_address": "pe-server-59ab63-1.us-west1-b.c.slice-cody.internal"
141+
},
142+
"role-letter": {
143+
"server": {
144+
"A": "pe-server-59ab63-0.us-west1-a.c.slice-cody.internal",
145+
"B": "pe-server-59ab63-1.us-west1-b.c.slice-cody.internal"
146+
},
147+
"postgresql": {
148+
"A": null,
149+
"B": null
150+
},
151+
"compilers": {
152+
"A": [
153+
"pe-compiler-59ab63-0.us-west1-a.c.slice-cody.internal"
154+
],
155+
"B": [
156+
"pe-compiler-59ab63-1.us-west1-b.c.slice-cody.internal"
157+
]
158+
}
159+
}
160+
}
161+
Successful on 1 target: pe-server-59ab63-0.us-west1-a.c.slice-cody.internal
162+
Ran on 1 target in 2.46 sec
163+
164+
### Adding compilers to Extra Large when disaster recovery has been enabled
165+
166+
Adding a Compiler to a deployment which has adopted the Extra Large deployment architecture with disaster recovery (DR) enabled is similar to Standard and Large but the value of `primary_postgresql_host` will no longer correspond to the Primary or Replica Puppet server since PuppetDB databases are now hosted externally. In the **Example** section, the value to use when targeting the **A** group is `pe-psql-65e03f-0.us-west1-a.c.slice-cody.internal`. You'll find the value at `role-letter.postgresql.A`.
167+
168+
bolt plan run peadm::get_peadm_config --targets <primary-server-fqdn>
169+
bolt plan run peadm::add_compiler primary_host=<primary-server-fqdn> compiler_host=<new-compiler-fqdn> avail_group_letter=<new-compiler-target-availability-group> primary_postgresql_host=<target-availability-group-postgresql-fqdn>
170+
171+
**Example**
172+
173+
% bolt task run peadm::get_peadm_config --targets pe-server-65e03f-0.us-west1-a.c.slice-cody.internal
174+
Started on pe-server-65e03f-0.us-west1-a.c.slice-cody.internal...
175+
Finished on pe-server-65e03f-0.us-west1-a.c.slice-cody.internal:
176+
{
177+
"params": {
178+
"primary_host": "pe-server-65e03f-0.us-west1-a.c.slice-cody.internal",
179+
"replica_host": "pe-server-65e03f-1.us-west1-b.c.slice-cody.internal",
180+
"primary_postgresql_host": "pe-psql-65e03f-0.us-west1-a.c.slice-cody.internal",
181+
"replica_postgresql_host": "pe-psql-65e03f-1.us-west1-b.c.slice-cody.internal",
182+
"compilers": [
183+
"pe-compiler-65e03f-0.us-west1-a.c.slice-cody.internal"
184+
],
185+
"compiler_pool_address": "puppet.pe-compiler-lb-65e03f.il4.us-west1.lb.slice-cody.internal",
186+
"internal_compiler_a_pool_address": "pe-server-65e03f-0.us-west1-a.c.slice-cody.internal",
187+
"internal_compiler_b_pool_address": "pe-server-65e03f-1.us-west1-b.c.slice-cody.internal"
188+
},
189+
"role-letter": {
190+
"server": {
191+
"A": "pe-server-65e03f-0.us-west1-a.c.slice-cody.internal",
192+
"B": "pe-server-65e03f-1.us-west1-b.c.slice-cody.internal"
193+
},
194+
"postgresql": {
195+
"A": "pe-psql-65e03f-0.us-west1-a.c.slice-cody.internal",
196+
"B": "pe-psql-65e03f-1.us-west1-b.c.slice-cody.internal"
197+
},
198+
"compilers": {
199+
"A": [
200+
"pe-compiler-65e03f-0.us-west1-a.c.slice-cody.internal"
201+
],
202+
"B": [
203+
"pe-compiler-65e03f-1.us-west1-b.c.slice-cody.internal"
204+
]
205+
}
206+
}
207+
}
208+
Successful on 1 target: pe-server-65e03f-0.us-west1-a.c.slice-cody.internal
209+
Ran on 1 target in 2.35 sec

0 commit comments

Comments
 (0)