Skip to content

Commit 86d2e99

Browse files
Merge pull request #5588 from DonOmalVindula/fix/24552
Remove local_member_host entry from kubernetes configuration
2 parents d6a2210 + e3c0cab commit 86d2e99

File tree

7 files changed

+471
-485
lines changed

7 files changed

+471
-485
lines changed

en/identity-server/5.10.0/docs/setup/deployment-guide.md

Lines changed: 134 additions & 136 deletions
Large diffs are not rendered by default.

en/identity-server/5.11.0/docs/setup/deployment-guide.md

Lines changed: 118 additions & 120 deletions
Large diffs are not rendered by default.

en/identity-server/6.0.0/docs/deploy/deployment-guide.md

Lines changed: 50 additions & 52 deletions
Large diffs are not rendered by default.

en/identity-server/6.1.0/docs/deploy/deployment-guide.md

Lines changed: 49 additions & 51 deletions
Large diffs are not rendered by default.

en/identity-server/7.0.0/docs/deploy/deployment-guide.md

Lines changed: 40 additions & 42 deletions
Original file line numberDiff line numberDiff line change
@@ -46,7 +46,7 @@ The above recommendations can change based on the expected concurrency and perfo
4646
<tbody>
4747
<tr class="odd">
4848
<th>Operating systems</th>
49-
<td><p>For information on tested operating systems, see <a href="{{base_path}}/deploy/environment-compatibility/#tested-operating-systems-and-jdks">Tested Operating Systems and JDKs</a>
49+
<td><p>For information on tested operating systems, see <a href="{{base_path}}/deploy/environment-compatibility/#tested-operating-systems-and-jdks">Tested Operating Systems and JDKs</a>
5050
.</p></td>
5151
</tr>
5252
<tr class="even">
@@ -82,15 +82,15 @@ The above recommendations can change based on the expected concurrency and perfo
8282
In a clustered deployment, all WSO2 Identity Server nodes are pointed to the same databases to ensure the integrity of the data. Also, you can configure multiple logical databases if you require to keep your data logically separated in the environment. The following tutorial demonstrates deployment with an identity database (`IDENTITY_DB`) and a user database (`UM_DB`).
8383

8484
!!! note
85-
Alternatively, you can create more databases to separate the data logically for each data type.
85+
Alternatively, you can create more databases to separate the data logically for each data type.
8686
This will not make a difference in performance and is not mandatory. Separating databases logically may help to have a different backup and scaling strategy when the deployment is large and complex.
8787
If you do wish to separate the data logically into separate databases, see [Set Up Separate Databases for Clustering]({{base_path}}/deploy/set-up-separate-databases-for-clustering).
8888

8989
!!! tip
9090
If you have configured the shared database correctly, the `deployment.toml` file in the `<IS_HOME>/repository/conf` directory should have the following configurations.
9191

9292
Following is a sample configuration. Therefore parameter values might be different.
93-
93+
9494
```toml
9595
[database.identity_db]
9696
type = "mysql"
@@ -99,7 +99,7 @@ In a clustered deployment, all WSO2 Identity Server nodes are pointed to the sam
9999
username = "regadmin"
100100
password = "regadmin"
101101
port = "3306"
102-
102+
103103
[database.shared_db]
104104
type = "mysql"
105105
hostname = "localhost"
@@ -121,7 +121,7 @@ The following diagram is a high-level component diagram showing how the system w
121121
{% include "../includes/registry-repositories.md" %}
122122

123123
<!--!!! info
124-
For more information about the registry,
124+
For more information about the registry,
125125
see [Work with the Registry](TO-DO:{{base_path}}/guides/working-with-the-registry).-->
126126

127127
In this cluster setup, we use the default h2 database as the local registry in each node individually. The governance and configuration registries should be mounted to share across all nodes. In the WSO2 Identity Server, config and governance registries are mounted by default.
@@ -162,29 +162,29 @@ The following configurations need to be done in both the WSO2 Identity Server no
162162
!!! info
163163
The simplest is the well-known address (WKA) based clustering method. It only suites where all the nodes are deployed on machines having static IP addresses. <!--For more information, see [About Membership Schemes]({{base_path}}/deploy/clustering-overview/#about-membership-schemes).-->
164164
Configurations for each membership scheme are listed below.
165-
166-
??? tip "Click to see the instructions for the WKA scheme"
165+
166+
??? tip "Click to see the instructions for the WKA scheme"
167167
Edit the `<IS_HOME>/repository/conf/deployment.toml` file to add the following configurations.
168-
Configure the `localMemberHost` and `localMemberPort` entries. Add the IP of the editing node itself.
168+
Configure the `localMemberHost` and `localMemberPort` entries. Add the IP of the editing node itself.
169169
```
170170
[clustering]
171171
membership_scheme = "wka"
172172
local_member_host = "192.168.2.1"
173173
local_member_port = "4000"
174174
members = ["192.168.2.1:4000", "192.168.2.2:4001"]
175-
```
176-
Under the `members` section, add the `hostName` and `port` for each WKA member. As we have only two nodes in our sample cluster configuration, we will configure both as WKA nodes.
177-
178-
You can also use IP address ranges for the `hostName`. For example, `192.168.1.2-10`. This should ensure that the cluster eventually recovers after failures. One shortcoming of doing this is that you can define a range only for the last portion of the IP address. You should also keep in mind that the smaller the range, the faster it discovers members since each node has to scan fewer potential members.
179-
180-
??? tip "Click to see the instructions for the AWS ECS membership scheme"
181-
175+
```
176+
Under the `members` section, add the `hostName` and `port` for each WKA member. As we have only two nodes in our sample cluster configuration, we will configure both as WKA nodes.
177+
178+
You can also use IP address ranges for the `hostName`. For example, `192.168.1.2-10`. This should ensure that the cluster eventually recovers after failures. One shortcoming of doing this is that you can define a range only for the last portion of the IP address. You should also keep in mind that the smaller the range, the faster it discovers members since each node has to scan fewer potential members.
179+
180+
??? tip "Click to see the instructions for the AWS ECS membership scheme"
181+
182182
1. Create a working AWS ECS Cluster. Note the following when creating a cluster.
183183
- Select the `EC2 instance` type.
184184
- Note the `name` and `VPC CIDR block` of the cluster, as you will require them later for configurations.
185-
- Ensure that the `Container instance IAM role` that you assign to the ECS cluster has the following permission policy attached.
185+
- Ensure that the `Container instance IAM role` that you assign to the ECS cluster has the following permission policy attached.
186186
```
187-
{ "Version": "2012-10-17",
187+
{ "Version": "2012-10-17",
188188
"Statement":
189189
[
190190
{
@@ -198,22 +198,22 @@ The following configurations need to be done in both the WSO2 Identity Server no
198198
}
199199
]
200200
}
201-
201+
202202
```
203-
- Make sure that the security group of the cluster instances has an inbound rule to allow incoming traffic on the Hazelcast default port range `(5701 - 5708)`. It is advised to restrict access to instances in the same security group for this inbound rule.
204-
205-
2. Create a `deployment.toml` file in a preferred directory and add the following configurations.
203+
- Make sure that the security group of the cluster instances has an inbound rule to allow incoming traffic on the Hazelcast default port range `(5701 - 5708)`. It is advised to restrict access to instances in the same security group for this inbound rule.
204+
205+
2. Create a `deployment.toml` file in a preferred directory and add the following configurations.
206206
```
207207
[clustering]
208208
membership_scheme = "aws-ecs"
209-
209+
210210
[clustering.properties]
211211
region = "us-east-1"
212212
clusterName = "ECS-IS-CLUSTER"
213213
hostHeader = "ec2"
214214
vpcCidrBlock = "10.0.*.*"
215215
tagValue = "a_tag_value"
216-
```
216+
```
217217
Under the `clustering.properties` section, set the `region`, `clusterName`, `tagValue` and `vpcCidrBlock` based on the AWS ECS cluster you created in the previous step. The `tagValue` is derived from the auto-generated tag `aws:cloudformation:stack-name` in the AWS cluster. If you want to use a custom tag, you'll need to update both the `tagKey` and the `tagValue` accordingly.
218218

219219
!!! note
@@ -222,7 +222,7 @@ The following configurations need to be done in both the WSO2 Identity Server no
222222
!!! note
223223
Once all the configurations are complete, build a docker image including the configurations. You can use this Docker image to create a `Task Definition`, and make sure to set the network mode to `host` in the definition. Then run a new `Service` or a `Task` on the `AWS ECS cluster` you created.
224224

225-
??? tip "Click to see the instructions for the AWS EC2 membership scheme"
225+
??? tip "Click to see the instructions for the AWS EC2 membership scheme"
226226

227227
When WSO2 products are deployed in clustered mode on Amazon EC2 instances, it is recommended to use the AWS clustering mode. Open the `deployment.toml` file (stored in the `<IS_HOME>/repository/conf/` directory) and make the following changes.
228228

@@ -244,12 +244,12 @@ The following configurations need to be done in both the WSO2 Identity Server no
244244
securityGroup = "security_group_name"
245245
region = "us-east-1"
246246
tagKey = "a_tag_key"
247-
tagValue = "a_tag_value"
247+
tagValue = "a_tag_value"
248248
```
249-
It's recommended to add all the nodes to the same security group. The AWS credentials and security group depend on your configurations in the Amazon EC2 instance. The `tagKey` and `tagValue` are optional and the rest of the above parameters are mandatory.
249+
It's recommended to add all the nodes to the same security group. The AWS credentials and security group depend on your configurations in the Amazon EC2 instance. The `tagKey` and `tagValue` are optional and the rest of the above parameters are mandatory.
250250

251251
3. To provide specific permissions for creating an access key and secret key for only this AWS clustering attempt, use the custom policy block given below.
252-
See the [AWS documentation](http://docs.aws.amazon.com/IAM/latest/UserGuide/tutorial_managed-policies.html) for details on how to add the custom IAM policy.
252+
See the [AWS documentation](http://docs.aws.amazon.com/IAM/latest/UserGuide/tutorial_managed-policies.html) for details on how to add the custom IAM policy.
253253
Attach this to the user account that will operate AWS clustering in your WSO2 IS. The access key and secret key can only be used to list EC2 instance details in the AWS account.
254254
```json
255255
{ "Version": "2012-10-17",
@@ -276,15 +276,13 @@ The following configurations need to be done in both the WSO2 Identity Server no
276276
| Parameter | Description | Example |
277277
|-----------|---------------|-----------|
278278
| `membershipScheme` | This is the membership scheme that will be used to manage the membership of nodes in a cluster. | `kubernetes` |
279-
| `local_member_host` | This is the member's hostname or IP address. Set it to the pod's local IP address. | `172.17.0.2` |
280279
| `local_member_port` | This is the TCP port used by this member and through which other members will contact this member. | `4000` |
281280
| `KUBERNETES_NAMESPACE` | This is the Kubernetes Namespace in which the pods are deployed. | `wso2-is` |
282281
| `KUBERNETES_SERVICES` | These are the Kubernetes Services that belong in the cluster. | `wso2is-service` |
283-
282+
284283
```toml
285284
[clustering]
286285
membership_scheme = "kubernetes"
287-
local_member_host = "172.17.0.2"
288286
local_member_port = "4000"
289287

290288
[clustering.properties]
@@ -293,14 +291,14 @@ The following configurations need to be done in both the WSO2 Identity Server no
293291
```
294292

295293
- In order to retrieve the pod IP address information from the Kubernetes api server, the Kubernetes membership scheme uses the pod's service account. Hence, the pods need to be associated with a service account that has permission to read the "endpoints" resource. Make sure the role you bind has the following permissions.
296-
294+
297295
```toml
298296
rules:
299-
- apiGroups: [""]
300-
verbs: ["get", "list"]
301-
resources: ["endpoints"]
297+
- apiGroups: [""]
298+
verbs: ["get", "list"]
299+
resources: ["endpoints"]
302300
```
303-
301+
304302
- Optionally, a Kubernetes token or basic authentication can be used to authenticate with the Kubernetes api server.
305303
The following properties can be set under `[clustering.properties]` accordingly.
306304
- `KUBERNETES_API_SERVER`: This is the Kubernetes API endpoint,e.g., `http://172.17.8.101:8080`. Alternatively, an https endpoint can be set via `KUBERNETES_SERVICE_HOST` and `KUBERNETES_SERVICE_PORT_HTTPS`.
@@ -342,7 +340,7 @@ The following configurations need to be done in both the WSO2 Identity Server no
342340
force_local_cache = true
343341
```
344342
Cache invalidation uses Hazelcast messaging to distribute the invalidation message over the cluster and invalidate the caches properly. This is used to minimize the coherence problem in a multi-node setup.
345-
343+
346344
**Typical clustered deployment cache scenarios**
347345

348346
<table>
@@ -378,7 +376,7 @@ The following configurations need to be done in both the WSO2 Identity Server no
378376
<td>Disabled</td>
379377
<td>
380378
<ul>
381-
<li>Invalidation clears only the caches in specific nodes. Other caches are
379+
<li>Invalidation clears only the caches in specific nodes. Other caches are
382380
cleared at cache expiration.</li>
383381
<li>Hazelcast communication is not used.</li>
384382
<li>As the decisions take time to propagate over nodes (default cache timeout is 15 minutes), there is a security risk in this method. To reduce the risk, reduce the default cache timeout period. To learn how to reduce the default cache timeout period, see [Configuring Cache Layers - timeout]({{base_path}}/deploy/performance/configure-cache-layers#timeout).</li>
@@ -395,7 +393,7 @@ The following configurations need to be done in both the WSO2 Identity Server no
395393
<ul>
396394
<li>The data are directly acquired from the database.</li>
397395
<li>Eliminates the security risks caused due to not having cache invalidation.</li>
398-
<li>This method will create a performance degradation due to the lack of
396+
<li>This method will create a performance degradation due to the lack of
399397
caching.</li>
400398
</ul>
401399
</td>
@@ -408,9 +406,9 @@ The following configurations need to be done in both the WSO2 Identity Server no
408406
<td>Enabled</td>
409407
<td>
410408
<ul>
411-
<li>To reduce the security risk created in the second scenario and to
412-
improve performance in comparison with the third scenario, disable the
413-
security-related caches and sustain the performance-related caches as
409+
<li>To reduce the security risk created in the second scenario and to
410+
improve performance in comparison with the third scenario, disable the
411+
security-related caches and sustain the performance-related caches as
414412
local caches.</li>
415413
<li>This requires identifying these caches depending on the use case.</li>
416414
</ul>
@@ -455,7 +453,7 @@ The following configurations need to be done in both the WSO2 Identity Server no
455453
!!! note
456454
The WSO2 Identity Server cluster uses this hostname and therefore it must be specified in the `/etc/hosts` file in each node so that internal calls will not be routed through the Load Balancer.
457455
458-
Example:
456+
Example:
459457
`192.168.2.1 wso2.is`
460458
461459
## Enable artifact synchronization

0 commit comments

Comments
 (0)