Skip to content

Commit e0677d9

Browse files
authored
CLD-476 HAProxy Load Balancer and Bug Fixes
* feature/cld-421: added docs folder and tutorial draft for getting started in k8s * feature/cld-421: fixed youre grammar * feature/cld-421: added apostrophe where it was needed * feature/cld-421: fixed spelling of occurred * feature/cld-421: bulleted list * feature/cld-421: readme updates and changes * feature/cld-421: fixed documentation from peer review * feature/cld-421: fixed bad code style * feature/cld-421: title rework * feature/cld-421: application lowercase * feature/cld-421: updates for bullet points and minikube command structure * feature/cld-421: in vs on * feature/cld-421: updates for bullet points and helm release period * feature/cld-421: bullet structure update for single host install * feature/cld-421: updated the list of ports to have more * feature/cld-421: updated cleaup hyperlink to work * feature/cld-421: added note for ML cluster * feature/cld-421: added link for split brain problem and updated toc additionally fixed some comma errors * feature/cld-421: Updates from peer review * feature/cld-421: fixed some grammar and removed virtual box reference from tutorial * feature/cld-421: added repository instead of cloning * feature/cld-421: docs update for values.yaml and -it in kubectl exec command * Adding official License and Notice files (#6) Co-authored-by: Vitaly Korolev <[email protected]> * CLD-413: Kubernetes EA documentation (#7) * CLD-413: Kubernetes EA documentation * Initial Commit for Marklogic Helm Chart (#3) CLD-392: Develop Helm Chart * use template directives for names * make ServiceAccount following Helm convention * split the headless service and ClusterIP service in to 2 yaml files * add helm test for testing connections * add emptyDir if disable the persistence PVC * add README with Chart usage * remove resource request and limit by default * add imagePullSecret * CLD-424: Using Terratest for E2E test * package helm chart for repo hosting * change chart version * CLD-456: Configure DNS search entries dynamically * set the storageClass to empty to use the default storage class * update e2e test script with Minikube * make ML_BOOTSTRAP_HOST adapt to the change of name in configmap * change the container name to marklogic-server * CLD-472: Use Resolvable FQDN for Marklogic hostname * remove ingress and roles that won't ship in EA1 release * add extraContainerPorts * disable turning on XDQP * update default image version to 10.0-9.1-centos-1.0.0-ea4 * CLD-504: Fix helm repo add url * Update README * update package * add nodeSelector to statefulSet * CLD-413: Document update for EA1 release (#15) * fix test failure * CLD-413: add parameters and how to access marklogic section in README * README review update * change README as per review * change README base on review * CLD 545: EA1 Documentation fixes (#16) * CLD-545: EA1 document changes commit * CLD-545: EA1 document changes commit * CLD-545: Spell Check Correction Co-authored-by: Sumanth Ravipati <[email protected]> * CLD-520: EA1 Documentation fixes (#17) Co-authored-by: Sumanth Ravipati <[email protected]> * CLD-548: Docker Images Changes (#19) Co-authored-by: Sumanth Ravipati <[email protected]> * update helm package for 1.0.0-ea1 (#21) Co-authored-by: Daniel Wasserlauf <[email protected]> Co-authored-by: Daniel Wasserlauf <[email protected]> Co-authored-by: Vitaly <[email protected]> Co-authored-by: Vitaly Korolev <[email protected]> Co-authored-by: Peng Zhou <[email protected]> Co-authored-by: IlanRosenbaum <[email protected]> Co-authored-by: Ilan <[email protected]> Co-authored-by: Sumanth Ravipati <[email protected]> * fix README for getting the password from secret. * add haproxytech/haproxy 1.17.3 as dependent chart * add support for exisitingConfigMap for haproxy * modify Chart File * marklogic config file for Haproxy * add clusterDomain * modify configMap for HAproxy to automatically create connection config * change connection and cookie setting that fixed MLCP connection issue * use stick-table to support multiple cookie for sticky session * add config for odbc * modify the haproxy chart to take the new ports in values file * Use Port Type to configure HaProxy * use custom clusterDomain for fqdn * update values file for HAProxy * Update haproxy config map for better support for cookie based session affinity for Admin Web Apps * remove externTrafficPolicy * Add support for restart HAproxy whenever perform upgrade via Helm * add template test for haproxy * reset the values file * remove deprecated files * restore values file to fix testing * update values.yaml documentation and option for haproxy * fix issue in haproxy template test * upgrade HAProxy Chart to version 1.18.0 * add documentation for HAProxy in README * update versions in Chart.yaml * remove support for DaemonSet * Add MarkLogic modification copyright notice for HAProxy chart * add targetPort for haproxy ports * refactor statistic page for haproxy * Support for HAProxy * remove unused values.yaml * remove certificates * update documentation * fix merge issues * remove unnecessary if check * fix merging issue * remove t.Parallel() to improve test stability * update documentation for HAProxy * increase e2e test time * fix merge mistake * remove duplicated volumeMounts for mladmin-secrets * bug fix for wallet-password * bugfix: read file into variable from shell * Fix E2E test: increase retry times for WaitUntilPodAvailable
1 parent c0b741d commit e0677d9

39 files changed

+2323
-35
lines changed

.gitignore

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,2 @@
11
.vscode
2-
values.yaml
32
test/test_results

README.md

Lines changed: 79 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -33,6 +33,15 @@
3333
- [Port Forward](#port-forward)
3434
- [Forward to Pod](#forward-to-pod)
3535
- [Forward to Service](#forward-to-service)
36+
- [HAPorxy Load Balancer](#haporxy-load-balancer)
37+
- [Enable HAProxy](#enable-haproxy)
38+
- [Configuration](#configuration)
39+
- [ConfigMap](#configmap)
40+
- [Modify Port Configuration](#modify-port-configuration)
41+
- [Automatic Reload Configuration](#automatic-reload-configuration)
42+
- [External Access](#external-access)
43+
- [Access HAProxy](#access-haproxy)
44+
- [Security](#security)
3645
- [Security Context](#security-context)
3746
- [Network Policy](#network-policy)
3847
- [Pod Priorty](#pod-priorty)
@@ -249,7 +258,7 @@ Identify the name of the secret.
249258

250259
2. Save the secret name from step 1 and get the admin password using the following script:
251260
```
252-
kubectl get secret SECRET_NAME -o jsonpath='{.data.marklogic-password}' | base64 --decode
261+
kubectl get secret SECRET_NAME -o jsonpath='{.data.password}' | base64 --decode
253262
```
254263
### Log Collection
255264

@@ -412,6 +421,75 @@ kubectl port-forward svc/marklogic 8000:8000
412421

413422
This pod can now be accessed via http://localhost:8001.
414423

424+
## HAPorxy Load Balancer
425+
426+
HAProxy is provided as a load balancer that is configured to supported cookie-based session affinity and multi-statement transaction that is needed by some of the MarkLogic client application like MLCP.
427+
428+
### Enable HAProxy
429+
430+
The HAProxy Load Balancer is disabled by default. To enable the HAProxy, provide the following config in your values file for your Chart installation:
431+
```
432+
haproxy:
433+
enabled: true
434+
```
435+
436+
### Configuration
437+
438+
#### ConfigMap
439+
440+
The HAProxy configuation is dynamically generated in ConfigMap with the name of "marklogic-haproxy". You can provide your own configuation by creating a new ConfigMap and set the "existingConfigmap" in values file to the name of the new ConfigMap.
441+
442+
#### Modify Port Configuration
443+
444+
By default, port 8000, 8001, 8002 are configuered to handle HTTP traffic.
445+
446+
You can modify the default ports selection for the HAProxy by providing your own configuration in values file. Below is the default configuration:
447+
```
448+
haproxy:
449+
ports:
450+
- name: app-service
451+
type: HTTP
452+
port: 8000
453+
- name: admin
454+
type: HTTP
455+
port: 8001
456+
- name: manage
457+
type: HTTP
458+
port: 8002
459+
- name: odbc
460+
type: TCP
461+
port: 5432
462+
```
463+
You can remove/add/modify the entry in ports by providing your own entry. For each entry, you need to specify the name, type and port. There are three type backend supported when configuring the loadbalancer:
464+
1. HTTP: configure the backend as HTTP proxy that handles HTTPC traffic. It also configured to handle Cookie based session affinity and multi-statement trasaction from MarkLogic Client.
465+
2. TCP: configure the backend as TCP proxy that handles TCP traffic.
466+
467+
#### Automatic Reload Configuration
468+
469+
When any change to the current deployment happens(change the backend ports, number of running MarkLogic nodes), the HAProxy will be restarted to load the new configuration by default. You can change this behavior by change the setting below:
470+
```
471+
haproxy:
472+
restartWhenUpgrade:
473+
enabled: false
474+
```
475+
Note: if restartWhenUpgrade.enabled set to false, you have to manually delete the HAProxy deployment to have the lastest configuration if you update the number of MarkLogic node.
476+
477+
#### External Access
478+
479+
By default, the HAProxy is configured to provide access within the Kubernetes cluster. You can configure the HAProxy to provide external access by setting the service type in values file below:
480+
```
481+
haproxy:
482+
service:
483+
type: LoadBalancer
484+
```
485+
Warning: By setting the haproxy service type to loadbalancer MarkLogic endpoint is exposed to public Internet. Please set the networkPolicy to limit the sources that can visit MarkLogic.
486+
487+
### Access HAProxy
488+
489+
The HAProxy can be accessed from service with the name of <RELEASE_NAME>-haproxy. For example, if the release name is marklogic, then the name of the service will be marklogic-haproxy.
490+
491+
# Security
492+
415493
## Security Context
416494

417495
Security context defines privilege and access control settings for a Pod or Container. By default security context for containers is enabled with runAsUser, runAsNonRoot, allowPrivilegeEscalation settings. To configure these values for containers, set the containerSecurityContext in the values.yaml file or using the `--set` flag. Additional security context settings can be added to containerSecurityContext configuration. Please refer [https://kubernetes.io/docs/tasks/configure-pod-container/security-context/](https://kubernetes.io/docs/tasks/configure-pod-container/security-context/).

charts/Chart.yaml

Lines changed: 7 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,14 @@
11
annotations:
22
category: Database
33
apiVersion: v2
4+
dependencies:
5+
- condition: haproxy.enabled
6+
name: haproxy
7+
repository: "file://./charts/haproxy"
8+
version: "1.18.0"
49
name: marklogic
510
description: MarkLogic Server is a multi-model database that has both NoSQL and trusted enterprise data management capabilities.
6-
appVersion: "10.0-9.5"
11+
appVersion: "11.0.0"
712
type: application
813
keywords:
914
- marklogic
@@ -12,4 +17,4 @@ keywords:
1217
sources:
1318
- https://github.com/marklogic/marklogic-kubernetes
1419
- https://www.marklogic.com/
15-
version: 1.0.0-ea2
20+
version: 1.0.0

charts/charts/haproxy/.helmignore

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
# Patterns to ignore when building packages.
2+
# This supports shell glob matching, relative path matching, and
3+
# negation (prefixed with !). Only one pattern per line.
4+
.DS_Store
5+
# Common VCS dirs
6+
.git/
7+
.gitignore
8+
.bzr/
9+
.bzrignore
10+
.hg/
11+
.hgignore
12+
.svn/
13+
# Common backup files
14+
*.swp
15+
*.bak
16+
*.tmp
17+
*.orig
18+
*~
19+
# Various IDEs
20+
.project
21+
.idea/
22+
*.tmproj
23+
.vscode/

charts/charts/haproxy/Chart.yaml

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
# Copyright 2020 HAProxy Technologies LLC
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# http://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.
14+
15+
apiVersion: v2
16+
name: haproxy
17+
description: A Helm chart for HAProxy on Kubernetes
18+
type: application
19+
version: 1.18.0
20+
appVersion: 2.6.9
21+
kubeVersion: ">=1.17.0-0"
22+
keywords:
23+
- haproxy
24+
home: https://github.com/haproxytech/helm-charts/tree/main/haproxy
25+
sources:
26+
- http://www.haproxy.org/
27+
icon: https://raw.githubusercontent.com/haproxytech/helm-charts/main/haproxy/chart-icon.png
28+
maintainers:
29+
- name: Dinko Korunic
30+
31+
engine: gotpl
32+
annotations:
33+
artifacthub.io/changes: |
34+
- Update base image to HAProxy 2.6.9
35+
- Update HPA API version (#172)
36+
- Fix HPA spec (#173)
37+
- Ensure api version capabilities of PodDisruptionBudget (#175)
38+
- Add additionalPorts to service (#177)

0 commit comments

Comments
 (0)