Skip to content

Commit 4e76a35

Browse files
vasil-cfvasil-cf
committed
fix: README.md
1 parent b5397ef commit 4e76a35

File tree

2 files changed

+214
-182
lines changed

2 files changed

+214
-182
lines changed

‎codefresh/README.md

Lines changed: 107 additions & 90 deletions
Original file line numberDiff line numberDiff line change
@@ -24,7 +24,6 @@ Helm chart for deploying [Codefresh On-Premises](https://codefresh.io/docs/docs/
2424
- [Configuration with ALB (Application Load Balancer)](#configuration-with-alb-application-load-balancer)
2525
- [Configuration with Private Registry](#configuration-with-private-registry)
2626
- [Configuration with multi-role CF-API](#configuration-with-multi-role-cf-api)
27-
- [Indexes in MongoDB](#indexes-in-mongodb)
2827
- [High Availability](#high-availability)
2928
- [Mounting private CA certs](#mounting-private-ca-certs)
3029
- [Installing on OpenShift](#installing-on-openshift)
@@ -36,6 +35,7 @@ Helm chart for deploying [Codefresh On-Premises](https://codefresh.io/docs/docs/
3635
- [X-Frame-Options response header](#x-frame-options-response-header)
3736
- [Image digests in containers](#image-digests-in-containers)
3837
- [Configuring OIDC Provider](#configuring-oidc-provider)
38+
- [Maintaining MongoDB indexes](#maintaining-mongodb-indexes)
3939
- [Upgrading](#upgrading)
4040
- [To 2.0.0](#to-2-0-0)
4141
- [To 2.0.12](#to-2-0-12)
@@ -80,8 +80,6 @@ See [Use OCI-based registries](https://helm.sh/docs/topics/registries/)
8080

8181
**Important:** only helm 3.8.0+ is supported
8282

83-
**Important:** Read about [Indexes in MongoDB](#indexes-in-mongodb) before installation
84-
8583
Edit default `values.yaml` or create empty `cf-values.yaml`
8684

8785
- Pass `sa.json` (as a single line) to `.Values.imageCredentials.password`
@@ -94,7 +92,7 @@ imageCredentials:
9492
password: '{ "type": "service_account", "project_id": "codefresh-enterprise", "private_key_id": ... }'
9593
```
9694
97-
- Specify `.Values.global.appUrl`, `.Values.global.firebaseUrl` and `.Values.global.firebaseSecret`
95+
- Specify `.Values.global.appUrl`, `.Values.global.firebaseUrl`, `.Values.global.firebaseSecret`, `.Values.global.env.MONGOOSE_AUTO_INDEX`, `.Values.global.env.MONGO_AUTOMATIC_INDEX_CREATION`
9896

9997
```yaml
10098
global:
@@ -119,9 +117,9 @@ global:
119117
# name: my-secret
120118
# key: firebase-secret
121119
122-
# -- Enable auto-index creation in MongoDB
120+
# -- Enable index creation in MongoDB
123121
# This is required for first-time installations!
124-
# For upgrades, you should set it to `false`!
122+
# Before usage in Production, you must set it to `false` or remove it!
125123
env:
126124
MONGOOSE_AUTO_INDEX: "true"
127125
MONGO_AUTOMATIC_INDEX_CREATION: "true"
@@ -181,6 +179,17 @@ ingress-nginx:
181179
--timeout 15m
182180
```
183181

182+
### ⚠️ **MANDATORY** Post-Installation Action Items
183+
184+
Once your Codefresh On-Prem instance is installed, configured, and confirmed to be ready for production use, the following variables must be set to `false` or removed:
185+
186+
```yaml
187+
global:
188+
env:
189+
MONGOOSE_AUTO_INDEX: "false"
190+
MONGO_AUTOMATIC_INDEX_CREATION: "false"
191+
```
192+
184193
## Chart Configuration
185194

186195
See [Customizing the Chart Before Installing](https://helm.sh/docs/intro/using_helm/#customizing-the-chart-before-installing). To see all configurable options with detailed comments, visit the chart's [values.yaml](./values.yaml), or run these configuration commands:
@@ -785,64 +794,6 @@ cfapi-test-reporting:
785794
enabled: true
786795
```
787796
788-
⚠️ ⚠️ ⚠️
789-
### Indexes in MongoDB
790-
⚠️ ⚠️ ⚠️
791-
792-
Indexes in MongoDB are essential for efficient query performance, especially as your data grows. Without proper indexes, MongoDB must perform full collection scans to find matching documents, which can significantly slow down operations and increase resource usage. For production environments, ensuring that all frequently queried fields are indexed is vital to maintain optimal performance and scalability.
793-
794-
Auto-index creation in MongoDB is disabled by default in Codefresh On-Prem to prevent unexpected performance issues in production environments during upgrades. When enabled, MongoDB will automatically create indexes for fields used in queries, which can lead to high CPU and disk usage, increased I/O, and potential service disruptions—especially on large datasets. By requiring manual index management, administrators can plan index creation during maintenance windows, ensuring system stability and predictable performance before upgrading Codefresh On-Prem.
795-
796-
It is critical to ensure that your MongoDB indexes are always aligned with the latest recommended state for your Codefresh On-Prem version. Outdated or missing indexes can lead to degraded performance, slow queries, and increased resource consumption. Always review release notes and update or create indexes as specified during upgrades or when new collections/fields are introduced. Regularly auditing and maintaining your indexes helps ensure optimal system reliability and scalability.
797-
798-
The indexes list is located at the [codefresh-io/codefresh-onprem-helm](https://github.com/codefresh-io/codefresh-onprem-helm/tree/release-2.8/indexes) repository.
799-
The indexes are stored in JSON files with keys and options specified.
800-
801-
The directory structure is:
802-
803-
```console
804-
codefresh-onprem-helm
805-
├── indexes
806-
│ ├── <DB_NAME> # MongoDB database name
807-
│ │ ├── <COLLECTION_NAME>.json # MongoDB indexes for the specified collection
808-
```
809-
810-
#### Enabling auto-index creation
811-
812-
For first-time installations, you **must** enable auto-index creation by setting the following values:
813-
814-
```yaml
815-
global:
816-
env:
817-
MONGOOSE_AUTO_INDEX: "true"
818-
MONGO_AUTOMATIC_INDEX_CREATION: "true"
819-
```
820-
821-
You **should** disable it for the next upgrades by setting these variables to `false`:
822-
823-
```yaml
824-
global:
825-
env:
826-
MONGOOSE_AUTO_INDEX: "false"
827-
MONGO_AUTOMATIC_INDEX_CREATION: "false"
828-
```
829-
830-
#### Creating Indexes manually
831-
832-
> **Note!** If you have a large amount of MongoDB data, it is recommended to create indexes manually. Enabling auto-index creation can cause performance degradation during the index creation process with large datasets.
833-
834-
Depending on your MongoDB service (e.g., Atlas, self-hosted), you can create indexes using the MongoDB shell or the Atlas UI.
835-
836-
Ref:
837-
- [Create an Index in Atlas DB](https://www.mongodb.com/docs/atlas/atlas-ui/indexes/#create-an-index)
838-
- [Create an Index with mongosh](https://www.mongodb.com/docs/manual/reference/method/db.collection.createIndex/)
839-
840-
##### ⚠️ Upgrading when the indexes are maintained manually
841-
842-
If you maintain indexes manually and you upgrade your On-Prem installation you must create all indexes which were introduced from your current version up to the version you upgrade to manually.
843-
844-
You can follow [Upgrading section](#upgrading) to see what changes were made for indexes in every specific release.
845-
846797
### High Availability
847798
848799
The chart installs the non-HA version of Codefresh by default. If you want to run Codefresh in HA mode, use the example values below.
@@ -1492,6 +1443,86 @@ To see all the claims supported by Codefresh OIDC provider, see `claims_supporte
14921443

14931444
Use [obtain-oidc-id-token](https://github.com/codefresh-io/steps/blob/822afc0a9a128384e76459c6573628020a2cf404/incubating/obtain-oidc-id-token/step.yaml#L27-L58) and [aws-sts-assume-role-with-web-identity](https://github.com/codefresh-io/steps/blob/822afc0a9a128384e76459c6573628020a2cf404/incubating/aws-sts-assume-role-with-web-identity/step.yaml#L29-L63) steps to exchange the OIDC token (JWT) for a cloud access token.
14941445

1446+
## Maintaining MongoDB Indexes
1447+
1448+
Sometimes, in new releases of Codefresh On-Prem, index requirements change. When this happens, it's mentioned in the [Upgrading section](#upgrading) for the specific release.
1449+
1450+
> ℹ️ If you're upgrading from version `X` to version `Y`, and index requirements were updated in any of the intermediate versions, you only need to align your indexes with the index requirements of version `Y`. To do that, follow [Index alignment](#index-alignment) instructions.
1451+
1452+
### Index alignment
1453+
1454+
The required index definitions for each release can be found at the following resources:
1455+
1456+
- `2.6` <https://github.com/codefresh-io/codefresh-onprem-helm/tree/release-2.6/indexes>
1457+
- `2.7` <https://github.com/codefresh-io/codefresh-onprem-helm/tree/release-2.7/indexes>
1458+
- `2.8` <https://github.com/codefresh-io/codefresh-onprem-helm/tree/release-2.8/indexes>
1459+
1460+
The indexes are stored in JSON files with keys and options specified.
1461+
1462+
The directory structure is:
1463+
1464+
```console
1465+
indexes
1466+
├── <DB_NAME> # MongoDB database name
1467+
│ ├── <COLLECTION_NAME>.json # MongoDB indexes for the specified collection
1468+
```
1469+
1470+
**Overview of the index alignment process:**
1471+
1472+
1. Identify the differences between the indexes in your MongoDB instance and the required index definitions.
1473+
2. Create any missing indexes one by one. (It's important not to create them in bulk.)
1474+
3. Then remove any unnecessary indexes.
1475+
1476+
> ⚠️ **Note! Any changes to indexes should be performed during a defined maintenance window or during periods of lowest traffic to MongoDB.**
1477+
>
1478+
> Building indexes during time periods where the target collection is under heavy write load can result in reduced write performance and longer index builds. ([*Source: MongoDB official documentation*](https://www.mongodb.com/docs/v6.0/core/index-creation/#index-build-impact-on-database-performance))
1479+
>
1480+
> Even minor changes to indexes (e.g., index removal) can cause brief but noticeable performance degradationю ([*Source: MongoDB official documentation*](https://www.mongodb.com/docs/v6.0/core/query-plans/#plan-cache-flushes))
1481+
1482+
#### Self-hosted MongoDB
1483+
1484+
For self-hosted MongoDB, follow the instructions below:
1485+
1486+
- Connect to the MongoDB server using the [mongosh](https://www.mongodb.com/docs/mongodb-shell/install/) shell. Open your terminal or command prompt and run the following command, replacing `<connection_string>` with the appropriate MongoDB connection string for your server:
1487+
1488+
```shell
1489+
mongosh "<connection_string>"
1490+
```
1491+
1492+
- Retrieve the list of indexes for a specific collection:
1493+
1494+
```js
1495+
db.getSiblingDB('<db_name>').getCollection('<collection_name>').getIndexes()
1496+
```
1497+
1498+
- Compare your indexes with the required indexes for the target release, and adjust them by creating any missing indexes or removing any unnecessary ones
1499+
1500+
**Index creation**
1501+
1502+
> ⚠ **Note! Always create indexes sequentially, one by one. Don't create them in bulk.**
1503+
1504+
- To create an index, use the `createIndex()` method:
1505+
1506+
```js
1507+
db.getSiblingDB('<db_name>').getCollection('<collection_name>').createIndex(<key_object>, <options_object>)
1508+
```
1509+
1510+
After executing the `createIndex()` command, you should see a result indicating that the index was created successfully.
1511+
1512+
**Index removal**
1513+
1514+
- To remove an index, use the `dropIndex()` method with either the `<index_name>` or a `<key_object>`:
1515+
1516+
```js
1517+
db.getSiblingDB('<db_name>').getCollection('<collection_name>').dropIndex('<index_name>' | <key_object>)
1518+
```
1519+
1520+
#### Atlas Database
1521+
1522+
If you're hosting MongoDB on [Atlas](https://www.mongodb.com/atlas/database), use the following [Manage Indexes](https://www.mongodb.com/docs/atlas/atlas-ui/indexes/) guide to View, Create or Remove indexes.
1523+
1524+
> ⚠️ **Important!** In Atlas in production environment the indexes must be created in *a rolling fashion* mode, i.e., with `Build index via rolling process` checkbox enabled.
1525+
14951526
## Upgrading
14961527

14971528
### To 2-0-0
@@ -2027,24 +2058,22 @@ cfapi:
20272058

20282059
### To 2-6-0
20292060

2061+
> ⚠️ **WARNING! MongoDB indexes changed!**
2062+
>
2063+
> Please, follow [Maintaining MongoDB indexes](#maintaining-mongodb-indexes) guide to meet index requirements **BEFORE** the upgrade process.
2064+
20302065
### [What's new in 2.6.x](https://codefresh.io/docs/docs/whats-new/on-prem-release-notes/#on-premises-version-26)
20312066

20322067
#### Affected values
20332068

20342069
[Image digests in containers](#image-digests-in-containers)
20352070

2036-
#### Auto-index creation in MongoDB
2037-
2038-
[Auto-index creation in MongoDB](#auto-index-creation-in-mongodb)
2039-
2040-
#### ⚠️ New indexes in MongoDB
2041-
2042-
If you maintain indexes manually (i.e. [Auto-index creation](#enabling-auto-index-creation) is off) you must create the following index **before** the upgrade:
2043-
2044-
- [Database: `read-models`, collection: `images-binaries`, index: `accountId_1_imageName_1`](https://github.com/codefresh-io/codefresh-onprem-helm/tree/release-2.8/indexes/read-models/images-binaries.json#L75-L94)
2045-
20462071
### To 2-7-0
20472072

2073+
> ⚠️ **WARNING! MongoDB indexes changed!**
2074+
>
2075+
> Please, follow [Maintaining MongoDB indexes](#maintaining-mongodb-indexes) guide to meet index requirements **BEFORE** the upgrade process.
2076+
20482077
### [What's new in 2.7.x](https://codefresh.io/docs/docs/whats-new/on-prem-release-notes/#on-premises-version-27)
20492078

20502079
#### Affected values
@@ -2074,17 +2103,12 @@ global:
20742103
- "value"
20752104
```
20762105

2077-
#### ⚠️ New indexes in MongoDB
2078-
2079-
If you maintain indexes manually (i.e. [Auto-index creation](#enabling-auto-index-creation) is off) you must create the following indexes **before or right after** the upgrade:
2080-
2081-
> ⚠ **Note!** In case if you create indexes **before** the upgrade, please, create `codefresh.feature-store-versioned` collection manually in advance.
2082-
2083-
- [Database: `codefresh`, collection: `feature-store-versioned`, index: `createdAt_1`](https://github.com/codefresh-io/codefresh-onprem-helm/tree/release-2.8/indexes/codefresh/feature-store-versioned.json#L2-L9)
2084-
- [Database: `codefresh`, collection: `feature-store-versioned`, index: `LDRedisStoreVersion_1__id_-1`](https://github.com/codefresh-io/codefresh-onprem-helm/tree/release-2.8/indexes/codefresh/feature-store-versioned.json#L10-L17)
2085-
20862106
### To 2-8-0
20872107

2108+
> ⚠️ **WARNING! MongoDB indexes changed!**
2109+
>
2110+
> Please, follow [Maintaining MongoDB indexes](#maintaining-mongodb-indexes) guide to meet index requirements **BEFORE** the upgrade process.
2111+
20882112
### [What's new in 2.8.x](https://codefresh.io/docs/docs/whats-new/on-prem-release-notes/#on-premises-version-28)
20892113

20902114
### ⚠️ ⚠️ ⚠️ Breaking changes. Read before upgrading!
@@ -2123,13 +2147,6 @@ mongodb:
21232147
enabled: false
21242148
```
21252149

2126-
#### ⚠️ New indexes in MongoDB
2127-
2128-
If you maintain indexes manually (i.e. [Auto-index creation](#enabling-auto-index-creation) is off) you must create the following indexes **before** the upgrade:
2129-
2130-
- [Database: `codefresh`, collection: `users`, index: `account_1__id_1`](https://github.com/codefresh-io/codefresh-onprem-helm/tree/release-2.8/indexes/codefresh/users.json#L2-L9)
2131-
- [Database: `codefresh`, collection: `users`, index: `role_1_account_1__id_1`](https://github.com/codefresh-io/codefresh-onprem-helm/tree/release-2.8/indexes/codefresh/users.json#L10-L17)
2132-
21332150
### PostgreSQL update
21342151

21352152
Default PostgreSQL image is changed from 13.x to 17.x

0 commit comments

Comments
 (0)