Skip to content

Commit 7327627

Browse files
gavineldergavinelder
committed
docs: Move memory tuning to an advanced topic
1 parent 1f5b5c7 commit 7327627

File tree

37 files changed

+402
-328
lines changed

37 files changed

+402
-328
lines changed

‎platform-enterprise_docs/enterprise-sidebar.json‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -58,7 +58,8 @@
5858
"enterprise/advanced-topics/custom-launch-container",
5959
"enterprise/advanced-topics/firewall-configuration",
6060
"enterprise/advanced-topics/seqera-container-images",
61-
"enterprise/advanced-topics/content-security-policy"
61+
"enterprise/advanced-topics/content-security-policy",
62+
"enterprise/advanced-topics/jvm-memory-tuning"
6263
]
6364
},
6465
"enterprise/general_troubleshooting"

‎platform-enterprise_docs/enterprise/advanced-topics/jvm-memory-tuning.md‎

Lines changed: 65 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,65 @@
1+
---
2+
title: "JVM memory tuning"
3+
description: Configure JVM memory parameters for Seqera Platform Enterprise deployments
4+
tags: [configuration, jvm, memory, tuning]
5+
---
6+
7+
# JVM memory tuning
8+
9+
:::warning
10+
JVM memory tuning is an advanced topic that may cause instability and performance issues.
11+
:::
12+
13+
Seqera Platform scales memory allocation based on resources allocated to the application. To best inform available memory, set memory requests and limits on your deployments. We recommend increasing memory allocation before manually configuring JVM settings.
14+
15+
If you wish to manually configure JVM memory, use the following baseline recommendations.
16+
17+
## Memory parameters
18+
19+
Set JVM memory parameters using the `JAVA_OPTS` environment variable. The following parameters control memory allocation:
20+
21+
| Parameter | Description |
22+
| -------------------------- | ----------------------------------------------------------------------------------------------------------------------------------- |
23+
| `-Xms` / `-Xmx` | Set the initial (`Xms`) and maximum (`Xmx`) heap size. The heap stores Java objects and should be 50-70% of total allocated memory. |
24+
| `-XX:MaxDirectMemorySize` | Set the maximum direct (off-heap) memory. Used for NIO operations, network buffers, and file I/O. |
25+
| `-XX:ActiveProcessorCount` | Set the number of CPUs available to the JVM. Should match the number of vCPUs allocated to the container. |
26+
27+
## Resource allocation guidelines
28+
29+
- **Heap (`-Xmx`)**: 50-70% of total allocated memory
30+
- **Direct memory**: 10-20% of total allocated memory
31+
- **Overhead** (metaspace, thread stacks, native memory): ~10% of total allocated memory
32+
33+
Ensure total JVM memory (heap + direct memory + overhead) does not exceed container memory limits.
34+
35+
## Example configurations
36+
37+
The following table provides example configurations for common deployment sizes. These are starting points and may need to be tuned based on your specific usage patterns.
38+
39+
| vCPU | RAM | Heap (`-Xmx`) | Direct Memory | `JAVA_OPTS` |
40+
| :--: | :---: | :-----------: | :-----------: | ------------------------------------------------------------------------------- |
41+
| 1 | 2 GB | 1 GB | 512 MB | `-XX:ActiveProcessorCount=1 -Xms500M -Xmx1000M -XX:MaxDirectMemorySize=512m` |
42+
| 1 | 4 GB | 2.5 GB | 800 MB | `-XX:ActiveProcessorCount=1 -Xms1000M -Xmx2500M -XX:MaxDirectMemorySize=800m` |
43+
| 2 | 2 GB | 1 GB | 512 MB | `-XX:ActiveProcessorCount=2 -Xms500M -Xmx1000M -XX:MaxDirectMemorySize=512m` |
44+
| 2 | 4 GB | 2 GB | 800 MB | `-XX:ActiveProcessorCount=2 -Xms1000M -Xmx2000M -XX:MaxDirectMemorySize=800m` |
45+
| 2 | 8 GB | 5 GB | 1.5 GB | `-XX:ActiveProcessorCount=2 -Xms2000M -Xmx5000M -XX:MaxDirectMemorySize=1500m` |
46+
| 3 | 2 GB | 1 GB | 512 MB | `-XX:ActiveProcessorCount=3 -Xms500M -Xmx1000M -XX:MaxDirectMemorySize=512m` |
47+
| 3 | 4 GB | 2 GB | 800 MB | `-XX:ActiveProcessorCount=3 -Xms1000M -Xmx2000M -XX:MaxDirectMemorySize=800m` |
48+
| 3 | 8 GB | 5 GB | 1.5 GB | `-XX:ActiveProcessorCount=3 -Xms2000M -Xmx5000M -XX:MaxDirectMemorySize=1500m` |
49+
| 3 | 16 GB | 11 GB | 2.5 GB | `-XX:ActiveProcessorCount=3 -Xms4000M -Xmx11000M -XX:MaxDirectMemorySize=2500m` |
50+
51+
## When to adjust memory settings
52+
53+
Adjust your JVM memory settings if you observe the following issues in your deployment:
54+
55+
**Increase heap memory (`-Xmx`)** if you see:
56+
57+
- `OutOfMemoryError: Java heap space` errors in logs
58+
- Garbage collection pauses affecting performance
59+
- Steadily growing memory usage under sustained load
60+
61+
**Increase direct memory (`MaxDirectMemorySize`)** if you see:
62+
63+
- `OutOfMemoryError: Direct buffer memory` errors in logs
64+
- High concurrent workflow launch rates (more than 100 simultaneous workflows)
65+
- Large configuration payloads or extensive API usage

‎platform-enterprise_versioned_docs/version-24.1/enterprise/_templates/docker/tower.env‎

Lines changed: 0 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -4,9 +4,6 @@ TOWER_JWT_SECRET=<Replace This With A Secret String containing at least 35 chara
44
TOWER_CRYPTO_SECRETKEY=<Replace This With Another Secret String>
55
TOWER_LICENSE=<YOUR TOWER LICENSE KEY>
66

7-
# Configuration to optimize JVM memory settings. See https://docs.seqera.io/platform-enterprise/24.1/enterprise/configuration/overview#backend-memory-requirements
8-
JAVA_OPTS="-Xms1000M -Xmx2000M -XX:MaxDirectMemorySize=800m -Dio.netty.maxDirectMemory=0 -Djdk.nio.maxCachedBufferSize=262144"
9-
107
# Compute environment settings
118
TOWER_ENABLE_PLATFORMS=altair-platform,awsbatch-platform,azbatch-platform,eks-platform,gke-platform,googlebatch-platform,k8s-platform,lsf-platform,moab-platform,slurm-platform,uge-platform
129

‎platform-enterprise_versioned_docs/version-24.1/enterprise/_templates/k8s/tower-cron.yml‎

Lines changed: 0 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -43,9 +43,6 @@ spec:
4343
env:
4444
- name: MICRONAUT_ENVIRONMENTS
4545
value: "prod,redis,cron"
46-
# Configuration to optimize JVM memory settings. See https://docs.seqera.io/platform-enterprise/24.1/enterprise/configuration/overview#backend-memory-requirements
47-
- name: JAVA_OPTS
48-
value: "-Xms1000M -Xmx2000M -XX:MaxDirectMemorySize=800m -Dio.netty.maxDirectMemory=0 -Djdk.nio.maxCachedBufferSize=262144"
4946
ports:
5047
- containerPort: 8080
5148
resources:

‎platform-enterprise_versioned_docs/version-24.1/enterprise/_templates/k8s/tower-svc.yml‎

Lines changed: 0 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -36,9 +36,6 @@ spec:
3636
env:
3737
- name: MICRONAUT_ENVIRONMENTS
3838
value: "prod,redis,ha"
39-
# Configuration to optimize JVM memory settings. See https://docs.seqera.io/platform-enterprise/24.1/enterprise/configuration/overview#backend-memory-requirements
40-
- name: JAVA_OPTS
41-
value: "-Xms1000M -Xmx2000M -XX:MaxDirectMemorySize=800m -Dio.netty.maxDirectMemory=0 -Djdk.nio.maxCachedBufferSize=262144"
4239
# TLS certificate for Studios
4340
#- name: TOWER_OIDC_PEM_PATH
4441
# value: '/data/certs/oidc.pem'

‎platform-enterprise_versioned_docs/version-24.1/enterprise/advanced-topics/jvm-memory-tuning.md‎

Lines changed: 65 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,65 @@
1+
---
2+
title: "JVM memory tuning"
3+
description: Configure JVM memory parameters for Seqera Platform Enterprise deployments
4+
tags: [configuration, jvm, memory, tuning]
5+
---
6+
7+
# JVM memory tuning
8+
9+
:::warning
10+
JVM memory tuning is an advanced topic that may cause instability and performance issues.
11+
:::
12+
13+
Seqera Platform scales memory allocation based on resources allocated to the application. To best inform available memory, set memory requests and limits on your deployments. We recommend increasing memory allocation before manually configuring JVM settings.
14+
15+
If you wish to manually configure JVM memory, use the following baseline recommendations.
16+
17+
## Memory parameters
18+
19+
Set JVM memory parameters using the `JAVA_OPTS` environment variable. The following parameters control memory allocation:
20+
21+
| Parameter | Description |
22+
| -------------------------- | ----------------------------------------------------------------------------------------------------------------------------------- |
23+
| `-Xms` / `-Xmx` | Set the initial (`Xms`) and maximum (`Xmx`) heap size. The heap stores Java objects and should be 50-70% of total allocated memory. |
24+
| `-XX:MaxDirectMemorySize` | Set the maximum direct (off-heap) memory. Used for NIO operations, network buffers, and file I/O. |
25+
| `-XX:ActiveProcessorCount` | Set the number of CPUs available to the JVM. Should match the number of vCPUs allocated to the container. |
26+
27+
## Resource allocation guidelines
28+
29+
- **Heap (`-Xmx`)**: 50-70% of total allocated memory
30+
- **Direct memory**: 10-20% of total allocated memory
31+
- **Overhead** (metaspace, thread stacks, native memory): ~10% of total allocated memory
32+
33+
Ensure total JVM memory (heap + direct memory + overhead) does not exceed container memory limits.
34+
35+
## Example configurations
36+
37+
The following table provides example configurations for common deployment sizes. These are starting points and may need to be tuned based on your specific usage patterns.
38+
39+
| vCPU | RAM | Heap (`-Xmx`) | Direct Memory | `JAVA_OPTS` |
40+
| :--: | :---: | :-----------: | :-----------: | ------------------------------------------------------------------------------- |
41+
| 1 | 2 GB | 1 GB | 512 MB | `-XX:ActiveProcessorCount=1 -Xms500M -Xmx1000M -XX:MaxDirectMemorySize=512m` |
42+
| 1 | 4 GB | 2.5 GB | 800 MB | `-XX:ActiveProcessorCount=1 -Xms1000M -Xmx2500M -XX:MaxDirectMemorySize=800m` |
43+
| 2 | 2 GB | 1 GB | 512 MB | `-XX:ActiveProcessorCount=2 -Xms500M -Xmx1000M -XX:MaxDirectMemorySize=512m` |
44+
| 2 | 4 GB | 2 GB | 800 MB | `-XX:ActiveProcessorCount=2 -Xms1000M -Xmx2000M -XX:MaxDirectMemorySize=800m` |
45+
| 2 | 8 GB | 5 GB | 1.5 GB | `-XX:ActiveProcessorCount=2 -Xms2000M -Xmx5000M -XX:MaxDirectMemorySize=1500m` |
46+
| 3 | 2 GB | 1 GB | 512 MB | `-XX:ActiveProcessorCount=3 -Xms500M -Xmx1000M -XX:MaxDirectMemorySize=512m` |
47+
| 3 | 4 GB | 2 GB | 800 MB | `-XX:ActiveProcessorCount=3 -Xms1000M -Xmx2000M -XX:MaxDirectMemorySize=800m` |
48+
| 3 | 8 GB | 5 GB | 1.5 GB | `-XX:ActiveProcessorCount=3 -Xms2000M -Xmx5000M -XX:MaxDirectMemorySize=1500m` |
49+
| 3 | 16 GB | 11 GB | 2.5 GB | `-XX:ActiveProcessorCount=3 -Xms4000M -Xmx11000M -XX:MaxDirectMemorySize=2500m` |
50+
51+
## When to adjust memory settings
52+
53+
Adjust your JVM memory settings if you observe the following issues in your deployment:
54+
55+
**Increase heap memory (`-Xmx`)** if you see:
56+
57+
- `OutOfMemoryError: Java heap space` errors in logs
58+
- Garbage collection pauses affecting performance
59+
- Steadily growing memory usage under sustained load
60+
61+
**Increase direct memory (`MaxDirectMemorySize`)** if you see:
62+
63+
- `OutOfMemoryError: Direct buffer memory` errors in logs
64+
- High concurrent workflow launch rates (more than 100 simultaneous workflows)
65+
- Large configuration payloads or extensive API usage

‎platform-enterprise_versioned_docs/version-24.1/enterprise/configuration/overview.mdx‎

Lines changed: 0 additions & 44 deletions
Original file line numberDiff line numberDiff line change
@@ -411,50 +411,6 @@ services:
411411
These default memory allocation limits are included in your Kubernetes manifests ([tower-svc.yml](../_templates/k8s/tower-svc.yml) and [tower-cron.yml](../_templates/k8s/tower-cron.yml)) and Docker Compose ([docker-compose.yml](../_templates/docker/docker-compose.yml)) configuration templates.
412412
:::
413413

414-
### JVM memory tuning
415-
416-
For production deployments, configure JVM memory parameters via the `JAVA_OPTS` environment variable. The following baseline configuration is suitable for most deployments:
417-
418-
```bash
419-
JAVA_OPTS: -Xms1000M -Xmx2000M -XX:MaxDirectMemorySize=800m -Dio.netty.maxDirectMemory=0 -Djdk.nio.maxCachedBufferSize=262144
420-
```
421-
422-
:::note
423-
These default JVM memory settings are included in the configuration templates provided in these docs:
424-
- Kubernetes: [tower-svc.yml](../_templates/k8s/tower-svc.yml) and [tower-cron.yml](../_templates/k8s/tower-cron.yml)
425-
- Docker Compose: [tower.env](../_templates/docker/tower.env)
426-
:::
427-
428-
**Parameter descriptions:**
429-
- **Heap memory** (`-Xms`/`-Xmx`): Memory pool for Java objects. Set initial (`Xms`) and maximum (`Xmx`) heap size.
430-
- **Direct memory** (`MaxDirectMemorySize`): Off-heap memory used for NIO operations, network buffers, and file I/O. Critical for handling concurrent workflow API operations.
431-
- **Netty memory accounting** (`io.netty.maxDirectMemory=0`): Disables Netty's internal tracking; relies on JVM direct memory limits instead.
432-
- **Buffer caching** (`jdk.nio.maxCachedBufferSize`): Limits size of cached NIO buffers to prevent excessive memory retention.
433-
434-
**When to adjust these values:**
435-
436-
Increase `MaxDirectMemorySize` if you observe:
437-
- `OutOfMemoryError: Direct buffer memory` in logs
438-
- High concurrent workflow launch rates (>100 simultaneous workflows)
439-
- Large configuration payloads or extensive API usage
440-
441-
Increase heap memory (`-Xmx`) if you observe:
442-
- `OutOfMemoryError: Java heap space` in logs
443-
- Garbage collection pauses affecting performance
444-
- Growing memory usage under sustained load
445-
446-
**Example: High-concurrency deployment**
447-
For deployments running 200+ concurrent workflows:
448-
```bash
449-
JAVA_OPTS: -Xms1000M -Xmx3000M -XX:MaxDirectMemorySize=1600m -Dio.netty.maxDirectMemory=0 -Djdk.nio.maxCachedBufferSize=262144
450-
```
451-
452-
Ensure container/pod memory limits are set higher than JVM limits to accommodate non-heap memory usage.
453-
454-
:::warning
455-
These are starting recommendations. Monitor your deployment's actual memory usage and adjust based on your specific workload patterns. Undersized memory allocation can cause OOM failures and service instability.
456-
:::
457-
458414
## Compute environments
459415

460416
Configuration values to enable computing platforms and customize Batch Forge resource naming.

‎platform-enterprise_versioned_docs/version-24.1/enterprise/upgrade.md‎

Lines changed: 0 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -15,17 +15,6 @@ The database volume is persistent on the local machine by default if you use the
1515
1. Download the latest versions of your deployment templates and update your Seqera container versions:
1616
- [docker-compose.yml](./_templates/docker/docker-compose.yml) for Docker Compose deployments
1717
- [tower-cron.yml](./_templates/k8s/tower-cron.yml) and [tower-svc.yml](./_templates/k8s/tower-svc.yml) for Kubernetes deployments
18-
1. **JVM memory configuration defaults (recommended)**: The following `JAVA_OPTS` environment variable is included in the deployment templates downloaded in the preceding step, to optimize JVM memory settings:
19-
20-
```bash
21-
JAVA_OPTS: -Xms1000M -Xmx2000M -XX:MaxDirectMemorySize=800m -Dio.netty.maxDirectMemory=0 -Djdk.nio.maxCachedBufferSize=262144
22-
```
23-
24-
These baseline values are suitable for most deployments running moderate concurrent workflow loads.
25-
26-
:::tip
27-
These are starting recommendations that may require tuning based on your deployment's workload. See [Backend memory requirements](./configuration/overview.mdx#backend-memory-requirements) for detailed guidance on when and how to adjust these values for your environment.
28-
:::
2918
1. Restart the application.
3019
1. If you're using a containerized database as part of your implementation:
3120
1. Stop the application.

‎platform-enterprise_versioned_docs/version-24.2/enterprise/_templates/docker/tower.env‎

Lines changed: 0 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -4,9 +4,6 @@ TOWER_JWT_SECRET=<Replace This With A Secret String containing at least 35 chara
44
TOWER_CRYPTO_SECRETKEY=<Replace This With Another Secret String>
55
TOWER_LICENSE=<YOUR TOWER LICENSE KEY>
66

7-
# Configuration to optimize JVM memory settings. See https://docs.seqera.io/platform-enterprise/24.2/enterprise/configuration/overview#backend-memory-requirements
8-
JAVA_OPTS="-Xms1000M -Xmx2000M -XX:MaxDirectMemorySize=800m -Dio.netty.maxDirectMemory=0 -Djdk.nio.maxCachedBufferSize=262144"
9-
107
# Compute environment settings
118
TOWER_ENABLE_PLATFORMS=altair-platform,awsbatch-platform,azbatch-platform,eks-platform,gke-platform,googlebatch-platform,k8s-platform,lsf-platform,moab-platform,slurm-platform,uge-platform
129

‎platform-enterprise_versioned_docs/version-24.2/enterprise/_templates/k8s/tower-cron.yml‎

Lines changed: 0 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -43,9 +43,6 @@ spec:
4343
env:
4444
- name: MICRONAUT_ENVIRONMENTS
4545
value: "prod,redis,cron"
46-
# Configuration to optimize JVM memory settings. See https://docs.seqera.io/platform-enterprise/24.2/enterprise/configuration/overview#backend-memory-requirements
47-
- name: JAVA_OPTS
48-
value: "-Xms1000M -Xmx2000M -XX:MaxDirectMemorySize=800m -Dio.netty.maxDirectMemory=0 -Djdk.nio.maxCachedBufferSize=262144"
4946
ports:
5047
- containerPort: 8080
5148
resources:

0 commit comments

Comments
 (0)