Skip to content

Commit 2eb939b

Browse files
Update README.md (#1354)
* Update README.md Dataproc clusters now have [cluster-level syslog](https://cloud.google.com/dataproc/docs/guides/logging#cluster-logs) collection enabled by default. To avoid log duplication, you should use opsagent_nosyslog.sh to prevent the Ops Agent from also collecting syslogs. * Update README.md * Update README.md * Update README.md * Update README.md
1 parent 84c2cba commit 2eb939b

File tree

1 file changed

+8
-2
lines changed

1 file changed

+8
-2
lines changed

opsagent/README.md

Lines changed: 8 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,11 @@ We provide two variants of this initialization action:
99
- `opsagent.sh` installs the Ops Agent. [By default](https://cloud.google.com/stackdriver/docs/solutions/agents/ops-agent/configuration#default), it collects syslogs and system (node) metrics.
1010
- `opsagent_nosyslog.sh` installs the Ops Agent and also specifies a user configuration in order to skip syslogs collection from your cluster nodes. If the user configuration is not specified, Ops Agent will collect syslogs besides the system (node) metrics. You can further customize this configuration to collect logs and metrics from other third-party applications.
1111

12-
If you are looking to match the behavior of Dataproc image versions up to 2.1 with `--metric-sources=monitoring-agent-defaults`, which did not ingest syslogs from Dataproc cluster nodes, please use `opsagent_nosyslog.sh`.
12+
⚠️ **Dataproc clusters now have cluster-level syslog collection enabled by default.** Starting **August 18, 2025,** new Dataproc clusters will have the property `dataproc.logging.syslog.enabled` set to `true`. This new default behavior can lead to log duplication if the Ops Agent is also configured to collect syslogs.
13+
14+
To prevent duplicate logs, we recommend using `opsagent_nosyslog.sh`. If you need to disable cluster-level syslog collection entirely, you can set the `dataproc.logging.syslog.enabled` property to `false` during cluster creation. For more details, refer to the [Dataproc Release Notes](https://cloud.google.com/dataproc/docs/release-notes#July_15_2025) and [Dataproc Logs documentation](https://cloud.google.com/dataproc/docs/guides/logging#cluster-logs).
15+
16+
If you are looking to match the behavior of Dataproc image versions up to 2.1 with `--metric-sources=monitoring-agent-defaults` without ingesting syslogs, please use `opsagent_nosyslog.sh` and additionally set the `dataproc.logging.syslog.enabled` property to `false` during cluster creation.
1317

1418
## Using this initialization action
1519

@@ -25,10 +29,12 @@ CLUSTER_NAME=<cluster_name>
2529
gcloud dataproc clusters create ${CLUSTER_NAME} \
2630
--image-version=2.2 \
2731
--region=${REGION} \
32+
--properties dataproc:dataproc.logging.syslog.enabled=false \
2833
--initialization-actions=gs://goog-dataproc-initialization-actions-${REGION}/opsagent/opsagent_nosyslog.sh
2934
```
3035

31-
## Install the Ops Agent with default configuration
36+
## Install the Ops Agent with default configuration (Not Recommended )
37+
This approach is not recommended from August 18, 2025 as the cluster-level syslog collection is enabled by default for newly created clusters.
3238

3339
```bash
3440
REGION=<region>

0 commit comments

Comments
 (0)