Skip to content

[observability] introduce prometheus-push mode #2317

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
wuchong merged 10 commits into from
Jan 29, 2026
Merged

[observability] introduce prometheus-push mode #2317

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
5ccc85d
[observability] introduce prometheus-push mode
zuston Jan 6, 2026
bc2a66e
adjust default options
zuston Jan 6, 2026
52e468f
fix part-1
Jan 28, 2026
f205db1
fix part-2
Jan 28, 2026
9efcd73
update options
Jan 28, 2026
157a65c
add docks
Jan 28, 2026
f754f3d
add doc for test
Jan 28, 2026
2d5c936
fix again
Jan 28, 2026
16c945f
add dep
Jan 28, 2026
27f0790
remove simpleclient_hotspot dependency
wuchong Jan 28, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
51 changes: 51 additions & 0 deletions fluss-common/src/main/java/org/apache/fluss/config/ConfigOptions.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1773,6 +1773,57 @@ public class ConfigOptions {
+ "the CoordinatorServer) it is advisable to use a port range "
+ "like 9250-9260.");

// ------------------------------------------------------------------------
// ConfigOptions for prometheus push gateway reporter
// ------------------------------------------------------------------------
public static final ConfigOption<String> METRICS_REPORTER_PROMETHEUS_PUSHGATEWAY_HOST_URL =
key("metrics.reporter.prometheus-pushgateway.hostUrl")
Copy link
Member

@wuchong wuchong Jan 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The configuration key is not aligned with Fluss format, I suggest to change them into

metrics.reporter.prometheus-push.url
metrics.reporter.prometheus-push.job-name
metrics.reporter.prometheus-push.random-job-name-suffix
metrics.reporter.prometheus-push.delete-on-shutdown
metrics.reporter.prometheus-push.filter-label-value-characters
metrics.reporter.prometheus-push.grouping-key

Use prometheus-push in the key as it is the identifier of metric.reporters=prometheus-push

.stringType()
.noDefaultValue()
.withDescription("The URL of the Prometheus PushGateway to push metrics to.");

public static final ConfigOption<String> METRICS_REPORTER_PROMETHEUS_PUSHGATEWAY_JOB_NAME =
key("metrics.reporter.prometheus-pushgateway.jobName")
.stringType()
.noDefaultValue()
.withDescription(
"The job name under which to push metrics to Prometheus PushGateway.");

public static final ConfigOption<Boolean>
METRICS_REPORTER_PROMETHEUS_PUSHGATEWAY_RANDOM_JOB_NAME_SUFFIX =
key("metrics.reporter.prometheus-pushgateway.randomJobNameSuffix")
.booleanType()
.defaultValue(false)
Copy link
Member

@wuchong wuchong Jan 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not true like Flink?

.withDescription(
"Whether to append a random suffix to the job name. "
+ "This is useful when multiple instances of the reporter "
+ "are running on the same host.");

public static final ConfigOption<Boolean>
METRICS_REPORTER_PROMETHEUS_PUSHGATEWAY_DELETE_ON_SHUTDOWN =
key("metrics.reporter.prometheus-pushgateway.deleteOnShutdown")
.booleanType()
.defaultValue(false)
.withDescription(
"Whether to delete metrics from PushGateway on shutdown.");
Copy link
Member

@wuchong wuchong Jan 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Flink sets the config to true by default, why do we use false?

Besides, maybe we can add a note like Flink for this config? Fluss will try its best to delete the metrics but this is not guaranteed.


public static final ConfigOption<Boolean>
METRICS_REPORTER_PROMETHEUS_PUSHGATEWAY_FILTER_LABEL_VALUE_CHARACTERS =
key("metrics.reporter.prometheus-pushgateway.filterLabelValueCharacters")
.booleanType()
.defaultValue(true)
.withDescription(
"Whether to filter characters in label values to conform to the "
+ "Prometheus character set ([a-zA-Z0-9:_]).");

public static final ConfigOption<String> METRICS_REPORTER_PROMETHEUS_PUSHGATEWAY_GROUPING_KEY =
key("metrics.reporter.prometheus-pushgateway.groupingKey")
.stringType()
.noDefaultValue()
.withDescription(
"The grouping key to use when pushing metrics to Prometheus PushGateway. "
+ "The format should be k1=v1;k2=v2.");
Copy link
Member

@wuchong wuchong Jan 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you update all the option description like Flink?

https://nightlies.apache.org/flink/flink-docs-master/docs/deployment/metric_reporters/#prometheuspushgateway


// ------------------------------------------------------------------------
// ConfigOptions for jmx reporter
// ------------------------------------------------------------------------
Expand Down
12 changes: 12 additions & 0 deletions fluss-metrics/fluss-metrics-prometheus/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,18 @@
<version>${prometheus.version}</version>
</dependency>

<dependency>
<groupId>io.prometheus</groupId>
<artifactId>simpleclient_pushgateway</artifactId>
<version>${prometheus.version}</version>
</dependency>

<dependency>
<groupId>io.prometheus</groupId>
<artifactId>simpleclient_hotspot</artifactId>
<version>${prometheus.version}</version>
</dependency>

<!-- test dependencies -->
<dependency>
<groupId>org.apache.fluss</groupId>
Expand Down
Loading
Loading