diff --git a/modules/manage/pages/cluster-maintenance/configure-client-connections.adoc b/modules/manage/pages/cluster-maintenance/configure-client-connections.adoc index d762ec17f..56a148202 100644 --- a/modules/manage/pages/cluster-maintenance/configure-client-connections.adoc +++ b/modules/manage/pages/cluster-maintenance/configure-client-connections.adoc @@ -6,17 +6,33 @@ Optimize the availability of your clusters by configuring and tuning properties. -// Don't display ListKafkaConnections in Cloud docs until support is added ifdef::env-cloud[] -TIP: Before you configure connection limits or reconnection settings, start by gathering detailed data about your client connections. Use the xref:reference:public-metrics-reference.adoc#redpanda_rpc_active_connections[`redpanda_rpc_active_connections` metric] to view current Kafka client connections. +[TIP] +==== +Before you configure connection limits or reconnection settings, start by gathering detailed data about your client connections. + +* Use the xref:reference:public-metrics-reference.adoc#redpanda_rpc_active_connections[`redpanda_rpc_active_connections` metric] to view current Kafka client connections. +* For clusters on v25.3 and later, use xref:reference:rpk/rpk-cluster/rpk-cluster-connections-list.adoc[`rpk cluster connections list`] or the `GET /v1/monitoring/kafka/connections` endpoint in the Data Plane API to identify: ++ +-- +** Which clients and applications are connected +** Long-lived connections and long-running requests +** Connections with no activity +** Whether any clients are causing excessive load +-- ++ +By reviewing connection details, you can make informed decisions about tuning connection limits and troubleshooting issues. + +See also: link:/api/doc/cloud-dataplane/operation/operation-monitoringservice_listkafkaconnections[Data Plane API reference], xref:manage:monitor-cloud.adoc#throughput[Monitor Redpanda Cloud] +==== endif::[] ifndef::env-cloud[] [TIP] ==== Before you configure connection limits or reconnection settings, start by gathering detailed data about your client connections. -* Internal metrics that follow the `vectorized_kafka_rpc_.\*connect.*` naming pattern provide details on Kafka client connection activity. For example, xref:reference:internal-metrics-reference.adoc#vectorized_kafka_rpc_active_connections[`vectorized_kafka_rpc_active_connections`] reports the current number of active connections. -* For Redpanda v25.3 and later, use `rpk cluster connections list` or the Admin API ListKafkaConnections endpoint to identify: +* Internal metrics that follow the `vectorized_kafka_rpc_.\*connect*` naming pattern provide details on Kafka client connection activity. For example, xref:reference:internal-metrics-reference.adoc#vectorized_kafka_rpc_active_connections[`vectorized_kafka_rpc_active_connections`] reports the current number of active connections. +* For Redpanda v25.3 and later, use xref:reference:rpk/rpk-cluster/rpk-cluster-connections-list.adoc[`rpk cluster connections list`] or the Admin API ListKafkaConnections endpoint to identify: + -- ** Which clients and applications are connected diff --git a/modules/manage/pages/cluster-maintenance/manage-throughput.adoc b/modules/manage/pages/cluster-maintenance/manage-throughput.adoc index 9a8519c74..dab8e1837 100644 --- a/modules/manage/pages/cluster-maintenance/manage-throughput.adoc +++ b/modules/manage/pages/cluster-maintenance/manage-throughput.adoc @@ -1,9 +1,16 @@ = Manage Throughput :description: Learn how to manage the throughput of Kafka traffic. :page-categories: Management, Networking + // tag::single-source[] -ifdef::env-cloud[:monitor-doc: manage:monitor-cloud.adoc#throughput] -ifndef::env-cloud[:monitor-doc: manage:monitoring.adoc#throughput] +ifdef::env-cloud[] +:monitor-doc: manage:monitor-cloud.adoc#throughput +:connected-clients-api-doc-ref: link:/api/doc/cloud-dataplane/operation/operation-monitoringservice_listkafkaconnections +endif::[] +ifndef::env-cloud[] +:monitor-doc: manage:monitoring.adoc#throughput +:connected-clients-api-doc-ref: link:/api/doc/admin/v2/operation/operation-redpanda-core-admin-v2-clusterservice-listkafkaconnections +endif::[] Redpanda supports throughput throttling on both ingress and egress independently, and allows configuration at the broker and client levels. This helps prevent clients from causing unbounded network and disk usage on brokers. You can configure limits at two levels: @@ -21,7 +28,13 @@ endif::[] You may find it helpful to check the xref:{monitor-doc}[current produce and consume throughput] of a client before you configure throughput quotas. -Use the `rpk cluster connections list` command or the link:/api/doc/admin/v2/operation/operation-redpanda-core-admin-v2-clusterservice-listkafkaconnections[ListKafkaConnections] Admin API endpoint to view detailed information about active Kafka client connections. +ifndef::env-cloud[] +Use the xref:reference:rpk/rpk-cluster/rpk-cluster-connections-list.adoc[`rpk cluster connections list`] command or the {connected-clients-api-doc-ref}[ListKafkaConnections] Admin API endpoint to view detailed information about active Kafka client connections. +endif::[] + +ifdef::env-cloud[] +Use the xref:reference:rpk/rpk-cluster/rpk-cluster-connections-list.adoc[`rpk cluster connections list`] command or the {connected-clients-api-doc-ref}[`GET /v1/monitoring/kafka/connections`] Data Plane API endpoint to view detailed information about active Kafka client connections. +endif::[] For example, to view a cluster's connected clients in order of highest current produce throughput, run: @@ -50,6 +63,7 @@ b41584f3-2662-4185-a4b8-0d8510f5c780 OPEN UNAUTHENTICATED perf-producer-clie ---- -- +ifndef::env-cloud[] curl:: + -- @@ -67,7 +81,7 @@ curl -s -X POST \ .Show example API response [%collapsible] ==== -[,json,role=no-copy,lines=54] +[,json,role=no-copy,lines=55] ---- { "connections": [ @@ -135,6 +149,95 @@ curl -s -X POST \ ---- ==== -- +endif::[] + +ifdef::env-cloud[] +Data Plane API:: ++ +-- +[,bash] +---- +curl \ + --request GET 'https:///v1/monitoring/kafka/connections' \ + --header "Authorization: Bearer $ACCESS_TOKEN" \ + --data '{ + "filter": "", + "order_by": "recent_request_statistics.produce_bytes desc" + }' +---- + +.Show example API response +[%collapsible] +==== +[,json,role=no-copy,lines=54] +---- +{ + "connections": [ + { + "node_id": 0, + "shard_id": 0, + "uid": "b20601a3-624c-4a8c-ab88-717643f01d56", + "state": "KAFKA_CONNECTION_STATE_OPEN", + "open_time": "2025-10-15T14:15:15.755065000Z", + "close_time": "1970-01-01T00:00:00.000000000Z", + "authentication_info": { + "state": "AUTHENTICATION_STATE_UNAUTHENTICATED", + "mechanism": "AUTHENTICATION_MECHANISM_UNSPECIFIED", + "user_principal": "" + }, + "listener_name": "", + "tls_info": { + "enabled": false + }, + "source": { + "ip_address": "127.0.0.1", + "port": 55012 + }, + "client_id": "perf-producer-client", + "client_software_name": "apache-kafka-java", + "client_software_version": "3.9.0", + "transactional_id": "my-tx-id", + "group_id": "", + "group_instance_id": "", + "group_member_id": "", + "api_versions": { + "18": 4, + "22": 3, + "3": 12, + "24": 3, + "0": 7 + }, + "idle_duration": "0s", + "in_flight_requests": { + "sampled_in_flight_requests": [ + { + "api_key": 0, + "in_flight_duration": "0.000406892s" + } + ], + "has_more_requests": false + }, + "total_request_statistics": { + "produce_bytes": "78927173", + "fetch_bytes": "0", + "request_count": "4853", + "produce_batch_count": "4849" + }, + "recent_request_statistics": { + "produce_bytes": "78927173", + "fetch_bytes": "0", + "request_count": "4853", + "produce_batch_count": "4849" + } + }, + ... + ], + "total_size": "9" +} +---- +==== +-- +endif::[] ====== To view connections for a specific client, you can use a filter expression: @@ -157,6 +260,7 @@ b20601a3-624c-4a8c-ab88-717643f01d56 OPEN UNAUTHENTICATED perf-producer-clie ---- -- +ifndef::env-cloud[] curl:: + -- @@ -234,6 +338,87 @@ curl -s -X POST \ ---- ==== -- +endif::[] + +ifdef::env-cloud[] +Data Plane API:: ++ +-- +[,bash] +---- +curl \ + --request GET 'https:///v1/monitoring/kafka/connections' \ + --header "Authorization: Bearer $ACCESS_TOKEN" \ + --data '{ + "filter": "client_id = \"perf-producer-client\"" + }' +---- + +.Show example API response +[%collapsible] +==== +[,json,lines=23] +---- +{ + "connections": [ + { + "node_id": 0, + "shard_id": 0, + "uid": "b41584f3-2662-4185-a4b8-0d8510f5c780", + "state": "KAFKA_CONNECTION_STATE_OPEN", + "open_time": "2025-10-15T14:15:15.219538000Z", + "close_time": "1970-01-01T00:00:00.000000000Z", + "authentication_info": { + "state": "AUTHENTICATION_STATE_UNAUTHENTICATED", + "mechanism": "AUTHENTICATION_MECHANISM_UNSPECIFIED", + "user_principal": "" + }, + "listener_name": "", + "tls_info": { + "enabled": false + }, + "source": { + "ip_address": "127.0.0.1", + "port": 55002 + }, + "client_id": "perf-producer-client", + "client_software_name": "apache-kafka-java", + "client_software_version": "3.9.0", + "transactional_id": "", + "group_id": "", + "group_instance_id": "", + "group_member_id": "", + "api_versions": { + "18": 4, + "3": 12, + "10": 4 + }, + "idle_duration": "7.743592270s", + "in_flight_requests": { + "sampled_in_flight_requests": [], + "has_more_requests": false + }, + "total_request_statistics": { + "produce_bytes": "0", + "fetch_bytes": "0", + "request_count": "3", + "produce_batch_count": "0" + }, + "recent_request_statistics": { + "produce_bytes": "0", + "fetch_bytes": "0", + "request_count": "3", + "produce_batch_count": "0" + } + }, + ... + ], + "total_size": "2" +} +---- +==== +-- +endif::[] ====== diff --git a/modules/manage/partials/monitor-health.adoc b/modules/manage/partials/monitor-health.adoc index 6ffc1d5c5..d6bab19bf 100644 --- a/modules/manage/partials/monitor-health.adoc +++ b/modules/manage/partials/monitor-health.adoc @@ -1,6 +1,13 @@ == Monitor for performance and health // tag::single-source[] +ifndef::env-cloud[] +:connected-clients-api-doc-ref: link:/api/doc/admin/v2/operation/operation-redpanda-core-admin-v2-clusterservice-listkafkaconnections +endif::[] +ifdef::env-cloud[] +:connected-clients-api-doc-ref: link:/api/doc/cloud-dataplane/operation/operation-monitoringservice_listkafkaconnections +endif::[] + This section provides guidelines and example queries using Redpanda's public metrics to optimize your system's performance and monitor its health. To help detect and mitigate anomalous system behaviors, capture baseline metrics of your healthy system at different stages (at start-up, under high load, in steady state) so you can set thresholds and alerts according to those baselines. @@ -125,11 +132,15 @@ For the consume rate, create a query to get the total consume rate across all to rate(redpanda_rpc_sent_bytes{redpanda_server="kafka"}[$__rate_interval]) ---- -// Don't display ListKafkaConnections in Cloud docs until support is added -ifndef::env-cloud[] + ==== Identify high-throughput clients -Use `rpk cluster connections list` or the ListKafkaConnections service in Admin API to identify which client connections are driving the majority of, or the change in, the produce or consume throughput for a cluster. +ifndef::env-cloud[] +Use xref:reference:rpk/rpk-cluster/rpk-cluster-connections-list.adoc[`rpk cluster connections list`] or the {connected-clients-api-doc-ref}[ListKafkaConnections] endpoint in Admin API to identify which client connections are driving the majority of, or the change in, the produce or consume throughput for a cluster. +endif::[] +ifdef::env-cloud[] +Use xref:reference:rpk/rpk-cluster/rpk-cluster-connections-list.adoc[`rpk cluster connections list`] or the {connected-clients-api-doc-ref}[`GET /v1/monitoring/kafka/connections`] endpoint in the Data Plane API to identify which client connections are driving the majority of, or the change in, the produce or consume throughput for a cluster. +endif::[] For example, to list connections with a current produce throughput greater than 1MB, run: @@ -150,6 +161,7 @@ b20601a3-624c-4a8c-ab88-717643f01d56 OPEN UNAUTHENTICATED perf-producer-clie ---- -- +ifndef::env-cloud[] curl:: + -- @@ -161,7 +173,7 @@ curl -s -X POST "localhost:9644/redpanda.core.admin.v2.ClusterService/ListKafkaC .Show example API response [%collapsible] ==== -[,bash,role=no-copy,lines=54] +[,bash,role=no-copy,lines=55] ---- { "connections": [ @@ -227,10 +239,93 @@ curl -s -X POST "localhost:9644/redpanda.core.admin.v2.ClusterService/ListKafkaC ---- ==== -- -====== +endif::[] -You can adjust the filter and sorting criteria as necessary. See the link:/api/doc/admin/v2/operation/operation-redpanda-core-admin-v2-clusterservice-listkafkaconnections[Admin API reference] for more details. +ifdef::env-cloud[] +Data Plane API:: ++ +-- +[,bash] +---- +curl \ + --request GET 'https:///v1/monitoring/kafka/connections' \ + --header "Authorization: Bearer $ACCESS_TOKEN" \ + --data '{"filter":"recent_request_statistics.produce_bytes > 1000000", "order_by":"recent_request_statistics.produce_bytes desc"}' | jq +---- + +.Show example API response +[%collapsible] +==== +[,bash,role=no-copy,lines=54] +---- +{ + "connections": [ + { + "node_id": 0, + "shard_id": 0, + "uid": "b20601a3-624c-4a8c-ab88-717643f01d56", + "state": "KAFKA_CONNECTION_STATE_OPEN", + "open_time": "2025-10-15T14:15:15.755065000Z", + "close_time": "1970-01-01T00:00:00.000000000Z", + "authentication_info": { + "state": "AUTHENTICATION_STATE_UNAUTHENTICATED", + "mechanism": "AUTHENTICATION_MECHANISM_UNSPECIFIED", + "user_principal": "" + }, + "listener_name": "", + "tls_info": { + "enabled": false + }, + "source": { + "ip_address": "127.0.0.1", + "port": 55012 + }, + "client_id": "perf-producer-client", + "client_software_name": "apache-kafka-java", + "client_software_version": "3.9.0", + "transactional_id": "my-tx-id", + "group_id": "", + "group_instance_id": "", + "group_member_id": "", + "api_versions": { + "18": 4, + "22": 3, + "3": 12, + "24": 3, + "0": 7 + }, + "idle_duration": "0s", + "in_flight_requests": { + "sampled_in_flight_requests": [ + { + "api_key": 0, + "in_flight_duration": "0.000406892s" + } + ], + "has_more_requests": false + }, + "total_request_statistics": { + "produce_bytes": "78927173", + "fetch_bytes": "0", + "request_count": "4853", + "produce_batch_count": "4849" + }, + "recent_request_statistics": { + "produce_bytes": "78927173", + "fetch_bytes": "0", + "request_count": "4853", + "produce_batch_count": "4849" + } + } + ] +} +---- +==== +-- endif::[] +====== + +You can adjust the filter and sorting criteria as necessary. === Latency diff --git a/modules/reference/pages/rpk/rpk-cluster/rpk-cluster-connections-list.adoc b/modules/reference/pages/rpk/rpk-cluster/rpk-cluster-connections-list.adoc index 0d780277e..2202afa68 100644 --- a/modules/reference/pages/rpk/rpk-cluster/rpk-cluster-connections-list.adoc +++ b/modules/reference/pages/rpk/rpk-cluster/rpk-cluster-connections-list.adoc @@ -1,12 +1,19 @@ = rpk cluster connections list +// tag::single-source[] Display statistics about current Kafka connections. This command displays a table of active and recently closed connections within the cluster. Use filtering and sorting to identify the connections of the client applications that you are interested in. See `--help` for the list of filtering arguments and sorting arguments. -In addition to the filtering shorthand CLI arguments (For example, `--client-id`, `--state`), you can also use the `--filter-raw` and `--order-by` arguments that take string expressions. To understand the syntax of these arguments, refer to the Admin API docs of the filter and order-by fields of the [ListKafkaConnections endpoint](https://docs.redpanda.com/api/doc/admin/version/11f41833-5783-4f1a-ad64-5957267abd52/operation/operation-redpanda-core-admin-v2-clusterservice-listkafkaconnections): +ifndef::env-cloud[] +In addition to filtering shorthand CLI arguments (For example, `--client-id`, `--state`), you can also use the `--filter-raw` and `--order-by` arguments that take string expressions. To understand the syntax of these arguments, refer to the Admin API docs of the filter and order-by fields of the link:/api/doc/admin/v2/operation/operation-redpanda-core-admin-v2-clusterservice-listkafkaconnections[ListKafkaConnections endpoint]. +endif::[] + +ifdef::env-cloud[] +In addition to filtering shorthand CLI arguments (For example, `--client-id`, `--state`), you can also use the `--filter-raw` and `--order-by` arguments that take string expressions. To understand the syntax of these arguments, refer to the Admin API docs of the filter and order-by fields of the link:/api/doc/cloud-dataplane/operation/operation-monitoringservice_listkafkaconnections[`GET /v1/monitoring/kafka/connections`] Data Plane API endpoint. +endif::[] By default only a subset of the per-connection data is printed. To see all of the available data, use `--format=json`. @@ -93,4 +100,6 @@ rpk cluster connections list --format=json --state="OPEN" |--profile |string |Profile to use. See xref:reference:rpk/rpk-profile.adoc[`rpk profile`] for more details. |-v, --verbose |- |Enable verbose logging. -|=== \ No newline at end of file +|=== + +// end::single-source[] \ No newline at end of file diff --git a/modules/reference/pages/rpk/rpk-cluster/rpk-cluster-connections.adoc b/modules/reference/pages/rpk/rpk-cluster/rpk-cluster-connections.adoc index 33343e5ca..dabb10806 100644 --- a/modules/reference/pages/rpk/rpk-cluster/rpk-cluster-connections.adoc +++ b/modules/reference/pages/rpk/rpk-cluster/rpk-cluster-connections.adoc @@ -1,4 +1,5 @@ = rpk cluster connections +// tag::single-source[] Manage and monitor cluster connections. @@ -31,4 +32,6 @@ connections, connection |--profile |string |Profile to use. See xref:reference:rpk/rpk-profile.adoc[`rpk profile`] for more details. |-v, --verbose |- |Enable verbose logging. -|=== \ No newline at end of file +|=== + +// end::single-source[] \ No newline at end of file