|
| 1 | +--- |
| 2 | +id: confluent-cloud |
| 3 | +title: Confluent Cloud |
| 4 | +sidebar_label: Confluent Cloud |
| 5 | +description: The Sumo Logic app for Confluent Cloud offers deep visibility into your environment with dashboards that monitor Kafka performance, consumer lag, producer latency, connector health, Flink compute pools, KSQL activity, and schema registry operations, enabling proactive troubleshooting and efficient data streaming. |
| 6 | +--- |
| 7 | + |
| 8 | +import useBaseUrl from '@docusaurus/useBaseUrl'; |
| 9 | + |
| 10 | +<img src={useBaseUrl('img/integrations/saas-cloud/confluent-cloud.png')} alt="Thumbnail icon" width="100"/> |
| 11 | + |
| 12 | +Confluent Cloud is a fully managed, cloud-native data streaming platform based on Apache Kafka. It enables real-time data processing, integration, and movement across applications and systems without managing infrastructure. With built-in support for connectors, ksqlDB, and stream governance, it simplifies building scalable, event-driven architectures. |
| 13 | + |
| 14 | +The Sumo Logic app for Confluent Cloud provides deep visibility into your Confluent Cloud environment based on key platform metrics. The app’s dashboards use preconfigured searches and filters to help you monitor Kafka cluster performance, topic-level throughput, consumer lag, producer latency, connector health, Flink compute pool utilization, KSQL activity, and schema registry operations—enabling proactive troubleshooting and streamlined data streaming operations. |
| 15 | + |
| 16 | +## Metrics type |
| 17 | + |
| 18 | +The Confluent Cloud app uses metrics: |
| 19 | + |
| 20 | +* [Confluent Cloud runtime metrics](https://api.telemetry.confluent.cloud/docs/descriptors/datasets/cloud). |
| 21 | + |
| 22 | +### Collecting metrics for the Confluent Cloud app |
| 23 | + |
| 24 | +Sumo Logic supports collecting metrics using the Cloud-to-Cloud integration framework: |
| 25 | + |
| 26 | +* Configure a [Confluent Cloud Metrics Source](/docs/send-data/hosted-collectors/cloud-to-cloud-integration-framework/confluent-cloud-metrics-source/) |
| 27 | + |
| 28 | +## Installing the Confluent Cloud app |
| 29 | + |
| 30 | +Now that you have set up a collection for **Confluent Cloud**, install the Sumo Logic app to use the pre-configured [dashboards](#viewing-the-confluent-cloud-dashboards) that provide visibility into your environment for real-time analysis of overall usage. |
| 31 | + |
| 32 | +import AppInstall from '../../reuse/apps/app-install-v2.md'; |
| 33 | + |
| 34 | +<AppInstall/> |
| 35 | + |
| 36 | +## Viewing the Confluent Cloud dashboards |
| 37 | + |
| 38 | +### Compute Pool |
| 39 | + |
| 40 | +The **Confluent Cloud - Compute Pool** dashboard provides details on the status and utilization of compute resources in your Confluent Cloud environment. |
| 41 | + |
| 42 | +Use this dashboard to: |
| 43 | +* Monitor the average Flink statement status to identify potential issues with stream processing jobs. |
| 44 | +* Track CPU utilization against the set limit to ensure optimal resource allocation and prevent performance bottlenecks. |
| 45 | +* Analyze trends in CPU minutes consumed over time to optimize cost and capacity planning for your Confluent Cloud deployment. |
| 46 | +* Compare current CPU usage across different compute pools to balance workloads and improve overall system efficiency. |
| 47 | + |
| 48 | +<img src={useBaseUrl('https://sumologic-app-data-v2.s3.amazonaws.com/dashboards/ConfluentCloud/Confluent-Cloud-Compute-Pool.png')} alt="Confluent Cloud - Compute Pool" style={{border: '1px solid gray'}} width="800" /> |
| 49 | + |
| 50 | +### Connectors |
| 51 | + |
| 52 | +The **Confluent Cloud - Connectors** dashboard provides details on the status, performance, and health of Confluent Cloud connectors. |
| 53 | + |
| 54 | +Use this dashboard to: |
| 55 | +* Monitor the status of connectors and their tasks to quickly identify any that are failed, paused, or provisioning. |
| 56 | +* Track the volume of sent and received records across different connectors to ensure data flow is as expected. |
| 57 | +* Analyze the average sent and received bytes to detect any unusual spikes or drops in data transfer. |
| 58 | +* Identify potential bottlenecks by correlating connector status with sent/received records and bytes. |
| 59 | +* Monitor dead letter queue records to catch any messages that failed to be processed correctly. |
| 60 | + |
| 61 | +<img src={useBaseUrl('https://sumologic-app-data-v2.s3.amazonaws.com/dashboards/ConfluentCloud/Confluent-Cloud-Connectors.png')} alt="Confluent Cloud - Connectors" style={{border: '1px solid gray'}} width="800" /> |
| 62 | + |
| 63 | +### Kafka Cluster |
| 64 | + |
| 65 | +The **Confluent Cloud - Kafka Cluster** dashboard provides details on key performance metrics and operational statistics for your Kafka cluster in Confluent Cloud. |
| 66 | + |
| 67 | +Use this dashboard to: |
| 68 | +* Monitor producer latency and cluster load to ensure optimal performance of your Kafka cluster. |
| 69 | +* Track active connections, request bytes, and response bytes to identify potential bottlenecks or unusual traffic patterns. |
| 70 | +* Analyze request count trends over time to understand usage patterns and plan for capacity needs. |
| 71 | + |
| 72 | +<img src={useBaseUrl('https://sumologic-app-data-v2.s3.amazonaws.com/dashboards/ConfluentCloud/Confluent-Cloud-Kafka-Cluster.png')} alt="Confluent Cloud - Kafka Cluster" style={{border: '1px solid gray'}} width="800" /> |
| 73 | + |
| 74 | +### Kafka Cluster Links |
| 75 | + |
| 76 | +The **Confluent Cloud – Kafka Cluster Link** dashboard provides key insights into cluster link performance and health. It tracks link task counts, mirror topic states, offset lags, data volume, and response metrics—helping monitor cross-cluster replication efficiency, detect mirror errors, and ensure seamless data delivery across environments. |
| 77 | + |
| 78 | +Use this dashboard to: |
| 79 | +* Monitor link counts and task activity to ensure proper setup and functioning of cluster replication. |
| 80 | +* Track mirror topic count, bytes, and offset lag to assess replication volume and delays. |
| 81 | +* Analyze destination response bytes for throughput visibility across clusters. |
| 82 | +* Detect mirror transition errors to troubleshoot sync failures efficiently. |
| 83 | + |
| 84 | +<img src={useBaseUrl('https://sumologic-app-data-v2.s3.amazonaws.com/dashboards/ConfluentCloud/Confluent-Cloud-Kafka-Cluster-Links.png')} alt="Confluent Cloud - Kafka Cluster Topic" style={{border: '1px solid gray'}} width="800" /> |
| 85 | + |
| 86 | +### Kafka Cluster Topic |
| 87 | + |
| 88 | +The **Confluent Cloud - Kafka Cluster Topic** dashboard provides details on topic-level metrics for Kafka clusters in Confluent Cloud, including consumer lag, data throughput, and partition information. |
| 89 | + |
| 90 | +Use this dashboard to: |
| 91 | +* Monitor consumer lag offsets to identify potential bottlenecks in data processing and ensure timely consumption of messages. |
| 92 | +* Analyze the relationship between received and sent bytes to detect any data loss or transmission issues within the Kafka cluster. |
| 93 | +* Track the number of received and sent records over time to understand topic usage patterns and optimize resource allocation. |
| 94 | +* Observe retained bytes and partition count to manage storage utilization and ensure proper topic scaling. |
| 95 | + |
| 96 | +<img src={useBaseUrl('https://sumologic-app-data-v2.s3.amazonaws.com/dashboards/ConfluentCloud/Confluent-Cloud-Kafka-Cluster-Topic.png')} alt="Confluent Cloud - Kafka Cluster Topic" style={{border: '1px solid gray'}} width="800" /> |
| 97 | + |
| 98 | +### KSQL |
| 99 | + |
| 100 | +The **Confluent Cloud - KSQL** dashboard provides details on query performance, data processing, and resource utilization for KSQL operations in Confluent Cloud. |
| 101 | + |
| 102 | +Use this dashboard to: |
| 103 | +* Monitor query saturation and identify potential bottlenecks in KSQL operations. |
| 104 | +* Track processing errors and query restarts to ensure smooth data flow and query execution. |
| 105 | +* Analyze storage utilization and streaming unit consumption to optimize resource allocation. |
| 106 | +* Correlate offset lag with processing errors to troubleshoot data processing delays. |
| 107 | +* Examine produced and consumed bytes to understand data throughput and potential network issues. |
| 108 | + |
| 109 | +<img src={useBaseUrl('https://sumologic-app-data-v2.s3.amazonaws.com/dashboards/ConfluentCloud/Confluent-Cloud-KSQL.png')} alt="Confluent Cloud - KSQL" style={{border: '1px solid gray'}} width="800" /> |
| 110 | + |
| 111 | +### Schema Registry |
| 112 | + |
| 113 | +The **Confluent Cloud - Schema Registry** dashboard provides details on schema operations, request counts, and traffic types for your Confluent Cloud Schema Registry. |
| 114 | + |
| 115 | +Use this dashboard to: |
| 116 | +* Monitor the distribution of schema operations (CREATE, READ, DELETE) to ensure proper usage and identify potential issues with schema management. |
| 117 | +* Analyze the ratio of public vs. private traffic to understand access patterns and potential security concerns. |
| 118 | +* Track schema count trends over time to manage growth and capacity planning for your Schema Registry. |
| 119 | +* Correlate request counts with operation types to identify unusual spikes or drops in schema-related activities. |
| 120 | + |
| 121 | +<img src={useBaseUrl('https://sumologic-app-data-v2.s3.amazonaws.com/dashboards/ConfluentCloud/Confluent-Cloud-Schema-Registry.png')} alt="Confluent Cloud - Schema Registry" style={{border: '1px solid gray'}} width="800" /> |
0 commit comments