|
| 1 | +--- |
| 2 | +slug: /use-cases/observability/clickstack/integrations/postgresql-metrics |
| 3 | +title: 'Monitoring PostgreSQL Metrics with ClickStack' |
| 4 | +sidebar_label: 'PostgreSQL Metrics' |
| 5 | +pagination_prev: null |
| 6 | +pagination_next: null |
| 7 | +description: 'Monitoring PostgreSQL Metrics with ClickStack' |
| 8 | +doc_type: 'guide' |
| 9 | +keywords: ['PostgreSQL', 'Postgres', 'metrics', 'OTEL', 'ClickStack', 'database monitoring'] |
| 10 | +--- |
| 11 | + |
| 12 | +import Image from '@theme/IdealImage'; |
| 13 | +import useBaseUrl from '@docusaurus/useBaseUrl'; |
| 14 | +import import_dashboard from '@site/static/images/clickstack/import-dashboard.png'; |
| 15 | +import finish_import from '@site/static/images/clickstack/postgres/import-dashboard.png'; |
| 16 | +import example_dashboard from '@site/static/images/clickstack/postgres/postgres-metrics-dashboard.png'; |
| 17 | +import { TrackedLink } from '@site/src/components/GalaxyTrackedLink/GalaxyTrackedLink'; |
| 18 | + |
| 19 | +# Monitoring PostgreSQL Metrics with ClickStack {#postgres-metrics-clickstack} |
| 20 | + |
| 21 | +:::note[TL;DR] |
| 22 | +This guide shows you how to monitor PostgreSQL performance metrics with ClickStack by configuring the OpenTelemetry collector's PostgreSQL receiver. You'll learn how to: |
| 23 | + |
| 24 | +- Configure the OTel collector to collect PostgreSQL metrics |
| 25 | +- Deploy ClickStack with your custom configuration |
| 26 | +- Use a pre-built dashboard to visualize PostgreSQL performance (transactions, connections, database size, cache hit ratios) |
| 27 | + |
| 28 | +A demo dataset with sample metrics is available if you want to test the integration before configuring your production PostgreSQL database. |
| 29 | + |
| 30 | +Time required: 10-15 minutes |
| 31 | +::: |
| 32 | + |
| 33 | +## Integration with existing PostgreSQL {#existing-postgres} |
| 34 | + |
| 35 | +This section covers configuring your existing PostgreSQL installation to send metrics to ClickStack by configuring the ClickStack OTel collector with the PostgreSQL receiver. |
| 36 | + |
| 37 | +If you would like to test the PostgreSQL metrics integration before configuring your own existing setup, you can test with our preconfigured demo dataset in the [following section](#demo-dataset). |
| 38 | + |
| 39 | +##### Prerequisites {#prerequisites} |
| 40 | +- ClickStack instance running |
| 41 | +- Existing PostgreSQL installation (version 9.6 or newer) |
| 42 | +- Network access from ClickStack to PostgreSQL (default port 5432) |
| 43 | +- PostgreSQL monitoring user with appropriate permissions |
| 44 | + |
| 45 | +<VerticalStepper headerLevel="h4"> |
| 46 | + |
| 47 | +#### Ensure monitoring user has required permissions {#monitoring-permissions} |
| 48 | + |
| 49 | +The PostgreSQL receiver requires a user with read access to statistics views. Grant the `pg_monitor` role to your monitoring user: |
| 50 | + |
| 51 | +```sql |
| 52 | +GRANT pg_monitor TO your_monitoring_user; |
| 53 | +``` |
| 54 | + |
| 55 | +#### Create custom OTel collector configuration {#create-custom-config} |
| 56 | + |
| 57 | +ClickStack allows you to extend the base OpenTelemetry collector configuration by mounting a custom configuration file and setting an environment variable. |
| 58 | + |
| 59 | +Create `postgres-metrics.yaml`: |
| 60 | + |
| 61 | +```yaml |
| 62 | +receivers: |
| 63 | + postgresql: |
| 64 | + endpoint: postgres-host:5432 |
| 65 | + transport: tcp |
| 66 | + username: otel_monitor |
| 67 | + password: ${env:POSTGRES_PASSWORD} |
| 68 | + databases: |
| 69 | + - postgres |
| 70 | + - your_application_db |
| 71 | + collection_interval: 30s |
| 72 | + tls: |
| 73 | + insecure: true |
| 74 | + |
| 75 | +processors: |
| 76 | + resourcedetection: |
| 77 | + detectors: [env, system, docker] |
| 78 | + timeout: 5s |
| 79 | + batch: |
| 80 | + timeout: 10s |
| 81 | + send_batch_size: 1024 |
| 82 | + |
| 83 | +exporters: |
| 84 | + clickhouse: |
| 85 | + endpoint: tcp://localhost:9000 |
| 86 | + database: default |
| 87 | + ttl: 96h |
| 88 | + |
| 89 | +service: |
| 90 | + pipelines: |
| 91 | + metrics/postgres: |
| 92 | + receivers: [postgresql] |
| 93 | + processors: [resourcedetection, batch] |
| 94 | + exporters: [clickhouse] |
| 95 | +``` |
| 96 | +
|
| 97 | +:::note |
| 98 | +The `tls: insecure: true` setting disables SSL verification for development/testing. For production PostgreSQL with SSL enabled, remove this line or configure proper certificates. |
| 99 | +::: |
| 100 | + |
| 101 | +#### Deploy ClickStack with custom configuration {#deploy-clickstack} |
| 102 | + |
| 103 | +Mount your custom configuration: |
| 104 | + |
| 105 | +```bash |
| 106 | +docker run -d \ |
| 107 | + --name clickstack-postgres \ |
| 108 | + -p 8123:8123 -p 9000:9000 -p 4317:4317 -p 4318:4318 \ |
| 109 | + -e HYPERDX_API_KEY=your-api-key \ |
| 110 | + -e CLICKHOUSE_PASSWORD=your-clickhouse-password \ |
| 111 | + -e POSTGRES_PASSWORD=secure_password_here \ |
| 112 | + -e CUSTOM_OTELCOL_CONFIG_FILE=/etc/otelcol-contrib/custom.config.yaml \ |
| 113 | + -v "$(pwd)/postgres-metrics.yaml:/etc/otelcol-contrib/custom.config.yaml:ro" \ |
| 114 | + clickhouse/clickstack:latest |
| 115 | +``` |
| 116 | + |
| 117 | +#### Verify metrics collection {#verify-metrics} |
| 118 | + |
| 119 | +Once configured, log into HyperDX and verify metrics are flowing: |
| 120 | + |
| 121 | +1. Navigate to the Metrics explorer |
| 122 | +2. Search for metrics starting with postgresql. (e.g., postgresql.backends, postgresql.commits) |
| 123 | +3. You should see metric data points appearing at your configured collection interval |
| 124 | + |
| 125 | +Once metrics are flowing, proceed to the [Dashboards and visualization](#dashboards) section to import the pre-built dashboard. |
| 126 | + |
| 127 | +</VerticalStepper> |
| 128 | + |
| 129 | +## Demo dataset {#demo-dataset} |
| 130 | + |
| 131 | +For users who want to test the PostgreSQL metrics integration before configuring their production systems, we provide a pre-generated dataset with realistic PostgreSQL metrics patterns. |
| 132 | + |
| 133 | +:::note[Database-level metrics only] |
| 134 | +This demo dataset includes database-level metrics only, matching the default PostgreSQL receiver configuration. Table-level and index-level metrics are disabled by default because they generate significantly more data. Database-level metrics provide comprehensive monitoring for most use cases. |
| 135 | +::: |
| 136 | + |
| 137 | +<VerticalStepper headerLevel="h4"> |
| 138 | + |
| 139 | +#### Download the sample metrics dataset {#download-sample} |
| 140 | + |
| 141 | +Download the pre-generated metrics files (24 hours of PostgreSQL metrics with realistic patterns): |
| 142 | + |
| 143 | +```bash |
| 144 | +# Download gauge metrics (connections, database size) |
| 145 | +curl -O https://datasets-documentation.s3.eu-west-3.amazonaws.com/clickstack-integrations/postgres/postgres-metrics-gauge.csv |
| 146 | +
|
| 147 | +# Download sum metrics (commits, rollbacks, operations) |
| 148 | +curl -O https://datasets-documentation.s3.eu-west-3.amazonaws.com/clickstack-integrations/postgres/postgres-metrics-sum.csv |
| 149 | +``` |
| 150 | + |
| 151 | +The dataset includes realistic patterns: |
| 152 | +- **Morning connection spike (08:00)** - Login rush |
| 153 | +- **Cache performance issue (11:00)** - Blocks_read spike |
| 154 | +- **Application bug (14:00-14:30)** - Rollback rate spikes to 15% |
| 155 | +- **Deadlock incidents (14:15, 16:30)** - Rare deadlocks |
| 156 | + |
| 157 | +#### Start ClickStack {#start-clickstack} |
| 158 | + |
| 159 | +Start a ClickStack instance: |
| 160 | + |
| 161 | +```bash |
| 162 | +docker run -d --name clickstack-postgres-demo \ |
| 163 | + -p 8080:8080 -p 4317:4317 -p 4318:4318 \ |
| 164 | + docker.hyperdx.io/hyperdx/hyperdx-all-in-one:latest |
| 165 | +``` |
| 166 | + |
| 167 | +Wait approximately 30 seconds for ClickStack to fully start. |
| 168 | + |
| 169 | +#### Load metrics into ClickStack {#load-metrics} |
| 170 | + |
| 171 | +Load the metrics directly into ClickHouse: |
| 172 | + |
| 173 | +```bash |
| 174 | +# Load gauge metrics |
| 175 | +cat postgres-metrics-gauge.csv | docker exec -i clickstack-postgres-demo \ |
| 176 | + clickhouse-client --query "INSERT INTO otel_metrics_gauge FORMAT CSVWithNames" |
| 177 | +
|
| 178 | +# Load sum metrics |
| 179 | +cat postgres-metrics-sum.csv | docker exec -i clickstack-postgres-demo \ |
| 180 | + clickhouse-client --query "INSERT INTO otel_metrics_sum FORMAT CSVWithNames" |
| 181 | +``` |
| 182 | + |
| 183 | +#### Verify metrics in HyperDX {#verify-metrics} |
| 184 | + |
| 185 | +Once loaded, the quickest way to see your metrics is through the pre-built dashboard. |
| 186 | + |
| 187 | +Proceed to the [Dashboards and visualization](#dashboards) section to import the dashboard and view many PostgreSQL metrics at once. |
| 188 | + |
| 189 | +:::note |
| 190 | +The demo dataset time range is November 10, 2025 00:00:00 to November 11, 2025 00:00:00. Make sure your time range in HyperDX matches this window. |
| 191 | +::: |
| 192 | + |
| 193 | +</VerticalStepper> |
| 194 | + |
| 195 | +## Dashboards and visualization {#dashboards} |
| 196 | + |
| 197 | +To help you get started monitoring PostgreSQL with ClickStack, we provide essential visualizations for PostgreSQL metrics. |
| 198 | + |
| 199 | +<VerticalStepper headerLevel="h4"> |
| 200 | + |
| 201 | +#### <TrackedLink href={useBaseUrl('/examples/postgres-metrics-dashboard.json')} download="postgres-metrics-dashboard.json" eventName="docs.postgres_metrics_monitoring.dashboard_download">Download</TrackedLink> the dashboard configuration {#download} |
| 202 | + |
| 203 | +#### Import the pre-built dashboard {#import-dashboard} |
| 204 | + |
| 205 | +1. Open HyperDX and navigate to the Dashboards section |
| 206 | +2. Click **Import Dashboard** in the upper right corner under the ellipses |
| 207 | + |
| 208 | +<Image img={import_dashboard} alt="Import dashboard button"/> |
| 209 | + |
| 210 | +3. Upload the `postgres-metrics-dashboard.json` file and click **Finish Import** |
| 211 | + |
| 212 | +<Image img={finish_import} alt="Finish import dialog"/> |
| 213 | + |
| 214 | +#### View the dashboard {#created-dashboard} |
| 215 | + |
| 216 | +The dashboard will be created with all visualizations pre-configured: |
| 217 | + |
| 218 | +<Image img={example_dashboard} alt="PostgreSQL metrics dashboard"/> |
| 219 | + |
| 220 | +:::note |
| 221 | +For the demo dataset, ensure the time range is set to November 10, 2025 00:00:00 - November 11, 2025 00:00:00. |
| 222 | +::: |
| 223 | + |
| 224 | +</VerticalStepper> |
| 225 | + |
| 226 | +## Troubleshooting {#troubleshooting} |
| 227 | + |
| 228 | +### Custom config not loading {#troubleshooting-not-loading} |
| 229 | + |
| 230 | +Verify the environment variable is set: |
| 231 | + |
| 232 | +```bash |
| 233 | +docker exec <container-name> printenv CUSTOM_OTELCOL_CONFIG_FILE |
| 234 | +``` |
| 235 | + |
| 236 | +Check the custom config file is mounted: |
| 237 | + |
| 238 | +```bash |
| 239 | +docker exec <container-name> cat /etc/otelcol-contrib/custom.config.yaml |
| 240 | +``` |
| 241 | + |
| 242 | +### No metrics appearing in HyperDX {#no-metrics} |
| 243 | + |
| 244 | +Verify PostgreSQL is accessible: |
| 245 | + |
| 246 | +```bash |
| 247 | +docker exec <clickstack-container> psql -h postgres-host -U otel_monitor -d postgres -c "SELECT 1" |
| 248 | +``` |
| 249 | + |
| 250 | +Check OTel collector logs: |
| 251 | + |
| 252 | +```bash |
| 253 | +docker exec <container> cat /etc/otel/supervisor-data/agent.log | grep -i postgres |
| 254 | +``` |
| 255 | + |
| 256 | +### Authentication errors {#auth-errors} |
| 257 | + |
| 258 | +Verify password is set correctly: |
| 259 | + |
| 260 | +```bash |
| 261 | +docker exec <clickstack-container> printenv POSTGRES_PASSWORD |
| 262 | +``` |
| 263 | + |
| 264 | +Test credentials directly: |
| 265 | + |
| 266 | +```bash |
| 267 | +psql -h postgres-host -U otel_monitor -d postgres -c "SELECT version();" |
| 268 | +``` |
| 269 | + |
| 270 | +## Next steps {#next-steps} |
| 271 | + |
| 272 | +After setting up PostgreSQL metrics monitoring: |
| 273 | + |
| 274 | +- Set up [alerts](/use-cases/observability/clickstack/alerts) for critical thresholds (connection limits, high rollback rates, low cache hit ratios) |
| 275 | +- Enable query-level monitoring with `pg_stat_statements` extension |
| 276 | +- Monitor multiple PostgreSQL instances by duplicating the receiver configuration with different endpoints and service names |
| 277 | + |
| 278 | +## Going to production {#going-to-production} |
| 279 | + |
| 280 | +This guide extends ClickStack's built-in OpenTelemetry Collector for quick setup. For production deployments, we recommend running your own OTel Collector and sending data to ClickStack's OTLP endpoint. See [Sending OpenTelemetry data](/use-cases/observability/clickstack/ingesting-data/opentelemetry) for production configuration. |
0 commit comments