3.0.0

What's New

This major release introduces several significant updates, new features, bug fixes, and enhancements:

Important Changes

• Documentation: The documentation has been updated and moved to https://pgwat.ch for improved readability and navigation. Special part dedicated to the pgwatch development has been added.
• Parallel Sinks: Support for simultaneous storing metric measurements to several destinations, e.g. database and file at once.
• Custom Storage: Leverage our remote sink (RPC) interface to seamlessly integrate any storage format or data stream, such as Parquet files or Kafka, directly into pgwatch.
• Windows Support Added: Windows support has been integrated, broadening the compatibility of the software.
• Docker Images Rename: The pgwatch2-postgres Docker image has been renamed to pgwatch-demo, pgwatch2-daemon to pgwatch.
• Enhanced Logging: An advanced logging system has been implemented to improve error tracking and monitoring.
• Grafana Upgrade: Support for the latest Grafana v9.x and v10 has been added, including a switch to Grafana's REST API for dashboard provisioning.
• Etcd v3 Migration: The system has been upgraded to Etcd v3, ensuring better performance and compatibility.
• PostgreSQL 17 Metrics: New metrics for PostgreSQL 17 have been introduced.
• Automatic Schema Rollout: A new automatic schema rollout mechanism has been implemented, streamlining database management.
• Build and Test Workflow: A new "Build and Test" workflow has been added to automate testing and building processes.
• WebUI Enhancements: The WebUI now includes options like test connections, a log tab, and a clone button for the database grid.
• REST API: New REST API endpoints have been added, including endpoints such as source, /metric, /preset, log.
• Docker Improvements: The Docker images and related scripts have been improved, including the introduction of a new Dockerfile based on the official Postgres image and Docker Compose support.
• Test Coverage: Test coverage statistics are now available to ensure code quality and reliability.
• Environment Variables Update: The PW2_ environment variable prefixes have been updated to PW3_.
• Code Refactoring: The main.go file has been streamlined by extracting functionality into separate packages.
• Deprecations: Removed support for InfluxDB, Graphite datastore, and other outdated features.

Dependency Updates

Numerous dependencies have been updated, including upgrades to Golang, PostgreSQL, Prometheus, and various Go modules. These upgrades ensure better performance, security, and compatibility with other tools.

This release represents a significant step forward in functionality, performance, and security, with a strong focus on expanding compatibility and improving the user experience.

Changelog

• [!] add Measurements group and use URIs for --sink options by @pashagolub in #368
• [!] add Grafana v11 dashboards by @muni106 in #407
• [!] add metrics for PostgreSQL 16 by @pashagolub in #246
• [!] add RPC sink support, closes #464 by @destrex271 in #465
• [!] add support for jackc/pgx, closes #171 by @pashagolub in #204
• [!] add support for application commands by @pashagolub in #504
• [!] add support for log tab in web UI, closes #170 by @pashagolub in #206
• [!] add support for PostgreSQL v17 by @pashagolub in #497
• [!] add support for the latest Grafana v9.x, closes #174 by @pashagolub in #186
• [!] add windows support, closes #3 by @pashagolub in #4
• [!] allow multiple parallel metric storages by @pashagolub in #313
• [!] bump Grafana to v10, closes #227 by @pashagolub in #232
• [!] decouple sources package from main, closes #357 by @pashagolub in #358
• [!] decouple metrics into separate package, closes #372 by @pashagolub in #408
• [!] differentiate the concepts of source and monitored database by @pashagolub in #472
• [!] extract psutil into separate package by @pashagolub in #144
• [!] extract configuration handling into separate package, closes #138 by @pashagolub in #139
• [!] fix Go linting, closes #142 by @pashagolub in #143
• [!] implement automatic schema rollout machinery, closes #180 by @pashagolub in #225
• [!] implement enhanced logging by @pashagolub in #146
• [!] implement new Dockerfile based on official Postgres image, closes #210 by @pashagolub in #226
• [!] implement upgrade machinery, closes #179 by @pashagolub in #451
• [!] implement webui authentication, closes #216 by @denys-holub in #251
• [!] move measurements reaper code out of main package, closes #418 by @pashagolub in #419
• [!] move metric batching to sinks from the gatherer by @pashagolub in #346
• [!] rename pgwatch3-postgres Docker image to pgwatch3, closes #20 by @pashagolub in #21
• [!] restructure code tree, closes #518 by @pashagolub in #521
• [!] restructure Docker related assets by @pashagolub in #535
• [!] switch configuration to connection string from separate parameters, closes #255 by @pashagolub in #322
• [!] switch to Grafana REST API for dashboard provisioning, closes #188 by @pashagolub in #189
• [!] transform Sphinx manual to mkdocs-material by @pashagolub in #476
• [!] upgrade to Etcd v3, resolves #59 by @pashagolub in #157
• [+] add "Build and Test" workflow, closes #5 by @pashagolub in #6
• [+] add /metric endpoint, closes #93 by @pashagolub in #113
• [+] add arm64 binaries to release assets, closes #512 by @pashagolub in #517
• [+] add convert app to migrate v2 metrics definitions to v3 by @pashagolub in #373
• [+] add pgbouncer_clients metric by @pashagolub in #440
• [+] add Preset configs tab, closes #135 by @denys-holub in #136
• [+] add RPCSyncRequest to api by @pashagolub in #528
• [+] add usename to replication metric by @pashagolub in #444
• [+] add bootstrap tests by @pashagolub in #277
• [+] add built-in http server, closes #28 by @pashagolub in #30
• [+] add clone button to database grid, closes #150 by @denys-holub in #159
• [+] add commands config [init|upgrade], closes #514 by @pashagolub in #515
• [+] add concurrency clause to Build GHA workflow by @pashagolub in #269
• [+] add Docker Compose file, closes #240 by @pashagolub in #242
• [+] add icon and logo for web ui, closes #66 by @denys-holub in #67
• [+] add log tab to the main window, closes #68 by @denys-holub in #71
• [+] add main metrics descriptions, closes #547 by @pashagolub in #548
• [+] add metrics tests by @pashagolub in #478
• [+] add new metrics for invalid and unused indexes by @pashagolub in #286
• [+] add pgAdmin4 to docker-compose.yml by @ondar in #432
• [+] add Prometheus service to Docker Compose file by @pashagolub in #314
• [+] add support for /PATCH preset endpoint, closes #163 by @pashagolub in #187
• [+] add support for GET /preset endpoint, closes #132 by @pashagolub in #134
• [+] add support for POST /preset endpoint by @pashagolub in #177
• [+] add test cases to webserver api by @muni106 in #397
• [+] add test coverage statistics, closes #273 by @pashagolub in #274
• [+] add tests for db package by @pashagolub in #463
• [+] add tests for sinks by @pashagolub in #449
• [+] add tests for postgres sources.ReaderWriter implementation by @pashagolub in #469
• [+] add tests for YAML sources.ReaderWriter implementation by @pashagolub in #470
• [+] add TimescaleDB metric storage based docker image, resolves #176 by @pashagolub in #183
• [+] add trigger on metric to update presets, closes #185 by @pashagolub in #370
• [+] add web UI start options, closes #236 by @pashagolub in #237
• [+] allow CORS from http://localhost:4000 for webui debugging by @pashagolub in #439
• [+] bump axios from 1.6.0 to 1.7.4 by @dependabot in #522
• [+] bump express from 4.19.2 to 4.21.0 in webui by @dependabot in #544
• [+] bump jackc/pgx/v5 from 5.6.0 to 5.7.1 by @dependabot in #543
• [+] bump micromatch from 4.0.5 to 4.0.8 by @dependabot in #520
• [+] bump prometheus/client_golang from 1.20.1 to 1.20.4 by @dependabot in #519 #539 #545
• [+] bump rollup from 2.79.1 to 2.79.2 in webui by @dependabot in #550
• [+] bump shirou/gopsutil/v4 from 4.24.7 to 4.24.9 by @dependabot in #538 #551
• [+] bump yaml.v2 to yaml.v3, closes #359 by @pashagolub in #360
• [+] bump webpack from 5.76.1 to 5.94.0 by @dependabot in #537
• [+] bump Go to v1.23 by @pashagolub in #555
• [+] create common error & loading components, closes #125 by @denys-holub in #126
• [+] get rid of control channel map, closes #434 by @pashagolub in #435
• [+] highlight active WebUI tab, closes #199 by @denys-holub in #200
• [+] implement GET /metric & DELETE /metric to web UI, closes #116 by @denys-holub in #117
• [+] implement POST /metric & PATCH /metric to web UI, closes #119 by @denys-holub in #120
• [+] improve Build & Test GHA workflow by @pashagolub in #481
• [+] improve Developer Documentation by @pashagolub in #487
• [+] improve Docker images and related scripts by @pashagolub in #158
• [+] improve Docker workflow by @pashagolub in #122
• [+] include webui into pgwatch3 binary, resolves #74 by @pashagolub in #75
• [+] make pgwatch2-daemon Docker image use alpine base image, closes #23 by @pashagolub in #25
• [+] move helpers to metrics definition by @pashagolub in #374
• [+] produce "latest" and semver tags for all Docker images by @pashagolub in #556
• [+] refactor sinks as a standalone package by @pashagolub in #345
• [+] refactor metric definitions handling by @pashagolub in #342
• [+] replace checkboxes in monitoring grid with switches, closes #160 by @denys-holub in #164
• [+] sync changes from pgwatch2 v1.10, closes #70 by @pashagolub in #118
• [+] sync GHA "Release" workflow by @pashagolub in #264
• [+] update Ubuntu to 22.04 in Docker files, closes #16 by @pashagolub in #17 #24
• [+] use embed package to serve webui files, closes #32 by @pashagolub in #33
• [+] use custom Postgres image for compose by @pashagolub in #503
• [+] webui test connection, closes #245 by @denys-holub in #259
• [*] add description to sink implementations by @pashagolub in #475
• [*] change ConnConfigCallback type to func(*pgxpool.Config) error by @pashagolub in #325
• [*] change URLs and packages to github.com/cybertec-postgresql/pgwatch by @pashagolub in #516
• [*] change web gui title to "pgwatch3 dashboard", closes #76 by @denys-holub in #82
• [*] decouple metrics related code into a separate package, closes #295 by @pashagolub in #297
• [*] define custom types instead of general array of object maps by @pashagolub in #145
• [*] dependencies upgrade by @denys-holub in #86
• [*] deprecate adhoc mode, closes #365 by @pashagolub in #366
• [*] extract database functionality from main.go, resolves #79 by @pashagolub in #80
• [*] fix typos, add idea folder to .gitignore by @ondar in #431
• [*] improve --help output by @pashagolub in #436
• [*] improve docker compose file by @pashagolub in #498
• [*] improve preset configs grid, closes #154 by @denys-holub in #155
• [*] make GridActions & GridToolbar components common, closes #129 by @denys-holub in #130
• [*] move DbStorageSchemaType related code to PostgresWriter by @pashagolub in #344
• [*] move psutil and logparser to sources package by @pashagolub in #363
• [*] move ReadPresetMetricsConfigFromFolder() to metrics package by @pashagolub in #343
• [*] move docker scripts to appropriate folder by @pashagolub in #477
• [*] move initialization code from db to appropriate packages by @pashagolub in #448
• [*] move metric conversion utility from the main tree by @pashagolub in #479
• [*] prettify config, remove unused opts by @pashagolub in #367
• [*] refactor db package by @pashagolub in #362
• [*] refactor command-line parsing by @pashagolub in #364
• [*] remove "NB!" from sources, fixes #221 by @pashagolub in #222
• [*] remove shirou/gopsutil dependancy from database.go, closes #172 by @pashagolub in #190
• [*] remove shopspring/decimal dependancy, resolves #173 by @pashagolub in #181
• [*] remove db password column from databases grid, closes #73 by @denys-holub in #81
• [*] remove deprecated "test data" mode, closes #271 by @pashagolub in #276
• [*] remove deprecated helpers by @pashagolub in #371
• [*] remove Graphite datastore support, closes #270 by @pashagolub in #272
• [*] remove internal statistics by @pashagolub in #430
• [*] remove references to deprecated metric files from docker-compose.yml by @Xunop in #382
• [*] remove unused code from GetAllActiveHostsFromConfigDB(), closes #266 by @pashagolub in #268
• [*] rename CmdOptions to Options by @pashagolub in #324
• [*] replace --no-helper-functions with --create-helpers , closes #557 by @pashagolub in #558
• [*] separate webui and webserver code, resolves #77 by @pashagolub in #78
• [*] update PW2_ environment variables prefixes to PW3_, closes #26 by @pashagolub in #27
• [*] update docker compose command to force recreate test database by @qazxcdswe123 in #398
• [*] update environment variables documentation, closes #424 by @pashagolub in #428
• [*] update monitored sources grid, closes #445 by @denys-holub in #473
• [*] update repository main page by @pashagolub in #263
• [*] use Metric term for definition and Measurement for data gathered by @pashagolub in #347
• [*] use Docker Compose V2 instead of V1 by @Massinja in #392
• [*] use envvar in GHA instead of deprecated set-output command, closes #35 by @pashagolub in #36
• [*] use system max identifier length instead of hard-coded value by @pashagolub in #293
• [-] apply --max-parallel-connections-per-db for connections by @pashagolub in #560
• [-] fix connection string used for resolved databases, fixes #554 by @pashagolub in #559
• [-] fix Patroni source resolving by @saletnik in #546
• [-] fix sources grid in WebUI, fixes #524 by @pashagolub in #525
• [-] fix update metrics in PostgreSQL, fixes #532 by @pashagolub in #534
• [-] fix webui if opened from a remote host, fixes #526 by @denys-holub in #527

New Contributors

• @Lana8Lee made their first contribution in #64
• @denys-holub made their first contribution in #82
• @AndrewJackson2020 made their first contribution in #355
• @Xunop made their first contribution in #382
• @Massinja made their first contribution in #392
• @muni106 made their first contribution in #397
• @qazxcdswe123 made their first contribution in #388
• @ondar made their first contribution in #431
• @destrex271 made their first contribution in #454
• @krisavi made their first contribution in #495
• @saletnik made their first contribution in #546

Full Changelog: https://github.com/cybertec-postgresql/pgwatch/commits/3.0.0