Releases: cybertec-postgresql/pgwatch
v4.0.0
What's New
This major release introduces several significant updates, new features, bug fixes, and enhancements
Important Changes
PostgreSQL 18 Metrics
total_XXX_timecolumns are added totable_statsmetric;read_bytes,write_bytes, andextend_bytesare added tostat_iometric;- use
pg_stat_ioforwal_statsmetric; - use
pg_ls_archive_statusdir()inarchiver_pending_countmetric; num_doneandslru_writtencolumns are added tocheckpointermetric;parallel_workers_to_launchandparallel_workers_launchedcolumns are added todb_statsmetric.
Grafana Dashboards
- New Grafana v12 dashboards have been added for both PostgreSQL and Prometheus sinks;
- Grafana v10 support is discontinued;
- New "Global Database Overview" dashboard with 26 panels, including new panels for replication, connections, and index usage (inspired by postgres.ai);
- New "Database Overview" now has 21 panels with time lag support and improved visualizations;
- New "Query Performance Analysis" dashboard with enhanced table with 17 metrics and 8 visualization panels (inspired by postgres.ai);
- New "Tables Overview" dashboard providing insights into table sizes, bloat, and index usage with treemap visualizations panels.
Metrics
- Realtime metrics have been deprecated and removed from the system.
- Metrics can now be loaded from a specified folder, allowing for better organization and management of metric definitions.
Sinks
- gRPC Sink Enhancements: Basic authentication support has been added to the gRPC sink, along with improved documentation for its usage.
Development and Contribution
- The development experience with Docker Compose has been enhanced, making it easier to set up and manage the environment.
- A new repository, pgwatch-contrib, has been created to host additional contributions and extensions for pgwatch from the community. The
rpcsubdirectory contains sample gRPC sinks implementations.
Changelog
[!]add Grafana v12 dashboards for Prometheus data source by @pashagolub in #933[!]add support for PostgreSQL v18 by @pashagolub in #903[!]deprecate realtime metrics by @pashagolub in #925[!]remove Grafana v10 dashboards due to EOL of Grafana v10 by @pashagolub in #934[!]remove obsoletedb_stats_aurorametric by @pashagolub in #930[*]add note in sizing docs about metrics returning thousands of rows by @0xgouda in #874[*]clarify the usage of docker compose, closes #879 by @pashagolub in #880[*]fixupdateSources()query parameters number by @0xgouda in #875[*]increase Postgres sink partitions to precreate from 0 to 3 by @kylorend3r in #960[*]mentionpgwatch-contrib/rpcin docs by @0xgouda in #922[*]misc minor updates to docs by @0xgouda in #871[*]removeuidfrom datasource by @smiyc in #952[*]remove hard coded metric defs from docs by @0xgouda in #870[*]update docs for metrics yaml folder configs by @0xgouda in #892[+]add "Query Performance Analysis" dashboard by @pashagolub in #916[+]add "Tables Overview" dashboard by @pashagolub in #918[+]add basic auth support to grpc sink by @0xgouda in #862[+]add docs for grpc sink by @0xgouda in #867[+]add PlantUML diagrams by @pashagolub in #926[+]add sample metrics YAML file, closes #883 by @pashagolub in #900[+]allow metrics loading from folder, closes #882 by @0xgouda in #889[+]bumpactions/checkoutfrom 4 to 5 by @dependabot[bot] in #888[+]bumpactions/download-artifactfrom 4 to 5 by @dependabot[bot] in #881[+]bumpactions/setup-gofrom 5 to 6 by @dependabot[bot] in #939[+]bumpactions/setup-nodefrom 4 to 5 by @dependabot[bot] in #936[+]bumpactions/setup-pythonfrom 5 to 6 by @dependabot[bot] in #938[+]bumpactions/stalefrom 9 to 10 by @dependabot[bot] in #937[+]bumpaxiosfrom 1.11.0 to 1.12.0 by @dependabot[bot] in #953[+]bumpdocker/dockerfrom 28.3.2 to 28.3.3 by @dependabot[bot] in #872[+]bumpgo.etcd.io/etcd/client/v3from 3.6.2 to 3.6.5 by @dependabot[bot] in #858 #868 #965[+]bumpgolang-jwt/jwt/v5from 5.2.3 to 5.3.0 by @dependabot[bot] in #876[+]bumpgoogle.golang.org/grpcfrom 1.73.0 to 1.75.1 by @dependabot[bot] in #865 #902 #950[+]bumpgoogle.golang.org/protobuffrom 1.36.6 to 1.36.9 by @dependabot[bot] in #887 #904 #948[+]bumpjackc/pgx/v5from 5.7.5 to 5.7.6 by @dependabot[bot] in #946[+]bumpprometheus/client_golangfrom 1.22.0 to 1.23.2 by @dependabot[bot] in #878 #941[+]bumpshirou/gopsutil/v4from 4.25.6 to 4.25.8 by @dependabot[bot] in #877 #923[+]bumpstretchr/testifyfrom 1.10.0 to 1.11.1 by @dependabot[bot] in #906 #912[+]bumptestcontainers/testcontainers-gofrom 0.38.0 to 0.39.0 by @dependabot[bot] in #964[+]bumpwebui/react-scriptsto v5.0.1 (pinned version) by @pashagolub in #866[+]improve "Global Database Overview" dashboard by @pashagolub in #869[+]improvechange_eventsmetric processing by @pashagolub in #932[+]improve development experience with docker compose by @pashagolub in #927[+]update docs with new v12 dashboards screenshots by @pashagolub in #943[-]do not fail on db resolve error, just log an error, fixes #890 by @pashagolub in #899[-]fix assignment to entry in nil map, fixes #968 by @pashagolub in #969[-]fix dashboard links to single query details, fixes #961 by @pashagolub in #970[-]fix Patroni resolver, fixes #962 by @pashagolub in #973[-]fix source YAML test introduced by #900 by @pashagolub in #901[-]fix weekly partition naming at year boundaries by @pashagolub in #974[-]move to next envelope in Postgres sink ifCOPYfails, fixes #908 by @pashagolub in #911[-]prevent concurrent map read and write inSourceConby @pashagolub in #971[-]removehost_configquerying fromGetSources()by @0xgouda in #863[-]remove double "git checkout" from "README.md" by @smiyc in #942[-]use proper datasource id in Prometheus dashboards by @pashagolub in #972
New Contributors
- @smiyc made their first contribution in #942
- @kylorend3r made their first contribution in #960
Full Changelog: v3.7.0...v4.0.0
v4.0.0-beta
What's New
This major release introduces several significant updates, new features, bug fixes, and enhancements
Important Changes
PostgreSQL 18 Metrics
total_XXX_timecolumns are added totable_statsmetric;read_bytes,write_bytes, andextend_bytesare added tostat_iometric;- use
pg_stat_ioforwal_statsmetric; - use
pg_ls_archive_statusdir()inarchiver_pending_countmetric; num_doneandslru_writtencolumns are added tocheckpointermetric;parallel_workers_to_launchandparallel_workers_launchedcolumns are added todb_statsmetric.
Grafana Dashboards
- New Grafana v12 dashboards have been added for both PostgreSQL and Prometheus sinks;
- Grafana v10 support is discontinued;
- New "Global Database Overview" dashboard with 26 panels, including new panels for replication, connections, and index usage (inspired by postgres.ai);
- New "Database Overview" now has 21 panels with time lag support and improved visualizations;
- New "Query Performance Analysis" dashboard with enhanced table with 17 metrics and 8 visualization panels (inspired by postgres.ai);
- New "Tables Overview" dashboard providing insights into table sizes, bloat, and index usage with treemap visualizations panels.
Metrics
- Realtime metrics have been deprecated and removed from the system.
- Metrics can now be loaded from a specified folder, allowing for better organization and management of metric definitions.
Sinks
- gRPC Sink Enhancements: Basic authentication support has been added to the gRPC sink, along with improved documentation for its usage.
Development and Contribution
- The development experience with Docker Compose has been enhanced, making it easier to set up and manage the environment.
- A new repository, pgwatch-contrib, has been created to host additional contributions and extensions for pgwatch from the community. The
rpcsubdirectory contains sample gRPC sinks implementations.
Changelog
- [!] add Grafana v12 dashboards for Prometheus data source by @pashagolub in #933
- [!] add support for PostgreSQL v18 by @pashagolub in #903
- [!] deprecate realtime metrics by @pashagolub in #925
- [!] remove Grafana v10 dashboards due to EOL of Grafana v10 by @pashagolub in #934
- [!] remove obsolete
db_stats_aurorametric by @pashagolub in #930 - [+] add "Query Performance Analysis" dashboard by @pashagolub in #916
- [+] add "Tables Overview" dashboard by @pashagolub in #918
- [+] add basic auth support to grpc sink by @0xgouda in #862
- [+] add docs for grpc sink by @0xgouda in #867
- [+] add PlantUML diagrams by @pashagolub in #926
- [+] add sample metrics YAML file, closes #883 by @pashagolub in #900
- [+] allow metrics loading from folder, closes #882 by @0xgouda in #889
- [+] bump
actions/checkoutfrom 4 to 5 by @dependabot[bot] in #888 - [+] bump
actions/download-artifactfrom 4 to 5 by @dependabot[bot] in #881 - [+] bump
actions/setup-gofrom 5 to 6 by @dependabot[bot] in #939 - [+] bump
actions/setup-nodefrom 4 to 5 by @dependabot[bot] in #936 - [+] bump
actions/setup-pythonfrom 5 to 6 by @dependabot[bot] in #938 - [+] bump
actions/stalefrom 9 to 10 by @dependabot[bot] in #937 - [+] bump
docker/dockerfrom 28.3.2 to 28.3.3 by @dependabot[bot] in #872 - [+] bump
go.etcd.io/etcd/client/v3from 3.6.2 to 3.6.4 by @dependabot[bot] in #858 #868 - [+] bump
golang-jwt/jwt/v5from 5.2.3 to 5.3.0 by @dependabot[bot] in #876 - [+] bump
google.golang.org/grpcfrom 1.73.0 to 1.75.0 by @dependabot[bot] in #865 #902 - [+] bump
google.golang.org/protobuffrom 1.36.6 to 1.36.8 by @dependabot[bot] in #887 #904 - [+] bump
prometheus/client_golangfrom 1.22.0 to 1.23.2 by @dependabot[bot] in #878 #941 - [+] bump
shirou/gopsutil/v4from 4.25.6 to 4.25.8 by @dependabot[bot] in #877 #923 - [+] bump
stretchr/testifyfrom 1.10.0 to 1.11.1 by @dependabot[bot] in #906 #912 - [+] bump
webui/react-scriptsto v5.0.1 (pinned version) by @pashagolub in #866 - [+] improve "Global Database Overview" dashboard by @pashagolub in #869
- [+] improve
change_eventsmetric processing by @pashagolub in #932 - [+] improve development experience with docker compose by @pashagolub in #927
- [+] update docs with new v12 dashboards screenshots by @pashagolub in #943
- [*] add note in sizing docs about metrics returning thousands of rows by @0xgouda in #874
- [*] clarify the usage of docker compose, closes #879 by @pashagolub in #880
- [*] fix
updateSources()query parameters number by @0xgouda in #875 - [*] mention
pgwatch-contrib/rpcin docs by @0xgouda in #922 - [*] misc minor updates to docs by @0xgouda in #871
- [*] remove hard coded metric defs from docs by @0xgouda in #870
- [*] update docs for metrics yaml folder configs by @0xgouda in #892
- [-] do not fail on db resolve error, just log an error, fixes #890 by @pashagolub in #899
- [-] fix source YAML test introduced by #900 by @pashagolub in #901
- [-] move to next envelope in Postgres sink if
COPYfails, fixes #908 by @pashagolub in #911 - [-] remove
host_configquerying fromGetSources()by @0xgouda in #863 - [-] remove double "git checkout" from "README.md" by @smiyc in #942
New Contributors
Full Changelog: v3.7.0...v4.0.0-beta
v3.7.0
What's Changed
[-]monitor all sources if--groupcmdopt is omitted, fixes #843 (#847)[*]improve stat-statements Grafana v11 Postgres dashboards by @f9n in #846
Full Changelog: v3.6.0...v3.7.0
What to do next?
- Star the repository
- Read the documentation
- Check out the blog
- Follow us on Linkedin
v3.6.0
What's Changed
[+]add "default" group processing by @pashagolub in #798[+]addmeasurements writtenlog message to RPC sink by @0xgouda in #829[+]add TLS encryption to RPC channel by @0xgouda in #796[+]bumpgo.etcd.io/etcd/v3from 3.6.0 to 3.6.2 by @dependabot[bot] in #789 #837[+]bumppashagolub/pgxmock/v4from 4.7.0 to 4.8.0 by @dependabot[bot] in #806[+]bumpshirou/gopsutil/v4from 4.25.5 to 4.25.6 by @dependabot[bot] in #827[+]bumptestcontainers/testcontainers-gofrom 0.37.0 to 0.38.0 by @dependabot[bot] in #839[+]optimize the size ofMeasurementEnvelopeby @pashagolub in #793[*]excludeMetricDeffromMeasurementEnvelopeby @pashagolub in #787[*]improve--batching-delayparameter description, closes #821 by @pashagolub in #828[-]fix--groupcommand-line option, closes #792 by @pashagolub in #794[-]fixinstance_upmetric, closes #759 by @pashagolub in #791[-]fixmaster onlypostgres source shutdown due to catchedInRecoverystate logic by @0xgouda in #813[-]fixReap()method to update Recovery cache after processing the previous value by @0xgouda in #815[-]fixReap()to not requireSource.MinDbSizeMbto be >= 8 by @0xgouda in #818[-]fixSource.Equal()presets comparison, closes #830 by @0xgouda in #831[-]fix typo inreapMetricMeasurements()log message by @0xgouda in #807[-]fix JSONB null casting errors inpostgres/v11dashboards by @f9n in #838[-]fix JSONB null casting errors in "Top Tables" and "Health-check" dashboards by @kmoppel in #833[-]improve Prometheus scrapping, fixes #749 by @pashagolub in #790[-]remove unnecessary check fromFetchMetric()by @0xgouda in #808
New Contributors
Full Changelog: v3.5.0...v3.6.0
What to do next?
- Star the repository
- Read the documentation
- Check out the blog
- Follow us on Linkedin
v3.5.0
What's Changed
[+]addpprofbuild profile for debugging by @pashagolub in #779[+]bumpshirou/gopsutil/v4from 4.25.4 to 4.25.5 by @dependabot in #774[+]decrease memory allocations formetrics.MeasurementEnvelopeby @pashagolub in #768[+]define sync metric operations types as constants by @0xgouda in #781 #782[+]move sources sync to the reaper by @pashagolub in #769[+]rewriteParseLogs()with timers and eliminate extra checks by @pashagolub in #770[+]switch tojson-iterator/gofromencoding/jsonby @pashagolub in #776[*]change json sink to use one global encoder by @0xgouda in #772[*]remove metrics definitions from Postgres sink by @pashagolub in #773[*]usepgx.Rowsdirectly inDetectConfigurationChanges()by @pashagolub in #775[-]add additional checks toLogParse()by @pashagolub in #784[-]remove misleadingVOLUME /var/lib/postgresqldeclaration by @pashagolub in #780
New Contributors
Full Changelog: v3.4.0...v3.5.0
What to do next?
- Star the repository
- Read the documentation
- Check out the blog
- Follow us on Linkedin
v3.4.0
What's Changed
[+]bumpjackc/pgx/v5from 5.7.4 to 5.7.5 by @dependabot in #763[+]bumpgo.etcd.io/etcd/v3from 3.5.21 to 3.6.0 by @dependabot in #762[*]use consistent password in install docs by @df7cb in #765[-]fixdbSourcesReaderWriter.updateDatabase()to includeis_enabledby @pashagolub in #760[-]fixQueryMeasurementsnil pointer dereference, fixes #766 by @pashagolub in #767
New Contributors
Full Changelog: v3.3.0...v3.4.0
What to do next?
- Star the repository
- Read the documentation
- Check out the blog
- Follow us on Linkedin
v3.3.0
What's Changed
[+]addpgwatch source resolvecommand by @pashagolub in #723[+]add a SystemD service template by @kmoppel in #704[+]add pgpool service to docker compose files by @pashagolub in #739[+]add replication node to docker compose file by @pashagolub in #734[+]add tests forpgwatch metric printcommands by @pashagolub in #708[+]add tests forconvert_metricsutility by @pashagolub in #744[+]add tests forpgwatchby @pashagolub in #748[+]add tests forwebserverby @pashagolub in #754[+]add tests for etcd-based Patroni clusters resolver by @pashagolub in #751[+]bumpcybertec-postgresql/pgx-migratorfrom 1.0.2 to 1.2.0 by @dependabot in #695 #727[+]bumpgo.etcd.io/etcd/client/v3from 3.5.20 to 3.5.21 by @dependabot in #700[+]bumpgolang-jwt/jwtto v5.2.2 by @pashagolub in #717[+]bumpgolangci/golangci-lint-actionfrom 6 to 7 by @dependabot in #690[+]bumphttp-proxy-middlewarefrom 2.0.7 to 2.0.9 by @dependabot in #742[+]bumppashagolub/pgxmock/v4from 4.6.0 to 4.7.0 by @dependabot in #726[+]bumpprometheus/client_golangfrom 1.21.1 to 1.22.0 by @dependabot in #715[+]bumpshirou/gopsutil/v4from 4.25.2 to 4.25.4 by @dependabot in #705 #753[+]bumptestcontainers/testcontainers-gofrom 0.35.0 to 0.37.0 by @dependabot in #701 #746[+]bump Go to v1.24 by @pashagolub in #698[+]improvesourcestest coverage by @pashagolub in #721[+]use newInstanceMetricCacheimplementation in theReaperby @pashagolub in #716[+]use new concurrent metric definitions implementation by @pashagolub in #709[-]fixpgwatcg source pingcommand by @pashagolub in #718[-]fix anonymous Grafana access in Docker, fixes #724 by @iuuuuan in #725[-]fix duplicated definition of symboldlopenin #758[-]fix growing measurement cache introduced in #716 by @pashagolub in #737[-]fix log message for rows written inJSONWriterby @pashagolub in #738[-]fix--metricscommand-option description, fixes #756 in #757[-]fix Pgpool-II support, closes #730 by @pashagolub in #740[*]deprecateIsSuperusersource option by @pashagolub in #706[*]drop support for discontinued Azure Single Server by @pashagolub in #736[*]fix hard codedepoch_nsmeasurement field name by @pashagolub in #712[*]fix oldarchives.format_overrides.formatgoreleaser option by @pashagolub in #694[*]movepsutilfunctionality toreaperpackage by @pashagolub in #714[*]refactorTryCreateMetricsFetchingHelpersby @pashagolub in #743[*]remove unuseddb.GetTableColumns()by @pashagolub in #707[*]replaceMetricFetchConfigwithSourceConninFetchMetrics()by @pashagolub in #741
New Contributors
Full Changelog: v3.2.0...v3.3.0
What to do next?
- Star the repository
- Read the documentation
- Check out the blog
- Follow us on Linkedin
v3.2.0
What's Changed
[!]refactorreaperpackage andsinks.MultiWriterby @pashagolub in #667[+]addReaper.WriteMonitoredSources()andQueryMeasurents()by @pashagolub in #678[+]add migration instructions for configuration database, closes #654 by @pashagolub in #655[+]allow to add EDB Postgres under monitoring, closes #649 by @pashagolub in #650[+]bump@babel/helpersfrom 7.20.13 to 7.26.10 by @dependabot in #669[+]bump@babel/runtimefrom 7.20.13 to 7.26.10 by @dependabot in #666[+]bumpaxiosfrom 1.7.4 to 1.8.2 by @dependabot in #668[+]bumpgo.etcd.io/etcd/client/v3from 3.5.18 to 3.5.20 by @dependabot in #662 #687[+]bumpjackc/pgx/v5from 5.7.2 to 5.7.4 by @dependabot in #688 #691[+]bumppashagolub/pgxmock/v4from 4.5.0 to 4.6.0 by @dependabot in #683[+]bumpprometheus/client_golangfrom 1.20.5 to 1.21.1 by @dependabot in #656 #660[+]bumpshirou/gopsutil/v4from 4.25.1 to 4.25.2 by @dependabot in #659[+]install Grafana withaptin Docker and bump to v10.4.15 by @pashagolub in #647[*]fix typo in documentation by @mookie- in #686[*]remove deprecated reaper constants by @pashagolub in #670[*]remove obsolete unreachable map by @pashagolub in #672[*]renamesources.MonitoredDatabasetoSourceConnby @pashagolub in #671[*]replace configMap arguments with interval by @pashagolub in #674[*]updating non-functional links in docs by @shubham-j-sde in #681[*]use dark style logo by @pashagolub in #651[*]use logging source field for log parsing instead of fmt modifiers by @pashagolub in #658[-]fix adding new metric to configuration db via web UI, fixes #664 by @pashagolub in #665[-]fix connection issues for discovered quoted db names, fixes #675 by @pashagolub in #676[-]fix datasource uid for Grafana v11 dashboards by @pashagolub in #689[-]fix duplicate object key inpostgres/v11/system-stats.jsonby @AjayPawarExist in #685[-]fix linter warning "impossible condition: nil != nil" by @pashagolub in #652[-]removepg_stat_file()dependencies from metrics by @pashagolub in #677[-]remove obsoleteundersizedDBsandrecoveryIgnoredDBsmaps by @pashagolub in #673
New Contributors
- @mookie- made their first contribution in #686
- @shubham-j-sde made their first contribution in #681
- @AjayPawarExist made their first contribution in #685
Full Changelog: v3.1.0...v3.2.0
What to do next?
- Star the repository
- Read the documentation
- Check out the blog
- Follow us on Linkedin
v3.1.0
What's Changed
[+]refactorconfigcommand and sub-commands by @pashagolub in #621[+]persist WebUI visible column settings, fixes #581 by @ay-bh in #635[+]make fallback logger use default options by @pashagolub in #620[+]improvepgwatch-demoDocker image by @pashagolub in #643[+]improvedocker composefiles by @pashagolub in #642[+]fail early if Prometheus sink cannot be served by @pashagolub in #627[+]bumpjackc/pgx/v5from 5.7.1 to 5.7.2 by @dependabot in #608[+]bumptestcontainers/testcontainers-gofrom 0.34.0 to 0.35.0 by @dependabot in #617[+]bumpshirou/gopsutil/v4from 4.24.11 to 4.25.1 by @dependabot in #609 #632[+]bumppashagolub/pgxmock/v4from 4.4.0 to 4.5.0 by @dependabot in #640[+]bumpgo.etcd.io/etcd/client/v3from 3.5.17 to 3.5.18 by @dependabot in #628[+]add packagesdoc.gofiles by @pashagolub in #623[+]add/readinessand/livenessendpoints, closes #611 by @pashagolub in #615[*]renormalize line endings by @pashagolub in #604[*]Prometheus Grafana v11 dashboard updates by @kmoppel in #626[*]mask password in the source connection string, fixes #610 by @tvecerakb in #618[*]align text fields to the left in WebUI grids by @pashagolub in #619[*]addinit_sqlfor metrics with external dependencies by @pashagolub in #641[-]remove passwords from sink log messages, fixes #612 by @pashagolub in #614[-]fix updating metric definition in database configuration, fixes #644 by @pashagolub in #645[-]fix duplicate source names in YAML configs, fixes #630 by @amrdb in #633[-]fix concatenated output ofpgwatch metric print-initby @agelwarg in #631[-]fix checkpointer dashboards for PostgreSQL 17, fixes #613 by @ay-bh in #636[-]fixPostgresWriter.EnsureMetricTime(), fixes #622 by @pashagolub in #624[-]fix "invalid command -- ping" for PgBouncer source, closes #625 by @pashagolub in #639
New Contributors
- @tvecerakb made their first contribution in #618
- @agelwarg made their first contribution in #631
- @amrdb made their first contribution in #633
- @kmoppel made their first contribution in #626
- @ay-bh made their first contribution in #635
Full Changelog: v3.0.0...v3.1.0
What to do next?
- Star the repository
- Read the documentation
- Check out the blog
- Follow us on Linkedin
3.0.0
What's New
This major release introduces several significant updates, new features, bug fixes, and enhancements:
Important Changes
- Documentation: The documentation has been updated and moved to https://pgwat.ch for improved readability and navigation. Special part dedicated to the pgwatch development has been added.
- Parallel Sinks: Support for simultaneous storing metric measurements to several destinations, e.g. database and file at once.
- Custom Storage: Leverage our remote sink (RPC) interface to seamlessly integrate any storage format or data stream, such as Parquet files or Kafka, directly into pgwatch.
- Windows Support Added: Windows support has been integrated, broadening the compatibility of the software.
- Docker Images Rename: The
pgwatch2-postgresDocker image has been renamed topgwatch-demo,pgwatch2-daemontopgwatch. - Enhanced Logging: An advanced logging system has been implemented to improve error tracking and monitoring.
- Grafana Upgrade: Support for the latest Grafana v9.x and v10 has been added, including a switch to Grafana's REST API for dashboard provisioning.
- Etcd v3 Migration: The system has been upgraded to Etcd v3, ensuring better performance and compatibility.
- PostgreSQL 17 Metrics: New metrics for PostgreSQL 17 have been introduced.
- Automatic Schema Rollout: A new automatic schema rollout mechanism has been implemented, streamlining database management.
- Build and Test Workflow: A new "Build and Test" workflow has been added to automate testing and building processes.
- WebUI Enhancements: The WebUI now includes options like test connections, a log tab, and a clone button for the database grid.
- REST API: New REST API endpoints have been added, including endpoints such as
source,/metric,/preset,log. - Docker Improvements: The Docker images and related scripts have been improved, including the introduction of a new Dockerfile based on the official Postgres image and Docker Compose support.
- Test Coverage: Test coverage statistics are now available to ensure code quality and reliability.
- Environment Variables Update: The
PW2_environment variable prefixes have been updated toPW3_. - Code Refactoring: The
main.gofile has been streamlined by extracting functionality into separate packages. - Deprecations: Removed support for InfluxDB, Graphite datastore, and other outdated features.
Dependency Updates
Numerous dependencies have been updated, including upgrades to Golang, PostgreSQL, Prometheus, and various Go modules. These upgrades ensure better performance, security, and compatibility with other tools.
This release represents a significant step forward in functionality, performance, and security, with a strong focus on expanding compatibility and improving the user experience.
Changelog
[!]addMeasurementsgroup and use URIs for--sinkoptions by @pashagolub in #368[!]add Grafana v11 dashboards by @muni106 in #407[!]add metrics for PostgreSQL 16 by @pashagolub in #246[!]add RPC sink support, closes #464 by @destrex271 in #465[!]add support forjackc/pgx, closes #171 by @pashagolub in #204[!]add support for application commands by @pashagolub in #504[!]add support for log tab in web UI, closes #170 by @pashagolub in #206[!]add support for PostgreSQL v17 by @pashagolub in #497[!]add support for the latest Grafana v9.x, closes #174 by @pashagolub in #186[!]add windows support, closes #3 by @pashagolub in #4[!]allow multiple parallel metric storages by @pashagolub in #313[!]bump Grafana to v10, closes #227 by @pashagolub in #232[!]decouplesourcespackage frommain, closes #357 by @pashagolub in #358[!]decouplemetricsinto separate package, closes #372 by @pashagolub in #408[!]differentiate the concepts of source and monitored database by @pashagolub in #472[!]extractpsutilinto separate package by @pashagolub in #144[!]extract configuration handling into separate package, closes #138 by @pashagolub in #139[!]fix Go linting, closes #142 by @pashagolub in #143[!]implement automatic schema rollout machinery, closes #180 by @pashagolub in #225[!]implement enhanced logging by @pashagolub in #146[!]implement new Dockerfile based on official Postgres image, closes #210 by @pashagolub in #226[!]implement upgrade machinery, closes #179 by @pashagolub in #451[!]implement webui authentication, closes #216 by @denys-holub in #251[!]move measurements reaper code out of main package, closes #418 by @pashagolub in #419[!]move metric batching to sinks from the gatherer by @pashagolub in #346[!]renamepgwatch3-postgresDocker image topgwatch3, closes #20 by @pashagolub in #21[!]restructure code tree, closes #518 by @pashagolub in #521[!]restructure Docker related assets by @pashagolub in #535[!]switch configuration to connection string from separate parameters, closes #255 by @pashagolub in #322[!]switch to Grafana REST API for dashboard provisioning, closes #188 by @pashagolub in #189[!]transformSphinxmanual tomkdocs-materialby @pashagolub in #476[!]upgrade to Etcd v3, resolves #59 by @pashagolub in #157[+]add "Build and Test" workflow, closes #5 by @pashagolub in #6[+]add/metricendpoint, closes #93 by @pashagolub in #113[+]addarm64binaries to release assets, closes #512 by @pashagolub in #517[+]addconvertapp to migrate v2 metrics definitions to v3 by @pashagolub in #373[+]addpgbouncer_clientsmetric by @pashagolub in #440[+]addPreset configstab, closes #135 by @denys-holub in #136[+]addRPCSyncRequesttoapiby @pashagolub in #528[+]addusenametoreplicationmetric by @pashagolub in #444[+]add bootstrap tests by @pashagolub in #277[+]add built-in http server, closes #28 by @pashagolub in #30[+]add clone button to database grid, closes #150 by @denys-holub in #159[+]add commandsconfig [init|upgrade], closes #514 by @pashagolub in #515[+]add concurrency clause to Build GHA workflow by @pashagolub in #269[+]add Docker Compose file, closes #240 by @pashagolub in #242[+]add icon and logo for web ui, closes #66 by @denys-holub in #67[+]add log tab to the main window, closes #68 by @denys-holub in #71[+]add main metrics descriptions, closes #547 by @pashagolub in #548[+]add metrics tests by @pashagolub in #478[+]add new metrics for invalid and unused indexes by @pashagolub in #286[+]add pgAdmin4 todocker-compose.ymlby @ondar in #432[+]add Prometheus service to Docker Compose file by @pashagolub in #314[+]add support for/PATCH presetendpoint, closes #163 by @pashagolub in #187[+]add support forGET /presetendpoint, closes #132 by @pashagolub in #134[+]add support forPOST /presetendpoint by @pashagolub in #177[+]add test cases to webserver api by @muni106 in #397[+]add test coverage statistics, closes #273 by @pashagolub in #274[+]add tests fordbpackage by @pashagolub in #463[+]add tests forsinksby @pashagolub in #449[+]add tests for postgressources.ReaderWriterimplementation by @pashagolub in #469[+]add tests for YAMLsources.ReaderWriterimplementation by @pashagolub in #470[+]add TimescaleDB metric storage based docker image, resolves #176 by @pashagolub in #183[+]add trigger onmetricto update presets, closes #185 by @pashagolub in #370[+]add web UI start options, closes #236 by @pashagolub in #237[+]allow CORS fromhttp://localhost:4000for webui debugging by @pashagolub in #439[+]bumpaxiosfrom 1.6.0 to 1.7.4 by @dependabot in #522[+]bumpexpressfrom 4.19.2 to 4.21.0 in webui by @dependabot in #544[+]bumpjackc/pgx/v5from 5.6.0 to 5.7.1 by @dependabot in #543[+]bumpmicromatchfrom 4.0.5 to 4.0.8 by @dependabot in #520[+]bumpprometheus/client_golangfrom 1.20.1 to 1.20.4 by @dependabot in #519 #539 #545[+]bumprollupfrom 2.79.1 to 2.79.2 in webui by @dependabot in #550[+]bumpshirou/gopsutil/v4from 4.24.7 to 4.24.9 by @dependabot in #538 #551[+]bumpyaml.v2toyaml.v3, closes #359 by @pashagolub in #360[+]bumpwebpackfrom 5.76.1 to 5.94.0 by @dependabot in #537[+]bump Go to v1.23 by @pashagolub in #555[+]create common error & loading components, closes #125 by @denys-holub in #126[+]get rid of control channel map, closes #434 by @pashagolub in #435[+]highlight active WebUI tab, closes #199 by @denys-holub in #200[+]implementGET /metric&DELETE /metricto web UI, closes #116 by @denys-holub in #117[+]implementPOST /metric&PATCH /metricto web UI, closes #119 by @denys-holub in #120[+]improveBuild & TestGHA workflow by @pashagolub in #481[+]improve Developer Documentation by @pashagolub in #487[+]improve Docker images and related scripts by @pashagolub in #158[+]improve Docker workflow by @pashagolub in #122[+]include webui into pgwatch3 binary, resolves #74 by @pashagolub in #75[+]makepgwatch2-daemonDocker image usealpinebase image, closes #23 by @pashagolub in #25[+]move helpers to metrics definition by @pashagolub in #374[+]produce "latest" and semver tags for all Docker images by @pashagolub in #556[+]refactorsinksas a standalone package by @pashagolub in #345[+]refactor metric definitions handling by @pashagolub in #342[+]replace checkboxes in monitoring grid with switches, closes #160 by @denys-holub in #164[+]sync changes from pgwatch2...