Skip to content

[!] add schema migration support for PostgreSQL sinks, closes #1110 #1111

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
pashagolub merged 3 commits into from
Jan 5, 2026
Merged

[!] add schema migration support for PostgreSQL sinks, closes #1110 #1111

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
364f139
[!] add schema migration support for PostgreSQL sinks, closes #1110
pashagolub Jan 4, 2026
b8e65f8
add more tests
pashagolub Jan 4, 2026
3ff0653
bump `cybertec-postgresql/pgx-migrator` to v1.3.0
pashagolub Jan 5, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions docs/howto/config_db_bootstrap.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,12 @@ Or you can use the `config init` command with the `--metrics` flag, since metric
pgwatch --metrics=postgresql://pgwatch:pgwatchadmin@localhost/pgwatch config init
```

If you're using a PostgreSQL sink for storing measurements, you can also initialize the sink database schema:

```terminal
pgwatch --sink=postgresql://pgwatch:pgwatchadmin@localhost/measurements config init
```

## Usage

You can now configure pgwatch to use the `pgwatch` database as the configuration database for storing monitored sources,
Expand Down
13 changes: 8 additions & 5 deletions docs/reference/cli_env.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ title: Command-Line Options & Environment Variables
pgwatch [OPTIONS] [config | metric | source]
```

When no command is specified, pgwatch starts the monitoring process.
When no command is specified, pgwatch starts the monitoring process.
It reads the configuration from the specified sources and metrics, then begins collecting measurements from the resolved databases.

## Options
Expand Down Expand Up @@ -194,16 +194,19 @@ It reads the configuration from the specified sources and metrics, then begins c
```

!!! info
To use `config` command, you need to specify the `-s`, `--sources` and\or `-m`, `--metrics` options.
To use `config` command, you need to specify at least one of the following: `-s`, `--sources`, `-m`, `--metrics`, or `--sink` options.

- `init`

Initialize the configuration database with the required tables and functions. If file is used, it will
be created in the specified location and filled with built-in defaults.
Initialize the configuration and/or sink database with the required tables and functions. If file is used, it will
be created in the specified location and filled with built-in defaults. Works with PostgreSQL-based sources,
metrics, and sink databases.

- `upgrade`

Upgrade the database to the latest version. File or folder based configurations are not supported yet.
Upgrade the configuration and/or sink database to the latest version by applying all pending migrations.
File or folder based configurations are not supported. The command will automatically detect which
databases (sources, metrics, sinks) require migrations and apply them.

### Manage metrics

Expand Down
23 changes: 15 additions & 8 deletions docs/tutorial/upgrading.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -89,8 +89,8 @@ There are no update or migration scripts for the built-in Grafana
dashboards as it would break possible user applied changes. If you know
that there are no user changes, then one can just delete or rename the
existing ones in bulk and import the latest JSON definitions.
See [here](../concept/long_term_installations.md) for
some more advice on how to manage dashboards.
See [some more advice](../concept/long_term_installations.md) on how to
manage dashboards.

### Updating the config / metrics DB version

Expand All @@ -105,31 +105,38 @@ problem-free, consisting of running something like:
sudo systemctl restart postgresql

For PostgreSQL major version upgrades one should read through the
according release notes (e.g.
[here](https://www.postgresql.org/docs/17/release-17.html#id-1.11.6.5.4))
according [release notes](https://www.postgresql.org/docs/current/release.html)
and be prepared for the unavoidable downtime.

### Updating the pgwatch schema

This is the pgwatch specific part, with some coupling between the
following components - Configuration DB SQL schema and pgwatch binary.
following components - Configuration DB SQL schema,
Sink DB SQL schema (if using PostgreSQL sink), and pgwatch binary.

First of all, the pgwatch binary needs to be updated to a newer version.
Then try to run the pgwatch as usual:

pgwatch --sources=postgresql://pgwatch:pgwatchadmin@localhost/pgwatch --sink=postgresql://pgwatch:pgwatchadmin@localhost/pgwatch_metrics

[ERROR] configuration needs upgrade, use "config upgrade" command
[ERROR] config database schema is outdated, please run migrations using `pgwatch config upgrade` command
exit status 4

If you see the above error message, then the pgwatch schema needs updating.
This is done by running the following command, which will apply all
the necessary SQL migrations to the configuration database:
the necessary SQL migrations to the configuration database and sink database:

pgwatch --sources=postgresql://pgwatch:pgwatchadmin@localhost/pgwatch config upgrade
pgwatch --sources=postgresql://pgwatch:pgwatchadmin@localhost/pgwatch --sink=postgresql://pgwatch:pgwatchadmin@localhost/pgwatch_metrics config upgrade

[INFO] Applying migration to config database...
[INFO] Applying migration named '00824 Refactor recommendations'...
[INFO] Applied migration named '00824 Refactor recommendations'
[INFO] Applying migration to sink database...
[INFO] All migrations applied successfully

!!! info
The `config upgrade` command will automatically detect which databases (sources, metrics, sinks) need migrations and apply them.
You only need to provide the connection strings for the databases you're using.

### Updating the metrics collector

Expand Down
32 changes: 16 additions & 16 deletions go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ module github.com/cybertec-postgresql/pgwatch/v3
go 1.25.0

require (
github.com/cybertec-postgresql/pgx-migrator v1.2.0
github.com/cybertec-postgresql/pgx-migrator v1.3.0
github.com/golang-jwt/jwt/v5 v5.3.0
github.com/gorilla/websocket v1.5.3
github.com/jackc/pgx/v5 v5.8.0
Expand Down Expand Up @@ -54,23 +54,23 @@ require (
github.com/gogo/protobuf v1.3.2 // indirect
github.com/golang/protobuf v1.5.4 // indirect
github.com/google/uuid v1.6.0 // indirect
github.com/grpc-ecosystem/grpc-gateway/v2 v2.27.3 // indirect
github.com/grpc-ecosystem/grpc-gateway/v2 v2.27.4 // indirect
github.com/jackc/pgpassfile v1.0.0 // indirect
github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761 // indirect
github.com/jackc/puddle/v2 v2.2.2 // indirect
github.com/klauspost/compress v1.18.1 // indirect
github.com/klauspost/compress v1.18.2 // indirect
github.com/lufia/plan9stats v0.0.0-20251013123823-9fd1530e3ec3 // indirect
github.com/magiconair/properties v1.8.10 // indirect
github.com/moby/docker-image-spec v1.3.1 // indirect
github.com/moby/go-archive v0.1.0 // indirect
github.com/moby/go-archive v0.2.0 // indirect
github.com/moby/patternmatcher v0.6.0 // indirect
github.com/moby/sys/sequential v0.6.0 // indirect
github.com/moby/sys/user v0.4.0 // indirect
github.com/moby/sys/userns v0.1.0 // indirect
github.com/moby/term v0.5.2 // indirect
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
github.com/modern-go/reflect2 v1.0.2 // indirect
github.com/morikuni/aec v1.0.0 // indirect
github.com/morikuni/aec v1.1.0 // indirect
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect
github.com/opencontainers/go-digest v1.0.0 // indirect
github.com/opencontainers/image-spec v1.1.1 // indirect
Expand All @@ -86,20 +86,20 @@ require (
go.etcd.io/etcd/api/v3 v3.6.7 // indirect
go.etcd.io/etcd/client/pkg/v3 v3.6.7 // indirect
go.opentelemetry.io/auto/sdk v1.2.1 // indirect
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.63.0 // indirect
go.opentelemetry.io/otel v1.38.0 // indirect
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.64.0 // indirect
go.opentelemetry.io/otel v1.39.0 // indirect
go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.37.0 // indirect
go.opentelemetry.io/otel/metric v1.38.0 // indirect
go.opentelemetry.io/otel/trace v1.38.0 // indirect
go.opentelemetry.io/otel/metric v1.39.0 // indirect
go.opentelemetry.io/otel/trace v1.39.0 // indirect
go.opentelemetry.io/proto/otlp v1.7.0 // indirect
go.uber.org/multierr v1.11.0 // indirect
go.yaml.in/yaml/v2 v2.4.3 // indirect
golang.org/x/crypto v0.45.0 // indirect
golang.org/x/net v0.47.0 // indirect
golang.org/x/sync v0.18.0 // indirect
golang.org/x/sys v0.38.0 // indirect
golang.org/x/text v0.31.0 // indirect
golang.org/x/crypto v0.46.0 // indirect
golang.org/x/net v0.48.0 // indirect
golang.org/x/sync v0.19.0 // indirect
golang.org/x/sys v0.39.0 // indirect
golang.org/x/text v0.32.0 // indirect
golang.org/x/time v0.12.0 // indirect
google.golang.org/genproto/googleapis/api v0.0.0-20251111163417-95abcf5c77ba // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20251111163417-95abcf5c77ba // indirect
google.golang.org/genproto/googleapis/api v0.0.0-20251222181119-0a764e51fe1b // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20251222181119-0a764e51fe1b // indirect
)
Loading
Loading