[!] add schema migration support for PostgreSQL sinks, closes #1110 (#1111)

- Add `admin.migration` table and migration infrastructure to sink
- Implement `Migrate()` and `NeedsMigration()` for `PostgresWriter` and
`MultiWriter`
- Add migration checks during sink initialization with error messages
- Extend `config init/upgrade` commands to support sink migrations
- `NewWriterFromPostgresConn` now returns nil writer on errors
- Add comprehensive test coverage for migration happy/unhappy paths
- Update docs to document sink database migration support

Migration pattern follows existing metrics database approach, ensuring
both config and sink databases can be upgraded independently using the
same `pgwatch config upgrade` command.

Author: pashagolub

docs/howto/config_db_bootstrap.md

@@ -50,6 +50,12 @@ Or you can use the `config init` command with the `--metrics` flag, since metric
 pgwatch --metrics=postgresql://pgwatch:pgwatchadmin@localhost/pgwatch config init
 ```
 
+If you're using a PostgreSQL sink for storing measurements, you can also initialize the sink database schema:
+
+```terminal
+pgwatch --sink=postgresql://pgwatch:pgwatchadmin@localhost/measurements config init
+```
+
 ## Usage
 
 You can now configure pgwatch to use the `pgwatch` database as the configuration database for storing monitored sources,

docs/reference/cli_env.md

@@ -8,7 +8,7 @@ title: Command-Line Options & Environment Variables
   pgwatch [OPTIONS] [config | metric | source]
 ```
 
-When no command is specified, pgwatch starts the monitoring process. 
+When no command is specified, pgwatch starts the monitoring process.
 It reads the configuration from the specified sources and metrics, then begins collecting measurements from the resolved databases.
 
 ## Options
@@ -194,16 +194,19 @@ It reads the configuration from the specified sources and metrics, then begins c
 ```
 
 !!! info
-    To use `config` command, you need to specify the `-s`, `--sources` and\or `-m`, `--metrics` options.
+    To use `config` command, you need to specify at least one of the following: `-s`, `--sources`, `-m`, `--metrics`, or `--sink` options.
 
 - `init`
 
-    Initialize the configuration database with the required tables and functions. If file is used, it will
-    be created in the specified location and filled with built-in defaults.
+    Initialize the configuration and/or sink database with the required tables and functions. If file is used, it will
+    be created in the specified location and filled with built-in defaults. Works with PostgreSQL-based sources,
+    metrics, and sink databases.
 
 - `upgrade`
 
-    Upgrade the database to the latest version. File or folder based configurations are not supported yet.
+    Upgrade the configuration and/or sink database to the latest version by applying all pending migrations.
+    File or folder based configurations are not supported. The command will automatically detect which
+    databases (sources, metrics, sinks) require migrations and apply them.
 
 ### Manage metrics
 

docs/tutorial/upgrading.md

@@ -89,8 +89,8 @@ There are no update or migration scripts for the built-in Grafana
 dashboards as it would break possible user applied changes. If you know
 that there are no user changes, then one can just delete or rename the
 existing ones in bulk and import the latest JSON definitions.
-See [here](../concept/long_term_installations.md) for
-some more advice on how to manage dashboards.
+See [some more advice](../concept/long_term_installations.md) on how to
+manage dashboards.
 
 ### Updating the config / metrics DB version
 
@@ -105,31 +105,38 @@ problem-free, consisting of running something like:
     sudo systemctl restart postgresql
 
 For PostgreSQL major version upgrades one should read through the
-according release notes (e.g.
-[here](https://www.postgresql.org/docs/17/release-17.html#id-1.11.6.5.4))
+according [release notes](https://www.postgresql.org/docs/current/release.html)
 and be prepared for the unavoidable downtime.
 
 ### Updating the pgwatch schema
 
 This is the pgwatch specific part, with some coupling between the
-following components - Configuration DB SQL schema and pgwatch binary.
+following components - Configuration DB SQL schema,
+Sink DB SQL schema (if using PostgreSQL sink), and pgwatch binary.
 
 First of all, the pgwatch binary needs to be updated to a newer version.
 Then try to run the pgwatch as usual:
 
     pgwatch --sources=postgresql://pgwatch:pgwatchadmin@localhost/pgwatch --sink=postgresql://pgwatch:pgwatchadmin@localhost/pgwatch_metrics
 
-    [ERROR] configuration needs upgrade, use "config upgrade" command
+    [ERROR] config database schema is outdated, please run migrations using `pgwatch config upgrade` command
     exit status 4
 
 If you see the above error message, then the pgwatch schema needs updating.
 This is done by running the following command, which will apply all
-the necessary SQL migrations to the configuration database:
+the necessary SQL migrations to the configuration database and sink database:
 
-    pgwatch --sources=postgresql://pgwatch:pgwatchadmin@localhost/pgwatch config upgrade
+    pgwatch --sources=postgresql://pgwatch:pgwatchadmin@localhost/pgwatch --sink=postgresql://pgwatch:pgwatchadmin@localhost/pgwatch_metrics config upgrade
 
+    [INFO] Applying migration to config database...
     [INFO] Applying migration named '00824 Refactor recommendations'...
     [INFO] Applied migration named '00824 Refactor recommendations'
+    [INFO] Applying migration to sink database...
+    [INFO] All migrations applied successfully
+
+!!! info
+    The `config upgrade` command will automatically detect which databases (sources, metrics, sinks) need migrations and apply them.
+    You only need to provide the connection strings for the databases you're using.
 
 ### Updating the metrics collector
 

go.mod

@@ -3,7 +3,7 @@ module github.com/cybertec-postgresql/pgwatch/v3
 go 1.25.0
 
 require (
-	github.com/cybertec-postgresql/pgx-migrator v1.2.0
+	github.com/cybertec-postgresql/pgx-migrator v1.3.0
 	github.com/golang-jwt/jwt/v5 v5.3.0
 	github.com/gorilla/websocket v1.5.3
 	github.com/jackc/pgx/v5 v5.8.0
@@ -54,23 +54,23 @@ require (
 	github.com/gogo/protobuf v1.3.2 // indirect
 	github.com/golang/protobuf v1.5.4 // indirect
 	github.com/google/uuid v1.6.0 // indirect
-	github.com/grpc-ecosystem/grpc-gateway/v2 v2.27.3 // indirect
+	github.com/grpc-ecosystem/grpc-gateway/v2 v2.27.4 // indirect
 	github.com/jackc/pgpassfile v1.0.0 // indirect
 	github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761 // indirect
 	github.com/jackc/puddle/v2 v2.2.2 // indirect
-	github.com/klauspost/compress v1.18.1 // indirect
+	github.com/klauspost/compress v1.18.2 // indirect
 	github.com/lufia/plan9stats v0.0.0-20251013123823-9fd1530e3ec3 // indirect
 	github.com/magiconair/properties v1.8.10 // indirect
 	github.com/moby/docker-image-spec v1.3.1 // indirect
-	github.com/moby/go-archive v0.1.0 // indirect
+	github.com/moby/go-archive v0.2.0 // indirect
 	github.com/moby/patternmatcher v0.6.0 // indirect
 	github.com/moby/sys/sequential v0.6.0 // indirect
 	github.com/moby/sys/user v0.4.0 // indirect
 	github.com/moby/sys/userns v0.1.0 // indirect
 	github.com/moby/term v0.5.2 // indirect
 	github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
 	github.com/modern-go/reflect2 v1.0.2 // indirect
-	github.com/morikuni/aec v1.0.0 // indirect
+	github.com/morikuni/aec v1.1.0 // indirect
 	github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect
 	github.com/opencontainers/go-digest v1.0.0 // indirect
 	github.com/opencontainers/image-spec v1.1.1 // indirect
@@ -86,20 +86,20 @@ require (
 	go.etcd.io/etcd/api/v3 v3.6.7 // indirect
 	go.etcd.io/etcd/client/pkg/v3 v3.6.7 // indirect
 	go.opentelemetry.io/auto/sdk v1.2.1 // indirect
-	go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.63.0 // indirect
-	go.opentelemetry.io/otel v1.38.0 // indirect
+	go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.64.0 // indirect
+	go.opentelemetry.io/otel v1.39.0 // indirect
 	go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.37.0 // indirect
-	go.opentelemetry.io/otel/metric v1.38.0 // indirect
-	go.opentelemetry.io/otel/trace v1.38.0 // indirect
+	go.opentelemetry.io/otel/metric v1.39.0 // indirect
+	go.opentelemetry.io/otel/trace v1.39.0 // indirect
 	go.opentelemetry.io/proto/otlp v1.7.0 // indirect
 	go.uber.org/multierr v1.11.0 // indirect
 	go.yaml.in/yaml/v2 v2.4.3 // indirect
-	golang.org/x/crypto v0.45.0 // indirect
-	golang.org/x/net v0.47.0 // indirect
-	golang.org/x/sync v0.18.0 // indirect
-	golang.org/x/sys v0.38.0 // indirect
-	golang.org/x/text v0.31.0 // indirect
+	golang.org/x/crypto v0.46.0 // indirect
+	golang.org/x/net v0.48.0 // indirect
+	golang.org/x/sync v0.19.0 // indirect
+	golang.org/x/sys v0.39.0 // indirect
+	golang.org/x/text v0.32.0 // indirect
 	golang.org/x/time v0.12.0 // indirect
-	google.golang.org/genproto/googleapis/api v0.0.0-20251111163417-95abcf5c77ba // indirect
-	google.golang.org/genproto/googleapis/rpc v0.0.0-20251111163417-95abcf5c77ba // indirect
+	google.golang.org/genproto/googleapis/api v0.0.0-20251222181119-0a764e51fe1b // indirect
+	google.golang.org/genproto/googleapis/rpc v0.0.0-20251222181119-0a764e51fe1b // indirect
 )