WPB-26487 backend background jobs hasql resource pool refactoring#5323
Conversation
There was a problem hiding this comment.
Pull request overview
This PR refactors the PostgreSQL connection pooling across multiple services/libraries by switching from hasql-pool to hasql-resource-pool, introducing a new Hasql.Pool.Extended.Pool wrapper that carries metrics and a rawPool accessor for code paths that still need the underlying pool (notably migrations).
Changes:
- Replace
hasql-poolwithhasql-resource-poolacross Cabal and Nix, including pin/override updates. - Update service/library effect stacks to use
Hasql.Pool.Extended.Pool(andrawPoolwhere a rawHasql.Pool.Poolis required). - Rework pool metrics collection (session/acquisition latency histograms + periodic pool stats snapshots).
Reviewed changes
Copilot reviewed 36 out of 36 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| services/galley/src/Galley/Run.hs | Use rawPool for migrations; adjust imports. |
| services/galley/src/Galley/Env.hs | Switch env pool type import to Hasql.Pool.Extended. |
| services/galley/src/Galley/App.hs | Update Polysemy Input pool type to the extended wrapper. |
| services/galley/galley.cabal | Swap dependency from hasql-pool to hasql-resource-pool. |
| services/galley/default.nix | Swap Nix dependency from hasql-pool to hasql-resource-pool. |
| services/brig/src/Brig/Run.hs | Use rawPool for migrations and in migration command. |
| services/brig/src/Brig/Index/Eval.hs | Introduce PostgresUsageException wrapper and updated interpreter for UsageError. |
| services/brig/src/Brig/CanonicalInterpreter.hs | Update Polysemy Input pool type to the extended wrapper. |
| services/brig/src/Brig/App.hs | Move Brig to Hasql.Pool.Extended import/aliasing. |
| services/brig/default.nix | Swap Nix dependency from hasql-pool to hasql-resource-pool. |
| services/brig/brig.cabal | Swap dependency from hasql-pool to hasql-resource-pool. |
| services/background-worker/src/Wire/Effects.hs | Update Polysemy Input pool type and wiring to the extended wrapper. |
| services/background-worker/src/Wire/BackgroundWorker/Env.hs | Update pool imports/aliases to Hasql.Pool.Extended. |
| services/background-worker/default.nix | Swap Nix dependency from hasql-pool to hasql-resource-pool. |
| services/background-worker/background-worker.cabal | Swap dependency from hasql-pool to hasql-resource-pool. |
| nix/manual-overrides.nix | Replace override of hasql-pool with hasql-resource-pool. |
| nix/haskell-pins.nix | Add pin for hasql-resource-pool. |
| libs/wire-subsystems/wire-subsystems.cabal | Swap dependency from hasql-pool to hasql-resource-pool. |
| libs/wire-subsystems/src/Wire/UserGroupStore/Postgres.hs | Replace explicit Hasql pool constraints with PGConstraints. |
| libs/wire-subsystems/src/Wire/TeamFeatureStore/Migration.hs | Switch to extended pool module aliasing. |
| libs/wire-subsystems/src/Wire/TeamCollaboratorsStore/Postgres.hs | Replace explicit Hasql pool constraints with PGConstraints. |
| libs/wire-subsystems/src/Wire/PostgresMigrations.hs | Wrap UsageError into PostgresMigrationError for throwing. |
| libs/wire-subsystems/src/Wire/Postgres.hs | Update pool usage and add observer-based metrics recording. |
| libs/wire-subsystems/src/Wire/MigrationLock.hs | Pull rawPool from the extended pool wrapper for lock sessions. |
| libs/wire-subsystems/src/Wire/MeetingsStore/Postgres.hs | Use runSession/PGConstraints instead of manual pool plumbing. |
| libs/wire-subsystems/src/Wire/Error.hs | Update UsageError pattern matching for new constructors. |
| libs/wire-subsystems/src/Wire/DomainRegistrationStore/Migration.hs | Switch to extended pool module aliasing. |
| libs/wire-subsystems/src/Wire/ConversationStore/Postgres.hs | Replace explicit Hasql pool constraints with PGConstraints. |
| libs/wire-subsystems/src/Wire/ConversationStore/Migration.hs | Switch to extended pool module aliasing. |
| libs/wire-subsystems/src/Wire/CodeStore/Migration.hs | Switch to extended pool module aliasing. |
| libs/wire-subsystems/src/Wire/AppStore/Postgres.hs | Replace explicit Hasql pool constraints with PGConstraints. |
| libs/wire-subsystems/default.nix | Swap Nix dependency from hasql-pool to hasql-resource-pool. |
| libs/extended/src/Hasql/Pool/Extended.hs | Reimplement pooling/metrics on top of hasql-resource-pool. |
| libs/extended/extended.cabal | Swap dependency from hasql-pool to hasql-resource-pool and remove uuid. |
| libs/extended/default.nix | Swap Nix dependency from hasql-pool to hasql-resource-pool and remove uuid. |
| changelog.d/0-release-notes/WPB-25325 | Add release note describing the pool backend switch and metrics changes. |
Co-authored-by: Copilot Autofix powered by AI <[email protected]>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 44 out of 45 changed files in this pull request and generated 4 comments.
Comments suppressed due to low confidence (1)
services/brig/src/Brig/Index/Options.hs:376
- This PR removes the
--pg-pool-aging-timeoutCLI option, butmake es-reindexstill passes it (seeMakefile:430). After this change, that make target will fail with an “unknown option” error. Either update the Makefile/docs to stop using the flag, or keep a deprecated alias flag that is accepted (and ignored) for backward compatibility.
akshaymankar
left a comment
There was a problem hiding this comment.
Overall looks good, minor changes requested.
The issues pointed out are resolved. Since Akshay is not available, this was approved by someone else.
https://wearezeta.atlassian.net/browse/WPB-26487
related to https://wearezeta.atlassian.net/browse/WPB-25325
Checklist
changelog.d