Skip to content

WPB-26489 backend adminless scheduled jobs model for deletion and reminder#5289

Open
battermann wants to merge 62 commits into
developfrom
WPB-26489-backend-adminless-scheduled-jobs-model-and-teardown-support
Open

WPB-26489 backend adminless scheduled jobs model for deletion and reminder#5289
battermann wants to merge 62 commits into
developfrom
WPB-26489-backend-adminless-scheduled-jobs-model-and-teardown-support

Conversation

@battermann

@battermann battermann commented Jun 25, 2026

Copy link
Copy Markdown
Contributor

https://wearezeta.atlassian.net/browse/WPB-26489

Checklist

  • Add a new entry in an appropriate subdirectory of changelog.d
  • Read and follow the PR guidelines

@zebot zebot added the ok-to-test Approved for running tests in CI, overrides not-ok-to-test if both labels exist label Jun 25, 2026
@battermann battermann changed the base branch from WPB-26487-backend-scheduled-jobs-http-control-plane-using-arbiter to develop June 29, 2026 10:13
@battermann battermann force-pushed the WPB-26489-backend-adminless-scheduled-jobs-model-and-teardown-support branch from 7737422 to 4e704c1 Compare June 29, 2026 12:28
@battermann battermann marked this pull request as ready for review June 29, 2026 12:30
@battermann battermann requested review from a team as code owners June 29, 2026 12:30
@battermann battermann requested a review from Copilot June 29, 2026 12:52

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces an Arbiter-backed scheduled-jobs subsystem across services, migrating the meetings-cleanup worker to run via Arbiter and adding initial “adminless” reminder/deletion scheduled jobs, along with an internal /i/jobs UI/API mount in Brig.

Changes:

  • Add Wire.API.Jobs, Wire.JobSubsystem, and Wire.JobStore to model/schedule jobs and persist minimal app-level metadata in Postgres.
  • Replace the standalone meetings cleanup cron thread with an Arbiter worker pool (scheduledJobs.pollInterval controls polling cadence).
  • Expose Arbiter’s internal UI + REST API through Brig’s internal routes under /i/jobs and /i/jobs/api/v1.

Reviewed changes

Copilot reviewed 56 out of 57 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
services/galley/src/Galley/Env.hs Add Galley env field for passing a connection string into the job subsystem.
services/galley/src/Galley/App.hs Wire JobSubsystem + JobStore interpreters and derive a Postgres connection string for Arbiter usage.
services/galley/galley.cabal Add arbiter-core dependency for Galley.
services/galley/default.nix Add arbiter-core Nix dependency for Galley.
services/brig/src/Brig/Run.hs Adjust internal API wiring to pass env into IAPI.servantSitemap.
services/brig/src/Brig/App.hs Initialize embedded Arbiter server/UI application and store it in Brig env.
services/brig/src/Brig/API/Internal.hs Mount internal /i/jobs Raw app into Brig internal routes.
services/brig/default.nix Add Arbiter packages (arbiter-core, arbiter-servant, arbiter-servant-ui).
services/brig/brig.cabal Add Arbiter packages to Brig build-depends.
services/background-worker/test/Test/Wire/Util.hs Extend test env with arbiterConnStr placeholder.
services/background-worker/test/Test/Wire/BackendNotificationPusherSpec.hs Extend test env with arbiterConnStr placeholder.
services/background-worker/src/Wire/MeetingsCleanupWorker.hs Refactor to expose runCleanupOldMeetings instead of owning the cron thread; move metrics increment into cleanup routine.
services/background-worker/src/Wire/Effects.hs Add JobSubsystem/JobStore effects + interpreters and extend migration handling.
services/background-worker/src/Wire/BackgroundWorker/ScheduledJobs.hs New scheduled-jobs dispatcher/worker wiring for meetings cleanup + adminless jobs via Arbiter.
services/background-worker/src/Wire/BackgroundWorker/Options.hs Add scheduledJobs config section (poll interval).
services/background-worker/src/Wire/BackgroundWorker/Env.hs Add arbiterConnStr to runtime env, derived from Postgres settings.
services/background-worker/src/Wire/BackgroundWorker.hs Start the unified scheduled-jobs worker instead of the old meetings-cleanup worker thread.
services/background-worker/src/Wire/AdminlessJobsWorker.hs New adminless reminder/deletion job handlers + scheduled job cleanup in app-level store.
services/background-worker/default.nix Add arbiter-core dependency for background-worker.
services/background-worker/background-worker.integration.yaml Add integration config for scheduledJobs.pollInterval.
services/background-worker/background-worker.cabal Expose new modules and add arbiter-core dependency.
postgres-schema.sql Update schema snapshot to include scheduled jobs table and related DB objects.
nix/manual-overrides.nix Add overrides/jailbreaks for Arbiter package set.
nix/haskell-pins.nix Add Arbiter source pin and package mapping.
libs/wire-subsystems/wire-subsystems.cabal Add Arbiter deps and expose new job subsystem/store modules; add cron dep.
libs/wire-subsystems/test/unit/Wire/ConversationSubsystem/InterpreterSpec.hs Add a JobSubsystem mock interpreter for conversation subsystem unit tests.
libs/wire-subsystems/src/Wire/Options/Keys.hs Fix ASN.1 imports to use crypton-asn1-* packages explicitly.
libs/wire-subsystems/src/Wire/JobSubsystem/Workers.hs New Arbiter worker runner helpers (recurring + one-off) with explicit defaults.
libs/wire-subsystems/src/Wire/JobSubsystem/Interpreter.hs New Polysemy interpreter to schedule jobs into Arbiter and start workers.
libs/wire-subsystems/src/Wire/JobSubsystem.hs New Polysemy effect and configuration types for scheduling/starting jobs.
libs/wire-subsystems/src/Wire/JobStore/Postgres.hs New Postgres-backed interpreter for persisting minimal scheduled job metadata.
libs/wire-subsystems/src/Wire/JobStore.hs New Polysemy effect for scheduled job metadata persistence.
libs/wire-subsystems/src/Wire/ConversationSubsystem/Update.hs Schedule adminless reminder/deletion jobs when last admin leaves; add reminder event emission.
libs/wire-subsystems/src/Wire/ConversationSubsystem/Interpreter.hs Add internal actions for adminless deletion/reminder execution paths.
libs/wire-subsystems/src/Wire/ConversationSubsystem/Action.hs Factor conversation deletion into reusable removeConversation.
libs/wire-subsystems/src/Wire/ConversationSubsystem.hs Add new internal conversation actions for adminless job execution.
libs/wire-subsystems/postgres-migrations/20260624095944-extend-job-model.sql Add scheduled_jobs table + indexes migration.
libs/wire-subsystems/default.nix Add Arbiter and cron dependencies for wire-subsystems.
libs/wire-api/wire-api.cabal Expose new Jobs API modules in wire-api.
libs/wire-api/test/unit/Test/Wire/API/Conversation.hs Extend event tests for new ConvAdminlessReminder type classification.
libs/wire-api/src/Wire/API/Team/FeatureFlags.hs Accept both legacy and current JSON keys for search visibility defaults.
libs/wire-api/src/Wire/API/Routes/Internal/Jobs.hs New internal route type for mounting the embedded jobs app.
libs/wire-api/src/Wire/API/Routes/Internal/Brig.hs Add Jobs app route into Brig internal API type.
libs/wire-api/src/Wire/API/Jobs.hs New job payload types, queue names, and registry definition for Arbiter.
libs/wire-api/src/Wire/API/Event/Conversation.hs Add ConvAdminlessReminder event type and corresponding event payload schema.
libs/types-common/src/Data/Id.hs Introduce ScheduledJobId and ScheduledJob id tag.
libs/extended/src/Hasql/Pool/Extended.hs Add shared postgresqlConnectionString helper and refactor pool init to use it.
integration/test/Test/AdminlessGroups.hs Add integration coverage for adminless scheduling and Arbiter execution path.
integration/test/Notifications.hs Add helper to detect adminless reminder notifications.
flake.nix Add Arbiter git input.
flake.lock Lock Arbiter input revision/hash.
docs/src/developer/reference/config-options.md Document scheduledJobs.pollInterval.
charts/wire-server/values.yaml Add Helm value for background-worker.config.scheduledJobs.pollInterval.
charts/wire-server/templates/background-worker/configmap.yaml Render scheduledJobs config into background-worker configmap.
changelog.d/2-features/WPB-26489 Feature changelog entry for Arbiter-backed scheduled jobs + adminless jobs.
changelog.d/0-release-notes/WPB-26489 Release notes for Arbiter scheduling and /i/jobs endpoints.
cabal.project Add source-repository-package entries for Arbiter packages.

Comment thread libs/extended/src/Hasql/Pool/Extended.hs Outdated
Comment thread libs/types-common/src/Data/Id.hs
Comment thread integration/test/Test/AdminlessGroups.hs Outdated
Comment thread integration/test/Test/AdminlessGroups.hs Outdated
Comment thread services/galley/src/Galley/Env.hs Outdated

@akshaymankar akshaymankar left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Haven't read everything, comments based on just trying to understand the arbiter stuff.

Comment thread cabal.project Outdated
Comment thread services/brig/src/Brig/App.hs Outdated
Comment thread services/brig/src/Brig/API/Internal.hs Outdated
Comment thread services/background-worker/src/Wire/Effects.hs Outdated
Comment thread services/background-worker/src/Wire/Effects.hs Outdated
Comment thread libs/wire-api/src/Wire/API/Routes/Internal/Jobs.hs Outdated
@battermann battermann force-pushed the WPB-26489-backend-adminless-scheduled-jobs-model-and-teardown-support branch 2 times, most recently from 25c5413 to c9ed418 Compare July 6, 2026 07:42
@battermann battermann changed the title WPB-26489 backend adminless scheduled jobs model and teardown support WPB-26489 backend adminless scheduled jobs model for deletion and reminder Jul 6, 2026
@battermann battermann force-pushed the WPB-26489-backend-adminless-scheduled-jobs-model-and-teardown-support branch 2 times, most recently from 5e49c76 to 9d7fbf5 Compare July 9, 2026 14:27
@battermann battermann changed the base branch from develop to WPB-25325-backend-background-jobs-hasql-resource-pool-refactoring July 9, 2026 14:28
@battermann battermann force-pushed the WPB-26489-backend-adminless-scheduled-jobs-model-and-teardown-support branch from f0df3e3 to 33c7c37 Compare July 9, 2026 16:03
@battermann battermann requested a review from Copilot July 9, 2026 16:03

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 50 out of 51 changed files in this pull request and generated 6 comments.

Comment thread services/background-worker/src/Wire/AdminlessJobsWorker.hs
Comment thread libs/wire-subsystems/src/Wire/JobSubsystem/ArbiterAdapter.hs
Comment thread libs/wire-subsystems/src/Wire/JobSubsystem/ArbiterAdapter.hs
Comment thread libs/wire-subsystems/src/Wire/JobSubsystem/ArbiterAdapter.hs
Comment thread libs/extended/src/Hasql/Pool/Extended.hs Outdated
Comment thread services/background-worker/src/Wire/BackgroundWorker/Options.hs
@battermann battermann force-pushed the WPB-26489-backend-adminless-scheduled-jobs-model-and-teardown-support branch from 33c7c37 to 35ea9ff Compare July 10, 2026 08:51
Base automatically changed from WPB-25325-backend-background-jobs-hasql-resource-pool-refactoring to develop July 10, 2026 12:37
@battermann battermann force-pushed the WPB-26489-backend-adminless-scheduled-jobs-model-and-teardown-support branch from dbb2409 to d4d78b4 Compare July 10, 2026 13:14
@battermann battermann requested a review from Copilot July 10, 2026 13:23

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 52 out of 53 changed files in this pull request and generated 2 comments.

Comment thread libs/wire-subsystems/src/Wire/JobSubsystem/Workers.hs Outdated
Comment thread libs/wire-subsystems/src/Wire/JobSubsystem/Interpreter.hs

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 53 out of 54 changed files in this pull request and generated 2 comments.

Comment thread services/background-worker/src/Wire/BackgroundWorker/Options.hs
Comment thread libs/wire-subsystems/src/Wire/ConversationSubsystem/Update.hs Outdated
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ok-to-test Approved for running tests in CI, overrides not-ok-to-test if both labels exist

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants