Skip to content

Background worker health status not exposed as metrics #962

Description

@hman38705

Labels: monitoring, reliability

Priority: High

Description

Background workers (email queue worker, blockchain sync worker, rate limit cleanup, newsletter cleanup) run as tokio tasks with no health metrics. Operators cannot determine from Prometheus whether any worker is running, stuck, or has crashed.

Acceptance Criteria

  • Add a Prometheus gauge worker_status{name} (1 = running, 0 = stopped) for each background worker
  • Update the gauge in each worker loop heartbeat and clear it on panic/exit
  • Add a Prometheus alert firing when any worker_status drops to 0 for more than 60 seconds

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

Stellar WaveIssues in the Stellar wave programmonitoringMetrics, alerting, and dashboardsreliabilityResilience, recovery, and uptime

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions