Skip to content

feat(data-retention): add automated pruning for audit logs and analyt…#931

Open
Adeyemi-cmd wants to merge 1 commit into
rinafcode:mainfrom
Adeyemi-cmd:Add_data_retention
Open

feat(data-retention): add automated pruning for audit logs and analyt…#931
Adeyemi-cmd wants to merge 1 commit into
rinafcode:mainfrom
Adeyemi-cmd:Add_data_retention

Conversation

@Adeyemi-cmd

Copy link
Copy Markdown

Close #848

PR Description

Summary

GDPR Article 5(1)(e) requires data to not be kept longer than necessary.
Audit logs and analytics events were accumulating indefinitely with no
automated pruning, degrading query performance and creating compliance risk.

Changes

Environment Variables

  • AUDIT_LOG_RETENTION_DAYS — audit log retention period (default 730 = 2 years)
  • ANALYTICS_RETENTION_DAYS — analytics event retention period (default 365 = 1 year)
  • Both added to .env.example, env.validation.ts (Joi), and retention.config.ts

AuditRetentionTask (src/audit-log/tasks/audit-retention.task.ts)

  • Rewritten to read AUDIT_LOG_RETENTION_DAYS from config
  • Deletes expired rows in batches of 1000 (do...while loop with .limit())
  • Increments deleted_count{table="audit_logs"} Prometheus counter per run
  • Retains daily 2 AM cron schedule and weekly report handler

AnalyticsRetentionTask (src/analytics/tasks/analytics-retention.task.ts) [NEW]

  • Reads ANALYTICS_RETENTION_DAYS from config (default 365)
  • Deletes AnalyticsEvent rows where timestamp < cutoff in batches of 1000
  • Runs daily at 2:30 AM (staggered from audit retention to avoid DB contention)
  • Increments deleted_count{table="analytics_events"} Prometheus counter

@drips-wave

drips-wave Bot commented Jun 28, 2026

Copy link
Copy Markdown

@Adeyemi-cmd Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits.

You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀

Learn more about application limits

@RUKAYAT-CODER

Copy link
Copy Markdown
Contributor

Great work on this PR
I really appreciate the effort you put into this—everything
kindly fix your workflow

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add data retention policy automation for audit logs and analytics events

2 participants