Skip to content

feat (backup): improve lsbackup tool#9693

Merged
matthewmcneely merged 2 commits intomainfrom
shiva/ls-backup
Apr 29, 2026
Merged

feat (backup): improve lsbackup tool#9693
matthewmcneely merged 2 commits intomainfrom
shiva/ls-backup

Conversation

@shiva-istari
Copy link
Copy Markdown
Contributor

@shiva-istari shiva-istari commented Apr 27, 2026

Fixes #9684

Improve lsbackup Tool

User-Facing Changes

CLI: dgraph lsbackup

Flag Type Description
--since-date string Only show backups on or after this date (YYYY-MM-DD or RFC3339)
--until-date string Only show backups on or before this date (YYYY-MM-DD or RFC3339)
--last-n-days int Only show backups from the last N calendar days (mutually exclusive with --since-date)
--summary bool Print summary statistics (total count, series count, oldest/newest, last full/incremental) to stderr
--verbose bool Behavior changed — now reads the full manifest.json to include predicate groups and drop operations. Without it, the lighter manifest_summary.json is used.

Example usage:

# List backups from the last 7 days with summary stats
dgraph lsbackup --location /data/backups --last-n-days 7 --summary

# List backups in a date range
dgraph lsbackup --location /data/backups --since-date 2026-01-01 --until-date 2026-03-31

# Include full group/predicate detail
dgraph lsbackup --location /data/backups --verbose

GraphQL: listBackups Query

New optional input fields on the listBackups admin query:

Field Type Description
sinceDate String Only return backups on or after this date
untilDate String Only return backups on or before this date
lastNDays Int Only return backups from the last N days (mutually exclusive with sinceDate)
fullManifest Boolean When true, reads the full manifest including predicate groups and drop ops. Default false.

Example query:

query {
  listBackups(input: {
    location: "/data/backups"
    sinceDate: "2026-01-01"
    untilDate: "2026-03-31"
    fullManifest: true
  }) {
    backupId backupNum type path encrypted since
    groups { groupId predicates }
  }
}

Backup Output: New manifest_summary.json

Every backup now writes a manifest_summary.json alongside manifest.json. This is a lightweight version that omits predicate groups and drop operations, making listing operations faster for large backup histories. The full manifest.json remains unchanged and is still used for restore.


Internal / Developer Changes

New types in worker package:

  • ManifestBase — shared fields between full and summary manifests
  • MasterManifestSummary — lightweight manifest container (no groups/drop ops)
  • BackupDateFilter — date-range filter with Since and Until pointers
  • BackupListStats — summary statistics struct

New functions in worker package:

  • FilterManifestsByDate() — filters manifests by a BackupDateFilter
  • ComputeBackupListStats() — computes aggregate stats over a manifest list

Signature changes:

  • ListBackupManifests() — new forceFull bool parameter (3rd arg)
  • ProcessListBackups() — new forceFull bool parameter (4th arg)

Files Changed

File Change
worker/backup_manifest.go Core types, filtering, stats, summary read/write
worker/backup.go Write manifest_summary.json during backup; updated signatures
worker/backup_handler.go Updated ProcessListBackups signature
backup/run.go New CLI flags, filter/summary logic
graphql/admin/list_backups.go GraphQL resolver wired with filter + fullManifest
graphql/admin/endpoints.go New GraphQL schema fields

New Test Files

File Coverage
backup/run_test.go Date parsing, filter construction
graphql/admin/list_backups_test.go GraphQL date filter, manifest conversion
worker/backup_manifest_test.go FilterManifestsByDate, ComputeBackupListStats, summary serialization
systest/backup/filesystem/backup_test.go E2E: summary manifest creation, GraphQL filter matrix, input validation
systest/integration2/backup_summary_compat_test.go Backward compat when manifest_summary.json is missing

@shiva-istari shiva-istari marked this pull request as ready for review April 27, 2026 10:37
@shiva-istari shiva-istari requested a review from a team as a code owner April 27, 2026 10:37
@github-actions github-actions Bot added area/testing Testing related issues area/graphql Issues related to GraphQL support on Dgraph. area/backup Related to binary backups area/core internal mechanisms go Pull requests that update Go code labels Apr 27, 2026
@blacksmith-sh

This comment has been minimized.

@matthewmcneely matthewmcneely changed the title feat (backup): imporve lsbackup tool feat (backup): improve lsbackup tool Apr 27, 2026
Comment thread graphql/admin/list_backups.go Outdated
Comment thread worker/backup.go Outdated
Comment thread worker/backup.go
@matthewmcneely
Copy link
Copy Markdown
Contributor

@shiva-istari Also, don't forget to open a PR in the dgraph-docs repos, lots of changes to both the cli and the api here.

@matthewmcneely matthewmcneely merged commit 76c508a into main Apr 29, 2026
26 checks passed
@matthewmcneely matthewmcneely deleted the shiva/ls-backup branch April 29, 2026 19:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/backup Related to binary backups area/core internal mechanisms area/graphql Issues related to GraphQL support on Dgraph. area/testing Testing related issues go Pull requests that update Go code

Development

Successfully merging this pull request may close these issues.

Enhance lsbackup with Date Filtering and Lightweight Listing

2 participants