Enhance `lsbackup` with Date Filtering and Lightweight Listing

[shiva-istari]

Problem

Listing backups via dgraph lsbackup or the listBackups GraphQL query requires parsing the full master manifest (manifest.json), which includes the groups map containing every predicate in every group. For clusters with vector indexes (HNSW), each vector predicate generates three supporting predicates (entry, keyword, dead), causing the manifest to grow to 500MB+.

This makes day-to-day backup listing extremely slow and memory-intensive, even when the operator only needs basic metadata (backup ID, timestamp, type, location).

There is also no way to filter backups by date the tool always returns every backup ever taken at the location.

Current Behavior

CLI (dgraph lsbackup -l <path>):

• Only flags: --location (required), --verbose
• Reads and fully deserialises the entire master manifest.json into memory
• Returns all backups with no filtering capability

GraphQL (listBackups query):

• Only inputs: location, credential fields, anonymous
• Same full manifest deserialization, no date filtering

Manifest struct (per backup entry):

{
  "type": "full|incremental",
  "since": 2280,
  "read_ts": 2281,
  "groups": { "1": ["predicate1", "predicate2", ...thousands more...] },
  "backup_id": "focused-borg",
  "backup_num": 1,
  "path": "dgraph.20260414.103045.123",
  "encrypted": false,
  "drop_operations": [...],
  "compression": "snappy"
}

The groups field is by far the largest and is not needed for listing.

Proposed Enhancements

1. Date-Range Filtering (CLI + GraphQL)

Backup directory paths already embed a parseable UTC timestamp:

dgraph.20260414.103045.123  →  2026-04-14 10:30:45.123 UTC

Format: dgraph. + time.Format("20060102.150405.000")

New CLI flags:

Flag	Type	Description
--since-date	string	Only show backups on or after this date (YYYY-MM-DD or RFC3339)
--until-date	string	Only show backups on or before this date
--last-n-days	int	Shorthand: show backups from the last N days

New GraphQL ListBackupsInput fields:

Field	Type	Description
sinceDate	DateTime	Filter backups on or after this date
untilDate	DateTime	Filter backups on or before this date

2. Summary Statistics

Add a --summary flag (or always print to stderr) showing at-a-glance backup health info:

Backup Summary:
  Total backups:              47
  Backup series:               3
  Oldest backup:              2026-01-15 08:00:12 UTC (full)
  Newest backup:              2026-04-14 10:30:45 UTC (incremental)
  Last full backup:           2026-04-01 00:00:05 UTC
  Last incremental backup:    2026-04-14 10:30:45 UTC

3. Lightweight Summary Manifest (Exclude groups)

The groups field in each manifest entry maps group IDs to their full list of predicates. This is the primary cause of the 500MB+ manifest size every backup snapshot repeats the entire predicate list for every group, and vector indexes (HNSW) multiply the count by 3x with supporting predicates (__entry, __keyword, __dead).

The groups field is internal to Dgraph's restore process — users never need it when listing backups. The same applies to drop_operations.

Proposal: Write a separate manifest_summary.json alongside the main manifest.json during CompleteBackup. This summary contains one entry per backup with groups excluded only the fields needed for listing:

{
  "Manifests": [
    {
      "path": "dgraph.20260101.000005.000",
      "type": "full",
      "backup_id": "focused-borg",
      "backup_num": 1,
      "encrypted": false,
      "read_ts": 1000,
      "since": 0
    },
    {
      "path": "dgraph.20260102.060000.000",
      "type": "incremental",
      "backup_id": "focused-borg",
      "backup_num": 2,
      "encrypted": false,
      "read_ts": 2000,
      "since": 1000
    }
  ]
}

The full manifest.json (with groups and drop_operations) remains untouched for restore. The listing code (lsbackup CLI and listBackups GraphQL) reads manifest_summary.json instead, keeping listing fast regardless of cluster size.

[matthewmcneely]

This looks like a great improvement. @raphael-istari can you have a look?

[amalistari]

Putting this in queue for @shiva-istari to pick up next after #9680

[raphael-istari]

Thanks @shiva-istari that aligns with customers request to "provide an alternate API to list backups by date".