Skip to content

[ENHANCEMENT] support folder quota #22

Description

@Vicente-Cheng

Folder quota would be a key feature to support k8s volumes.


Implementation Plan

Context

Arctic Wolf NFS server needs folder quota support for Kubernetes PVC size enforcement. Each PVC maps to a first-level subdirectory under the export root, and the quota limits the total byte usage of that directory.

Design Decisions

  • Granularity: First-level subdirectory quota (one per PVC)
  • Quota dimension: Bytes only (file/inode count is reported from the real filesystem but not enforced)
  • Calculation: Tracking-based (in-memory counters + redb persistence)
  • Error code: NFS3ERR_DQUOT (69)
  • Persistence: redb embedded KV store (crash-safe ACID, zero external dependencies, pure Rust)
  • Startup strategy: Trust stored values, start serving immediately, reconcile in background
  • FSSTAT: Bytes come from quota (or libc::statvfs when no quota); files always come from libc::statvfs
  • Tracked operations: write, remove, setattr_size (truncate down only), rename (cross-quota-dir only)
    • Not tracked: create, mkdir, rmdir, symlink, mknod, setattr_size extend (sparse file, write will catch real usage)

Implementation Stages

Each stage is an independent commit that builds and passes tests on its own.


Stage 1: Config infrastructure

Commit: config: add quota configuration and size parsing helper

  • Cargo.toml: add redb = \"3\" dependency
  • src/config.rs:
    • Add QuotaConfig struct (enabled: bool, db_path: PathBuf)
    • Add pub quota: QuotaConfig to Config
    • Add parse_size() helper (supports B, KB, MB, GB, TB)
    • Unit tests for parse_size variants and TOML parsing

Verification: make build, make test


Stage 2: QuotaManager core module

Commit: fsal: add QuotaManager with redb persistence

  • src/fsal/quota.rs (new):
    • QuotaManager struct (redb database + in-memory cache)
    • QuotaEntry struct (limit, usage)
    • redb table definition (key = first-level dir name, value = (limit_bytes, usage_bytes))
    • Methods: new, resolve_quota_dir, check_quota, add_usage, sub_usage, set_quota, remove_quota, get_quota_info
    • Unit tests: CRUD, redb persistence across reopen, boundary checks
  • src/fsal/mod.rs: add pub mod quota;

Verification: make test; redb file can be created and reopened


Stage 3: statvfs trait method (without quota wiring yet)

Commit: fsal: add statvfs trait method with real filesystem stats

  • src/fsal/mod.rs:
    • Add FsStats struct (bytes + files fields)
    • Add async fn statvfs() to Filesystem trait
  • src/fsal/local/mod.rs: implement statvfs() using libc::statvfs
  • src/nfs/fsstat.rs: replace hardcoded values with filesystem.statvfs()
  • Unit tests

Verification: FSSTAT returns real filesystem stats; df shows accurate values


Stage 4: Wire QuotaManager into LocalFilesystem

Commit: fsal: integrate QuotaManager into LocalFilesystem

  • src/fsal/local/mod.rs:
    • Add quota_manager: Option<QuotaManager> field
    • new() accepts Option<&QuotaConfig>
    • Update statvfs(): for paths inside a quota directory, return quota-based bytes with real file counts; fall back to libc::statvfs otherwise
  • src/fsal/mod.rs: BackendConfig carries optional quota config
  • src/main.rs: pass &config.quota to the backend
  • Tests

Verification: Server starts and loads the quota DB; FSSTAT inside a quota directory returns quota-based values


Stage 5: Quota enforcement + NFS error mapping

Commit: fsal: enforce folder quota on write/remove/truncate/rename

  • src/fsal/local/mod.rs: add quota checks and usage updates:
    • write(): check max(0, offset + len - current_size) before write; add actual delta after
    • remove(): capture file size before delete; subtract after
    • setattr_size(): if truncating down, subtract the delta (extend is not tracked)
    • rename(): if crossing quota directories, check target and transfer usage
  • NFS handler error mapping (\"Quota exceeded\"NFS3ERR_DQUOT):
    • src/nfs/write.rs
    • src/nfs/setattr.rs
    • src/nfs/rename.rs
  • Unit tests: over-limit rejection, space freeing, cross-quota move

Verification: make test; manual test confirms DQUOT is returned on over-limit writes


Stage 6: Background reconciliation

Commit: fsal: add background quota reconciliation task

  • src/fsal/quota.rs: add scan_and_reconcile() (per directory) and reconcile_all()
  • src/fsal/local/mod.rs: add start_quota_reconciliation() that spawns a background task
  • src/main.rs: call after server startup, without blocking service
  • Tests

Verification: Server logs show reconciliation progress; out-of-band filesystem changes are corrected after the next scan


Stage 7: Integration tests

Commit: test: add NFS folder quota integration tests

  • tests/test_nfs_quota.py:
    • WRITE up to the limit → assert NFS3ERR_DQUOT (69)
    • REMOVE a file → subsequent WRITE succeeds
    • FSSTAT returns quota-based byte values
    • Truncate down frees quota
  • Update tests/configs/ if a quota DB path is needed

Verification: make nfstest


Critical Files

  • Cargo.toml — redb dependency
  • src/config.rs — QuotaConfig, parse_size
  • src/fsal/quota.rs (new) — QuotaManager + redb
  • src/fsal/mod.rs — FsStats, statvfs trait
  • src/fsal/local/mod.rs — quota enforcement
  • src/nfs/fsstat.rs — dynamic space reporting
  • src/nfs/write.rs, src/nfs/setattr.rs, src/nfs/rename.rs — DQUOT error mapping
  • src/main.rs — config wiring, background reconciliation

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Type

No type

Fields

No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions