Skip to content

fix(3.1.2): grandfather pre-3.0.43 datasets in dataset-size check#831

Merged
FileSystemGuy merged 1 commit into
mainfrom
fix/grandfather-pre-3.0.43-datasets-3.1.2
Jul 22, 2026
Merged

fix(3.1.2): grandfather pre-3.0.43 datasets in dataset-size check#831
FileSystemGuy merged 1 commit into
mainfrom
fix/grandfather-pre-3.0.43-datasets-3.1.2

Conversation

@FileSystemGuy

Copy link
Copy Markdown
Contributor

Problem

A submitter re-validating against 3.0.46 is blocked on rule 3.1.2:

dataset size mismatch: actual files 4710038 < minimum required 4710039

Their dataset was generated with mlpstorage 3.0.24 (Jun 28), before the floor→ceil datagen alignment in f9d414d (first shipped in 3.0.43). Datagen back then sized datasets with integer floor division in rules/utils.py; the validator now re-derives the threshold with ceil, so it rejects a valid pre-alignment dataset by exactly one file. The submitter did nothing wrong — the official tool sized the dataset and reported it compliant.

Fix

Relax the rule 3.1.2 threshold from ceil to ceil - 1 for the v3.0 round (validator only; rules/utils.py datagen/runtime verifier stays on ceil).

Why ceil()-1 and not floor()

Datagen floored via integer // (exact), while this check recomputes the same formula as a float chain — so float drift can push floor() one above the recorded count and still reject the dataset we're grandfathering. ceil-1:

  • equals floor() for non-integer ratios (no change in the common case),
  • grants one file of slack exactly at the integer boundary where that drift bites, absorbing the bounded ≤1-file divergence,
  • stays int-vs-int (no "N < N" display bug),
  • still rejects genuinely undersized runs (short by >1 file).

Testing

All four CI suites pass (2938 + 890 + 238 + 228, zero failures). No assertion flipped, including test_true_undersized_submission_still_fails_3_1_2 (short by ~50k files, far beyond the 1-file relaxation).

Follow-up

Revert to plain math.ceil once the v3.0 round closes — post-3.0.43 datagen ceils, so newer datasets pass either way. Tracked inline in the code comment.

🤖 Generated with Claude Code

…heck

Datasets generated before the floor->ceil datagen alignment (commit
f9d414d, first shipped in mlpstorage 3.0.43) were sized with integer
floor division in rules/utils.py. Rule 3.1.2 in the submission checker
now re-derives the threshold with ceil, so it rejects a valid
pre-alignment dataset by exactly one file (a submitter hit "actual files
4710038 < minimum required 4710039" on a Jun-28 run made with 3.0.24).

Relax the 3.1.2 threshold to ceil-minus-one for the v3.0 round. We can't
just floor it: datagen floored via integer `//` while this check
recomputes the same formula as a float chain, so float drift can push
floor() one above the recorded count and still reject. ceil-1 equals
floor() for non-integer ratios and grants one file of slack exactly at
the integer boundary where that drift bites, absorbing the bounded
<=1-file divergence. Stays int-vs-int (no "N < N" display bug) and still
rejects genuinely undersized runs (short by >1 file).

Validator-only change; rules/utils.py (datagen/runtime verifier) stays
on ceil. Revert to plain ceil once the v3.0 round closes -- post-3.0.43
datagen ceils, so newer datasets pass either way.
@FileSystemGuy
FileSystemGuy requested a review from a team July 22, 2026 19:12
@github-actions

Copy link
Copy Markdown

MLCommons CLA bot All contributors have signed the MLCommons CLA ✍️ ✅

@FileSystemGuy
FileSystemGuy merged commit 1a236dc into main Jul 22, 2026
4 checks passed
@FileSystemGuy
FileSystemGuy deleted the fix/grandfather-pre-3.0.43-datasets-3.1.2 branch July 22, 2026 19:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant