Bump torch version, make tests deterministic#80
Merged
Conversation
Signed-off-by: Romeo Kienzler <[email protected]>
Co-Authored-By: Claude Opus 4.7 <[email protected]> Signed-off-by: Romeo Kienzler <[email protected]>
Defensive on Linux per _warn_mp_context_on_linux. Co-Authored-By: Claude Opus 4.7 <[email protected]> Signed-off-by: Romeo Kienzler <[email protected]>
Bounds widened/shifted due to combined effect of torch>=2.10 numerics and --mp_context spawn re-seeding DataLoader workers per run. Signed-off-by: Romeo Kienzler <[email protected]>
Adds --deterministic [true|warn] to the train subcommand and threads it through Lightning's Trainer(deterministic=...). The integration tests now run with --deterministic warn and CUBLAS_WORKSPACE_CONFIG=:4096:8, which produced Std=0 across 5 seed=0 calibration runs on case14_ieee. OPF CI bounds in test_train_opf retightened to ±1% around the new deterministic point estimates (replacing the wide bounds calibrated under CUDA non-determinism). Signed-off-by: Romeo Kienzler <[email protected]>
Co-Authored-By: Claude Opus 4.8 <[email protected]> Signed-off-by: Romeo Kienzler <[email protected]>
Co-Authored-By: Claude Opus 4.8 <[email protected]> Signed-off-by: Romeo Kienzler <[email protected]>
romeokienzler
force-pushed
the
bump_torch
branch
from
7f691bd to
ba86225
Compare
romeokienzler
changed the title
albanpuech
approved these changes
Jul 1, 2026
albanpuech
left a comment
albanpuech
left a comment
Collaborator
There was a problem hiding this comment.
This is great! I just have a very final request:
Could we launch the regression test with the old pytorch version and see what we get in terms of metrics? That would be to see the impact of bumping the version on the metrics
| choices=["true", "warn"], | ||
| help=( | ||
| "Enable deterministic CUDA/cuDNN algorithms via Lightning Trainer(deterministic=...). " | ||
| "Pass --deterministic (alone) for 'warn' mode, or --deterministic true for strict. " |
Collaborator
There was a problem hiding this comment.
can you say what warn is in the help please. would be helpful for others
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.