UNOMI-970: Fix IT ProgressListener ETA using live suite pace#834
Merged
sergehuber merged 3 commits intoJul 21, 2026
Merged
Conversation
Re-evaluate remaining time from elapsed/completed after every test and treat historical timings as hints so early skips no longer collapse ETA.
Address review findings on the live-pace ETA calculation: - Derive live pace from the average of substantive completed test durations instead of elapsed/completedCount, which chronically inflated the ETA after any failed test (wall time counted, but the failure was excluded from the denominator) and could still collapse the ETA when many fast assume/skip completions preceded a block of heavy tests. - Make computeScale's skew guard one-directional so a genuine per-test regression (historically fast, now much slower) is no longer silently excluded from raising the ETA. - Replace the untyped (observed, cached) long[] pairs with a TimingSample record to remove ordering ambiguity. - Fix fallbackAverageMs's averaging to be consistent with the new averagePositive/averageAtLeast helper. - Stop TestTimingCache.save() from blending near-instant assume/skip durations into a test's historical average, which would otherwise gradually erode the hint used by estimateRemainingMs. Adds regression tests for each fix (including a mutation-tested check that the old elapsed/completedCount bug is actually caught) plus edge-case coverage for null/empty inputs.
- Fix ProgressListener to override testAssumptionFailure so JUnit Assume-based skips (e.g. RolloverIT's backend-capability gating) are excluded from the timing cache/live pace the same way hard failures already were — previously they silently flowed through testFinished as full successes, contradicting the documented behavior. - Make estimateRemainingMs's historical-hint reweighting genuinely bidirectional: remove the Math.max(rateEtaMs, hintShapedEtaMs) floor that made hints raise-only, so historically lighter remaining work can now lower the ETA below flat live pace, not just raise it. - Consolidate ProgressListener's two always-together-mutated lists (completedDurations, observedVsCached) into a single CompletedSample per completed test, removing an untyped-parallel-collection hazard without changing TestTimingCache's tested public API. - Cap the true-cold-start ETA placeholder (COLD_START_PLACEHOLDER_CAP_MS) so a stalled start with no successes/no cache can't balloon the displayed ETA unboundedly with elapsed time. - Harden TestTimingCache.save()/load(): distinguish expected I/O failures from unexpected RuntimeExceptions (the latter now logged at WARN with a full stack trace instead of silently swallowed), clean up orphaned temp files on a failed write, and log malformed cache entries instead of dropping them silently. - Fix stale/misleading Javadoc (MAX_PAIR_SKEW's own example was already excluded by a different gate; the elapsed/cold-start parameter docs didn't match the actual guard). - Add ProgressListenerTest exercising the real JUnit RunListener wiring (testFailure/testAssumptionFailure/testFinished/testIgnored), plus new TestTimingCacheTest coverage for the bidirectional hint fix, the slowdown-boost ramp at scale, and the capped cold-start placeholder. Co-Authored-By: Claude Sonnet 5 <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
remaining × elapsed / completed) after every test.MIN_SCALEfactor that chronically under-reported ETA after early assumes/skips.itests/README.md.Fixes: https://issues.apache.org/jira/browse/UNOMI-970
Relates to: UNOMI-912, UNOMI-968
Test plan
mvn -Pintegration-tests -pl itests -Dtest=TestTimingCacheTest surefire:test -Drat.skip=true -Ddocker.skip=true