Move test repetition down to the test case level#1528
Move test repetition down to the test case level#1528harlanhaskins wants to merge 8 commits intomainfrom
Conversation
harlanhaskins
requested review from
briancroom,
grynspan,
jerryjrchen and
stmontgomery
as code owners
harlanhaskins
force-pushed
the
harlan/getting-those-reps-in
branch
from
f926e06 to
03ba197
Compare
|
Converting to a draft for now; there's follow-on discussions we need to have about behavioral changes before we can actually sink this repetition down |
harlanhaskins
force-pushed
the
harlan/getting-those-reps-in
branch
from
03ba197 to
8b09896
Compare
harlanhaskins
force-pushed
the
harlan/getting-those-reps-in
branch
from
8b09896 to
2f6657b
Compare
harlanhaskins
force-pushed
the
harlan/getting-those-reps-in
branch
from
69ebb08 to
47a6688
Compare
stmontgomery
added
enhancement
| /// The value of the `--repeat-until` argument. | ||
| public var repeatUntil: String? | ||
|
|
||
| var usePerTestCaseRepetition: Bool = false |
There was a problem hiding this comment.
nit: Can we include documentation aligning with the other Bool properties in this type?
|
|
||
| /// Whether to perform test repetition at the plan level or on a per-test- | ||
| /// case basis. | ||
| public var shouldUseLegacyPlanLevelRepetition: Bool = true |
There was a problem hiding this comment.
Let's include @_spi(Experimental) too, and that would be something we remove if the proposal becomes accepted. (This will mean initially, it will have that plus inherited @_spi(ForToolsIntegrationOnly), and the latter will remain even post-acceptance.)
| /// The test case that is running on the current task, if any. | ||
| var testCase: Test.Case? | ||
|
|
||
| /// The current iteration of the test repetition policy. |
There was a problem hiding this comment.
| /// The current iteration of the test repetition policy. | |
| /// The current iteration of the test repetition policy, if any. |
| } | ||
| } | ||
|
|
||
| static var currentIteration: Int? { |
There was a problem hiding this comment.
Can you add documentation for this property?
There was a problem hiding this comment.
| // MARK: - Current test, test case, and iteration |
| /// - Parameters: | ||
| /// - testCase: The test case to run. | ||
| /// - step: The runner plan step associated with this test case. | ||
| /// - context: Context for the test run. |
There was a problem hiding this comment.
Looks like the context parameter still exists on this overload, might be an unintentional copy/paste
There was a problem hiding this comment.
Or perhaps, you meant to remove the parameter from the function signature?
| /// - test: The test being executed. | ||
| /// - testCase: The test case being iterated. |
There was a problem hiding this comment.
Params aren't in this function
2 participants
This changes the runner such that "repeat until success" or the other repetition policies repeat only the affected test case, rather than repeating the entire test plan.
Fixes #1392
Fixes rdar://130508488
Motivation:
The previous behavior was unexpected and does not match the behavior of XCTest. This changes the behavior to align the behaviors (in the way that users expect).
Modifications:
Checklist: