Reframe CI: per-extension isolated validation#14
Open
vm-pranavan wants to merge 2 commits into
Open
Conversation
…sions - Add partial_test_compat category to skip_list.yaml for extensions that pass most tests but have a few MW 1.43-incompatible test files. Cargo and PageForms are promoted from load-only to full validation. - Update parse_yaml.py to load partial_test_compat entries and attach exclude_tests file lists to manifest entries. - Update run_isolated_tests.py to enumerate test files and exclude specific ones (PHPUnit 9 compat — no --exclude-file flag).
vm-pranavan
force-pushed
the
ci/add-extension-tests
branch
from
600a211 to
cfa395f
Compare
….43.yaml Remove SemanticBreadcrumbLinks, SemanticFormsSelect, and SemanticTasks from the semantic-ecosystem group since they don't exist in the upstream 1.43.yaml manifest. Also improve error reporting to show stdout when stderr is empty.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Overview
This PR updates our CI setup to run tests for extensions like
CargoandPageFormsthat we were previously skipping entirely due to a few minor incompatibilities with MediaWiki 1.43 / PHP 8.3. Instead of skipping their entire test suites, we can now run all tests while excluding only the specific files that are failing.I've also cleaned up the coexistence testing configuration to remove some old extensions that don't exist in our current manifest, and improved how we capture logs when coexistence checks fail.
Key Changes
Partial Test Exclusions for PHPUnit
partial_test_compatsection in .ci/skip_list.yaml to declare which files should be bypassed for each extension.load_skip_listandbuild_manifestin scripts/parse_yaml.py to parse this list and pass the excluded file paths in the manifest.run_phpunitin .ci/run_isolated_tests.py to support exclusions. Since PHPUnit 9 doesn't have an--exclude-fileflag, we dynamically walk the directory, filter out the excluded paths, and pass the remaining test files individually.CargoFeedFormatTest.php(fails because MW 1.43 changed heading HTML markup).PFFormLinkerTest.php(environment-dependent assertion) andPFHelperFormActionTest.php(references the deprecated/removedSkinTemplateclass).Coexistence Group Cleanup & Diagnostics
SemanticBreadcrumbLinks,SemanticFormsSelect, andSemanticTasksfrom thesemantic-ecosystemgroup in .ci/coexistence_groups.yaml since they are not present in the1.43.yamlmanifest.verify.stdoutifverify.stderris empty when a coexistence group fails to load, which helps capture PHP errors and warnings.