Skip to content

Tighten reportability checks and workload scoping in study comparison #9

Description

@lullu57

Summary

Run-level reportability and study-level filtering are too loose. Study comparison does not enforce run-group workload scoping, and summary.json reportability is weaker than the documented contract while still being used as the admission gate for study aggregation.

Evidence

  • src/kernel_tuner/analysis/reporting.py:392
  • src/kernel_tuner/analysis/reporting.py:403
  • src/kernel_tuner/analysis/comparison.py:232
  • src/kernel_tuner/analysis/comparison.py:291
  • configs/studies/validation_phase.yaml:120
  • docs/specs/analysis_and_reporting.md:79

Why This Matters

Study-level evidence can silently mix workload classes or trust a run summary that did not fully re-check reportability requirements. That weakens paper-facing conclusions.

Expected Fix

  • Enforce run-group workload scoping in study comparison.
  • Tighten run-level reportability checks to match the documented contract.
  • Make study admission depend on the stronger contract, not a loose boolean.

Acceptance Criteria

  • Study run-groups respect configured workload scoping.
  • is_reportable reflects the documented policy, not a reduced subset of checks.
  • Tests cover study filtering and reportability enforcement.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions