Skip to content

Clean-baseline patch replay as the official grading path + fresh-execution release gates #6

Description

@ElbertePlinio

Make clean-baseline patch replay the official grading path — the agent's workspace is never what gets graded. Panel-unanimous prerequisite for any published multi-model leaderboard.

  • Replay the captured patch (app/lib/core/patch_capture.dart) into a fresh baseline copy before hidden grading; stop grading inside the agent workspace
  • Grade-time assertion that hidden fixtures were never readable from the agent workspace (isolation asserted in the release gate, not only at workspace assembly)
  • Release gates verify fresh execution: current hidden-test digests, admission cleanliness, sandbox-enforced provenance (release_report.dart today mostly re-checks stored claims)
  • Fold the manual tasks/flutter/*/deepswe/ replay artifacts into runner behavior, or archive them

Note: patches + manual replay artifacts already exist, so this is a moderate lift, not a rebuild.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions