Explore durable completion criteria for long-running Agent work

[phodal]

Context

Better Harness already models several parts of reliable goal completion:

• references/loop-engineering/patterns/goal-completion.md requires observable
  acceptance criteria, a predeclared verifier, and verified completion rather
  than the executor's own summary.
• models/agent-work-loop.md evaluates relevant verification, post-repair
  revalidation, delivery acceptance, and recovery within one Task Episode.
• roadmap.md positions Better Harness as an evidence and control plane, with
  durable verification and structured stop behavior planned as part of the
  Loop runtime direction.

These foundations describe what a reliable Loop should do, but they do not yet
define a durable, project-owned completion boundary that an Agent can apply
throughout a long-running task.

Problem

For a short task, acceptance criteria and validation commands may remain clear
inside the active conversation.

For longer work, execution, verification, and completion decisions often remain
in transient Agent context. This creates several failure modes:

• acceptance criteria drift across rounds, compaction, handoffs, or sessions;
• a check passes, but a later relevant change silently invalidates the result;
• commands are executed without proving which completion claim they support;
• cheap local checks are treated as final acceptance;
• expensive checks are repeated without regard to their appropriate timing;
• unavailable environments or permissions are silently treated as skipped;
• the same failure is retried without a new hypothesis or evidence;
• the Agent's completion summary becomes a substitute for objective proof;
• post-merge, post-deployment, or delayed validation is disconnected from the
  original task.

As a result, an Agent may perform substantial work and still lack a defensible
answer to a basic question:

What must be true, and what current evidence must exist, before this task is
allowed to stop?

Concept

Explore a repository-local, Agent-facing, executable verification contract.

The contract would define, at a conceptual level:

• the claims that must be true before a goal is considered complete;
• the conditions under which each claim applies;
• the method used to verify each applicable claim;
• the evidence that must be retained;
• the time boundary or verification horizon for that evidence;
• the conditions that invalidate previously collected evidence;
• the required response to failure, repeated failure, or lack of progress; and
• the conditions under which the Agent may continue, re-plan, block, or stop.

This issue intentionally remains format-neutral.

It does not select a public filename, document syntax, DSL, product name,
host integration, or runtime implementation.

Desired properties

Project-owned and durable

The completion boundary should survive conversation compaction, Agent handoff,
and multi-session execution. It must not exist only in an implementation
Agent's working memory.

Claim-based

Validation should prove explicit claims about the requested outcome rather than
produce an unrelated list of successful commands.

A passing command is evidence only when its relationship to an applicable
completion claim is known.

Scoped and conditional

Not every check applies to every task.

Applicability should be derived from the goal, affected surfaces, project-owned
constraints, delivery boundary, and material risk. A check that does not apply
must be distinguishable from one that was applicable but not observed.

Executable and evidence-bound

The contract should identify a reproducible verification route and the evidence
needed to review its result.

Prose such as "ensure quality" or "test thoroughly" is not sufficient.

Freshness-aware

Evidence must remain tied to the relevant task state, revision, environment,
and verification boundary.

A later change that can affect a prior result should make that result
insufficient for final completion until it is revalidated.

Long-horizon aware

Different evidence may be appropriate during local iteration, at a meaningful
milestone, before final delivery, or after an external event or observation
window.

The model must not present delayed or unavailable evidence as current success.

Fail-closed

Missing, stale, blocked, partial, or unobserved required evidence must not be
converted into a successful completion result.

Recovery-aware

Verification failure should inform the next Loop decision.

Repeated execution without a new hypothesis, new evidence, or measurable
progress should lead to re-planning, escalation, blocking, rollback, or an
explicit stop reason rather than mechanical retry.

Host-neutral and non-authorizing

The contract should describe provider-neutral completion obligations.

Its presence must not grant permission to mutate files, deploy, publish, access
credentials, modify external systems, or bypass the selected host's approval
and safety boundaries.

Expected control behavior

A future design should support the following behavior without requiring one
specific Agent host:

1. At task intake, derive a bounded set of applicable completion claims from the
   user goal, project-owned context, and the verification contract.
2. During execution, collect reviewable evidence for those claims at the
   appropriate point in the task.
3. When relevant state changes, invalidate or supersede evidence that is no
   longer sufficient.
4. When verification fails, feed the bounded failure result into the next
   execution or re-planning decision.
5. Before stopping, evaluate the required claims against current evidence
   rather than the executor's self-report.
6. When required proof depends on an unavailable external boundary, return an
   explicit unresolved or blocked result.
7. When later observation is required, preserve the relationship between the
   original task, the pending verification, and the later result without
   claiming that same-window completion proves later effectiveness.

Proposed acceptance criteria

• AC-1: Add a reviewable design specification under docs/specs/ before
  selecting a filename, syntax, or public format.
• AC-2: The specification defines the minimum conceptual model for a
  completion claim, applicability, verification method, evidence requirement,
  time boundary, freshness, failure response, and stop behavior.
• AC-3: Required evidence can be bound to one Task Episode and the relevant
  project state without copying private task content into public artifacts.
• AC-4: An executor summary, plan completion, command count, changed-file
  count, or elapsed number of rounds cannot independently satisfy a completion
  claim.
• AC-5: A previously passing result cannot satisfy final completion after a
  relevant invalidating change unless the contract explicitly proves that the
  result remains valid.
• AC-6: An applicable check that cannot run because of missing access,
  environment, authority, or external state produces an explicit unresolved or
  blocked result rather than success.
• AC-7: Repeated failure or repeated rounds without new completion evidence
  produces a structured re-plan, escalation, rollback, blocked, or stop
  decision.
• AC-8: Immediate completion evidence and later outcome evidence remain
  separate. Same-window validation must not automatically prove longitudinal
  effectiveness.
• AC-9: The design remains provider-neutral and cannot expand mutation or
  external-action authority.
• AC-10: Better Harness can distinguish that such a contract is merely
  present, reachable by the relevant workflow, actually exercised in a Task
  Episode, or supported by a later comparable outcome. Static file presence
  alone must not prove Agent behavior.

Candidate proof scenarios

The first specification or prototype should be tested against a small number of
bounded scenarios:

Evidence becomes stale

A relevant check passes, the Agent makes another change that can affect the
result, and the task attempts to stop without rerunning the check.

Expected behavior: completion is denied or remains unresolved.

Required verification is unavailable

A task requires an external environment, permission, device, service, or human
decision that is not available.

Expected behavior: the result is blocked or unresolved, not silently treated as
successful.

Repeated failure without progress

The same verification failure appears across multiple attempts without a new
hypothesis or new evidence.

Expected behavior: the Loop changes strategy, escalates, rolls back, or stops
with a structured reason.

Later verification remains pending

The implementation and immediate checks pass, but an outcome can only be
observed after merge, deployment, a scheduled run, or a defined observation
window.

Expected behavior: immediate completion and later outcome verification remain
separate claims.

Relationship to existing Better Harness concepts

This proposal should extend rather than replace the current model:

• Goal Completion Pattern continues to own the composition of understanding,
  execution, verification, and delivery.
• Agent Work Loop continues to own Task Episode review and the distinction
  between configured, exercised, and outcome-supported evidence.
• Loop State and a future durable runtime may retain execution state and
  structured stop reasons.
• This proposal focuses only on the project-owned completion obligations that
  those mechanisms evaluate or execute.

The specification should reference canonical owners instead of duplicating
their full schemas.

Non-goals

• Do not standardize or announce a public filename in this issue.
• Do not define a public interchange format or ecosystem standard yet.
• Do not replace AGENTS.md, architecture or design documentation,
  specifications, tests, CI, release policy, or human acceptance.
• Do not create one universal checklist that applies to every repository and
  every task.
• Do not require every expensive check to run after every edit.
• Do not treat an Agent-authored judgment as equivalent to deterministic,
  environment, artifact, human, or independently reviewed evidence.
• Do not authorize deployment, release, credential access, external writes, or
  other high-risk effects.
• Do not claim later effectiveness from same-window repair validation.
• Do not build another general-purpose Coding Agent runtime as part of this
  issue.

Open design questions

• What is the canonical ownership boundary: repository, scoped module, task, or
  a composition of these?
• How should project-level and scoped contracts compose without silently
  weakening required checks?
• How should this concept relate to AGENTS.md, design documentation,
  specifications, issue acceptance criteria, and existing test or CI commands?
• How should an Agent select applicable claims without turning file paths,
  framework detection, or asset presence into unsupported conclusions?
• What changes invalidate previous evidence, and how can that relationship be
  represented without requiring a full dependency graph?
• When is deterministic verification required, and when is an independent
  model or human judgment appropriate?
• What minimum evidence must be retained, and how should privacy, redaction, and
  storage lifetime be handled?
• How should a host expose completion, unresolved, blocked, and later-pending
  results without implying capability parity across providers?
• What is the smallest read-only or report-only proof that can validate the
  model before any runtime or stop-gate integration is considered?