Skip to content

Gate C: 50%-prefix early warning (registered, never tested) #8

Description

@solarkyle

Gate C was registered but never run: can the workspace signal flag an incoming error from only the first half of the generation?

The prefix features already exist in the published traces (https://huggingface.co/datasets/solarkyle/jspace-hallucination-campaign) — this is a CPU-only analysis over existing data.

A useful submission: AUROC of prefix-only features vs full-trace features vs logprob baseline, per source, with the same grouped-CI treatment used in the Stage 2 report. Watch for the answer-readout confound (see README): report varying-truth slices separately.

Metadata

Metadata

Assignees

No one assigned

    Labels

    analysis ideaNew analysis over the published traces/features — usually no GPU neededhelp wantedExtra attention is needed

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions