Skip to content

Add ART and Verifiers trajectory interop helpers#692

Open
nboufeld wants to merge 1 commit into
OpenPipe:mainfrom
nboufeld:codex/verifiers-portability-bounty
Open

Add ART and Verifiers trajectory interop helpers#692
nboufeld wants to merge 1 commit into
OpenPipe:mainfrom
nboufeld:codex/verifiers-portability-bounty

Conversation

@nboufeld
Copy link
Copy Markdown

Summary

  • adds lightweight ART <-> Verifiers rollout portability helpers
  • converts ART Trajectory objects into Verifiers-style rollout state with per-choice trajectory steps
  • converts Verifiers rollout states back into ART Trajectory / TrajectoryGroup objects, with an optional choice_factory for preserving trainable assistant choices when callers have the original response payload
  • documents the helpers in the README

Bounty fit

This targets the hard scope from the Algora ART-E / ART bounty: two-way portability between ART and Verifiers tasks.

Bounty: https://algora.io/PrimeIntellect-ai/bounties/cxptwFoEv2up9Qjm

Verification

  • PYTHONPATH=src /tmp/art-interop-venv/bin/python -m pytest tests/unit/test_verifiers_interop.py -q
  • /tmp/art-interop-venv/bin/ruff format src/art/verifiers.py tests/unit/test_verifiers_interop.py
  • /tmp/art-interop-venv/bin/ruff check src/art/verifiers.py tests/unit/test_verifiers_interop.py
  • git diff --check

Note: I used a minimal local venv for this pure-Python adapter test because the repo's current uv config attempted to resolve/build heavy optional GPU dependencies in this environment.

@nboufeld nboufeld force-pushed the codex/verifiers-portability-bounty branch from b29167d to bb90893 Compare May 22, 2026 18:19
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: bb90893ea5

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/art/verifiers.py Outdated
Comment on lines +157 to +159
steps = state.get("trajectory") or []
if isinstance(steps, list) and steps:
return _steps_to_messages_and_choices(steps, choice_factory=choice_factory)
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Preserve completion tail when rebuilding from trajectory steps

When state["trajectory"] is non-empty, this branch returns _steps_to_messages_and_choices(...) and never merges state["completion"]. In states that include completion messages after the last recorded step (for example, environment/tool turns appended at rollout end), those tail messages are silently dropped on import, so ART↔Verifiers round-trips lose transcript content.

Useful? React with 👍 / 👎.

Comment thread src/art/verifiers.py
Comment on lines +55 to +57
"is_completed": True,
"is_truncated": False,
"stop_condition": "art_trajectory_imported",
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Propagate truncation status to state-level flag

This sets is_truncated to False unconditionally, even when imported choices mark a truncated finish (e.g., finish_reason == "length" on the final step). Consumers that read only the state-level truncation flag will misclassify truncated rollouts as complete, which can affect filtering and reward/metrics analysis.

Useful? React with 👍 / 👎.

@nboufeld nboufeld force-pushed the codex/verifiers-portability-bounty branch from bb90893 to 759ce24 Compare May 22, 2026 18:27
@nboufeld
Copy link
Copy Markdown
Author

Updated after the automated review feedback. The conversion now propagates state-level is_truncated when any imported choice/step is truncated, and verifiers_state_to_trajectory preserves state-level completion tail messages that appear after the last recorded trajectory step.\n\nRe-verified locally:\n- PYTHONPATH=src /tmp/art-interop-venv/bin/python -m pytest tests/unit/test_verifiers_interop.py -q -> 5 passed\n- /tmp/art-interop-venv/bin/ruff check src/art/verifiers.py tests/unit/test_verifiers_interop.py -> passed\n- git diff --check -> passed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant