fix(evaluator): emit evaluation_context in publish-to-intake

[shanaiabuggy]

Summary

The evaluator's publish-to-intake path (added in #456) still emits the deprecated experiment_context on ATIF ingest. As part of the Experiment → Evaluation rename, intake deprecated experiment_context in favor of evaluation_context. This migrates the boundary mapping to the canonical shape.

Per mapping.py's own design note ("a later rename is a one-file change"), the wire migration lives entirely in that module:

• ExperimentContextParam → EvaluationContextParam
• run_task_to_experiment_context → run_task_to_evaluation_context, now emitting {"evaluation_id": …, "test_case_id": …} under the "evaluation_context" ATIF body key
• tests updated to match (unit test_mapping.py/test_publish.py + the integration trace read)

Left for the evaluator team

The public publish_to_intake(experiment_id=…) parameter and PublishReport.experiment_id field are intentionally unchanged — renaming that external surface to evaluation_id is your call. The value already flows through to evaluation_context.evaluation_id, so functionally this is complete.

Not urgent / non-breaking

experiment_context still works as a deprecated alias in intake (evaluation_context takes precedence when both are sent), so publish isn't broken today — this is a forward migration before the alias is removed. No SDK version bump needed: the plugin uses the in-repo workspace SDK, which already exposes EvaluationContextParam.

Validation

• In-repo SDK confirmed to export EvaluationContextParam (TypedDict: evaluation_id, test_case_id).
• mapping.py compiles; no stale experiment_context/ExperimentContextParam references remain in the changed files.
• Unit + integration test assertions updated in lockstep. (Full suite runs in CI — the plugin needs the whole workspace built.)

🤖 Generated with Claude Code

Summary by CodeRabbit

• Bug Fixes
  • Updated the evaluation data sent to Intake to use evaluation context.
  • Evaluation records now include the evaluation ID and test case ID for clearer traceability.
  • Updated published trajectory and score metadata to consistently reflect the new evaluation context.
  • Improved integration coverage to verify evaluation context is preserved end-to-end through the publishing flow.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: da3ddba1-7c47-4c26-88cb-97e7cbed84df

📥 Commits

Reviewing files that changed from the base of the PR and between bdaf12f and 0686ccb.

📒 Files selected for processing (1)

• plugins/nemo-evaluator/src/nemo_evaluator/intake/mapping.py

🚧 Files skipped from review as they are similar to previous changes (1)

• plugins/nemo-evaluator/src/nemo_evaluator/intake/mapping.py

---

📝 Walkthrough

Walkthrough

The intake mapping replaces experiment_context with evaluation_context, using evaluation_id and test_case_id. Unit, publishing, and integration tests validate the updated payload and trace metadata.

Changes

Evaluation context migration

Layer / File(s)	Summary
Update intake context mapping plugins/nemo-evaluator/src/nemo_evaluator/intake/mapping.py	Replaces the experiment context type and helper, then emits evaluation_context with evaluation_id and test_case_id.
Validate evaluation context propagation plugins/nemo-evaluator/tests/intake/test_mapping.py, plugins/nemo-evaluator/tests/intake/test_publish.py, plugins/nemo-evaluator/tests/integration/test_publish_to_intake.py	Updates unit, publish, and round-trip assertions to verify evaluation context payloads and trace metadata.

Possibly related PRs

• NVIDIA-NeMo/nemo-platform#618: Deprecates experiment_context in favor of canonical evaluation_context fields.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title accurately summarizes the main change: emitting evaluation_context in publish-to-intake.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch sbuggy/evaluator-publish-evaluation-context

---

_{Comment @coderabbitai help to get the list of available commands.}

[github-actions]

Suite	Lines Covered	Line Rate	Branch Rate
Unit Tests	25533/32734	78.0%	62.6%
Integration Tests	14731/31383	46.9%	19.2%

[SandyChapman]

Thanks for this @shanaiabuggy ! I'll follow up with the experiment_id -> evaluation_id swap once this lands.