Skip to content

fix(eval): report task slugs with traces - #521

Merged
jdchawla29 merged 2 commits into
mainfrom
codex/trace-task-identity
Jul 30, 2026
Merged

fix(eval): report task slugs with traces#521
jdchawla29 merged 2 commits into
mainfrom
codex/trace-task-identity

Conversation

@jdchawla29

@jdchawla29 jdchawla29 commented Jul 30, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • materialize Task.slug during Pydantic validation, deriving a stable default from task id and args when omitted
  • use the validated slug directly across tasksets, sync, local/cloud trace reporting, hosted rollout submissions, and Harbor export
  • include task_slug in the trace-enter report for local and cloud execution
  • cover default, explicit, and invalid slug behavior plus both reporting paths with focused tests

Why

The trace reporting boundary carried execution identifiers but omitted logical task identity. Unlinked SDK traces therefore reached the platform with no stable task key and fell back to UUID-derived labels or Unknown Task.

This makes task_slug the portable task identity across execution environments while leaving group_id as execution-group identity. The model now guarantees that every valid Task has a slug, so consumers do not need independent fallback logic.

Validation

  • uv run pytest -p no:cacheprovider hud/eval/tests/test_task.py hud/eval/tests/test_job.py hud/eval/tests/test_hosted.py — 37 passed
  • uv run pyright — 0 errors
  • uv run ruff check on changed files
  • uv run ruff format on changed files
  • uv lock --check

Note

Medium Risk
Changes wire contracts (task_slug on trace enter, slug on rollout submit) and make slug required on all tasks; platform must accept new fields, and older pydantic or clients assuming optional slug may need updates.

Overview
Every valid Task now has a materialized slug at validation time (stable default from task id plus args hash when omitted), replacing optional slug and removed default_slug() fallbacks across the eval stack.

Platform and local reporting send logical task identity: trace_enter includes task_slug, hosted /rollouts/submit adds slug, and taskset upload/sync/Harbor export use task.slug directly for names and run receipts.

Dependency: pydantic minimum raised to 2.10 in pyproject.toml. Tests cover default/explicit/invalid slugs and both trace-enter and hosted submit payloads.

Reviewed by Cursor Bugbot for commit 556fe6d. Bugbot is set up for automated code reviews on this repo. Configure here.

@jdchawla29
jdchawla29 marked this pull request as ready for review July 30, 2026 22:05

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 556fe6d287

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread hud/eval/task.py

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 556fe6d. Configure here.

Comment thread hud/eval/sync.py
@jdchawla29
jdchawla29 merged commit bee52fc into main Jul 30, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant