chore(experiments): Rename all child resources of Experiment entity to "Evaluation"#612
Merged
Conversation
Signed-off-by: shanaiabuggy <[email protected]>
Contributor
|
🌿 Preview your docs: https://nvidia-preview-sb-ase-512.docs.buildwithfern.com/nemo-platform |
Contributor
|
Contributor
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughAdds workspace-scoped Evaluation CRUD, pinning, session listing, rollup hydration, authorization, OpenAPI contracts, backend coverage, seed tooling, and Studio routes/views while retaining persisted Experiment storage mappings. ChangesEvaluation platform migration
Studio evaluation experience
Sequence Diagram(s)sequenceDiagram
participant User
participant Studio
participant Intake
participant ClickHouse
User->>Studio: Open evaluation detail
Studio->>Intake: List evaluation sessions
Intake->>ClickHouse: Query evaluation-scoped sessions
ClickHouse-->>Intake: Return session rows
Intake-->>Studio: Return evaluation sessions
Studio-->>User: Render sessions and traces
Possibly related PRs
Suggested reviewers: 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 9
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (2)
services/core/auth/src/nmp/core/auth/assets/static-authz.yaml (1)
114-148: 🔒 Security & Privacy | 🔴 Critical | ⚡ Quick winAdd
intake.evaluationsto the permission registry
services/core/auth/src/nmp/core/auth/assets/static-authz.yamlreferencesintake.evaluations.*in roles and endpoints, but the registry has nointake.evaluationsblock. That leaves undeclared permissions and breaks authz validation.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@services/core/auth/src/nmp/core/auth/assets/static-authz.yaml` around lines 114 - 148, The permission registry is missing the intake.evaluations entry even though roles and endpoints reference intake.evaluations.*. Update the static authz registry block in static-authz.yaml by adding an intake.evaluations section with the needed actions and descriptions, matching the pattern used by intake.evaluator-results, annotations, experiment-groups, and experiments so authz validation can resolve the permissions.web/packages/studio/src/components/dataViews/ExperimentGroupDataView/index.tsx (1)
206-208: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick winLeftover "experiment" wording in evaluation-domain UI.
Aria-labels ("Pin/Unpin experiment") and the error fallback ("Failed to load experiments.") weren't updated even though this view now operates on evaluations.
✏️ Suggested text fix
- aria-label={isPinned ? 'Unpin experiment' : 'Pin experiment'} + aria-label={isPinned ? 'Unpin evaluation' : 'Pin evaluation'}- return <ErrorMessage message="Failed to load experiments." />; + return <ErrorMessage message="Failed to load evaluations." />;Also applies to: 409-411
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@web/packages/studio/src/components/dataViews/ExperimentGroupDataView/index.tsx` around lines 206 - 208, Update the leftover experiment-specific copy in ExperimentGroupDataView so the evaluation-domain UI uses evaluation wording instead of experiment wording. In the component’s aria-labels around togglePin and the error fallback around the loading/error state, replace “Pin/Unpin experiment” and “Failed to load experiments.” with evaluation-appropriate text, and make the same copy change in both referenced spots so the labels stay consistent.
🧹 Nitpick comments (1)
web/packages/studio/src/routes/EvaluationTraceDetailRoute/index.test.tsx (1)
31-35: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winStale test description.
Test title still says "renders the experiment context panel" but now asserts
'Evaluation Context'. Update description for consistency.✏️ Fix
- it('renders the experiment context panel', async () => { + it('renders the evaluation context panel', async () => {🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@web/packages/studio/src/routes/EvaluationTraceDetailRoute/index.test.tsx` around lines 31 - 35, Update the test name in EvaluationTraceDetailRoute’s test suite so it matches the current assertion. The spec currently uses the outdated description “renders the experiment context panel” while asserting for “Evaluation Context”; rename the test around renderTraceDetail and screen.findByText to describe the Evaluation Context panel consistently.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@docs/cli/reference.mdx`:
- Line 5524: The filter example in the CLI reference is being mangled by MDX
link-like syntax, causing run_count to disappear from the rendered help text.
Update the CLI help-text generator or escaping logic around the filter examples
used by the reference docs so the run_count numeric-range example is emitted
literally, then regenerate the docs via the CLI reference generation flow. Use
the existing filter example text in the CLI help source and the reference doc
generation path to locate the fix.
In `@openapi/ga/individual/platform.openapi.yaml`:
- Around line 3310-3315: Keep the existing /experiments intake routes available
as deprecated aliases instead of renaming them to /evaluations. Update the
OpenAPI definition around
create_evaluation_apis_intake_v2_workspaces__workspace__evaluations_post so the
new /evaluations path is added without removing the old experiments-based
endpoints, and mark the old routes as deprecated to preserve compatibility for
existing API/SDK clients.
In `@openapi/ga/openapi.yaml`:
- Around line 3671-3687: The list_evaluation_sessions endpoint schema is missing
the documented 400 response for invalid status filter values. Update the
responses block for list_evaluation_sessions in the OpenAPI spec to include a
400 entry alongside the existing 200, 404, 503, and 422 responses, matching the
backend behavior when an invalid status is provided.
- Around line 3310-3313: The OpenAPI rename from the Experiments routes to the
Evaluations routes removed the old
`/apis/intake/v2/workspaces/{workspace}/experiments*` paths without a
compatibility alias. Update the `openapi/ga/openapi.yaml` definitions for the
`Evaluations` endpoints so the existing `/experiments` operations remain
available as deprecated aliases alongside the new `/evaluations` paths,
preserving the same handlers and request/response shapes for backwards
compatibility.
In `@services/intake/scripts/spans/seed_experiment_rollup_data.py`:
- Around line 12-13: The metadata.seeded_by reference points to a nonexistent
script, so update the path in the seeding metadata to the actual script name
used by seed_experiment_rollup_data.py. Locate the seeded_by assignment in the
same area as the docstring reference to seed_evaluations_demo.py, and change
only the bad script path while leaving the separate docstring link untouched.
In `@services/intake/scripts/spans/seed_experiments_demo.py`:
- Around line 32-33: The docstring and seeded_by metadata in
seed_experiments_demo.py still refer to the old script name, so update both
references from seed_evaluations_demo.py to seed_experiments_demo.py. Locate the
docstring near the top of the script and the seeded_by assignment in the
metadata section, and make sure the script name is consistent everywhere in that
file.
In `@services/intake/src/nmp/intake/entities/experiments.py`:
- Around line 62-68: The Evaluation entity type change in
Evaluation.__entity_type__ from the old storage key to "evaluation" will orphan
existing persisted rows that were written as "experiment". Fix this by either
keeping the legacy type in Evaluation or adding backward-compatible lookup
support in the entity loading path so both keys resolve. If you choose
migration, add a backfill for existing rows and ensure the entity lookup logic
in the experiments entities module still returns prior records.
In
`@web/packages/studio/src/components/dataViews/ExperimentGroupDataView/useExperimentGroupEvaluations.ts`:
- Line 162: The toast error text in useExperimentGroupEvaluations is using stale
“experiment” wording even though the hook now works with evaluations. Update the
toast.error messages in the pin and unpin error handlers to reference
“evaluation” instead of “experiment” so the wording stays consistent with the
rest of the hook and the surrounding symbols like onError, pin, and unpin.
In `@web/packages/studio/src/routes/groups/experimentRoutes.tsx`:
- Around line 33-36: The route entry for ROUTES.workspace.experiment is using
the wrong error panel title, since this path renders ExperimentRoute for the
experiment groups listing rather than an evaluation page. Update the
errorElement in experimentRoutes.tsx so the ErrorPanel title stays group-related
(for example, “Experiment Groups”) and does not use “Evaluation”; keep the
change localized to the route config that wires ROUTES.workspace.experiment to
ExperimentRoute.
---
Outside diff comments:
In `@services/core/auth/src/nmp/core/auth/assets/static-authz.yaml`:
- Around line 114-148: The permission registry is missing the intake.evaluations
entry even though roles and endpoints reference intake.evaluations.*. Update the
static authz registry block in static-authz.yaml by adding an intake.evaluations
section with the needed actions and descriptions, matching the pattern used by
intake.evaluator-results, annotations, experiment-groups, and experiments so
authz validation can resolve the permissions.
In
`@web/packages/studio/src/components/dataViews/ExperimentGroupDataView/index.tsx`:
- Around line 206-208: Update the leftover experiment-specific copy in
ExperimentGroupDataView so the evaluation-domain UI uses evaluation wording
instead of experiment wording. In the component’s aria-labels around togglePin
and the error fallback around the loading/error state, replace “Pin/Unpin
experiment” and “Failed to load experiments.” with evaluation-appropriate text,
and make the same copy change in both referenced spots so the labels stay
consistent.
---
Nitpick comments:
In `@web/packages/studio/src/routes/EvaluationTraceDetailRoute/index.test.tsx`:
- Around line 31-35: Update the test name in EvaluationTraceDetailRoute’s test
suite so it matches the current assertion. The spec currently uses the outdated
description “renders the experiment context panel” while asserting for
“Evaluation Context”; rename the test around renderTraceDetail and
screen.findByText to describe the Evaluation Context panel consistently.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: 8a51b0dd-f5d8-46f0-be69-48f9c46ef8b7
⛔ Files ignored due to path filters (34)
sdk/python/nemo-platform/.nmpcontext/openapi.yamlis excluded by!sdk/**sdk/python/nemo-platform/.nmpcontext/stainless.yamlis excluded by!sdk/**sdk/python/nemo-platform/api.mdis excluded by!sdk/**sdk/python/nemo-platform/src/nemo_platform/_client.pyis excluded by!sdk/**sdk/python/nemo-platform/src/nemo_platform/cli/commands/api/__init__.pyis excluded by!sdk/**sdk/python/nemo-platform/src/nemo_platform/cli/commands/api/evaluations/__init__.pyis excluded by!sdk/**sdk/python/nemo-platform/src/nemo_platform/cli/commands/api/evaluations/sessions.pyis excluded by!sdk/**sdk/python/nemo-platform/src/nemo_platform/resources/evaluations/__init__.pyis excluded by!sdk/**sdk/python/nemo-platform/src/nemo_platform/resources/evaluations/api.mdis excluded by!sdk/**sdk/python/nemo-platform/src/nemo_platform/resources/evaluations/evaluations.pyis excluded by!sdk/**sdk/python/nemo-platform/src/nemo_platform/resources/evaluations/sessions.pyis excluded by!sdk/**sdk/python/nemo-platform/src/nemo_platform/resources/experiment_groups/experiment_groups.pyis excluded by!sdk/**sdk/python/nemo-platform/src/nemo_platform/resources/experiments/api.mdis excluded by!sdk/**sdk/python/nemo-platform/src/nemo_platform/types/evaluations/__init__.pyis excluded by!sdk/**sdk/python/nemo-platform/src/nemo_platform/types/evaluations/evaluation_create_params.pyis excluded by!sdk/**sdk/python/nemo-platform/src/nemo_platform/types/evaluations/evaluation_filter_param.pyis excluded by!sdk/**sdk/python/nemo-platform/src/nemo_platform/types/evaluations/evaluation_list_params.pyis excluded by!sdk/**sdk/python/nemo-platform/src/nemo_platform/types/evaluations/evaluation_response.pyis excluded by!sdk/**sdk/python/nemo-platform/src/nemo_platform/types/evaluations/evaluation_responses_page.pyis excluded by!sdk/**sdk/python/nemo-platform/src/nemo_platform/types/evaluations/evaluation_session_filter_param.pyis excluded by!sdk/**sdk/python/nemo-platform/src/nemo_platform/types/evaluations/evaluation_session_response.pyis excluded by!sdk/**sdk/python/nemo-platform/src/nemo_platform/types/evaluations/evaluation_session_responses_page.pyis excluded by!sdk/**sdk/python/nemo-platform/src/nemo_platform/types/evaluations/evaluation_update_params.pyis excluded by!sdk/**sdk/python/nemo-platform/src/nemo_platform/types/evaluations/evaluator_aggregate.pyis excluded by!sdk/**sdk/python/nemo-platform/src/nemo_platform/types/evaluations/metric_stat_filters_param.pyis excluded by!sdk/**sdk/python/nemo-platform/src/nemo_platform/types/evaluations/number_filter_param.pyis excluded by!sdk/**sdk/python/nemo-platform/src/nemo_platform/types/evaluations/session_list_params.pyis excluded by!sdk/**sdk/python/nemo-platform/src/nemo_platform/types/experiment_groups/experiment_group_create_params.pyis excluded by!sdk/**sdk/python/nemo-platform/src/nemo_platform/types/experiment_groups/experiment_group_response.pyis excluded by!sdk/**sdk/python/nemo-platform/src/nemo_platform/types/experiment_groups/experiment_group_update_params.pyis excluded by!sdk/**sdk/python/nemo-platform/tests/api_resources/evaluations/__init__.pyis excluded by!sdk/**sdk/python/nemo-platform/tests/api_resources/evaluations/test_sessions.pyis excluded by!sdk/**sdk/python/nemo-platform/tests/api_resources/test_evaluations.pyis excluded by!sdk/**sdk/stainless.yamlis excluded by!sdk/**
📒 Files selected for processing (55)
docs/cli/reference.mdxdocs/fern/snippets/_snippets/cli-summary.mdxopenapi/ga/individual/platform.openapi.yamlopenapi/ga/openapi.yamlopenapi/openapi.yamlpackages/nemo_platform_ext/src/nemo_platform_ext/cli/commands/api/__init__.pypackages/nemo_platform_ext/src/nemo_platform_ext/cli/commands/api/evaluations/__init__.pypackages/nemo_platform_ext/src/nemo_platform_ext/cli/commands/api/evaluations/sessions.pyservices/core/auth/src/nmp/core/auth/assets/static-authz.yamlservices/intake/scripts/spans/seed_experiment_rollup_data.pyservices/intake/scripts/spans/seed_experiments_demo.pyservices/intake/src/nmp/intake/api/v2/evaluations/endpoints.pyservices/intake/src/nmp/intake/api/v2/evaluations/schemas.pyservices/intake/src/nmp/intake/entities/experiments.pyservices/intake/src/nmp/intake/service.pyservices/intake/src/nmp/intake/spans/clickhouse_migrations.pyservices/intake/src/nmp/intake/spans/evaluation_rollup_repository.pyservices/intake/src/nmp/intake/spans/evaluation_session_repository.pyservices/intake/src/nmp/intake/spans/ingest/experiment_context_validation.pyservices/intake/tests/conftest.pyservices/intake/tests/integration/spans/test_experiment_metric_sort.pyservices/intake/tests/integration/spans/test_experiment_rollups.pyservices/intake/tests/integration/spans/test_experiment_sessions.pyservices/intake/tests/integration/test_experiments_crud.pyservices/intake/tests/test_experiment_default_sort.pyservices/intake/tests/test_experiment_metric_filter.pyservices/intake/tests/test_experiment_optimization_fields.pyservices/intake/tests/test_experiment_rollup_repository.pyservices/intake/tests/test_experiment_session_schemas.pyservices/intake/tests/test_experiment_sort.pyservices/intake/tests/test_experiment_sort_endpoint.pyservices/intake/tests/test_spans_clickhouse_migrations.pyweb/packages/common/src/hooks/useStudioDataViewState/filterFieldMap.integration.test.tsxweb/packages/studio/src/components/ExperimentGroupEditModal/index.tsxweb/packages/studio/src/components/dataViews/EvaluationSessionsDataView/Empty.tsxweb/packages/studio/src/components/dataViews/EvaluationSessionsDataView/index.test.tsxweb/packages/studio/src/components/dataViews/EvaluationSessionsDataView/index.tsxweb/packages/studio/src/components/dataViews/ExperimentGroupDataView/Empty.tsxweb/packages/studio/src/components/dataViews/ExperimentGroupDataView/index.tsxweb/packages/studio/src/components/dataViews/ExperimentGroupDataView/useExperimentGroupEvaluations.test.tsweb/packages/studio/src/components/dataViews/ExperimentGroupDataView/useExperimentGroupEvaluations.tsweb/packages/studio/src/components/dataViews/ExperimentGroupDataView/useSortErrorRecovery.test.tsweb/packages/studio/src/components/dataViews/ExperimentGroupDataView/useSortErrorRecovery.tsweb/packages/studio/src/constants/links.tsweb/packages/studio/src/constants/routes.tsweb/packages/studio/src/routes/EvaluationDetailRoute/EvaluationDetailMetrics.tsxweb/packages/studio/src/routes/EvaluationDetailRoute/index.tsxweb/packages/studio/src/routes/EvaluationTraceDetailRoute/index.test.tsxweb/packages/studio/src/routes/EvaluationTraceDetailRoute/index.tsxweb/packages/studio/src/routes/ExperimentGroupDetailRoute/index.tsxweb/packages/studio/src/routes/ExperimentRoute/ExperimentGroupCard.tsxweb/packages/studio/src/routes/WorkspaceLayout/WorkspaceSideNav.tsxweb/packages/studio/src/routes/groups/experimentRoutes.tsxweb/packages/studio/src/routes/utils.tsweb/packages/studio/src/tests/title-change.test.tsx
Signed-off-by: shanaiabuggy <[email protected]>
shanaiabuggy
marked this pull request as draft
July 13, 2026 17:38
Signed-off-by: shanaiabuggy <[email protected]> # Conflicts: # openapi/ga/individual/platform.openapi.yaml # openapi/ga/openapi.yaml # openapi/openapi.yaml # sdk/python/nemo-platform/.nmpcontext/openapi.yaml # services/intake/src/nmp/intake/spans/ingest/evaluation_context_validation.py # web/packages/studio/src/components/dataViews/ExperimentGroupDataView/index.tsx # web/packages/studio/src/constants/routes.ts # web/packages/studio/src/tests/title-change.test.tsx
Signed-off-by: shanaiabuggy <[email protected]>
Signed-off-by: shanaiabuggy <[email protected]>
shanaiabuggy
changed the title
Signed-off-by: shanaiabuggy <[email protected]>
shanaiabuggy
marked this pull request as ready for review
July 14, 2026 16:02
Signed-off-by: shanaiabuggy <[email protected]>
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
web/packages/studio/src/components/IntakeDetail/IntakeComponents/traceKeyValues.test.tsx (1)
69-75: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick winRename the remaining user-facing experiment label.
This test still expects
Experiment ID, while the PR renames the child resource to Evaluation. Update the context descriptor/rendered label toEvaluation IDand assert that value here; otherwise the UI will show “Evaluation Context” containing stale Experiment terminology.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@web/packages/studio/src/components/IntakeDetail/IntakeComponents/traceKeyValues.test.tsx` around lines 69 - 75, Update the context descriptor used by buildExperimentContextEntries to label the child resource “Evaluation ID” instead of “Experiment ID,” then change the expected label in the “includes evaluation context entries when present” test to match. Ensure the rendered evaluation context no longer exposes stale Experiment terminology.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@services/intake/src/nmp/intake/service.py`:
- Around line 54-58: Remove the router-level tag argument from the Experiments
RouterConfig entry so it no longer adds the duplicate “Experiments” OpenAPI
group; preserve the existing router, description, and route-level tags.
---
Outside diff comments:
In
`@web/packages/studio/src/components/IntakeDetail/IntakeComponents/traceKeyValues.test.tsx`:
- Around line 69-75: Update the context descriptor used by
buildExperimentContextEntries to label the child resource “Evaluation ID”
instead of “Experiment ID,” then change the expected label in the “includes
evaluation context entries when present” test to match. Ensure the rendered
evaluation context no longer exposes stale Experiment terminology.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: d888c512-1352-41fc-99b2-7d65834c4063
📒 Files selected for processing (22)
plugins/nemo-evaluator/tests/integration/test_publish_to_intake.pyservices/core/auth/src/nmp/core/auth/assets/static-authz.yamlservices/intake/scripts/spans/seed_experiment_rollup_data.pyservices/intake/scripts/spans/seed_experiments_demo.pyservices/intake/src/nmp/intake/api/v2/experiments/endpoints.pyservices/intake/src/nmp/intake/api/v2/experiments/schemas.pyservices/intake/src/nmp/intake/service.pyservices/intake/tests/conftest.pyservices/intake/tests/integration/spans/test_atif_ingest.pyservices/intake/tests/integration/spans/test_chat_completions_ingest.pyservices/intake/tests/integration/test_experiments_crud.pyservices/intake/tests/test_experiment_default_sort.pyservices/intake/tests/test_experiment_metric_filter.pyservices/intake/tests/test_experiment_session_schemas.pyservices/intake/tests/test_experiment_sort.pyservices/intake/tests/test_experiment_sort_endpoint.pyweb/packages/studio/src/components/IntakeDetail/IntakeComponents/traceKeyValues.test.tsxweb/packages/studio/src/components/IntakeDetail/README.mdweb/packages/studio/src/components/IntakeDetail/TraceDetailView.tsxweb/packages/studio/src/routes/EvaluationTraceDetailRoute/index.test.tsxweb/packages/studio/src/routes/IntakeTraceDetailRoute/index.test.tsxweb/packages/studio/src/routes/WorkspaceLayout/WorkspaceSideNav.tsx
🚧 Files skipped from review as they are similar to previous changes (8)
- web/packages/studio/src/routes/WorkspaceLayout/WorkspaceSideNav.tsx
- web/packages/studio/src/routes/EvaluationTraceDetailRoute/index.test.tsx
- services/intake/tests/conftest.py
- services/intake/tests/test_experiment_sort.py
- services/intake/tests/test_experiment_session_schemas.py
- services/intake/tests/test_experiment_sort_endpoint.py
- services/core/auth/src/nmp/core/auth/assets/static-authz.yaml
- services/intake/tests/integration/test_experiments_crud.py
Signed-off-by: shanaiabuggy <[email protected]> # Conflicts: # openapi/ga/individual/platform.openapi.yaml # openapi/ga/openapi.yaml # openapi/openapi.yaml # sdk/python/nemo-platform/.nmpcontext/openapi.yaml # sdk/python/nemo-platform/src/nemo_platform/resources/evaluations/evaluations.py # sdk/python/nemo-platform/src/nemo_platform/types/evaluations/evaluation_list_params.py # sdk/python/nemo-platform/tests/api_resources/test_evaluations.py # services/intake/src/nmp/intake/api/v2/experiments/schemas.py # web/packages/studio/src/routes/EvaluationDetailRoute/EvaluationDetailMetrics.tsx
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In
`@web/packages/studio/src/components/dataViews/ExperimentGroupDataView/index.tsx`:
- Around line 217-225: Update the pin button’s accessible label near the Tooltip
in the experiment group data view, replacing “Pin experiment” and “Unpin
experiment” with the corresponding “Pin evaluation” and “Unpin evaluation”
terminology while preserving the existing isPinned behavior.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: e2fa0a69-3572-4f6e-8b24-a937947ba615
⛔ Files ignored due to path filters (12)
sdk/python/nemo-platform/.nmpcontext/openapi.yamlis excluded by!sdk/**sdk/python/nemo-platform/src/nemo_platform/resources/evaluations/evaluations.pyis excluded by!sdk/**sdk/python/nemo-platform/src/nemo_platform/resources/experiment_groups/experiment_groups.pyis excluded by!sdk/**sdk/python/nemo-platform/src/nemo_platform/types/evaluations/evaluation_create_params.pyis excluded by!sdk/**sdk/python/nemo-platform/src/nemo_platform/types/evaluations/evaluation_filter_param.pyis excluded by!sdk/**sdk/python/nemo-platform/src/nemo_platform/types/evaluations/evaluation_list_params.pyis excluded by!sdk/**sdk/python/nemo-platform/src/nemo_platform/types/evaluations/evaluation_response.pyis excluded by!sdk/**sdk/python/nemo-platform/src/nemo_platform/types/evaluations/evaluation_update_params.pyis excluded by!sdk/**sdk/python/nemo-platform/src/nemo_platform/types/experiment_groups/experiment_group_create_params.pyis excluded by!sdk/**sdk/python/nemo-platform/src/nemo_platform/types/experiment_groups/experiment_group_response.pyis excluded by!sdk/**sdk/python/nemo-platform/src/nemo_platform/types/experiment_groups/experiment_group_update_params.pyis excluded by!sdk/**sdk/python/nemo-platform/tests/api_resources/test_evaluations.pyis excluded by!sdk/**
📒 Files selected for processing (12)
openapi/ga/individual/platform.openapi.yamlopenapi/ga/openapi.yamlopenapi/openapi.yamlservices/intake/src/nmp/intake/api/v2/experiments/endpoints.pyservices/intake/src/nmp/intake/api/v2/experiments/schemas.pyservices/intake/src/nmp/intake/entities/experiments.pyservices/intake/tests/test_experiment_default_sort.pyservices/intake/tests/test_experiment_optimization_fields.pyweb/packages/studio/src/components/IntakeDetail/IntakeComponents/traceKeyValues.test.tsxweb/packages/studio/src/components/dataViews/EvaluationSessionsDataView/index.tsxweb/packages/studio/src/components/dataViews/ExperimentGroupDataView/index.tsxweb/packages/studio/src/routes/EvaluationDetailRoute/EvaluationDetailMetrics.tsx
🚧 Files skipped from review as they are similar to previous changes (10)
- web/packages/studio/src/routes/EvaluationDetailRoute/EvaluationDetailMetrics.tsx
- web/packages/studio/src/components/IntakeDetail/IntakeComponents/traceKeyValues.test.tsx
- services/intake/tests/test_experiment_optimization_fields.py
- web/packages/studio/src/components/dataViews/EvaluationSessionsDataView/index.tsx
- services/intake/tests/test_experiment_default_sort.py
- services/intake/src/nmp/intake/api/v2/experiments/schemas.py
- openapi/ga/openapi.yaml
- openapi/ga/individual/platform.openapi.yaml
- services/intake/src/nmp/intake/api/v2/experiments/endpoints.py
- openapi/openapi.yaml
Contributor
There was a problem hiding this comment.
Caution
Inline review comments failed to post. This is likely due to GitHub's internal server error or limits when posting large numbers of comments. If you are seeing this consistently it is likely a permissions issue. Please check "Moderation" -> "Code review limits" under your organization settings.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In
`@web/packages/studio/src/components/dataViews/ExperimentGroupDataView/index.tsx`:
- Around line 217-225: Update the pin button’s accessible label near the Tooltip
in the experiment group data view, replacing “Pin experiment” and “Unpin
experiment” with the corresponding “Pin evaluation” and “Unpin evaluation”
terminology while preserving the existing isPinned behavior.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: e2fa0a69-3572-4f6e-8b24-a937947ba615
⛔ Files ignored due to path filters (12)
sdk/python/nemo-platform/.nmpcontext/openapi.yamlis excluded by!sdk/**sdk/python/nemo-platform/src/nemo_platform/resources/evaluations/evaluations.pyis excluded by!sdk/**sdk/python/nemo-platform/src/nemo_platform/resources/experiment_groups/experiment_groups.pyis excluded by!sdk/**sdk/python/nemo-platform/src/nemo_platform/types/evaluations/evaluation_create_params.pyis excluded by!sdk/**sdk/python/nemo-platform/src/nemo_platform/types/evaluations/evaluation_filter_param.pyis excluded by!sdk/**sdk/python/nemo-platform/src/nemo_platform/types/evaluations/evaluation_list_params.pyis excluded by!sdk/**sdk/python/nemo-platform/src/nemo_platform/types/evaluations/evaluation_response.pyis excluded by!sdk/**sdk/python/nemo-platform/src/nemo_platform/types/evaluations/evaluation_update_params.pyis excluded by!sdk/**sdk/python/nemo-platform/src/nemo_platform/types/experiment_groups/experiment_group_create_params.pyis excluded by!sdk/**sdk/python/nemo-platform/src/nemo_platform/types/experiment_groups/experiment_group_response.pyis excluded by!sdk/**sdk/python/nemo-platform/src/nemo_platform/types/experiment_groups/experiment_group_update_params.pyis excluded by!sdk/**sdk/python/nemo-platform/tests/api_resources/test_evaluations.pyis excluded by!sdk/**
📒 Files selected for processing (12)
openapi/ga/individual/platform.openapi.yamlopenapi/ga/openapi.yamlopenapi/openapi.yamlservices/intake/src/nmp/intake/api/v2/experiments/endpoints.pyservices/intake/src/nmp/intake/api/v2/experiments/schemas.pyservices/intake/src/nmp/intake/entities/experiments.pyservices/intake/tests/test_experiment_default_sort.pyservices/intake/tests/test_experiment_optimization_fields.pyweb/packages/studio/src/components/IntakeDetail/IntakeComponents/traceKeyValues.test.tsxweb/packages/studio/src/components/dataViews/EvaluationSessionsDataView/index.tsxweb/packages/studio/src/components/dataViews/ExperimentGroupDataView/index.tsxweb/packages/studio/src/routes/EvaluationDetailRoute/EvaluationDetailMetrics.tsx
🚧 Files skipped from review as they are similar to previous changes (10)
- web/packages/studio/src/routes/EvaluationDetailRoute/EvaluationDetailMetrics.tsx
- web/packages/studio/src/components/IntakeDetail/IntakeComponents/traceKeyValues.test.tsx
- services/intake/tests/test_experiment_optimization_fields.py
- web/packages/studio/src/components/dataViews/EvaluationSessionsDataView/index.tsx
- services/intake/tests/test_experiment_default_sort.py
- services/intake/src/nmp/intake/api/v2/experiments/schemas.py
- openapi/ga/openapi.yaml
- openapi/ga/individual/platform.openapi.yaml
- services/intake/src/nmp/intake/api/v2/experiments/endpoints.py
- openapi/openapi.yaml
🛑 Comments failed to post (1)
web/packages/studio/src/components/dataViews/ExperimentGroupDataView/index.tsx (1)
217-225: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
Rename the pin button’s accessible label to Evaluation.
The tooltip now uses Evaluation terminology, but the existing button label still says “Pin experiment” / “Unpin experiment” at Line 230. Update those labels so screen-reader users receive the same terminology as other users.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@web/packages/studio/src/components/dataViews/ExperimentGroupDataView/index.tsx` around lines 217 - 225, Update the pin button’s accessible label near the Tooltip in the experiment group data view, replacing “Pin experiment” and “Unpin experiment” with the corresponding “Pin evaluation” and “Unpin evaluation” terminology while preserving the existing isPinned behavior.
Signed-off-by: shanaiabuggy <[email protected]>
Signed-off-by: shanaiabuggy <[email protected]>
shanaiabuggy
commented
Jul 14, 2026
Signed-off-by: shanaiabuggy <[email protected]>
Signed-off-by: shanaiabuggy <[email protected]>
BrianNewsom
approved these changes
Jul 15, 2026
shanaiabuggy
enabled auto-merge
July 15, 2026 17:08
This was referenced Jul 15, 2026
This was referenced Jul 22, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Rename the child resource everywhere user-facing (API paths/schemas, SDK, CLI, RBAC, docs, Studio) to Evaluation. Keep the stored entity as Experiment (entity_type="experiment", parent_experiment_id); the API layer maps Evaluation ⇄ the Experiment entity so existing rows keep working.
include_in_schema=Falseso the old URL keeps working but doesn't clutter the docsSummary by CodeRabbit
summaryvsdetailedmode, filtering, and pagination.evaluation_count, updated descriptions, and evaluation-aware sorting.