Skip to content

fix(intake): Do not read unneeded trace input/output for list view#334

Merged
BrianNewsom merged 1 commit into
mainfrom
brnewsom/trace-index-perf-fixes
Jun 15, 2026
Merged

fix(intake): Do not read unneeded trace input/output for list view#334
BrianNewsom merged 1 commit into
mainfrom
brnewsom/trace-index-perf-fixes

Conversation

@BrianNewsom

@BrianNewsom BrianNewsom commented Jun 15, 2026

Copy link
Copy Markdown
Contributor

Summary by CodeRabbit

  • Bug Fixes
    • Updated trace read/list responses to no longer expose root-level input/output payload fields.
    • Removed span input/output from trace rollup/value aggregation so only the supported aggregate fields are returned.
    • Kept root identity details (such as source format, root span ID, and name) in responses.
  • Tests
    • Adjusted ClickHouse SQL expectation tests and updated row-mapping assertions to reflect the new query projections and None values for omitted payload fields.

@BrianNewsom
BrianNewsom requested review from a team as code owners June 15, 2026 18:44
@github-actions github-actions Bot added the fix label Jun 15, 2026
@coderabbitai

coderabbitai Bot commented Jun 15, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 7eb86146-2e9e-4e7b-bc0f-c909005acd81

📥 Commits

Reviewing files that changed from the base of the PR and between 53f99a4 and f16279d.

📒 Files selected for processing (2)
  • services/intake/src/nmp/intake/spans/trace_repository.py
  • services/intake/tests/test_traces_clickhouse_repository.py
🚧 Files skipped from review as they are similar to previous changes (1)
  • services/intake/tests/test_traces_clickhouse_repository.py

📝 Walkthrough

Walkthrough

TRACE_COLUMNS and _trace_select_columns drop input/output. _trace_index_sql switches from a subquery wrapper to direct FROM trace_index AS trace_roots FINAL and adds cost documentation. Span rollup columns filtered via _CURRENT_SPAN_VALUE_COLUMNS. Tests updated to match new SQL shape and expect None for trace.input/trace.output.

Changes

Trace index root payload removal

Layer / File(s) Summary
Trace column definitions and SQL generation
services/intake/src/nmp/intake/spans/trace_repository.py
TRACE_COLUMNS drops input/output; _trace_select_columns removes traces.input and traces.output projections; _trace_index_sql adds comments explaining pagination cost rationale and replaces subquery FROM with direct FINAL selection.
Configurable span value columns for rollup aggregation
services/intake/src/nmp/intake/spans/trace_repository.py
_CURRENT_SPAN_VALUE_COLUMNS excludes input/output from span rollup; current_spans_sql reformatted for clarity.
Test assertions and fixtures
services/intake/tests/test_traces_clickhouse_repository.py
Summary and detailed tests assert FROM trace_index AS trace_roots FINAL and verify root_input/root_output absent; span aggregate block excludes argMax for input/output; mapping tests and _trace_row fixture updated to expect None for trace.input/trace.output.

Suggested labels

refactor

Suggested reviewers

  • shanaiabuggy
🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed Title accurately describes the main change: removing unused trace input/output fields from list view queries for performance.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch brnewsom/trace-index-perf-fixes

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
services/intake/src/nmp/intake/spans/trace_repository.py (1)

266-274: ⚠️ Potential issue | 🟠 Major

Stop detailed rollups from selecting span payload columns.

_trace_aggregates_sql() calls current_spans_sql() without limiting columns, so the generated query includes argMax() for input and output despite aggregating only trace-level metrics. Pass a lean value-column set to exclude payload columns.

Proposed fix
 def _trace_aggregates_sql(table: str) -> tuple[str, dict[str, Any]]:
@@
         FROM {
         current_spans_sql(
             table,
+            value_columns=("status", "attributes_string", "attributes_number", "is_deleted"),
             extra_where_sql=(
                 "(span_versions.workspace, span_versions.source_format, span_versions.trace_id) IN "
                 "(SELECT workspace, source_format, id FROM page_traces)"
             ),
         )
     } AS {source_alias}
@@
-def current_spans_sql(table: str, *, extra_where_sql: str | None = None) -> str:
+def current_spans_sql(
+    table: str,
+    *,
+    extra_where_sql: str | None = None,
+    value_columns: tuple[str, ...] = _CURRENT_SPAN_VALUE_COLUMNS,
+) -> str:
@@
         *[
             f"argMax({source_alias}.{column}, ({source_alias}.event_ts, {source_alias}.is_deleted)) AS {column}"
-            for column in _CURRENT_SPAN_VALUE_COLUMNS
+            for column in value_columns
         ],

Add regression assertions in tests:

     assert "sumIf" in client.queries[1]
     assert "groupUniqArrayIf" in client.queries[1]
+    assert "span_versions.input" not in client.queries[1]
+    assert "span_versions.output" not in client.queries[1]
     assert "count() AS span_count" in client.queries[1]

Verify the rendered detailed query excludes input/output column selections.

Also applies to: 324-331

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@services/intake/src/nmp/intake/spans/trace_repository.py` around lines 266 -
274, The `_trace_aggregates_sql()` function calls `current_spans_sql()` without
specifying which columns to select, causing unnecessary payload column
aggregations like `argMax()` for `input` and `output` columns even though only
trace-level metrics are needed. Pass a lean set of value columns to the
`current_spans_sql()` call that excludes the payload columns `input` and
`output`. This issue applies to two locations: the main call site around lines
266-274 (in the outer FROM clause of the detailed trace aggregates query) and
the second location at lines 324-331. Modify both calls to `current_spans_sql()`
to include a column parameter that specifies only the necessary trace-level
metric columns.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Outside diff comments:
In `@services/intake/src/nmp/intake/spans/trace_repository.py`:
- Around line 266-274: The `_trace_aggregates_sql()` function calls
`current_spans_sql()` without specifying which columns to select, causing
unnecessary payload column aggregations like `argMax()` for `input` and `output`
columns even though only trace-level metrics are needed. Pass a lean set of
value columns to the `current_spans_sql()` call that excludes the payload
columns `input` and `output`. This issue applies to two locations: the main call
site around lines 266-274 (in the outer FROM clause of the detailed trace
aggregates query) and the second location at lines 324-331. Modify both calls to
`current_spans_sql()` to include a column parameter that specifies only the
necessary trace-level metric columns.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: e356bdbe-5086-40af-890d-3bfa2aecdf5f

📥 Commits

Reviewing files that changed from the base of the PR and between cb77cb9 and 4ea7a66.

📒 Files selected for processing (2)
  • services/intake/src/nmp/intake/spans/trace_repository.py
  • services/intake/tests/test_traces_clickhouse_repository.py

@github-actions

github-actions Bot commented Jun 15, 2026

Copy link
Copy Markdown
Contributor
Suite Lines Covered Line Rate Branch Rate
Unit Tests 19462/25829 75.3% 60.9%
Integration Tests 11372/24601 46.2% 20.2%

@BrianNewsom
BrianNewsom force-pushed the brnewsom/trace-index-perf-fixes branch 2 times, most recently from 60e0867 to 53f99a4 Compare June 15, 2026 19:02
@BrianNewsom
BrianNewsom force-pushed the brnewsom/trace-index-perf-fixes branch from 53f99a4 to f16279d Compare June 15, 2026 19:05
@BrianNewsom
BrianNewsom added this pull request to the merge queue Jun 15, 2026
Merged via the queue into main with commit dbff951 Jun 15, 2026
51 checks passed
@BrianNewsom
BrianNewsom deleted the brnewsom/trace-index-perf-fixes branch June 15, 2026 23:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants