Skip to content

fix(intake): remove implicit 30-day lookback from span/trace lists; s…#580

Merged
BrianNewsom merged 1 commit into
mainfrom
intake-remove-implicit-lookback/brnewsom
Jul 6, 2026
Merged

fix(intake): remove implicit 30-day lookback from span/trace lists; s…#580
BrianNewsom merged 1 commit into
mainfrom
intake-remove-implicit-lookback/brnewsom

Conversation

@BrianNewsom

@BrianNewsom BrianNewsom commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

…eed Studio browse filter

The span and trace list endpoints silently added started_at >= now()-30d whenever no time filter was given. Any id-scoped query (trace detail spans, old bookmarks, SDK calls) returned silently empty results once the data aged past 30 days, with nothing in the request or response hinting a filter was applied.

The API now returns exactly what was asked for. The browsing default moves to Studio, where it belongs to the UX: the intake traces/spans tables seed a visible, clearable 30-day started_at filter chip via a new defaultColumnFilters option on useStudioDataViewState (written to the URL on mount so chips, shared links, and clearing behave like user-set filters).

Behavior change for API consumers: unscoped list queries now scan and return full history instead of the last 30 days.

Summary by CodeRabbit

  • New Features
    • Spans and traces now seed a default 30-day “started_at” filter in the UI so recent results appear automatically.
  • Bug Fixes
    • Removed the backend’s implicit time limit for spans and traces when no time bounds are provided.
    • Updated UI empty-state and “Clear Filters” behavior so the seeded date filter isn’t treated as user-applied, and clearing removes it.
  • Tests
    • Added/updated integration and unit tests to verify no implicit time bounds, deterministic timestamps, and correct seeded filter query parameters.

@BrianNewsom
BrianNewsom requested review from a team as code owners July 6, 2026 21:29
@github-actions github-actions Bot added the fix label Jul 6, 2026
@coderabbitai

coderabbitai Bot commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 4cfb8990-b22e-4bd2-a265-3344e31b33ac

📥 Commits

Reviewing files that changed from the base of the PR and between df6e6cf and d9b6212.

📒 Files selected for processing (10)
  • services/intake/src/nmp/intake/spans/api/spans.py
  • services/intake/src/nmp/intake/spans/api/traces.py
  • services/intake/tests/integration/spans/test_atif_ingest.py
  • services/intake/tests/integration/spans/test_chat_completions_ingest.py
  • services/intake/tests/test_traces_api.py
  • web/packages/studio/src/components/IntakeLists/IntakeSpansTable.test.tsx
  • web/packages/studio/src/components/IntakeLists/IntakeSpansTable.tsx
  • web/packages/studio/src/components/IntakeLists/IntakeTracesTable.test.tsx
  • web/packages/studio/src/components/IntakeLists/IntakeTracesTable.tsx
  • web/packages/studio/src/components/IntakeLists/defaultStartedAtFilter.ts
💤 Files with no reviewable changes (3)
  • services/intake/tests/integration/spans/test_chat_completions_ingest.py
  • services/intake/src/nmp/intake/spans/api/spans.py
  • services/intake/src/nmp/intake/spans/api/traces.py
🚧 Files skipped from review as they are similar to previous changes (7)
  • web/packages/studio/src/components/IntakeLists/IntakeTracesTable.test.tsx
  • web/packages/studio/src/components/IntakeLists/IntakeSpansTable.test.tsx
  • services/intake/tests/test_traces_api.py
  • services/intake/tests/integration/spans/test_atif_ingest.py
  • web/packages/studio/src/components/IntakeLists/IntakeTracesTable.tsx
  • web/packages/studio/src/components/IntakeLists/defaultStartedAtFilter.ts
  • web/packages/studio/src/components/IntakeLists/IntakeSpansTable.tsx

📝 Walkthrough

Walkthrough

Removes implicit started_at defaults from intake spans/traces APIs and adds tests for explicit bounds. The frontend now seeds the 30-day started_at filter into URL state before rendering IntakeSpansTable and IntakeTracesTable.

Changes

Backend: remove default lookback

Layer / File(s) Summary
Spans API default lookback removal
services/intake/src/nmp/intake/spans/api/spans.py
Removes the implicit started_at lookback constant, helper, and helper calls from spans list/group endpoints.
Traces API default lookback removal
services/intake/src/nmp/intake/spans/api/traces.py
Removes the implicit started_at lookback constant, helper, and helper call from trace listing.
Backend tests validating no implicit time bound
services/intake/tests/test_traces_api.py, services/intake/tests/integration/spans/test_atif_ingest.py, services/intake/tests/integration/spans/test_chat_completions_ingest.py
Adds a unit test asserting _trace_filter leaves started_at bounds unset, freezes ingest timestamps, and removes a stale lookback comment.

Frontend: seeded default started_at filter

Layer / File(s) Summary
defaultStartedAtFilter utility and seeding hook
web/packages/studio/src/components/IntakeLists/defaultStartedAtFilter.ts
Defines the default started_at filter shape, 30-day factory, equality check, and URL-seeding hook.
IntakeSpansTable seeded default filter
web/packages/studio/src/components/IntakeLists/IntakeSpansTable.tsx, web/packages/studio/src/components/IntakeLists/IntakeSpansTable.test.tsx
Seeds the default filter before mount, gates rendering until seeding completes, excludes the seeded default from empty-state logic, and verifies the outgoing filter value and clear-filters behavior.
IntakeTracesTable seeded default filter
web/packages/studio/src/components/IntakeLists/IntakeTracesTable.tsx, web/packages/studio/src/components/IntakeLists/IntakeTracesTable.test.tsx
Seeds the default filter before mount, gates rendering until seeding completes, excludes the seeded default from empty-state logic, and verifies the outgoing filter value and clear-filters behavior.

Sequence Diagram(s)

sequenceDiagram
  participant Browser
  participant IntakeTable as IntakeSpansTable/IntakeTracesTable
  participant SeedHook as useSeededStartedAtFilter
  participant URL as URL search params
  participant API as Intake API

  Browser->>IntakeTable: mount
  IntakeTable->>SeedHook: useSeededStartedAtFilter(defaultStartedAtFilter)
  SeedHook->>URL: check filters param
  alt filters missing
    SeedHook->>URL: write default started_at filter (replace)
  end
  SeedHook-->>IntakeTable: seeded=true
  IntakeTable->>API: request list with filter[started_at][$gte]
  API-->>IntakeTable: results
  Browser->>IntakeTable: click clear-filters
  IntakeTable->>API: request list with started_at=null
  API-->>IntakeTable: results
Loading

Possibly related PRs

Suggested labels: refactor

Suggested reviewers: shanaiabuggy

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title matches the main change: removing the implicit 30-day lookback from intake span and trace lists.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch intake-remove-implicit-lookback/brnewsom

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
web/packages/studio/src/components/IntakeLists/IntakeSpansTable.tsx (1)

150-166: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Duplicate default-filter/hasActiveFilters wiring vs IntakeTracesTable.

Same useState(makeDefaultStartedAtFilter) + hasActiveFilters pattern is repeated in IntakeTracesTable.tsx (lines 52-62). Consider extracting a small shared hook (e.g. useDefaultStartedAtFilter(fixedFilter?)) into defaultStartedAtFilter.ts returning { defaultStartedAtFilter, hasActiveFilters } to avoid drift between the two call sites.

♻️ Sketch of shared hook
+// in defaultStartedAtFilter.ts
+export const useDefaultStartedAtFilter = (
+  fixedFilter?: unknown,
+  debouncedColumnFilters: { id: string; value: unknown }[] = []
+) => {
+  const [defaultStartedAtFilter] = useState(() =>
+    fixedFilter ? null : makeDefaultStartedAtFilter()
+  );
+  const hasActiveFilters = debouncedColumnFilters.some(
+    (filter) =>
+      defaultStartedAtFilter === null || !isDefaultStartedAtFilter(filter, defaultStartedAtFilter)
+  );
+  return { defaultStartedAtFilter, hasActiveFilters };
+};
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@web/packages/studio/src/components/IntakeLists/IntakeSpansTable.tsx` around
lines 150 - 166, The default started_at filter and hasActiveFilters logic is
duplicated between IntakeSpansTable and IntakeTracesTable, so extract the shared
behavior into a small hook or helper such as useDefaultStartedAtFilter in
defaultStartedAtFilter.ts. Have the shared symbol return both
defaultStartedAtFilter and hasActiveFilters, and update IntakeSpansTable and
IntakeTracesTable to consume it so the
fixedFilter/makeDefaultStartedAtFilter/isDefaultStartedAtFilter behavior stays
consistent in one place.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@web/packages/studio/src/components/IntakeLists/IntakeSpansTable.tsx`:
- Around line 150-166: The default started_at filter and hasActiveFilters logic
is duplicated between IntakeSpansTable and IntakeTracesTable, so extract the
shared behavior into a small hook or helper such as useDefaultStartedAtFilter in
defaultStartedAtFilter.ts. Have the shared symbol return both
defaultStartedAtFilter and hasActiveFilters, and update IntakeSpansTable and
IntakeTracesTable to consume it so the
fixedFilter/makeDefaultStartedAtFilter/isDefaultStartedAtFilter behavior stays
consistent in one place.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 945acadc-17d8-4bfe-9c4d-f5ca749d1c92

📥 Commits

Reviewing files that changed from the base of the PR and between 954c403 and cefcf21.

📒 Files selected for processing (13)
  • services/intake/src/nmp/intake/spans/api/spans.py
  • services/intake/src/nmp/intake/spans/api/traces.py
  • services/intake/tests/integration/spans/test_atif_ingest.py
  • services/intake/tests/integration/spans/test_chat_completions_ingest.py
  • services/intake/tests/test_spans_api.py
  • services/intake/tests/test_traces_api.py
  • web/packages/common/src/hooks/useStudioDataViewState/index.test.tsx
  • web/packages/common/src/hooks/useStudioDataViewState/index.ts
  • web/packages/studio/src/components/IntakeLists/IntakeSpansTable.test.tsx
  • web/packages/studio/src/components/IntakeLists/IntakeSpansTable.tsx
  • web/packages/studio/src/components/IntakeLists/IntakeTracesTable.test.tsx
  • web/packages/studio/src/components/IntakeLists/IntakeTracesTable.tsx
  • web/packages/studio/src/components/IntakeLists/defaultStartedAtFilter.ts
💤 Files with no reviewable changes (3)
  • services/intake/tests/integration/spans/test_chat_completions_ingest.py
  • services/intake/src/nmp/intake/spans/api/spans.py
  • services/intake/src/nmp/intake/spans/api/traces.py

@BrianNewsom
BrianNewsom force-pushed the intake-remove-implicit-lookback/brnewsom branch from cefcf21 to d3828c4 Compare July 6, 2026 21:38
@BrianNewsom
BrianNewsom force-pushed the intake-remove-implicit-lookback/brnewsom branch from d3828c4 to df6e6cf Compare July 6, 2026 22:14

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
web/packages/studio/src/components/IntakeLists/defaultStartedAtFilter.ts (1)

19-32: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

No direct unit tests for these pure helpers.

makeDefaultStartedAtFilter and isDefaultStartedAtFilter are pure, easily-testable logic (date math, equality contract) but tests are only mentioned for the table components downstream. Direct tests here would pin the date-window and equality behavior independent of table rendering.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@web/packages/studio/src/components/IntakeLists/defaultStartedAtFilter.ts`
around lines 19 - 32, Add direct unit tests for the pure helpers in
defaultStartedAtFilter, specifically makeDefaultStartedAtFilter and
isDefaultStartedAtFilter, instead of relying only on downstream table tests.
Cover the date-window behavior in makeDefaultStartedAtFilter by asserting the
returned started_at filter uses the expected lookback and start-of-day
normalization, and cover the equality contract in isDefaultStartedAtFilter by
checking it returns true only for an untouched seeded default and false for
changed id or value cases.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@web/packages/studio/src/components/IntakeLists/defaultStartedAtFilter.ts`:
- Around line 19-32: Add direct unit tests for the pure helpers in
defaultStartedAtFilter, specifically makeDefaultStartedAtFilter and
isDefaultStartedAtFilter, instead of relying only on downstream table tests.
Cover the date-window behavior in makeDefaultStartedAtFilter by asserting the
returned started_at filter uses the expected lookback and start-of-day
normalization, and cover the equality contract in isDefaultStartedAtFilter by
checking it returns true only for an untouched seeded default and false for
changed id or value cases.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 5a18c10b-5ebb-4b4c-b201-f1ac639b68cc

📥 Commits

Reviewing files that changed from the base of the PR and between d3828c4 and df6e6cf.

📒 Files selected for processing (10)
  • services/intake/src/nmp/intake/spans/api/spans.py
  • services/intake/src/nmp/intake/spans/api/traces.py
  • services/intake/tests/integration/spans/test_atif_ingest.py
  • services/intake/tests/integration/spans/test_chat_completions_ingest.py
  • services/intake/tests/test_traces_api.py
  • web/packages/studio/src/components/IntakeLists/IntakeSpansTable.test.tsx
  • web/packages/studio/src/components/IntakeLists/IntakeSpansTable.tsx
  • web/packages/studio/src/components/IntakeLists/IntakeTracesTable.test.tsx
  • web/packages/studio/src/components/IntakeLists/IntakeTracesTable.tsx
  • web/packages/studio/src/components/IntakeLists/defaultStartedAtFilter.ts
💤 Files with no reviewable changes (3)
  • services/intake/tests/integration/spans/test_chat_completions_ingest.py
  • services/intake/src/nmp/intake/spans/api/traces.py
  • services/intake/src/nmp/intake/spans/api/spans.py
🚧 Files skipped from review as they are similar to previous changes (6)
  • services/intake/tests/test_traces_api.py
  • web/packages/studio/src/components/IntakeLists/IntakeSpansTable.test.tsx
  • web/packages/studio/src/components/IntakeLists/IntakeTracesTable.test.tsx
  • web/packages/studio/src/components/IntakeLists/IntakeSpansTable.tsx
  • web/packages/studio/src/components/IntakeLists/IntakeTracesTable.tsx
  • services/intake/tests/integration/spans/test_atif_ingest.py

@github-actions

github-actions Bot commented Jul 6, 2026

Copy link
Copy Markdown
Contributor
Suite Lines Covered Line Rate Branch Rate
Unit Tests 23269/30434 76.5% 61.2%
Integration Tests N/A N/A N/A

…eed Studio browse filter

The span and trace list endpoints silently added started_at >= now()-30d
whenever no time filter was given. Any id-scoped query (trace detail spans,
old bookmarks, SDK calls) returned silently empty results once the data aged
past 30 days, with nothing in the request or response hinting a filter was
applied.

The API now returns exactly what was asked for. The browsing default moves to
Studio, where it belongs to the UX: the intake traces/spans tables seed a
visible, clearable 30-day started_at filter chip via a new
defaultColumnFilters option on useStudioDataViewState (written to the URL on
mount so chips, shared links, and clearing behave like user-set filters).

Behavior change for API consumers: unscoped list queries now scan and return
full history instead of the last 30 days.

Co-Authored-By: Claude Fable 5 <[email protected]>
Signed-off-by: Brian Newsom <[email protected]>
@BrianNewsom
BrianNewsom force-pushed the intake-remove-implicit-lookback/brnewsom branch from df6e6cf to d9b6212 Compare July 6, 2026 22:38
@BrianNewsom
BrianNewsom added this pull request to the merge queue Jul 6, 2026
Merged via the queue into main with commit 9d16a12 Jul 6, 2026
53 of 54 checks passed
@BrianNewsom
BrianNewsom deleted the intake-remove-implicit-lookback/brnewsom branch July 6, 2026 23:05
arpitsardhana pushed a commit that referenced this pull request Jul 9, 2026
…eed Studio browse filter (#580)

The span and trace list endpoints silently added started_at >= now()-30d
whenever no time filter was given. Any id-scoped query (trace detail spans,
old bookmarks, SDK calls) returned silently empty results once the data aged
past 30 days, with nothing in the request or response hinting a filter was
applied.

The API now returns exactly what was asked for. The browsing default moves to
Studio, where it belongs to the UX: the intake traces/spans tables seed a
visible, clearable 30-day started_at filter chip via a new
defaultColumnFilters option on useStudioDataViewState (written to the URL on
mount so chips, shared links, and clearing behave like user-set filters).

Behavior change for API consumers: unscoped list queries now scan and return
full history instead of the last 30 days.

Signed-off-by: Brian Newsom <[email protected]>
Co-authored-by: Claude Fable 5 <[email protected]>
arpitsardhana pushed a commit that referenced this pull request Jul 9, 2026
…TL (#587)

test_atif_ingest_accepts_example_trajectory_and_reconstructs_read_side_data
seeded spans at a fixed `_BASE_TIME = 2026-01-15`. The spans/traces ClickHouse
tables carry `TTL toDate(start_time) + INTERVAL 90 DAY`, so once that fixed date
is >90 days in the past the rows fall out of retention and the read returns
nothing (assert 0 == 7). Because TTL eviction runs on async background merges,
the failure is flaky — it passes only when no merge evicts the rows before the
read, which is why it intermittently blocked the merge queue.

Seed relative to `now()` (~45 days back): beyond the removed 30-day list
lookback that this test exercises, yet safely inside the 90-day TTL, and no
longer a fixed date that rots. Use a .500000 microsecond offset so no derived
timestamp lands on a whole second, which the span read-side returns without a
fractional part and would fail the exact-timestamp assertions (the pre-#580
`now()-2h` seed avoided this; the fixed µs=0 base introduced in #580 did not).

Full module passes repeatedly.

Signed-off-by: Sandy Chapman <[email protected]>
Zenodia pushed a commit to Zenodia/nemo-platform that referenced this pull request Jul 10, 2026
…TL (NVIDIA-NeMo#587)

test_atif_ingest_accepts_example_trajectory_and_reconstructs_read_side_data
seeded spans at a fixed `_BASE_TIME = 2026-01-15`. The spans/traces ClickHouse
tables carry `TTL toDate(start_time) + INTERVAL 90 DAY`, so once that fixed date
is >90 days in the past the rows fall out of retention and the read returns
nothing (assert 0 == 7). Because TTL eviction runs on async background merges,
the failure is flaky — it passes only when no merge evicts the rows before the
read, which is why it intermittently blocked the merge queue.

Seed relative to `now()` (~45 days back): beyond the removed 30-day list
lookback that this test exercises, yet safely inside the 90-day TTL, and no
longer a fixed date that rots. Use a .500000 microsecond offset so no derived
timestamp lands on a whole second, which the span read-side returns without a
fractional part and would fail the exact-timestamp assertions (the pre-NVIDIA-NeMo#580
`now()-2h` seed avoided this; the fixed µs=0 base introduced in NVIDIA-NeMo#580 did not).

Full module passes repeatedly.

Signed-off-by: Sandy Chapman <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants