Skip to content

Fix datadog_metrics handling of Prometheus +Inf histogram buckets - #26135

Open
anvithsg2004 wants to merge 2 commits into
vectordotdev:masterfrom
anvithsg2004:fix/datadog-metrics-inf-bucket
Open

Fix datadog_metrics handling of Prometheus +Inf histogram buckets#26135
anvithsg2004 wants to merge 2 commits into
vectordotdev:masterfrom
anvithsg2004:fix/datadog-metrics-inf-bucket

Conversation

@anvithsg2004

Copy link
Copy Markdown

What does this PR do?

This fixes an issue where Prometheus histogram observations that fall into the implicit +Inf bucket can be lost by the datadog_metrics sink and result in an empty Datadog sketch payload.

The Prometheus parser previously removed the +Inf bucket after converting cumulative bucket counts to per-bucket counts. This caused valid observations above the highest finite bucket to be discarded.

This PR:

  • Preserves the +Inf bucket and its observations in the Prometheus parser.
  • Handles the edge case where a histogram contains only a +Inf bucket during DDSketch interpolation.
  • Filters empty sketches after histogram/distribution conversion so empty Datadog sketch payloads are not sent.
  • Filters already-empty Sketch metrics as well.

Why?

For example, a histogram such as:

test_overflow_bucket{le="1.0"} 0
test_overflow_bucket{le="+Inf"} 4
test_overflow_count 4
test_overflow_sum 20000

represents four valid observations above 1.0.

Previously, the +Inf bucket was removed, leaving no bucket observations to insert into the DDSketch. This could produce an empty sketch, which was subsequently encoded as an empty SketchPayload and rejected by Datadog with HTTP 400 Bad Request.

The +Inf bucket is now preserved, allowing the observations to be represented in the DDSketch.

Tests

Added and updated tests covering:

  • Prometheus histograms containing +Inf buckets.
  • DDSketch conversion with finite and +Inf buckets.
  • Histograms containing only a +Inf bucket.
  • Filtering of empty sketches and aggregated histograms.
  • Successful normalization of histograms containing +Inf observations.

@anvithsg2004
anvithsg2004 requested a review from a team as a code owner August 18, 2026 12:41
@github-actions github-actions Bot added domain: sources Anything related to the Vector's sources domain: sinks Anything related to the Vector's sinks domain: core Anything related to core crates i.e. vector-core, core-common, etc labels Aug 18, 2026
@github-actions

github-actions Bot commented Aug 18, 2026

Copy link
Copy Markdown
Contributor

All contributors have signed the CLA ✍️ ✅
Posted by the CLA Assistant Lite bot.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: aa0cd0170c

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread lib/vector-core/src/metrics/ddsketch.rs
@anvithsg2004
anvithsg2004 force-pushed the fix/datadog-metrics-inf-bucket branch from aa0cd01 to 4c1df75 Compare August 18, 2026 12:47
@anvithsg2004

Copy link
Copy Markdown
Author

I have read the CLA Document and I hereby sign the CLA

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 4c1df75ddb

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread src/sources/prometheus/parser.rs
@anvithsg2004
anvithsg2004 force-pushed the fix/datadog-metrics-inf-bucket branch from 4c1df75 to 48d9b4f Compare August 18, 2026 12:57

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 48d9b4fb86

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread lib/vector-core/src/metrics/ddsketch.rs
@anvithsg2004
anvithsg2004 force-pushed the fix/datadog-metrics-inf-bucket branch from 48d9b4f to cc619bb Compare August 18, 2026 13:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

domain: core Anything related to core crates i.e. vector-core, core-common, etc domain: sinks Anything related to the Vector's sinks domain: sources Anything related to the Vector's sources

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant