Custom events, PostHog exporter, async identify, and hardened event pipeline#37
Open
naji247 wants to merge 3 commits into
Open
Custom events, PostHog exporter, async identify, and hardened event pipeline#37naji247 wants to merge 3 commits into
naji247 wants to merge 3 commits into
Conversation
…vent pipeline Features: - publish_custom_event(): publish agentcat:custom events against a tracked server or an MCP session ID string, with support for message/user intent, parameters, response, duration, error state, tags, and properties. Session IDs derived from MCP session IDs are deterministic (SHA-256 -> KSUID), so custom events correlate with SDK-tracked sessions across languages. - PostHog telemetry exporter: /batch capture events, person profile updates, $exception events on errors, and optional AI-observability spans behind enable_ai_tracing. Session IDs mapped to deterministic UUIDv7. - identify hooks may now be async; sync hooks keep working unchanged. - Identity change detection: identify events are published only when the resolved identity actually changes; user_data is merged across calls and identities are cached (LRU, 1000 entries). Fixes: - redact_sensitive_information is now applied to live events. Previously the redaction pass silently skipped the event model, so custom redaction functions never ran on published events. Async redaction functions are now supported and awaited. - The low-level Server integration now captures full structured error data (stack frames, chained exceptions) instead of only the error message. - Event truncation applies per-field limits (user intent, error message, resource and server/client names, stack frames, response text blocks) and raises general limits to match the published event size contract (32KB per string, depth 10, breadth 100, 100KB total). Reliability: - The request path is fully fail-open: failures in identify hooks, tag and property callbacks, error capture, or event publishing are logged and dropped without affecting the customer's tool call - including exceptions whose own __str__ raises. - The queue worker survives poisoned events (cyclic structures, NaN/Inf, bytes, objects with raising __repr__) and continues delivering subsequent events. - publish_custom_event raises only its documented ValueError/TypeError contract on invalid arguments. Tests: 611 passed, 4 skipped, including new failure-injection, custom event, PostHog, identify, and truncation suites.
The identify hook already runs on every captured request; an agentcat:identify event is now published each time it returns a valid identity, instead of only when the identity changed. Merge semantics are unchanged: user_id/user_name are overwritten and user_data keys accumulate across calls, with the LRU cache retained for merge continuity across server-object recreation. Stateless mode now bypasses the identity cache and publishes the raw identity per request, so user_data from different actors sharing a server instance can no longer merge together. Tests updated to the new semantics, including a stateless no-cross-request-merge case. Full suite: 611 passed, 4 skipped.
Code-quality pass over the branch; no behavior changes. - exceptions reuses logging.safe_error_string instead of a local copy. - Async redaction functions now cost one event loop per event instead of one per redacted string; sync paths are unchanged. - Redaction rebuilds the event with model_construct, skipping re-validation of data that came from an already-valid model (serialization verified identical). - PostHog exporter computes the session UUID once per export. - publish_custom_event normalizes its optional payload once instead of guarding every field access. - Event-type validator generalizes to the set of SDK-defined event types instead of special-casing one constant. - Removed dead IdentityCache.__contains__ and a redundant second parse of error results on the low-level path. Full suite: 611 passed, 4 skipped.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR brings the Python SDK to feature parity with the TypeScript SDK and hardens the event pipeline so that no SDK failure can ever surface into a customer's MCP server. It adds custom event publishing, a PostHog exporter, async identify support with per-request identify events and identity merging, fixes a bug where redaction functions never ran on published events, and adds an extensive failure-injection test suite.
New features
publish_custom_eventPublish
agentcat:customevents for actions outside the MCP request lifecycle:The first argument accepts either a tracked server or an MCP session ID string. String session IDs are converted with the same deterministic SHA-256 → KSUID derivation used by the TypeScript and Go SDKs, so custom events correlate with tracked sessions across languages and server restarts.
PostHog exporter
Joins the existing OTLP, Datadog, and Sentry exporters:
Sends capture events with mapped names (
mcp_tool_call,mcp_initialize, ...), person profile updates from identified actors,$exceptionevents on errors, and optional$ai_spanAI-observability events. Session IDs are mapped to deterministic UUIDv7 values.Async identify + identity merging
identifyhooks may now beasync def; sync hooks work unchanged.agentcat:identifyevent is published each time it returns an identity. Identities merge across calls —user_id/user_nameare overwritten by the newest result whileuser_datakeys accumulate — with an LRU cache (1000 sessions) providing merge continuity across server-object recreation. Actor attribution on individual events is unaffected.user_datafrom different actors sharing a server instance can never merge together.Bug fixes
redact_sensitive_informationfunctions never ran on the live publish path. The three e2e tests documenting this were previously marked expected-failure and now pass. Async redaction functions are also supported and awaited.Serverpath. The low-level integration recorded only the error message; it now captures the same structured data (stack frames, chained exceptions, in-app detection) as the FastMCP integrations.publish_custom_eventargument errors are confined to the documentedValueError/TypeErrorcontract.Truncation alignment
Event truncation now applies dedicated per-field limits before the total-size pass — user intent and error messages (2048 chars), resource/server/client names (256), stack frames (50, keeping first and last 25), response text blocks (32KB) — and the general limits were aligned with the published event contract: 32KB per string, depth 10, breadth 100, 100KB total (previously 10KB / 5 / 500).
Reliability hardening
The SDK's contract is that analytics must never break the host server. This PR closes every gap found in an adversarial review of the request path:
__str__raises, callbacks returning non-dict values, or objects with a raising__bool__.__repr__, redaction functions that raise sync or async) and keeps delivering subsequent events.Each fix is covered by
tests/test_failure_injection.py(38 tests), several of which exercise real Streamable HTTP transports end to end.Compatibility notes
agentcat:identifyevent is published for every captured request where the identify hook returns an identity. Per-event actor fields are unchanged, and identify events now carry the accumulated (merged)user_data.redact_sensitive_informationwill now see redaction actually applied to published events.publish_custom_event,CustomEventData,PostHogExporterConfig.Testing