π MethodProxy breaks async generator methods β streaming and observability are mutually exclusive#37
Closed
MFA-X-AI wants to merge 1 commit into
Closed
π MethodProxy breaks async generator methods β streaming and observability are mutually exclusive#37MFA-X-AI wants to merge 1 commit into
MFA-X-AI wants to merge 1 commit into
Conversation
MethodProxy.__call__ unconditionally awaited the wrapped method, so any async generator method (e.g. LLMProtocol.generate_stream) called through the observability proxy raised TypeError. Dispatch on the method kind at call time and return a matching wrapper that emits CALL/EXCEPTION/RESULT around iteration, with RESULT carrying a chunk count.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Every module resolved through the exchange is wrapped in the observability proxy, and
MethodProxy.__call__unconditionally doesawait self._method(...). Async generator methods βLLMProtocol.generate_streambeing the flagship β return an async generator object, which can't be awaited, so any streaming call through the proxy raisesTypeError: object async_generator can't be used in 'await' expression. Since the proxy wrapping is unconditional, streaming currently works only by bypassing the exchange entirely (e.g. unwrapping_obj), which silently exits the observability layer. This PR makes the proxy transparent for all four method shapes: plain async, async generator, and sync generator methods all forward correctly and emit CALL/RESULT/EXCEPTION events.Changes
MethodProxy.__call__is no longer unconditionallyasyncβ it dispatches on the wrapped method's kind viainspect.isasyncgenfunction/inspect.isgeneratorfunctionand returns a matching wrapper (coroutine, async generator, or sync generator)CALLbefore the first chunk,EXCEPTION(then re-raise) if the stream fails mid-flight, andRESULTwith{"stream": True, "chunks": n}after exhaustion β chunk contents are not recorded, keeping event volume boundedGeneratorExit) is treated as normal termination, not an errortests/core/test_proxy_generator_methods.pycovering async/sync generators, mid-stream exceptions, early close, listener-free operation, unchanged plain-method behavior, andMockLLM.generate_streamend-to-end through aProxyTechnical Details
async def __call__: a coroutine that returns an async generator would force callers intoawait-then-iterate, breaking duck-typing against unproxied modules. Plaindef __call__returning the un-awaited coroutine is behaviorally identical for existing callers (await proxy.method(...)works unchanged).call_idpairing.Verification
Without the fix, 6 of the 7 new tests fail with the
TypeErrorabove; with it, all pass.