Add Functions gRPC core helpers#282
Conversation
Expose low-level protobuf codecs and single work-item execution helpers for Azure Functions Durable JS gRPC consolidation without adding Functions metadata support. Co-authored-by: Copilot <[email protected]>
Expose stable byte-oriented worker processing methods and endpoint/taskHub client options for Azure Functions Durable JS gRPC consolidation. Co-authored-by: Copilot <[email protected]>
Co-authored-by: Copilot <[email protected]>
Replace the work-item-executor extraction and base64 protobuf helpers with a minimal, Python-aligned implementation. processOrchestratorRequest and processEntityBatchRequest now reuse the existing worker execution path via an in-process capturing stub (mirroring durabletask-python PR #155's null-stub pattern) instead of refactoring the worker. Removed the base64 helper module, the object-level execute* methods, the V2 EntityRequest host path, and the related exports and tests. Co-authored-by: Copilot <[email protected]>
There was a problem hiding this comment.
Pull request overview
Adds a minimal “single work-item” execution surface to the Durable Task JS worker so host integrations (e.g., Azure Functions) can execute one orchestrator/entity batch request per invocation without running the long-lived gRPC streaming loop. This aligns with the SDK’s role as a low-level Durable Task Scheduler (sidecar) client/worker implementation.
Changes:
- Added byte-level helper methods on
TaskHubGrpcWorkerto process serializedOrchestratorRequest/EntityBatchRequestpayloads and return serialized responses. - Introduced an in-process
CapturingSidecarStubthat captures completion payloads instead of sending them over gRPC. - Added end-to-end tests and README documentation for the new host-integration surface.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
| README.md | Documents the new low-level host integration APIs and clarifies the non–Durable Functions programming model scope. |
| packages/durabletask-js/test/functions-grpc-support.spec.ts | Adds e2e-style tests covering the new byte processors for orchestrations and entities. |
| packages/durabletask-js/src/worker/task-hub-grpc-worker.ts | Implements the new public byte-processing methods and the capturing sidecar stub. |
Co-authored-by: Copilot <[email protected]>
Co-authored-by: Copilot <[email protected]>
…ator Co-authored-by: Copilot <[email protected]>
…t glue Co-authored-by: Copilot <[email protected]>
…ease pipeline - Add DurableOrchestrationContext/DurableEntityContext (context.df.* adapters) + wrapOrchestrator/wrapEntity - Add RetryOptions, callHttp (throws), parentInstanceId; align newGuid/callSubOrchestrator with v3 signatures - Add v3 client query-return types (DurableOrchestrationStatus, OrchestrationRuntimeStatus, EntityStateResponse, PurgeHistoryResult) - Add client getStatus/getStatusAll/getStatusBy/readEntityState/purgeInstanceHistory/startNew/waitForCompletionOrCreateCheckStatusResponse - Add deprecated DurableOrchestrationClient alias - Wire host-provided maxGrpcMessageSizeInBytes into gRPC channel options (Python parity) - Remove dead CapturingSidecarStub.abandonRequest field (keep required no-op method) - Add durable-functions package to build/release pipelines
| "dependencies": { | ||
| "@azure/functions": "^4.16.1", | ||
| "@grpc/grpc-js": "^1.14.4", | ||
| "@microsoft/durabletask-js": "0.3.0" |
There was a problem hiding this comment.
TODO: remember to bump this to 0.4.0, and we need to release @microsoft/durabletask-js first
also confirm whether @azure/functions 4.16.1 has the extension changes required,
There was a problem hiding this comment.
Release-readiness findings (version bump to 0.4.0 is done in code; the two open items below are release-time actions):
1) @azure/functions 4.16.1 floor — confirmed adequate.
- v4.16.0 carried the substantive change: bumped
@azure/function-extensions-base0.2.0 → 0.3.0 and dropped Node 18 from CI (which lines up with this package's Node >= 22 floor). - v4.16.1 is a security-servicing release over 4.16.0 (MSRC updates); v4.16.2 is functionally identical to 4.16.1 (only reclaimed the npm
latesttag from an erroneously published 3.6.0). - This package uses only stable generic-binding APIs (
trigger.generic/input.genericcarryingdurableRequiresGrpc), available since v4.0, and does not depend on@azure/function-extensions-basedirectly. So^4.16.1is a sound "recent + security-patched" floor (it resolves to 4.16.2 today). The actual durable-gRPC enablement (durableRequiresGrpc→ gRPC channel) is host-extension-side, not gated by the npm library version.
2) Core release ordering — a real blocker to verify before publish.
- The core package (
@microsoft/durabletask-js) on this branch still declaresversion: 0.3.0, but it adds new public APIs this package relies on (processOrchestratorRequest,addNamedOrchestrator,CapturingSidecarStub, …). The published0.3.0on npm does NOT contain those. - This package's dependency is pinned to
"@microsoft/durabletask-js": "0.3.0". Locally the monorepo workspace links the in-tree core (which has the new APIs), so E2E passes; but once published to npm,0.3.0would resolve to the old published core and fail at runtime (processOrchestratorRequest is not a function). - Release checklist: (a) bump the core version (e.g. 0.4.0) and publish
@microsoft/durabletask-jsfirst; (b) update this package's dep from0.3.0to the new core version (0.4.0/^0.4.0); (c) then publish[email protected].
Leaving this thread open until the core is released and the dependency is bumped.
Drop rootDir, outDir, include, and exclude since they are inherited from ./tsconfig.json. Keep only the baseUrl + paths override that points @microsoft/durabletask-js imports at the built dist output. Co-authored-by: Copilot App <[email protected]>
…ment call style and update tests
…to DurableFunctionsClient
…m/microsoft/durabletask-js into pr/282/yunchuwang/functions-grpc-support
… Functions - Introduced `EntityId` class for classic v3 entity identifiers. - Enhanced `DurableEntityContext` to include `isNewlyConstructed` and `entityId` properties. - Implemented `signalEntity` method in `DurableEntityContext` for signaling other entities. - Updated `DurableOrchestrationContext` to track and set custom status. - Added unit tests for new features and behaviors.
…ayload in DurableFunctionsClient
Collapse the 4.0.0-alpha.0 changelog to a single bullet and rewrite the README to focus on what the package supports and why it is needed, dropping the implementation-plan / phase-status / open-questions sections. Co-authored-by: Copilot <[email protected]>
…onContext type aliases Co-authored-by: Copilot <[email protected]>
…rator Co-authored-by: Copilot App <[email protected]>
… context.log to classic orchestration context Co-authored-by: Copilot App <[email protected]>
…3-compatible) Co-authored-by: Copilot App <[email protected]>
…InstanceHistoryBy (v3-compat) Co-authored-by: Copilot App <[email protected]>
andystaples
left a comment
There was a problem hiding this comment.
Review summary
Two things ship here: (1) the advertised core gRPC byte-processors (processOrchestratorRequest / processEntityBatchRequest + CapturingSidecarStub) in task-hub-grpc-worker.ts, and (2) an entire new [email protected] package (packages/azure-functions-durable) — the intended next-major replacement for Azure/azure-functions-durable-js v3.
The core helpers are clean, and reusing the existing execution path with an in-process capturing stub is a nice low-risk approach. The v3 compat adapters (context.df.* for orchestrations/entities and the client method surface) are genuinely broad and well documented.
My main concern is the "drop-in / run unchanged" framing in the README: this is a strong compatibility shim for the common method surface, but several heavily-used v3 patterns and the entire durable-client authoring model are not covered — and there's one silent-failure bug in the orchestrator adapter. Details are inline; grouped here:
Bug (please fix)
wrapOrchestratorpicks classic-vs-native by parameter arity, which silently mis-runs a native single-arg orchestrator (async function*(ctx)) — the body never executes and no error is thrown.
Behavioral breaks vs v3 (worth closing or documenting)
createTimerreturns a non-cancelableTask; theTask.any+timeoutTask.cancel()timeout pattern breaks, and theTaskresult shape changed (isCompleted/isFaulted/result→isComplete/isFailed/getResult()).- Orchestration/entity contexts no longer extend
InvocationContext. - Missing top-level exports:
DummyOrchestrationContext/DummyEntityContext(breaks users' tests), error types (DurableError/AggregatedError/TaskFailedError) forinstanceof,ManagedIdentityTokenSource/TokenSource; theapp.client.*durable-client trigger helpers andsetExceptionPropertiesProviderare also gone. - Client deltas:
getStatusreturns| undefinedand ignoresshowHistory;createCheckStatusResponsenow requires a definedrequest;startNewdrops theversionoption.
Nits
parseJsondoesn't guard""(JSON.parse("")throws).- Core: a version-mismatch abandon makes
processOrchestratorRequestthrow a generic error. - Spurious
deserializeBinaryreorder in the generated proto.d.ts. engines.node >= 22vs@types/node@^18; the Node-22 floor is a notable consumer bump.
Overall a promising foundation — I'd fix the arity bug and soften/annotate the compatibility claims before this reads as a transparent v3 replacement. Leaving this as comments (non-blocking).
… empty-string parseJson Two self-contained fixes in the durable-functions v4 compat package: - rewind(): delegate to core TaskHubGrpcClient.rewindInstance() instead of throwing 'rewind is not yet supported'; the optional reason defaults to an empty string. - parseJson(): treat empty-string serialized input/output/customStatus as undefined so getStatusAll/getStatusBy no longer throw 'Unexpected end of JSON input'. Adds one regression test per fix (durable-functions unit suite: 67 tests, all green). Co-authored-by: Copilot App <[email protected]>
Brings the PR branch up to date with main so the compat layer can build against the current core public API: cancellable TimerTask (#293), native AggregateError WhenAll semantics (#302), rewind support (#296), and related fixes. Clean auto-merge (proto + worker). Co-authored-by: Copilot App <[email protected]>
Resolves andystaples' review comments plus a ListAll E2E bug on packages/azure-functions-durable: - #1 orchestrator arity: detect generator kind (sync=classic v3, async=core native) instead of an arity heuristic; driver now handles sync+async generators so single-param native orchestrators actually execute. - #2 parseJson tolerates non-JSON serializedOutput (fixes getStatusAll 400). - #3 createCheckStatusResponse accepts undefined request (baseUrl fallback). - #4 createTimer returns TimerTask (cancelable) in its type signature. - #5 re-export TaskFailedError from core + compat; document removed v3-only exports in CHANGELOG/README. - #6/#7 document getStatus + classic-context breaking changes. - #8 bump @types/node to ^22 to match engines.node>=22. - #9 set version 0.4.0 + document release order. - #10 distinct abandon error for the single work-item execution path. - #11 revert proto codegen churn to origin/main. Adds regression tests for items #1/#2/#3/#5/#10. Co-authored-by: Copilot App <[email protected]>
…, startNew version
- (1) orchestration-context: detect classic vs core-native orchestrators by generator/async kind (async* and async => native; sync generator => classic; plain sync => arity fallback) and drive both sync/async generators; add end-to-end regression tests through the real core executor.
- (2) createTimer already returns cancelable TimerTask (type now visible); add type/Task.any cancel test.
- (3) doc-only: classic contexts no longer extend InvocationContext (entity ctx is {df}); add replay-nondeterminism rationale to CHANGELOG/README.
- (4) createCheckStatusResponse already accepts undefined request with baseUrl fallback (v3-faithful).
- (5) getStatus returns non-optional DurableOrchestrationStatus and throws on not-found (v3 parity); showHistory populates core history; showInput maps to fetchPayloads; startNew forwards v3 version option. Docs note gRPC-path boundaries.
Co-authored-by: Copilot App <[email protected]>
- getStatus throws on not-found with a v3-style 'DurableClient error:' message (verified v3 DurableClient.getStatus throws on the extension's HTTP 404; returns non-optional DurableOrchestrationStatus). - showInput now gates only the top-level input (payloads always fetched); output/customStatus are always returned, matching v3. - showHistoryOutput now strips input/result payloads from history entries when falsy and keeps them when true. history stays core HistoryEvent[] (v3 types history as Array<unknown>). - startNew forwards the v3 version option (already wired to core scheduleNewOrchestration). Co-authored-by: Copilot App <[email protected]>
v3 DurableClient.getStatus throws on the extension's HTTP 404 with 'This usually means we could not find any data associated with the instanceId provided: <id>.'. Mirror that sentence verbatim (only the HTTP-404-specific first sentence is replaced, since the gRPC path has no HTTP 404). Update the not-found tests to assert the v3 message. Co-authored-by: Copilot App <[email protected]>
The single-work-item abandon path is JS-specific for the Azure Functions host integration. durabletask-python has no equivalent single-work-item helper, and its worker-loop abandon hands the item back over a real sidecar stub rather than no-opping, so the prior 'matches the Python provider, whose null stub no-ops' note was inaccurate. Comment-only change. Co-authored-by: Copilot App <[email protected]>
The prior note referenced durabletask-python behavior, which is fragile: the Python single-work-item support lives on an unmerged fork PR and its details can drift. Trim the comment to the JS-only rationale for the no-op abandon on the single-work-item host path. Comment-only change. Co-authored-by: Copilot App <[email protected]>
…pers for v3 parity Restores the 10 client-starter registration helpers (http, timer, storageBlob, storageQueue, serviceBusQueue, serviceBusTopic, eventHub, eventGrid, cosmosDB, generic). Each adds the durableClient input binding and injects a DurableFunctionsClient as the handler's 2nd arg, so (trigger, client, context) handlers work without manual df.input.durableClient() + df.getClient(context). Co-authored-by: Copilot App <[email protected]>
Surface the classic v3 client.startNew alias alongside the canonical scheduleNewOrchestration in the README examples, and add a verified alias -> core method table. Docs-only; these aliases already ship in client.ts. Co-authored-by: Copilot App <[email protected]>
What
Adds the minimal host-integration surface so Azure Functions can drive a single durable work item per invocation, without running the long-lived gRPC worker loop. This mirrors the Azure Functions consolidation done for Python in durabletask-python PR #155.
API
Two methods on
TaskHubGrpcWorker:processOrchestratorRequest(request: Uint8Array): Promise<Uint8Array>processEntityBatchRequest(request: Uint8Array): Promise<Uint8Array>Each deserializes a TaskHubSidecarService protobuf payload, executes one work item, and returns the serialized response. Host integrations own any transport encoding (for example base64); base64 stays out of the core SDK.
Design (aligned with Python #155)
Rather than refactoring the worker, these methods reuse the worker's existing internal execution path (
_executeOrchestratorInternal/_executeEntityInternal) and pass an in-processCapturingSidecarStubthat records the completion payload instead of sending it over gRPC. This is the same null-stub pattern Python uses (AzureFunctionsNullStub).Deliberately NOT included (kept minimal)
execute*public methods (the byte processors are the single entry point).EntityRequesthost path (not present in Python fix: Reset customStatus on continue-as-new in InMemoryOrchestrationBackend #155; the worker's internal V2 handling for the DTS backend is unchanged).metadataGenerator.Tests
packages/durabletask-js/test/functions-grpc-support.spec.tscovers both byte processors end to end.