feat: connect orchestrator and frontend#1564
Open
Scra3 wants to merge 175 commits into
Open
Conversation
…premature deps, add smoke test - Rewrite CLAUDE.md with project overview and architecture principles, remove changelog - Remove unused dependencies (ai-proxy, sequelize, zod) per YAGNI - Add smoke test so CI passes Co-Authored-By: Claude Opus 4.6 <[email protected]>
… document system architecture
- Lint now covers src and test directories
- Replace require() with import, use stronger assertion (toHaveLength)
- Add System Architecture section describing Front/Orchestrator/Executor/Agent
- Mark Architecture Principles as planned (not yet implemented)
- Remove redundant test/.gitkeep
- Make index.ts a valid module with export {}
Co-Authored-By: Claude Opus 4.6 <[email protected]>
…erver (#1504) Co-authored-by: Claude Opus 4.6 (1M context) <[email protected]> Co-authored-by: alban bertolini <[email protected]>
…+ DatabaseStore) (#1506)
…xecutor factories (#1510)
…ain (#1512) Co-authored-by: alban bertolini <[email protected]>
- Remove McpClient.tools property, loadTools() returns local array - Rename closeConnections() → dispose() - Rename testConnections() → checkConnection() - Add McpServers type export - Rename mcpServerConfigs → toolConfigs in create-ai-provider - Update all tests accordingly Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
…eability Add sourceId to McpToolRef so that persisted execution data (pendingData, executionParams) tracks which MCP server provided the tool. This fixes tool lookup on re-entry (confirmation flow) when multiple servers expose tools with the same name. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
… collection schema - Replace non-null assertion with explicit McpToolNotFoundError when AI selects a tool name that doesn't match any available tool - Resolve related collection name from parent schema before looking up the related schema in cache, fixing cases where relation name differs from target collection name Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Merge main into feature branch, resolve conflicts by taking main's versions of router.ts, create-ai-provider.ts and their tests. Remove deleted mcp-config-checker.ts. Bump workflow-executor internal deps to match main. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
…TCH body
The server validator (Joi) only accepts { status } — extra fields cause a
400 ValidationFailedError. The error message is still logged locally via
stepErrorMessage on failure.
Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
…nt camelCase deserialization The JSON:API deserializer in agent-client converts all attribute keys to camelCase (card_number → cardNumber). ReadRecordStepExecutor looks up values by original field name, so snake_case fields came back undefined and were stripped from executionResult. Add restoreFieldNames() in AgentClientAgentPort.getRecord to reverse the camelCase mapping using the original field names from the query, ensuring executors receive values keyed by the field name they requested. Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
![macroscopeapp[bot]](https://avatars.githubusercontent.com/in/900172?s=60&v=4){kind=small}
…d and values Same camelCase deserialization issue as getRecord: apply restoreFieldNames to related records using both primaryKeyFields and requested fields so that extractRecordId finds the correct PK values and callers receive snake_case keys. Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
…lues Same camelCase deserialization issue as getRecord/getRelatedData: agent-client's JSON:API deserializer converts response keys to camelCase. Restore original names using the input values keys, which are already in the caller's original format. Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
…client to 1.39.5 Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
…pending data When the executor picks up a guidance step from the polling loop (before the user has submitted anything), it now returns awaiting-input instead of throwing StepStateError. This keeps the step pending so the user trigger can process it with the submitted data. Also makes userInput optional so users can submit a guidance step without providing any text input. Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
…restoreFieldNames Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
… rethrow path Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
…ect to fix OpenAI rejection
OpenAI requires type: "object" at the root of a tool schema. Using
z.discriminatedUnion directly as schema serialized to anyOf, causing
a 400 "got type: None" error on update-data steps with ≥2 fields.
Wrap in z.object({ input: ... }) (same pattern as the frontend) so
the root is always type: "object". Switch to z.union for the field
variants — discriminatedUnion brought no benefit for LLM selection.
Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
… root schema shape Add two tests to prevent the OpenAI 400 regression from silently re-appearing: one asserting the root schema is a ZodObject (not a union), and one covering the multi-field z.union path with a flat payload rejection check. Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
…apper for update-record-field
The integration tests were still passing the old flat args shape
{ fieldName, value, reasoning } to the mock model. After wrapping
the tool schema in z.object({ input: ... }), the executor now
destructures result.input — so the mocks need the wrapper too.
Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
…r) when finding pending step After a back change, errored steps are stored as done:false + context.error instead of done:true. The find() must exclude them so the executor picks the next genuinely pending step and not the already-failed one. Also includes a temporary throw in ReadRecordStepExecutor for manual front-end error testing. Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
… first non-done The orchestrator is the source of truth for which step to execute next. Always pick the last entry in workflowHistory rather than scanning for the first non-done/non-cancelled/non-errored step. Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
Closed
3 tasks
| run: ServerHydratedWorkflowRun, | ||
| err: WorkflowExecutorError, | ||
| ): MalformedRunInfo { | ||
| const pending = run.workflowHistory.at(-1) ?? null; |
There was a problem hiding this comment.
🟡 Medium adapters/forest-server-workflow-port.ts:134
In toMalformedInfo, accessing run.workflowHistory.at(-1) throws TypeError: Cannot read properties of undefined (reading 'at') when workflowHistory is null or undefined. Since this method is only called from error handlers during malformed run recovery, a missing workflowHistory causes the error handling itself to crash. Consider guarding the access with optional chaining: run.workflowHistory?.at(-1).
Suggested change
| const pending = run.workflowHistory.at(-1) ?? null; | |
| const pending = run.workflowHistory?.at(-1) ?? null; |
🚀 Reply "fix it for me" or copy this AI Prompt for your agent:
In file packages/workflow-executor/src/adapters/forest-server-workflow-port.ts around line 134:
In `toMalformedInfo`, accessing `run.workflowHistory.at(-1)` throws `TypeError: Cannot read properties of undefined (reading 'at')` when `workflowHistory` is `null` or `undefined`. Since this method is only called from error handlers during malformed run recovery, a missing `workflowHistory` causes the error handling itself to crash. Consider guarding the access with optional chaining: `run.workflowHistory?.at(-1)`.
Evidence trail:
packages/workflow-executor/src/adapters/forest-server-workflow-port.ts line 134: `run.workflowHistory.at(-1)` without optional chaining
packages/workflow-executor/src/adapters/server-types.ts line 120: `workflowHistory: ServerStepHistory[]` (non-nullable type, but data from network)
packages/workflow-executor/src/adapters/forest-server-workflow-port.ts lines 63-76: catch block calling `toMalformedInfo` at line 69 when error is `WorkflowExecutorError`
packages/workflow-executor/src/adapters/forest-server-workflow-port.ts lines 107-128: `toDispatch` throws `InvalidStepDefinitionError` at lines 109 and 118 BEFORE `workflowHistory` is accessed
packages/workflow-executor/src/adapters/forest-server-workflow-port.ts lines 55-57: data comes from server HTTP response with no runtime type validation
… run) at(-1) picks the orchestrator's current step but must still return null when that step is already done — the run is complete and nothing to execute. Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
Scra3
changed the base branch from
main
to
feat/prd-214-setup-workflow-executor-package
Scra3
changed the base branch from
feat/prd-214-setup-workflow-executor-package
to
main
The model sometimes returns snake_case (first_name) when the actual displayName is camelCase or unseparated (firstname). Add a normalized fallback that strips separators and lowercases before matching, so a hallucinated variation doesn't kill an otherwise correct step. Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
…allback Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
…fails before timeout (#1587)
…ape (#1583) fix(workflow-executor): fix MCP step path — type shape and id matching MCP-typed workflow steps failed end-to-end before this change. Two independent bugs on the path getMcpServerConfigs → loadRemoteTools → getFilteredTools: 1. Port type mismatch (PRD-357). The orchestrator's /liana/mcp-server-configs-with-details endpoint returns Record<string, ToolConfig>, but the executor port was typed McpConfiguration[] and runner.fetchRemoteTools called .map on it — every MCP step crashed with "TypeError: configs.map is not a function" before reaching loadRemoteTools. Tests masked the bug by mocking mockResolvedValue([]) at 8 call sites, which matched the wrong type and short-circuited the buggy branch. The port now returns McpServers. The { configs } wrap to langchain's McpConfiguration shape lives in runner.fetchRemoteTools, one site, after the empty-record short-circuit. 2. Tool id matching (PRD-362). The executor's filter compared tool.sourceId (server display name) against step.mcpServerId (DB id written by the frontend), so any workflow that specified an MCP server failed with NoMcpToolsError regardless of configuration. The stable DB id from each ToolConfig entry is now threaded through ai-proxy (McpClient, ForestIntegrationClient, and the integration factories) onto RemoteTool.mcpServerId, and the executor matches by id. NoMcpToolsError's technical message now includes the requested id and the loaded id list so misconfigurations are diagnosable from structured logs; the user-facing message stays generic per the dual-message convention. The new tool-side field is named mcpServerId (not id) to read honestly at the consumer site — tool.mcpServerId === step.mcpServerId expresses the FK relationship plainly. McpServerConfig.id and ForestIntegrationConfig.id keep id since those mirror the wire shape (which itself mirrors the ai_mcp_configs PK column). Also drops the now-unused McpConfiguration re-export from the port and barrel (used only internally by AiModelPort/adapters, imported directly from @forestadmin/ai-proxy), drops a wire-shape comment that the type and route URL already document, and renames a test config key from "data-gouv" to "mcp-server-1" to keep the test free of real-world references. fixes PRD-357 fixes PRD-362
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
MAIN BRANCH TO INTRODUCE WORKFLOW EXECUTOR.
Note
Add
@forestadmin/workflow-executorpackage and connect it to the agent frontendpackages/workflow-executorpackage: a standalone HTTP + polling service that executes AI-driven workflow steps (condition, read-record, update-record, trigger-action, load-related-record, mcp, guidance) against a Forest Admin orchestrator.WorkflowExecutorProxyRouteto the agent that proxies/_internal/workflow-executions/:runIdGET and POST requests to the configured executor URL whenworkflowExecutorUrlis set in agent options.AiClient,createBaseChatModel,getAiConfiguration, andvalidateAiConfigurationsto@forestadmin/ai-proxy, centralizing model instantiation and MCP connection lifecycle; Zendesk, Kolar, and Snowflake tools now propagate an optionalmcpServerId.forest-workflow-executor), in-memory and database-backed run stores (SQLite/Postgres via Sequelize + Umzug migrations), JWT-authenticated Koa HTTP server, structured logging, retry helpers, and graceful SIGTERM/SIGINT shutdown.workflowExecutorUrlon the agent — omitting it leaves behavior unchanged, but once set, all/_internal/workflow-executionstraffic is forwarded externally and auth headers are relayed as-is.Macroscope summarized 74e4b8f.