Skip to content

Commit b382d1a

Browse files
kitlangtonkitlangton
authored
docs(effect): track schema migration progress with concrete file checklists (#23242)
1 parent 23f3147 commit b382d1a

1 file changed

Lines changed: 254 additions & 28 deletions

File tree

‎packages/opencode/specs/effect/schema.md‎

Lines changed: 254 additions & 28 deletions
Original file line numberDiff line numberDiff line change
@@ -1,12 +1,19 @@
11
# Schema migration
22

3-
Practical reference for migrating data types in `packages/opencode` from Zod-first definitions to Effect Schema with Zod compatibility shims.
3+
Practical reference for migrating data types in `packages/opencode` from
4+
Zod-first definitions to Effect Schema with Zod compatibility shims.
45

56
## Goal
67

7-
Use Effect Schema as the source of truth for domain models, IDs, inputs, outputs, and typed errors.
8+
Use Effect Schema as the source of truth for domain models, IDs, inputs,
9+
outputs, and typed errors. Keep Zod available at existing HTTP, tool, and
10+
compatibility boundaries by exposing a `.zod` static derived from the Effect
11+
schema via `@/util/effect-zod`.
812

9-
Keep Zod available at existing HTTP, tool, and compatibility boundaries by exposing a `.zod` field derived from the Effect schema.
13+
The long-term driver is `specs/effect/http-api.md` — once the HTTP server
14+
moves to `@effect/platform`, every Schema-first DTO can flow through
15+
`HttpApi` / `HttpRouter` without a zod translation layer, and the entire
16+
`effect-zod` walker plus every `.zod` static can be deleted.
1017

1118
## Preferred shapes
1219

@@ -24,17 +31,14 @@ export class Info extends Schema.Class<Info>("Foo.Info")({
2431
}
2532
```
2633

27-
If the class cannot reference itself cleanly during initialization, use the existing two-step pattern:
34+
If the class cannot reference itself cleanly during initialization, use the
35+
two-step `withStatics` pattern:
2836

2937
```ts
30-
const _Info = Schema.Struct({
38+
export const Info = Schema.Struct({
3139
id: FooID,
3240
name: Schema.String,
33-
})
34-
35-
export const Info = Object.assign(_Info, {
36-
zod: zod(_Info),
37-
})
41+
}).pipe(withStatics((s) => ({ zod: zod(s) })))
3842
```
3943

4044
### Errors
@@ -49,27 +53,89 @@ export class NotFoundError extends Schema.TaggedErrorClass<NotFoundError>()("Foo
4953

5054
### IDs and branded leaf types
5155

52-
Keep branded/schema-backed IDs as Effect schemas and expose `static readonly zod` for compatibility when callers still expect Zod.
56+
Keep branded/schema-backed IDs as Effect schemas and expose
57+
`static readonly zod` for compatibility when callers still expect Zod.
58+
59+
### Refinements
60+
61+
Reuse named refinements instead of re-spelling `z.number().int().positive()`
62+
in every schema. The `effect-zod` walker translates the Effect versions into
63+
the corresponding zod methods, so JSON Schema output (`type: integer`,
64+
`exclusiveMinimum`, `pattern`, `format: uuid`, …) is preserved.
65+
66+
```ts
67+
const PositiveInt = Schema.Number.check(Schema.isInt()).check(Schema.isGreaterThan(0))
68+
const NonNegativeInt = Schema.Number.check(Schema.isInt()).check(Schema.isGreaterThanOrEqualTo(0))
69+
const HexColor = Schema.String.check(Schema.isPattern(/^#[0-9a-fA-F]{6}$/))
70+
```
71+
72+
See `test/util/effect-zod.test.ts` for the full set of translated checks.
5373

5474
## Compatibility rule
5575

56-
During migration, route validators, tool parameters, and any existing Zod-based boundary should consume the derived `.zod` schema instead of maintaining a second hand-written Zod schema.
76+
During migration, route validators, tool parameters, and any existing
77+
Zod-based boundary should consume the derived `.zod` schema instead of
78+
maintaining a second hand-written Zod schema.
5779

5880
The default should be:
5981

6082
- Effect Schema owns the type
6183
- `.zod` exists only as a compatibility surface
62-
- new domain models should not start Zod-first unless there is a concrete boundary-specific need
84+
- new domain models should not start Zod-first unless there is a concrete
85+
boundary-specific need
6386

6487
## When Zod can stay
6588

6689
It is fine to keep a Zod-native schema temporarily when:
6790

68-
- the type is only used at an HTTP or tool boundary
91+
- the type is only used at an HTTP or tool boundary and is not reused elsewhere
6992
- the validator depends on Zod-only transforms or behavior not yet covered by `zod()`
7093
- the migration would force unrelated churn across a large call graph
7194

72-
When this happens, prefer leaving a short note or TODO rather than silently creating a parallel schema source of truth.
95+
When this happens, prefer leaving a short note or TODO rather than silently
96+
creating a parallel schema source of truth.
97+
98+
## Escape hatches
99+
100+
The walker in `@/util/effect-zod` exposes three explicit escape hatches for
101+
cases the pure-Schema path cannot express. Each one stays in the codebase
102+
only as long as its upstream or local dependency requires it — inline
103+
comments document when each can be deleted.
104+
105+
### `ZodOverride` annotation
106+
107+
Replaces the entire derivation with a hand-crafted zod schema. Used when:
108+
109+
- the target carries external `$ref` metadata (e.g.
110+
`config/model-id.ts` points at `https://models.dev/...`)
111+
- the target is a zod-only schema that cannot yet be expressed as Schema
112+
(e.g. `ConfigAgent.Info`, `ConfigPermission.Info`, `Log.Level`)
113+
114+
### `ZodPreprocess` annotation
115+
116+
Wraps the derived zod schema with `z.preprocess(fn, inner)`. Used by
117+
`config/permission.ts` to inject `__originalKeys` before parsing, because
118+
`Schema.StructWithRest` canonicalises output (known fields first, catchall
119+
after) and destroys the user's original property order — which permission
120+
rule precedence depends on.
121+
122+
Tracked upstream as `effect:core/wlh553`: "Schema: add preserveInputOrder
123+
(or pre-parse hook) for open structs." Once that lands, `ZodPreprocess` and
124+
the `__originalKeys` hack can both be deleted.
125+
126+
### Local `DeepMutable<T>` in `config/config.ts`
127+
128+
`Schema.Struct` produces `readonly` types. Some consumer code (notably the
129+
`Config` service) mutates `Info` objects directly, so a readonly-stripping
130+
utility is needed when casting the derived zod schema's output type.
131+
132+
`Types.DeepMutable` from effect-smol would be a drop-in, but it widens
133+
`unknown` to `{}` in the fallback branch — a bug that affects any schema
134+
using `Schema.Record(String, Schema.Unknown)`.
135+
136+
Tracked upstream as `effect:core/x228my`: "Types.DeepMutable widens unknown
137+
to `{}`." Once that lands, the local `DeepMutable` copy can be deleted and
138+
`Types.DeepMutable` used directly.
73139

74140
## Ordering
75141

@@ -81,19 +147,179 @@ Migrate in this order:
81147
4. Service-local internal models
82148
5. Route and tool boundary validators that can switch to `.zod`
83149

84-
This keeps shared types canonical first and makes boundary updates mostly mechanical.
85-
86-
## Checklist
87-
88-
- [ ] Shared `schema.ts` leaf models are Effect Schema-first
89-
- [ ] Exported `Info` / `Input` / `Output` types use `Schema.Class` where appropriate
90-
- [ ] Domain errors use `Schema.TaggedErrorClass`
91-
- [ ] Migrated types expose `.zod` for back compatibility
92-
- [ ] Route and tool validators consume derived `.zod` instead of duplicate Zod definitions
93-
- [ ] New domain models default to Effect Schema first
150+
This keeps shared types canonical first and makes boundary updates mostly
151+
mechanical.
152+
153+
## Progress tracker
154+
155+
### `src/config/` ✅ complete
156+
157+
All of `packages/opencode/src/config/` has been migrated. Files that still
158+
import `z` do so only for local `ZodOverride` bridges or for `z.ZodType`
159+
type annotations — the `export const <Info|Spec>` values are all Effect
160+
Schema at source.
161+
162+
- [x] skills, formatter, console-state, mcp, lsp, permission (leaves), model-id, command, plugin, provider
163+
- [x] server, layout
164+
- [x] keybinds
165+
- [x] permission#Info
166+
- [x] agent
167+
- [x] config.ts root
168+
169+
### `src/*/schema.ts` leaf modules
170+
171+
These are the highest-priority next targets. Each is a small, self-contained
172+
schema module with a clear domain.
173+
174+
- [ ] `src/control-plane/schema.ts`
175+
- [ ] `src/permission/schema.ts`
176+
- [ ] `src/project/schema.ts`
177+
- [ ] `src/provider/schema.ts`
178+
- [ ] `src/pty/schema.ts`
179+
- [ ] `src/question/schema.ts`
180+
- [ ] `src/session/schema.ts`
181+
- [ ] `src/sync/schema.ts`
182+
- [ ] `src/tool/schema.ts`
183+
184+
### Session domain
185+
186+
Major cluster. Message + event types flow through the SSE API and every SDK
187+
output, so byte-identical SDK surface is critical.
188+
189+
- [ ] `src/session/compaction.ts`
190+
- [ ] `src/session/message-v2.ts`
191+
- [ ] `src/session/message.ts`
192+
- [ ] `src/session/prompt.ts`
193+
- [ ] `src/session/revert.ts`
194+
- [ ] `src/session/session.ts`
195+
- [ ] `src/session/status.ts`
196+
- [ ] `src/session/summary.ts`
197+
- [ ] `src/session/todo.ts`
198+
199+
### Provider domain
200+
201+
- [ ] `src/provider/auth.ts`
202+
- [ ] `src/provider/models.ts`
203+
- [ ] `src/provider/provider.ts`
204+
205+
### Tool schemas
206+
207+
Each tool declares its parameters via a zod schema. Tools are consumed by
208+
both the in-process runtime and the AI SDK's tool-calling layer, so the
209+
emitted JSON Schema must stay byte-identical.
210+
211+
- [ ] `src/tool/apply_patch.ts`
212+
- [ ] `src/tool/bash.ts`
213+
- [ ] `src/tool/codesearch.ts`
214+
- [ ] `src/tool/edit.ts`
215+
- [ ] `src/tool/glob.ts`
216+
- [ ] `src/tool/grep.ts`
217+
- [ ] `src/tool/invalid.ts`
218+
- [ ] `src/tool/lsp.ts`
219+
- [ ] `src/tool/multiedit.ts`
220+
- [ ] `src/tool/plan.ts`
221+
- [ ] `src/tool/question.ts`
222+
- [ ] `src/tool/read.ts`
223+
- [ ] `src/tool/registry.ts`
224+
- [ ] `src/tool/skill.ts`
225+
- [ ] `src/tool/task.ts`
226+
- [ ] `src/tool/todo.ts`
227+
- [ ] `src/tool/tool.ts`
228+
- [ ] `src/tool/webfetch.ts`
229+
- [ ] `src/tool/websearch.ts`
230+
- [ ] `src/tool/write.ts`
231+
232+
### HTTP route boundaries
233+
234+
Every file in `src/server/routes/` uses hono-openapi with zod validators for
235+
route inputs/outputs. Migrating these individually is the last step; most
236+
will switch to `.zod` derived from the Schema-migrated domain types above,
237+
which means touching them is largely mechanical once the domain side is
238+
done.
239+
240+
- [ ] `src/server/error.ts`
241+
- [ ] `src/server/event.ts`
242+
- [ ] `src/server/projectors.ts`
243+
- [ ] `src/server/routes/control/index.ts`
244+
- [ ] `src/server/routes/control/workspace.ts`
245+
- [ ] `src/server/routes/global.ts`
246+
- [ ] `src/server/routes/instance/index.ts`
247+
- [ ] `src/server/routes/instance/config.ts`
248+
- [ ] `src/server/routes/instance/event.ts`
249+
- [ ] `src/server/routes/instance/experimental.ts`
250+
- [ ] `src/server/routes/instance/file.ts`
251+
- [ ] `src/server/routes/instance/mcp.ts`
252+
- [ ] `src/server/routes/instance/permission.ts`
253+
- [ ] `src/server/routes/instance/project.ts`
254+
- [ ] `src/server/routes/instance/provider.ts`
255+
- [ ] `src/server/routes/instance/pty.ts`
256+
- [ ] `src/server/routes/instance/question.ts`
257+
- [ ] `src/server/routes/instance/session.ts`
258+
- [ ] `src/server/routes/instance/sync.ts`
259+
- [ ] `src/server/routes/instance/tui.ts`
260+
261+
The bigger prize for this group is the `@effect/platform` HTTP migration
262+
described in `specs/effect/http-api.md`. Once that lands, every one of
263+
these files changes shape entirely (`HttpApi.endpoint(...)` and friends),
264+
so the Schema-first domain types become a prerequisite rather than a
265+
sibling task.
266+
267+
### Everything else
268+
269+
Small / shared / control-plane / CLI. Mostly independent; can be done
270+
piecewise.
271+
272+
- [ ] `src/acp/agent.ts`
273+
- [ ] `src/agent/agent.ts`
274+
- [ ] `src/bus/bus-event.ts`
275+
- [ ] `src/bus/index.ts`
276+
- [ ] `src/cli/cmd/tui/config/tui-migrate.ts`
277+
- [ ] `src/cli/cmd/tui/config/tui-schema.ts`
278+
- [ ] `src/cli/cmd/tui/config/tui.ts`
279+
- [ ] `src/cli/cmd/tui/event.ts`
280+
- [ ] `src/cli/ui.ts`
281+
- [ ] `src/command/index.ts`
282+
- [ ] `src/control-plane/adaptors/worktree.ts`
283+
- [ ] `src/control-plane/types.ts`
284+
- [ ] `src/control-plane/workspace.ts`
285+
- [ ] `src/file/index.ts`
286+
- [ ] `src/file/ripgrep.ts`
287+
- [ ] `src/file/watcher.ts`
288+
- [ ] `src/format/index.ts`
289+
- [ ] `src/id/id.ts`
290+
- [ ] `src/ide/index.ts`
291+
- [ ] `src/installation/index.ts`
292+
- [ ] `src/lsp/client.ts`
293+
- [ ] `src/lsp/lsp.ts`
294+
- [ ] `src/mcp/auth.ts`
295+
- [ ] `src/patch/index.ts`
296+
- [ ] `src/plugin/github-copilot/models.ts`
297+
- [ ] `src/project/project.ts`
298+
- [ ] `src/project/vcs.ts`
299+
- [ ] `src/pty/index.ts`
300+
- [ ] `src/skill/index.ts`
301+
- [ ] `src/snapshot/index.ts`
302+
- [ ] `src/storage/db.ts`
303+
- [ ] `src/storage/storage.ts`
304+
- [ ] `src/sync/index.ts`
305+
- [ ] `src/util/fn.ts`
306+
- [ ] `src/util/log.ts`
307+
- [ ] `src/util/update-schema.ts`
308+
- [ ] `src/worktree/index.ts`
309+
310+
### Do-not-migrate
311+
312+
- `src/util/effect-zod.ts` — the walker itself. Stays zod-importing forever
313+
(it's what emits zod from Schema). Goes away only when the `.zod`
314+
compatibility layer is no longer needed anywhere.
94315

95316
## Notes
96317

97-
- Use `@/util/effect-zod` for all Schema -> Zod conversion.
98-
- Prefer one canonical schema definition. Avoid maintaining parallel Zod and Effect definitions for the same domain type.
99-
- Keep the migration incremental. Converting the domain model first is more valuable than converting every boundary in the same change.
318+
- Use `@/util/effect-zod` for all Schema → Zod conversion.
319+
- Prefer one canonical schema definition. Avoid maintaining parallel Zod and
320+
Effect definitions for the same domain type.
321+
- Keep the migration incremental. Converting the domain model first is more
322+
valuable than converting every boundary in the same change.
323+
- Every migrated file should leave the generated SDK output (`packages/sdk/
324+
openapi.json` and `packages/sdk/js/src/v2/gen/types.gen.ts`) byte-identical
325+
unless the change is deliberately user-visible.

0 commit comments

Comments
 (0)