Skip to content

Commit 78fc6da

Browse files
committed
docs: migrate all book/docs examples from JSON actions to TypedAST actions
- docs/ZYN_GRAMMAR_SPEC.md: rewrite as v3.0 for TypedAST action system - book/README.md: update description and example to TypedAST syntax - book/01-introduction.md: update example, architecture diagram, chapter link - book/02-getting-started.md: rewrite calculator grammar with fold_left_ops, named bindings - book/04-grammar-syntax.md: replace fold_binary with fold_left_ops + make_pair pattern - book/05-semantic-actions.md: complete rewrite covering all 5 action kinds - book/08-zig-example.md: major rewrite - named bindings, no split pattern, consolidated op rules - book/09-reference.md: replace Command Reference with TypedAST Action Reference - book/13-async-runtime.md: update async grammar example with is_async field - book/15-building-dsls.md: update ArtLang/ChartLang grammar snippets - book/16-image-pipeline-dsl.md: update all grammar snippets and Key Concepts section Old JSON command blocks (get_text, fold_binary, define, get_all_children, $1/$2) replaced throughout with named bindings, intern(), fold_left_ops(), prepend_list(), and TypedAST construct expressions.
1 parent baa8377 commit 78fc6da

11 files changed

Lines changed: 1321 additions & 1926 deletions

book/01-introduction.md

Lines changed: 25 additions & 35 deletions
Original file line numberDiff line numberDiff line change
@@ -4,8 +4,8 @@
44

55
Zyn (ZynPEG) is a grammar-driven language frontend system that transforms source code into a typed abstract syntax tree (TypedAST). It combines three powerful concepts:
66

7-
1. **PEG Parsing** - Uses Pest-compatible Parser Expression Grammars for syntax definition
8-
2. **Declarative Semantics** - JSON command blocks describe how to build AST nodes
7+
1. **PEG Parsing** - Parser Expression Grammars with Packrat memoization for O(n) parsing
8+
2. **TypedAST Actions** - Typed construct expressions build AST nodes directly at parse time
99
3. **Universal TypedAST** - A target representation that supports multiple programming paradigms
1010

1111
## The Problem Zyn Solves
@@ -32,24 +32,20 @@ Source Code → Zyn Grammar → TypedAST → Compiler Backend
3232

3333
A Zyn grammar file (`.zyn`) contains:
3434

35-
1. **Grammar rules** - PEG syntax defining what to parse
36-
2. **Semantic actions** - JSON blocks defining what AST nodes to create
35+
1. **Grammar rules** - PEG syntax defining what to parse, with named bindings
36+
2. **Semantic actions** - TypedAST construct expressions defining what AST nodes to create
3737

3838
```zyn
39-
// Grammar rule
39+
// Atomic rule (@) captures matched text automatically
40+
// The binding name 'integer_literal' holds that text in the action
4041
integer_literal = @{ "-"? ~ ASCII_DIGIT+ }
41-
-> TypedExpression {
42-
"get_text": true,
43-
"parse_int": true,
44-
"define": "int_literal",
45-
"args": { "value": "$result" }
46-
}
42+
-> TypedExpression::IntLiteral { value: integer_literal }
4743
```
4844

4945
When this rule matches, Zyn:
50-
1. Extracts the matched text (`get_text`)
51-
2. Parses it as an integer (`parse_int`)
52-
3. Calls `create_int_literal(value)` on the AST builder
46+
1. Captures the matched text automatically (the `@` modifier)
47+
2. Evaluates the action expression, constructing a `TypedExpression::IntLiteral` node
48+
3. Returns the node directly to the parent rule — no separate AST building pass needed
5349

5450
## The TypedAST
5551

@@ -91,37 +87,31 @@ Grammar changes automatically propagate to AST construction - no separate files
9187
```
9288
┌─────────────────┐
9389
│ Source File │
94-
(*.zig)
90+
│ (*.zynml)
9591
└────────┬────────┘
9692
9793
98-
┌─────────────────┐ ┌─────────────────┐
99-
│ Zyn Grammar │────▶│ Pest Parser │
100-
│ (*.zyn) │ │ (generated) │
101-
└─────────────────┘ └────────┬────────┘
102-
103-
104-
┌─────────────────┐
105-
│ Command │
106-
│ Interpreter │
107-
└────────┬────────┘
108-
109-
110-
┌─────────────────┐
111-
│ TypedAST │
112-
│ Builder │
113-
└────────┬────────┘
114-
115-
94+
┌─────────────────┐ ┌──────────────────────┐
95+
│ Zyn Grammar │────▶│ GrammarInterpreter │
96+
│ (*.zyn) │ │ (Packrat memoized │
97+
└─────────────────┘ │ PEG, O(n) time) │
98+
└──────────┬───────────┘
99+
100+
│ TypedAST actions
101+
│ evaluated inline
102+
116103
┌─────────────────┐
117104
│ TypedProgram │
118-
│ (JSON/Binary) │
105+
│ (direct, no │
106+
│ JSON/VM) │
119107
└────────┬────────┘
120108
121109
122110
┌─────────────────┐
123111
│ Compiler │
124112
│ Backend │
113+
│ (HIR → SSA → │
114+
│ Native Code) │
125115
└─────────────────┘
126116
```
127117

@@ -132,7 +122,7 @@ In the following chapters, you'll learn:
132122
- [Chapter 2](./02-getting-started.md): Set up your environment and write your first grammar
133123
- [Chapter 3](./03-using-the-cli.md): Use the CLI for compilation and REPL testing
134124
- [Chapter 4](./04-grammar-syntax.md): Master PEG syntax for parsing
135-
- [Chapter 5](./05-semantic-actions.md): Use JSON commands to build AST nodes
125+
- [Chapter 5](./05-semantic-actions.md): Use TypedAST actions to build AST nodes
136126
- [Chapter 6](./06-typed-ast.md): Understand the TypedAST structure
137127
- [Chapter 7](./07-typed-ast-builder.md): Use the builder API directly
138128
- [Chapter 8](./08-zig-example.md): Walk through a complete Zig implementation

book/02-getting-started.md

Lines changed: 50 additions & 51 deletions
Original file line numberDiff line numberDiff line change
@@ -35,44 +35,40 @@ Create `calc.zyn`:
3535
entry_point: "main",
3636
}
3737
38-
// Program structure
39-
program = { SOI ~ expr ~ EOI }
38+
// Program structure: wrap the expression in a main function
39+
program = { SOI ~ e:expr ~ EOI }
4040
-> TypedProgram {
41-
"commands": [
42-
{ "define": "program_expr", "args": { "expr": "$1" } }
43-
]
41+
declarations: [
42+
TypedDeclaration::Function {
43+
name: intern("main"),
44+
params: [],
45+
return_type: Type::Named { name: intern("i64") },
46+
body: Some(TypedBlock {
47+
stmts: [TypedStatement::Return { value: Some(e) }],
48+
}),
49+
is_async: false,
50+
}
51+
],
4452
}
4553
46-
// Expression with addition/subtraction
47-
expr = { term ~ ((add_op | sub_op) ~ term)* }
48-
-> TypedExpression {
49-
"fold_binary": { "operand": "term", "operator": "add_op|sub_op" }
50-
}
54+
// Expression with addition/subtraction (left-associative)
55+
expr = { first:term ~ rest:expr_rest* }
56+
-> fold_left_ops(first, rest)
57+
58+
expr_rest = { op:add_sub_op ~ operand:term }
59+
-> make_pair(op, operand)
5160
52-
// Terms are atoms or parenthesized expressions
61+
add_sub_op = @{ "+" | "-" }
62+
63+
// Terms are atoms or parenthesized expressions — passthrough
5364
term = { integer | paren_expr }
54-
-> TypedExpression {
55-
"get_child": { "index": 0 }
56-
}
5765
5866
// Parenthesized expression (silent rule - doesn't create node)
5967
paren_expr = _{ "(" ~ expr ~ ")" }
6068
61-
// Integer literal
69+
// Integer literal — atomic rule captures text automatically
6270
integer = @{ ASCII_DIGIT+ }
63-
-> TypedExpression {
64-
"get_text": true,
65-
"parse_int": true,
66-
"define": "int_literal",
67-
"args": { "value": "$result" }
68-
}
69-
70-
// Operators
71-
add_op = { "+" }
72-
-> String { "get_text": true }
73-
74-
sub_op = { "-" }
75-
-> String { "get_text": true }
71+
-> TypedExpression::IntLiteral { value: integer }
7672
7773
// Whitespace handling
7874
WHITESPACE = _{ " " | "\t" | "\n" }
@@ -113,13 +109,11 @@ This block defines metadata about your language:
113109

114110
### Grammar Rules
115111

116-
Rules follow PEG syntax with semantic action blocks:
112+
Rules follow PEG syntax with optional semantic actions:
117113

118114
```zyn
119115
rule_name = { pattern }
120-
-> ResultType {
121-
// JSON commands
122-
}
116+
-> TypedAST::Variant { field: value, ... }
123117
```
124118

125119
| Syntax | Meaning |
@@ -155,32 +149,37 @@ rule_name = { pattern }
155149

156150
## Semantic Actions
157151

158-
Each rule can have a semantic action block that describes how to build AST nodes:
152+
Each rule can have a semantic action that builds a TypedAST node directly from parsed bindings:
159153

160154
```zyn
155+
// Atomic rule (@) — matched text is available via the binding name 'integer'
161156
integer = @{ ASCII_DIGIT+ }
162-
-> TypedExpression {
163-
"get_text": true, // Get matched text
164-
"parse_int": true, // Parse as integer
165-
"define": "int_literal",
166-
"args": { "value": "$result" }
167-
}
157+
-> TypedExpression::IntLiteral { value: integer }
168158
```
169159

170-
The `->` arrow connects the grammar rule to its semantic action:
171-
- `TypedExpression` is the expected result type
172-
- The JSON block contains commands to execute
160+
The `->` arrow connects the grammar rule to its action:
161+
- `TypedExpression::IntLiteral` is the TypedAST variant to construct
162+
- `value: integer` sets the field using the captured text from the binding
163+
164+
Named bindings in the pattern make values available to the action:
165+
166+
```zyn
167+
fn_param = { name:identifier ~ ":" ~ ty:type_expr }
168+
-> TypedParameter { name: intern(name), ty: ty }
169+
// ^^^^ ^^
170+
// binding 'name' binding 'ty'
171+
```
173172

174-
### Common Commands
173+
### Common Action Patterns
175174

176-
| Command | Purpose |
177-
|---------|---------|
178-
| `get_text` | Extract matched text |
179-
| `get_child` | Get a specific child node |
180-
| `get_all_children` | Collect all child nodes |
181-
| `parse_int` | Parse text as integer |
182-
| `define` | Call an AST builder method |
183-
| `fold_binary` | Build left-associative binary expressions |
175+
| Pattern | Description |
176+
|---------|-------------|
177+
| `TypedX::Y { field: binding }` | Construct a TypedAST node from bindings |
178+
| `-> binding` | Passthrough — return a binding directly |
179+
| `-> fold_left_ops(first, rest)` | Build a left-associative binary expression tree |
180+
| `-> prepend_list(first, rest)` | Combine first element + rest Vec into a Vec |
181+
| `-> intern(name)` | Intern a string binding into the arena |
182+
| `-> if cond { ... } else { ... }` | Branch on a boolean binding |
184183

185184
## Project Structure
186185

book/04-grammar-syntax.md

Lines changed: 49 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -200,43 +200,68 @@ PEG handles operator precedence through grammar structure, not precedence tables
200200

201201
### Left-Associative Operators
202202

203-
Build a chain of rules from lowest to highest precedence:
203+
Build a chain of rules from lowest to highest precedence. Each level uses `fold_left_ops` with a companion `rest` rule that packages `(op, operand)` pairs using `make_pair`:
204204

205205
```zyn
206206
// Lowest precedence: logical OR
207-
expr = { logical_or }
207+
expr = { e:logical_or }
208+
-> e
208209
209-
logical_or = { logical_and ~ ("or" ~ logical_and)* }
210-
-> TypedExpression {
211-
"fold_binary": { "operand": "logical_and", "operator": "or" }
212-
}
210+
logical_or = { first:logical_and ~ rest:logical_or_rest* }
211+
-> fold_left_ops(first, rest)
213212
214-
logical_and = { comparison ~ ("and" ~ comparison)* }
215-
-> TypedExpression {
216-
"fold_binary": { "operand": "comparison", "operator": "and" }
217-
}
213+
logical_or_rest = { op:or_op ~ operand:logical_and }
214+
-> make_pair(op, operand)
218215
219-
comparison = { addition ~ (("==" | "!=" | "<" | ">") ~ addition)* }
220-
-> TypedExpression {
221-
"fold_binary": { "operand": "addition", "operator": "==|!=|<|>" }
222-
}
216+
or_op = @{ "or" }
223217
224-
addition = { multiplication ~ (("+" | "-") ~ multiplication)* }
225-
-> TypedExpression {
226-
"fold_binary": { "operand": "multiplication", "operator": "+|-" }
227-
}
218+
logical_and = { first:comparison ~ rest:logical_and_rest* }
219+
-> fold_left_ops(first, rest)
228220
229-
multiplication = { unary ~ (("*" | "/") ~ unary)* }
230-
-> TypedExpression {
231-
"fold_binary": { "operand": "unary", "operator": "*|/" }
232-
}
221+
logical_and_rest = { op:and_op ~ operand:comparison }
222+
-> make_pair(op, operand)
223+
224+
and_op = @{ "and" }
225+
226+
comparison = { first:addition ~ rest:comparison_rest* }
227+
-> fold_left_ops(first, rest)
228+
229+
comparison_rest = { op:comparison_op ~ operand:addition }
230+
-> make_pair(op, operand)
231+
232+
comparison_op = @{ "==" | "!=" | "<=" | ">=" | "<" | ">" }
233+
234+
addition = { first:multiplication ~ rest:addition_rest* }
235+
-> fold_left_ops(first, rest)
236+
237+
addition_rest = { op:add_op ~ operand:multiplication }
238+
-> make_pair(op, operand)
239+
240+
add_op = @{ "+" | "-" }
241+
242+
multiplication = { first:unary ~ rest:multiplication_rest* }
243+
-> fold_left_ops(first, rest)
244+
245+
multiplication_rest = { op:mul_op ~ operand:unary }
246+
-> make_pair(op, operand)
247+
248+
mul_op = @{ "*" | "/" }
233249
234250
// Highest precedence: unary then atoms
235-
unary = { ("-" | "!") ~ unary | atom }
251+
unary = { unary_with_op | atom }
252+
253+
unary_with_op = { op:unary_op ~ operand:atom }
254+
-> TypedExpression::Unary { op: op, operand: Box::new(operand) }
255+
256+
unary_op = @{ "-" | "!" }
236257
237-
atom = { number | identifier | "(" ~ expr ~ ")" }
258+
atom = { number | identifier | paren_expr }
259+
260+
paren_expr = _{ "(" ~ expr ~ ")" }
238261
```
239262

263+
`fold_left_ops` takes the `first` value and the `rest` Vec of `(op, operand)` pairs and builds a left-associative binary tree. For input `a + b - c` it produces `Binary(-, Binary(+, a, b), c)`.
264+
240265
### Right-Associative Operators
241266

242267
Use recursion for right-associativity:

0 commit comments

Comments
 (0)