Skip to content

Commit b73ae87

Browse files
darmiedarmie
committed
docs: Add comprehensive Zyn tutorial book
Add a 9-chapter tutorial covering: - Introduction to Zyn and its architecture - Getting started with your first grammar - Using the CLI (compile, REPL, workflows) - PEG grammar syntax and patterns - JSON semantic actions and commands - TypedAST structure and design - TypedAST Builder fluent API - Complete Zig grammar walkthrough - Command and API reference Also adds GitHub Actions workflow to automatically publish the book to the repository wiki on changes to book/ directory.
1 parent f6363bc commit b73ae87

11 files changed

Lines changed: 3845 additions & 0 deletions

‎.github/workflows/wiki.yml‎

Lines changed: 53 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,53 @@
1+
name: Publish Wiki
2+
3+
on:
4+
push:
5+
branches:
6+
- main
7+
paths:
8+
- 'book/**'
9+
workflow_dispatch: # Allow manual trigger
10+
11+
permissions:
12+
contents: write
13+
14+
jobs:
15+
publish-wiki:
16+
runs-on: ubuntu-latest
17+
steps:
18+
- name: Checkout repository
19+
uses: actions/checkout@v4
20+
21+
- name: Prepare wiki content
22+
run: |
23+
mkdir -p wiki
24+
25+
# Copy book files to wiki directory
26+
cp book/README.md wiki/Home.md
27+
cp book/01-introduction.md wiki/01-Introduction.md
28+
cp book/02-getting-started.md wiki/02-Getting-Started.md
29+
cp book/03-using-the-cli.md wiki/03-Using-the-CLI.md
30+
cp book/04-grammar-syntax.md wiki/04-Grammar-Syntax.md
31+
cp book/05-semantic-actions.md wiki/05-Semantic-Actions.md
32+
cp book/06-typed-ast.md wiki/06-The-TypedAST.md
33+
cp book/07-typed-ast-builder.md wiki/07-TypedAST-Builder.md
34+
cp book/08-zig-example.md wiki/08-Zig-Example.md
35+
cp book/09-reference.md wiki/09-Reference.md
36+
37+
# Fix internal links for wiki format
38+
# GitHub wiki uses spaces as hyphens in URLs
39+
sed -i 's|\./01-introduction\.md|01-Introduction|g' wiki/*.md
40+
sed -i 's|\./02-getting-started\.md|02-Getting-Started|g' wiki/*.md
41+
sed -i 's|\./03-using-the-cli\.md|03-Using-the-CLI|g' wiki/*.md
42+
sed -i 's|\./04-grammar-syntax\.md|04-Grammar-Syntax|g' wiki/*.md
43+
sed -i 's|\./05-semantic-actions\.md|05-Semantic-Actions|g' wiki/*.md
44+
sed -i 's|\./06-typed-ast\.md|06-The-TypedAST|g' wiki/*.md
45+
sed -i 's|\./07-typed-ast-builder\.md|07-TypedAST-Builder|g' wiki/*.md
46+
sed -i 's|\./08-zig-example\.md|08-Zig-Example|g' wiki/*.md
47+
sed -i 's|\./09-reference\.md|09-Reference|g' wiki/*.md
48+
49+
- name: Upload wiki
50+
uses: Andrew-Chen-Wang/github-wiki-action@v4
51+
with:
52+
path: wiki/
53+
token: ${{ secrets.GITHUB_TOKEN }}

‎book/01-introduction.md‎

Lines changed: 138 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,138 @@
1+
# Chapter 1: Introduction
2+
3+
## What is Zyn?
4+
5+
Zyn (ZynPEG) is a grammar-driven language frontend system that transforms source code into a typed abstract syntax tree (TypedAST). It combines three powerful concepts:
6+
7+
1. **PEG Parsing** - Uses Pest-compatible Parser Expression Grammars for syntax definition
8+
2. **Declarative Semantics** - JSON command blocks describe how to build AST nodes
9+
3. **Universal TypedAST** - A target representation that supports multiple programming paradigms
10+
11+
## The Problem Zyn Solves
12+
13+
Building a language frontend traditionally requires:
14+
15+
```
16+
Source Code → Lexer → Parser → AST Builder → Type Checker → IR
17+
```
18+
19+
Each stage requires significant code:
20+
- Lexer rules and token definitions
21+
- Parser with precedence handling
22+
- AST node types and construction logic
23+
- Visitor patterns for tree traversal
24+
25+
Zyn collapses the parser and AST builder into a single declarative specification:
26+
27+
```
28+
Source Code → Zyn Grammar → TypedAST → Compiler Backend
29+
```
30+
31+
## How It Works
32+
33+
A Zyn grammar file (`.zyn`) contains:
34+
35+
1. **Grammar rules** - PEG syntax defining what to parse
36+
2. **Semantic actions** - JSON blocks defining what AST nodes to create
37+
38+
```zyn
39+
// Grammar rule
40+
integer_literal = @{ "-"? ~ ASCII_DIGIT+ }
41+
-> TypedExpression {
42+
"get_text": true,
43+
"parse_int": true,
44+
"define": "int_literal",
45+
"args": { "value": "$result" }
46+
}
47+
```
48+
49+
When this rule matches, Zyn:
50+
1. Extracts the matched text (`get_text`)
51+
2. Parses it as an integer (`parse_int`)
52+
3. Calls `create_int_literal(value)` on the AST builder
53+
54+
## The TypedAST
55+
56+
The TypedAST is a universal intermediate representation that can express:
57+
58+
- **Multiple paradigms**: OOP, functional, procedural
59+
- **Rich type system**: Generics, traits, enums, structs
60+
- **Language features**: Pattern matching, async/await, error handling
61+
62+
Every node carries:
63+
- **Type information** - Full type annotation
64+
- **Source spans** - Location for error reporting
65+
- **Semantic data** - Names, operators, modifiers
66+
67+
```rust
68+
pub struct TypedNode<T> {
69+
pub node: T, // The actual AST node
70+
pub ty: Type, // Type annotation
71+
pub span: Span, // Source location
72+
}
73+
```
74+
75+
## Why Use Zyn?
76+
77+
### 1. Rapid Prototyping
78+
Define a new language syntax in hours, not weeks. The grammar and semantics live in one file.
79+
80+
### 2. Correctness
81+
Declarative specifications are easier to verify than imperative AST construction code.
82+
83+
### 3. Reusability
84+
The TypedAST can target multiple backends: JIT compilation, LLVM, interpreters.
85+
86+
### 4. Maintainability
87+
Grammar changes automatically propagate to AST construction - no separate files to update.
88+
89+
## Architecture Overview
90+
91+
```
92+
┌─────────────────┐
93+
│ Source File │
94+
│ (*.zig) │
95+
└────────┬────────┘
96+
│
97+
▼
98+
┌─────────────────┐ ┌─────────────────┐
99+
│ Zyn Grammar │────▶│ Pest Parser │
100+
│ (*.zyn) │ │ (generated) │
101+
└─────────────────┘ └────────┬────────┘
102+
│
103+
▼
104+
┌─────────────────┐
105+
│ Command │
106+
│ Interpreter │
107+
└────────┬────────┘
108+
│
109+
▼
110+
┌─────────────────┐
111+
│ TypedAST │
112+
│ Builder │
113+
└────────┬────────┘
114+
│
115+
▼
116+
┌─────────────────┐
117+
│ TypedProgram │
118+
│ (JSON/Binary) │
119+
└────────┬────────┘
120+
│
121+
▼
122+
┌─────────────────┐
123+
│ Compiler │
124+
│ Backend │
125+
└─────────────────┘
126+
```
127+
128+
## Next Steps
129+
130+
In the following chapters, you'll learn:
131+
132+
- [Chapter 2](./02-getting-started.md): Set up your environment and write your first grammar
133+
- [Chapter 3](./03-using-the-cli.md): Use the CLI for compilation and REPL testing
134+
- [Chapter 4](./04-grammar-syntax.md): Master PEG syntax for parsing
135+
- [Chapter 5](./05-semantic-actions.md): Use JSON commands to build AST nodes
136+
- [Chapter 6](./06-typed-ast.md): Understand the TypedAST structure
137+
- [Chapter 7](./07-typed-ast-builder.md): Use the builder API directly
138+
- [Chapter 8](./08-zig-example.md): Walk through a complete Zig implementation

‎book/02-getting-started.md‎

Lines changed: 206 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,206 @@
1+
# Chapter 2: Getting Started
2+
3+
## Prerequisites
4+
5+
Before using Zyn, ensure you have:
6+
7+
- Rust toolchain (1.70+)
8+
- The `zyntax` CLI tool
9+
10+
```bash
11+
# Build zyntax from source
12+
cargo build --release
13+
14+
# Verify installation
15+
./target/release/zyntax --help
16+
```
17+
18+
## Your First Grammar
19+
20+
Let's create a minimal calculator language that supports:
21+
- Integer literals
22+
- Addition and subtraction
23+
- Parentheses for grouping
24+
25+
### Step 1: Create the Grammar File
26+
27+
Create `calc.zyn`:
28+
29+
```zyn
30+
// Calculator Language Grammar
31+
@language {
32+
name: "Calc",
33+
version: "1.0",
34+
file_extensions: [".calc"],
35+
entry_point: "main",
36+
}
37+
38+
// Program structure
39+
program = { SOI ~ expr ~ EOI }
40+
-> TypedProgram {
41+
"commands": [
42+
{ "define": "program_expr", "args": { "expr": "$1" } }
43+
]
44+
}
45+
46+
// Expression with addition/subtraction
47+
expr = { term ~ ((add_op | sub_op) ~ term)* }
48+
-> TypedExpression {
49+
"fold_binary": { "operand": "term", "operator": "add_op|sub_op" }
50+
}
51+
52+
// Terms are atoms or parenthesized expressions
53+
term = { integer | paren_expr }
54+
-> TypedExpression {
55+
"get_child": { "index": 0 }
56+
}
57+
58+
// Parenthesized expression (silent rule - doesn't create node)
59+
paren_expr = _{ "(" ~ expr ~ ")" }
60+
61+
// Integer literal
62+
integer = @{ ASCII_DIGIT+ }
63+
-> TypedExpression {
64+
"get_text": true,
65+
"parse_int": true,
66+
"define": "int_literal",
67+
"args": { "value": "$result" }
68+
}
69+
70+
// Operators
71+
add_op = { "+" }
72+
-> String { "get_text": true }
73+
74+
sub_op = { "-" }
75+
-> String { "get_text": true }
76+
77+
// Whitespace handling
78+
WHITESPACE = _{ " " | "\t" | "\n" }
79+
```
80+
81+
### Step 2: Create a Test File
82+
83+
Create `test.calc`:
84+
85+
```
86+
1 + 2 + 3
87+
```
88+
89+
### Step 3: Compile and Run
90+
91+
```bash
92+
zyntax compile --grammar calc.zyn --source test.calc --run
93+
```
94+
95+
## Understanding the Grammar
96+
97+
### Language Metadata
98+
99+
```zyn
100+
@language {
101+
name: "Calc",
102+
version: "1.0",
103+
file_extensions: [".calc"],
104+
entry_point: "main",
105+
}
106+
```
107+
108+
This block defines metadata about your language:
109+
- `name`: Language identifier
110+
- `version`: Grammar version
111+
- `file_extensions`: Associated file types
112+
- `entry_point`: The function to execute (for JIT compilation)
113+
114+
### Grammar Rules
115+
116+
Rules follow PEG syntax with semantic action blocks:
117+
118+
```zyn
119+
rule_name = { pattern }
120+
-> ResultType {
121+
// JSON commands
122+
}
123+
```
124+
125+
| Syntax | Meaning |
126+
|--------|---------|
127+
| `{ }` | Normal rule (creates parse node) |
128+
| `@{ }` | Atomic rule (no whitespace handling) |
129+
| `_{ }` | Silent rule (matches but creates no node) |
130+
131+
### Operators
132+
133+
| Operator | Meaning | Example |
134+
|----------|---------|---------|
135+
| `~` | Sequence | `a ~ b` matches a then b |
136+
| `|` | Choice | `a | b` matches a or b |
137+
| `*` | Zero or more | `a*` matches "", "a", "aa", ... |
138+
| `+` | One or more | `a+` matches "a", "aa", ... |
139+
| `?` | Optional | `a?` matches "" or "a" |
140+
| `!` | Not predicate | `!a` succeeds if a fails |
141+
| `&` | And predicate | `&a` succeeds if a matches (no consume) |
142+
143+
### Built-in Rules
144+
145+
| Rule | Matches |
146+
|------|---------|
147+
| `SOI` | Start of input |
148+
| `EOI` | End of input |
149+
| `ANY` | Any single character |
150+
| `ASCII_DIGIT` | 0-9 |
151+
| `ASCII_ALPHA` | a-z, A-Z |
152+
| `ASCII_ALPHANUMERIC` | a-z, A-Z, 0-9 |
153+
| `WHITESPACE` | Define whitespace handling |
154+
| `COMMENT` | Define comment syntax |
155+
156+
## Semantic Actions
157+
158+
Each rule can have a semantic action block that describes how to build AST nodes:
159+
160+
```zyn
161+
integer = @{ ASCII_DIGIT+ }
162+
-> TypedExpression {
163+
"get_text": true, // Get matched text
164+
"parse_int": true, // Parse as integer
165+
"define": "int_literal",
166+
"args": { "value": "$result" }
167+
}
168+
```
169+
170+
The `->` arrow connects the grammar rule to its semantic action:
171+
- `TypedExpression` is the expected result type
172+
- The JSON block contains commands to execute
173+
174+
### Common Commands
175+
176+
| Command | Purpose |
177+
|---------|---------|
178+
| `get_text` | Extract matched text |
179+
| `get_child` | Get a specific child node |
180+
| `get_all_children` | Collect all child nodes |
181+
| `parse_int` | Parse text as integer |
182+
| `define` | Call an AST builder method |
183+
| `fold_binary` | Build left-associative binary expressions |
184+
185+
## Project Structure
186+
187+
A typical Zyn project looks like:
188+
189+
```
190+
my-language/
191+
├── grammar/
192+
│ └── mylang.zyn # Grammar definition
193+
├── examples/
194+
│ ├── hello.mylang # Example source files
195+
│ └── test.mylang
196+
└── tests/
197+
└── parser_tests.rs # Test cases
198+
```
199+
200+
## Next Steps
201+
202+
Now that you have a working grammar:
203+
204+
1. [Chapter 4](./04-grammar-syntax.md): Learn advanced grammar patterns
205+
2. [Chapter 5](./05-semantic-actions.md): Master semantic actions
206+
3. [Chapter 8](./08-zig-example.md): Study a complete real-world example

0 commit comments

Comments
 (0)