Skip to content

Commit b407fc8

Browse files
committed
feat: Add multi-language runtime support to zyntax_embed
Add grammar registry and load_module API to both ZyntaxRuntime and TieredRuntime for polyglot application support: New methods: - register_grammar(language, grammar) - Register grammar by language ID - register_grammar_file(language, path) - Compile and register from .zyn - register_grammar_zpeg(language, path) - Load and register from .zpeg - get_grammar(language) - Retrieve registered grammar - languages() - List all registered languages - has_language(language) - Check if language is registered - language_for_extension(ext) - Get language for file extension - load_module(language, source) - Parse and compile by language - load_module_file(path) - Load with auto-detected language - functions() - List all loaded function names - has_function(name) - Check if function exists Usage: runtime.register_grammar("zig", grammar)?; runtime.load_module("zig", "pub fn add(a: i32, b: i32) i32 { ... }")?; runtime.load_module_file("./src/math.zig")?; // Auto-detect from extension
1 parent e69c33d commit b407fc8

3 files changed

Lines changed: 585 additions & 0 deletions

File tree

book/12-embedding-sdk.md

Lines changed: 108 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,7 @@ This chapter covers how to embed the Zyntax JIT runtime in your Rust application
77
The Zyntax Embedding SDK provides:
88

99
- **Language Grammar Interface**: Parse source code using any `.zyn` grammar
10+
- **Multi-Language Runtime**: Register multiple grammars and compile from different languages
1011
- **JIT Compilation**: Compile TypedAST to native code at runtime
1112
- **Multi-Tier Optimization**: Automatic optimization of hot code paths
1213
- **Bidirectional Interop**: Seamless conversion between Rust and Zyntax types
@@ -132,6 +133,113 @@ impl AstHostFunctions for MyCustomBuilder {
132133
}
133134
```
134135

136+
## Multi-Language Runtime
137+
138+
The runtime supports registering multiple language grammars and loading modules by language name. This enables polyglot applications where different parts of your codebase can use different languages.
139+
140+
### Registering Grammars
141+
142+
```rust
143+
use zyntax_embed::{ZyntaxRuntime, LanguageGrammar};
144+
145+
let mut runtime = ZyntaxRuntime::new()?;
146+
147+
// Register grammars by language name
148+
runtime.register_grammar("zig", LanguageGrammar::compile_zyn_file("grammars/zig.zyn")?);
149+
runtime.register_grammar("python", LanguageGrammar::compile_zyn_file("grammars/python.zyn")?);
150+
runtime.register_grammar("calc", LanguageGrammar::compile_zyn_file("grammars/calc.zyn")?);
151+
152+
// Or use convenience methods
153+
runtime.register_grammar_file("haxe", "grammars/haxe.zyn")?;
154+
runtime.register_grammar_zpeg("lua", "grammars/lua.zpeg")?;
155+
156+
// Query registered languages
157+
println!("Languages: {:?}", runtime.languages()); // ["zig", "python", "calc", "haxe", "lua"]
158+
println!("Has Python: {}", runtime.has_language("python")); // true
159+
```
160+
161+
### Loading Modules by Language
162+
163+
```rust
164+
// Load modules specifying the language
165+
let functions = runtime.load_module("zig", r#"
166+
pub fn add(a: i32, b: i32) i32 {
167+
return a + b;
168+
}
169+
170+
pub fn factorial(n: i32) i32 {
171+
if (n <= 1) return 1;
172+
return n * factorial(n - 1);
173+
}
174+
"#)?;
175+
176+
println!("Loaded functions: {:?}", functions); // ["add", "factorial"]
177+
178+
// Load from another language
179+
runtime.load_module("calc", "def multiply(a, b) = a * b")?;
180+
181+
// Call any function regardless of source language
182+
let sum: i32 = runtime.call("add", &[10.into(), 32.into()])?;
183+
let fact: i32 = runtime.call("factorial", &[5.into()])?;
184+
```
185+
186+
### Auto-Detection from File Extension
187+
188+
When grammars declare file extensions in their `@language` metadata, the runtime automatically maps extensions to languages:
189+
190+
```rust
191+
// The zig.zyn grammar declares: file_extensions: [".zig"]
192+
runtime.register_grammar("zig", LanguageGrammar::compile_zyn_file("zig.zyn")?);
193+
194+
// Load files - language auto-detected from extension
195+
runtime.load_module_file("./src/math.zig")?; // Uses "zig" grammar
196+
runtime.load_module_file("./lib/utils.py")?; // Uses "python" grammar
197+
198+
// Query extension mappings
199+
println!("Language for .zig: {:?}", runtime.language_for_extension(".zig")); // Some("zig")
200+
println!("Language for .py: {:?}", runtime.language_for_extension("py")); // Some("python")
201+
```
202+
203+
### Cross-Language Function Calls
204+
205+
All modules loaded into the same runtime share a common execution environment. Functions can call each other across language boundaries:
206+
207+
```rust
208+
// Load core utilities in Zig
209+
runtime.load_module("zig", r#"
210+
pub fn square(x: i32) i32 { return x * x; }
211+
pub fn cube(x: i32) i32 { return x * x * x; }
212+
"#)?;
213+
214+
// Load a DSL that uses the Zig functions via extern declarations
215+
runtime.load_module("calc", r#"
216+
extern fn square(x: i32) i32;
217+
extern fn cube(x: i32) i32;
218+
219+
def sum_of_powers(a, b) = square(a) + cube(b)
220+
"#)?;
221+
222+
// The calc function calls into the Zig implementation
223+
let result: i32 = runtime.call("sum_of_powers", &[3.into(), 2.into()])?;
224+
assert_eq!(result, 17); // square(3) + cube(2) = 9 + 8 = 17
225+
```
226+
227+
### TieredRuntime Multi-Language Support
228+
229+
The `TieredRuntime` also supports multi-language modules with the same API:
230+
231+
```rust
232+
use zyntax_embed::TieredRuntime;
233+
234+
let mut runtime = TieredRuntime::production()?;
235+
236+
runtime.register_grammar("zig", LanguageGrammar::compile_zyn_file("zig.zyn")?);
237+
runtime.load_module("zig", "pub fn compute(x: i32) i32 { return x * 2; }")?;
238+
239+
// Hot functions are automatically optimized regardless of source language
240+
let result: i32 = runtime.call("compute", &[21.into()])?;
241+
```
242+
135243
## Runtime Options
136244

137245
### ZyntaxRuntime (Single-Tier JIT)

crates/zyntax_embed/README.md

Lines changed: 69 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,7 @@ Apache-2.0
1111
`zyntax_embed` provides a high-level Rust API for embedding the Zyntax compiler as a JIT runtime. It supports:
1212

1313
- **Language Grammar Interface**: Parse source code using any `.zyn` grammar
14+
- **Multi-Language Runtime**: Register multiple grammars and compile from different languages
1415
- **Compiler Integration**: Compile and execute Zyntax code at runtime
1516
- **Multi-Tier JIT**: Automatic optimization of hot code paths
1617
- **Async/Await**: Promise-based async operations with `.then()` and `.catch()`
@@ -118,6 +119,74 @@ let json = grammar.parse_to_json(source_code)?;
118119
println!("{}", json);
119120
```
120121

122+
## Multi-Language Runtime
123+
124+
Register multiple language grammars and load modules by language name:
125+
126+
```rust
127+
use zyntax_embed::{ZyntaxRuntime, LanguageGrammar};
128+
129+
fn main() -> Result<(), Box<dyn std::error::Error>> {
130+
let mut runtime = ZyntaxRuntime::new()?;
131+
132+
// Register language grammars
133+
runtime.register_grammar("zig", LanguageGrammar::compile_zyn_file("grammars/zig.zyn")?);
134+
runtime.register_grammar("python", LanguageGrammar::compile_zyn_file("grammars/python.zyn")?);
135+
runtime.register_grammar("calc", LanguageGrammar::compile_zyn_file("grammars/calc.zyn")?);
136+
137+
// Load modules by language name
138+
runtime.load_module("zig", r#"
139+
pub fn add(a: i32, b: i32) i32 {
140+
return a + b;
141+
}
142+
"#)?;
143+
144+
runtime.load_module("calc", "def multiply(a, b) = a * b")?;
145+
146+
// Call functions from any loaded module
147+
let sum: i32 = runtime.call("add", &[10.into(), 32.into()])?;
148+
println!("Sum: {}", sum); // 42
149+
150+
Ok(())
151+
}
152+
```
153+
154+
### Auto-detection from File Extension
155+
156+
```rust
157+
// Register grammars (extensions are auto-detected from grammar metadata)
158+
runtime.register_grammar("zig", LanguageGrammar::compile_zyn_file("zig.zyn")?);
159+
160+
// Load files - language detected from extension
161+
runtime.load_module_file("./src/math.zig")?; // Uses "zig" grammar
162+
runtime.load_module_file("./src/utils.py")?; // Uses "python" grammar
163+
164+
// Query registered languages
165+
println!("Languages: {:?}", runtime.languages());
166+
println!("Language for .zig: {:?}", runtime.language_for_extension(".zig"));
167+
```
168+
169+
### Cross-Language Interop
170+
171+
Functions from different languages share the same runtime and can call each other:
172+
173+
```rust
174+
// Load math utilities in Zig
175+
runtime.load_module("zig", r#"
176+
pub fn square(x: i32) i32 { return x * x; }
177+
"#)?;
178+
179+
// Load DSL that uses the Zig function
180+
runtime.load_module("calc", r#"
181+
extern fn square(x: i32) i32;
182+
def sum_of_squares(a, b) = square(a) + square(b)
183+
"#)?;
184+
185+
// Call the DSL function that internally uses Zig
186+
let result: i32 = runtime.call("sum_of_squares", &[3.into(), 4.into()])?;
187+
assert_eq!(result, 25); // 9 + 16
188+
```
189+
121190
## Runtime Options
122191

123192
### ZyntaxRuntime (Simple JIT)

0 commit comments

Comments
 (0)