Skip to content

Commit e9a5365

Browse files
timfennisclaude
andauthored
πŸš€ Bytecode VM (#93)
## Summary Replaces the tree-walk interpreter with a bytecode-compiled stack-based VM as the single execution path (~192 commits). ### Compiler & VM - Bytecode compiler translates the annotated AST into a flat `OpCode` sequence - Stack-based VM executes bytecode with support for locals, globals, upvalues, and closures - Upvalue hoisting in the analyser enables mutable closed-over variables - Control flow: `if`/`while`/`for`/`break`/`continue`/`return` all compile to jump instructions - Tuple destructuring via `Unpack` opcode - Map/set/list comprehensions compile to inline loops - Op-assignment (`+=`, etc.) for identifiers, index expressions, and destructuring - Closure capture and upvalue resolution (including nested closures) - Memoization support for pure functions ### Stdlib & dispatch - `FunctionRegistry` + `vm_native` field on `Function` for bridge-free native dispatch - Arithmetic, comparison, boolean, trig/math operators migrated to VM-native dispatch - HOF callbacks (`map`, `filter`, `fold`, `all`, `any`, etc.) work with VM closures - Dynamic dispatch for overloaded functions at runtime ### Infrastructure - `ndc_vm` crate with compiler and VM - `vm-trace` feature for source-annotated instruction tracing - -disassemble` CLI subcommand - All existing tests pass against the VM backend ## Benchmarks (v0.2.1 tree-walk vs VM) | Benchmark | v0.2.1 (tree-walk) | VM | Speedup | |:---|---:|---:|---:| | ackermann | stack overflow | 155.0 ms | - | | bigint | 9.3 ms | 6.7 ms | 1.4x | | closures | 248.5 ms | 78.6 ms | **3.2x** | | fibonacci | 318.0 ms | 80.1 ms | **4.0x** | | hof_pipeline | 84.8 ms | 36.7 ms | **2.3x** | | map_ops | 74.7 ms | 28.7 ms | **2.6x** | | matrix_mul | 133.1 ms | 67.0 ms | **2.0x** | | perlin | 216.5 ms | 67.5 ms | **3.2x** | | pi_approx | 103.8 ms | 32.3 ms | **3.2x** | | print_heavy | 8.9 ms | 5.6 ms | 1.6x | | quicksort | 192.0 ms | 75.2 ms | **2.6x** | | sieve | 439.4 ms | 125.2 ms | **3.5x** | | string_concat | 16.0 ms | 13.4 ms | 1.2x | πŸ€– Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Sonnet 4.6 <[email protected]>
1 parent 601c499 commit e9a5365

167 files changed

Lines changed: 11557 additions & 7564 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

β€Ž.cargo/config.tomlβ€Ž

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,7 @@ rustflags = [
1212
"-Wclippy::dbg_macro",
1313
"-Wclippy::debug_assert_with_mut_call",
1414
"-Wclippy::doc_markdown",
15-
"-Wclippy::empty_enum",
15+
"-Wclippy::empty_enums",
1616
"-Wclippy::enum_glob_use",
1717
"-Wclippy::exit",
1818
"-Wclippy::expl_impl_clone_on_copy",
@@ -76,7 +76,7 @@ rustflags = [
7676
"-Wtrivial-numeric-casts",
7777
"-Wunused-crate-dependencies",
7878
"-Wunused-qualifications",
79-
"-Wclippy::as_conversions",
79+
"-Aclippy::as_conversions",
8080
"-Wclippy::as_pointer_underscore",
8181
"-Wclippy::doc_include_without_cfg",
8282
"-Wclippy::get_unwrap",

β€ŽCLAUDE.mdβ€Ž

Lines changed: 108 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,108 @@
1+
# CLAUDE.md
2+
3+
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
4+
5+
## Before commit
6+
- Run `cargo fmt`
7+
- Run `cargo clippy` and fix all warnings
8+
- Ensure all tests pass (`cargo test`)
9+
- Do not leave `TODO` comments in code β€” either fix the issue immediately or open a GitHub issue and record it in `TODO.md`
10+
11+
## Common Commands
12+
13+
```bash
14+
# Build
15+
cargo build
16+
17+
# Run all tests
18+
cargo test
19+
20+
# Run a single test by name (substring match on the filename)
21+
cargo test test_001_math_001_addition
22+
23+
# Run benchmarks
24+
cargo bench -p benches
25+
26+
# Start REPL
27+
cargo run --bin ndc
28+
29+
# Run a .ndc script
30+
cargo run --bin ndc -- script.ndc
31+
32+
# Disassemble bytecode
33+
cargo run --bin ndc -- disassemble script.ndc
34+
35+
# Show documentation (optionally filtered by query)
36+
cargo run --bin ndc -- docs [query] [--no-color]
37+
38+
# Profile with perf (requires release-with-debug profile in Cargo.toml)
39+
cargo build --profile release-with-debug
40+
hyperfine --warmup 3 './target/release-with-debug/ndc script.ndc'
41+
perf stat ./target/release-with-debug/ndc script.ndc
42+
perf record -g --call-graph=dwarf -o /tmp/out.perf ./target/release-with-debug/ndc script.ndc
43+
perf report -i /tmp/out.perf --stdio --no-children --percent-limit=1
44+
```
45+
46+
## Manual
47+
48+
User-facing language documentation lives in `manual/src/`. It is an mdBook project. The entry point is `manual/src/SUMMARY.md`.
49+
50+
When making changes that affect language behaviour or runtime semantics, update the relevant manual page.
51+
52+
## Architecture
53+
54+
This is a custom language interpreter ("Andy C++") with a bytecode VM backend:
55+
56+
```
57+
Source β†’ [Lexer] β†’ Tokens β†’ [Parser] β†’ AST β†’ [Analyser] β†’ Annotated AST
58+
↓
59+
[Compiler]
60+
↓
61+
[Bytecode VM] β†’ Value
62+
```
63+
64+
### Git Workflow
65+
- Prefer short commit messages, only use multiple lines in case of unrelated changes
66+
- Pull request titles must start with an emoji
67+
68+
### Crate Layout
69+
70+
| Crate | Role |
71+
|---|---|
72+
| `ndc_lexer` | Tokenisation, `Span` (offset+length) |
73+
| `ndc_parser` | AST (`Expression`, `ExpressionLocation`), parser |
74+
| `ndc_core` | `Number` (BigInt/Rational/Complex), `StaticType`, `FunctionRegistry`, ordering, hashing |
75+
| `ndc_interpreter` | Semantic analyser, `Interpreter` facade (compile + run via VM) |
76+
| `ndc_vm` | Bytecode `Compiler` and stack-based `Vm` |
77+
| `ndc_stdlib` | Built-in functions registered via `FunctionRegistry` |
78+
| `ndc_lsp` | LSP backend (hover, inlay hints) |
79+
| `ndc_bin` | CLI entry point, REPL, syntax highlighting |
80+
81+
### Key Concepts
82+
83+
**Single execution path** β€” The bytecode VM in `ndc_vm` is the only execution path. `ndc_interpreter` acts as a facade: it runs the semantic analyser, compiles to bytecode via `ndc_vm::Compiler`, and executes via `ndc_vm::Vm`. `vm_bridge.rs` handles value conversion between `ndc_interpreter::Value` and `ndc_vm::Value`.
84+
85+
**Value types** β€” `ndc_interpreter/src/value.rs` and `ndc_vm/src/value.rs` are separate enums. The VM `Value` is constrained to 16 bytes (`Int(i64)`, `Float(f64)`, `Bool`, `None`, `Object(Box<Object>)`).
86+
87+
**Function overloading** β€” Functions are matched by name and arity. The semantic analyser produces `Binding::Resolved` (exact compile-time match) or `Binding::Dynamic(Vec<ResolvedVar>)` (runtime dispatch among candidates). Binary operators like `+` are parsed as `Expression::Call`.
88+
89+
**Semantic analyser** β€” `ndc_interpreter/src/semantic/analyser.rs` infers `StaticType` and resolves function bindings. `StaticType::Any` is the fallback when inference fails.
90+
91+
**`FunctionRegistry`** β€” Lives in `ndc_core`. Holds all registered built-in functions as `Rc<NativeFunction>`. Replaces the old `Environment`-based function registry. At runtime, natives are passed to the VM as global slots.
92+
93+
**Persistent REPL** β€” The `Interpreter` keeps `repl_state: Option<(Vm, Compiler)>` so variables declared on one REPL line are visible on subsequent lines (resume-from-halt pattern).
94+
95+
### Test Infrastructure
96+
97+
The `tests` crate auto-generates one test function per `.ndc` file at build time via `tests/build.rs`. For every `.ndc` file under `tests/programs/`, a single Rust test function is generated:
98+
- `test_<path>` β€” runs via `Interpreter::run_str` (VM)
99+
100+
Test directives are comments inside `.ndc` files:
101+
```ndc
102+
// expect-output: 42 ← assert stdout equals this
103+
// expect-error: divide ← assert error message contains this substring
104+
```
105+
106+
### Compiler Tests
107+
108+
`compiler_tests/` validates the bytecode compiler by asserting exact `OpCode` sequences. Use these when adding new VM instructions.

0 commit comments

Comments
Β (0)