Skip to content

perf(stdlib): make enumerate return a lazy iterator 🦥 - #138

Merged
timfennis merged 2 commits into
masterfrom
feature/lazy-enumerate
May 20, 2026
Merged

perf(stdlib): make enumerate return a lazy iterator 🦥#138
timfennis merged 2 commits into
masterfrom
feature/lazy-enumerate

Conversation

@timfennis

@timfennis timfennis commented May 19, 2026

Copy link
Copy Markdown
Owner

Context

enumerate() was eager: it drained the input into a Vec<(i64, Value)> and returned a list. That wastes a full allocation when the caller only consumes part of the result (enumerate().take(N), enumerate().find(...)) and made enumerating an unbounded source like (0..) impossible.

Changes

  • New EnumerateIter in ndc_vm/src/iterator.rs, modelled on TakeIter: wraps a ValueIter, yields (index, value) tuples lazily, propagates size_hint, and supports deep_copy for Shared sources.
  • Re-export EnumerateIter from ndc_vm.
  • enumerate in ndc_stdlib/src/sequence.rs now returns Iterator<Value> and constructs an EnumerateIter instead of .collect()-ing a Vec.
  • Existing test updated to materialise with .list() (mirrors 008_iterators/005_take.ndc); added assertions covering an unbounded range and composition with .map.

Benchmarks

Compared b820599 (lazy, this PR) against its parent 2f4a0df (eager). Each program runs the release ndc binary via hyperfine --warmup 3. Programs live in benches/programs/enumerate_*.ndc.

Scenario Eager (before) Lazy (after) Speedup
enumerate(1M).take(10).list() 154.4 ms 26.3 ms 5.88×
enumerate(1M).find(...) 266.4 ms 158.8 ms 1.68×
for (i, v) in enumerate(500k) 198.6 ms 110.3 ms 1.80×
enumerate(500k).list() (full materialisation) 85.3 ms 63.5 ms 1.34×

Full materialisation is faster too, which I hadn't expected — the eager path was paying a Vec<Value> allocation plus per-element tuple boxing, and the iterator path skips the intermediate Vec.

🤖 Generated with Claude Code

@timfennis
timfennis requested a review from tosti007 May 20, 2026 09:34
@timfennis
timfennis merged commit a818a9a into master May 20, 2026
1 check passed
@timfennis
timfennis deleted the feature/lazy-enumerate branch May 20, 2026 12:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant