Skip to content

RUBY-3893 Fix heap use-after-free in put_array#371

Merged
jamis merged 2 commits into
mongodb:masterfrom
jamis:3893-use-after-free
Jun 8, 2026
Merged

RUBY-3893 Fix heap use-after-free in put_array#371
jamis merged 2 commits into
mongodb:masterfrom
jamis:3893-use-after-free

Conversation

@jamis

@jamis jamis commented Jun 3, 2026

Copy link
Copy Markdown
Contributor

Description

  • rb_bson_byte_buffer_put_array cached the array's raw backing pointer via RARRAY_PTR before the loop, then called rb_funcall twice per element (bson_type and to_bson). Either callback could execute arbitrary Ruby, trigger GC compaction, and free the old backing store — leaving a dangling pointer that subsequent iterations dereferenced. AddressSanitizer confirms heap-use-after-free at write.c:665.
  • Replace the raw pointer walk with rb_ary_entry, which re-derives each element from the live array object and is safe across Ruby callbacks.
  • Snapshot the array length before the loop and raise RuntimeError if it changes mid-iteration, rather than silently producing corrupted BSON output.

RARRAY_PTR is unsafe to hold across any call that may invoke the Ruby
runtime or GC. The loop body called rb_funcall twice per element (for
bson_type and to_bson), so a callback that mutated the array and
triggered GC compaction could free the backing store while the cached
pointer was still in use.

Replace the raw pointer walk with rb_ary_entry, which re-derives the
element from the live array object on every access. Also snapshot the
array length before the loop and raise RuntimeError if it changes,
rather than silently producing corrupted BSON output.
@jamis jamis requested a review from a team as a code owner June 3, 2026 21:49
@jamis jamis requested review from comandeo-mongo and Copilot June 3, 2026 21:49

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR hardens BSON array serialization in the native extension by eliminating an unsafe cached RARRAY_PTR traversal that could become dangling across Ruby callbacks/GC, and adds a spec to verify the new failure mode when an array is mutated during serialization.

Changes:

  • Rework rb_bson_byte_buffer_put_array to fetch each element with rb_ary_entry rather than walking a cached backing pointer.
  • Snapshot the initial array length and raise if the array size changes mid-serialization.
  • Add an RSpec example that mutates the array size during element serialization and asserts an exception is raised.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File Description
ext/bson/write.c Makes C-level array serialization resilient to Ruby callbacks/GC by avoiding cached backing pointers and detecting size mutations.
spec/bson/array_spec.rb Adds regression coverage for array-size mutation during serialization.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread ext/bson/write.c Outdated
Comment thread ext/bson/write.c Outdated
Keep original_len as long and raise RangeError for arrays exceeding
INT32_MAX, preventing truncation from silently corrupting the
modification check. Mark element as volatile VALUE so the conservative
GC can see it on the C stack during ALLOC_N calls in pvt_put_array_index.
@jamis jamis merged commit 9f0891b into mongodb:master Jun 8, 2026
48 of 51 checks passed
@jamis jamis deleted the 3893-use-after-free branch June 8, 2026 14:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants