Describe the bug
Checked get_many can overflow the thread stack when processing a deeply nested but otherwise valid JSON payload.
This can happen even when the requested path is shallow: while searching for a requested field, get_many recursively validates and skips preceding, unrelated JSON values. A single deeply nested value can therefore abort the entire Rust process instead of returning a recoverable parsing error.
To Reproduce
use sonic_rs::{get_many, PointerTree};
fn main() {
std::thread::Builder::new()
.name("get_many_repro".into())
.stack_size(1024 * 1024)
.spawn(|| {
let depth = 10_000;
let nested = format!("{}0{}", "[".repeat(depth), "]".repeat(depth));
let json = format!(r#"{{"ignored":{nested},"wanted":1}}"#);
let mut tree = PointerTree::new();
tree.add_path(["wanted"]);
println!("{:?}", get_many(&json, &tree));
})
.unwrap()
.join()
.unwrap();
}
A runnable reproduction is also available here:
https://github.com/starcatmeow/sonic-rs/blob/rayli/repro-get-many-stack-overflow/examples/repro_get_many_stack_overflow.rs
Run:
git clone \
--branch rayli/repro-get-many-stack-overflow \
--single-branch \
https://github.com/starcatmeow/sonic-rs.git \
sonic-rs-get-many-repro
cd sonic-rs-get-many-repro
cargo run --release --example repro_get_many_stack_overflow
Observed output:
thread 'get_many_repro' has overflowed its stack
fatal runtime error: stack overflow, aborting
The debug build also reproduces the issue:
cargo run --example repro_get_many_stack_overflow
Expected behavior
get_many should return a parsing or recursion-limit error before exhausting the stack.
A deeply nested, unrelated value should not be able to terminate a process that is only attempting to extract a shallow field.
sonic-rs version:
0.5.9 / current main.
Environment:
aarch64-apple-darwin
rustc 1.96.1
- Reproduced in both debug and release builds with a 1 MiB worker-thread stack.
Additional context
This is particularly important for services processing externally supplied events, since a Rust stack overflow aborts the process and cannot be handled like a normal parsing error.
There appear to be multiple recursive paths involved:
get_many_rec recursively traverses matching PointerTree paths.
- Checked
get_many recursively skips and validates selected or unmatched values.
- Container type-mismatch validation can also recursively skip the actual value.
Only limiting the PointerTree traversal would therefore not cover the shallow-path reproduction above.
A similar stack-overflow issue was previously reported for deserialization in #208 and addressed in #213.
I would be happy to prepare a PR to fix this. Before doing so, I would like to understand which approach maintainers would prefer:
- Add a fixed maximum depth for
get_many, similar to the existing deserializer recursion limit (for example, 255 levels), and return RecursionLimitExceeded once reached.
- Allow callers to configure the maximum depth explicitly.
- Provide an optional feature or opt-out mechanism for callers that need to process deeper inputs and can ensure sufficient stack space.
Any guidance on the preferred default and API/feature direction would be appreciated.
Describe the bug
Checked
get_manycan overflow the thread stack when processing a deeply nested but otherwise valid JSON payload.This can happen even when the requested path is shallow: while searching for a requested field,
get_manyrecursively validates and skips preceding, unrelated JSON values. A single deeply nested value can therefore abort the entire Rust process instead of returning a recoverable parsing error.To Reproduce
A runnable reproduction is also available here:
https://github.com/starcatmeow/sonic-rs/blob/rayli/repro-get-many-stack-overflow/examples/repro_get_many_stack_overflow.rs
Run:
git clone \ --branch rayli/repro-get-many-stack-overflow \ --single-branch \ https://github.com/starcatmeow/sonic-rs.git \ sonic-rs-get-many-repro cd sonic-rs-get-many-repro cargo run --release --example repro_get_many_stack_overflowObserved output:
The debug build also reproduces the issue:
Expected behavior
get_manyshould return a parsing or recursion-limit error before exhausting the stack.A deeply nested, unrelated value should not be able to terminate a process that is only attempting to extract a shallow field.
sonic-rs version:
0.5.9/ currentmain.Environment:
aarch64-apple-darwinrustc 1.96.1Additional context
This is particularly important for services processing externally supplied events, since a Rust stack overflow aborts the process and cannot be handled like a normal parsing error.
There appear to be multiple recursive paths involved:
get_many_recrecursively traverses matchingPointerTreepaths.get_manyrecursively skips and validates selected or unmatched values.Only limiting the
PointerTreetraversal would therefore not cover the shallow-path reproduction above.A similar stack-overflow issue was previously reported for deserialization in #208 and addressed in #213.
I would be happy to prepare a PR to fix this. Before doing so, I would like to understand which approach maintainers would prefer:
get_many, similar to the existing deserializer recursion limit (for example, 255 levels), and returnRecursionLimitExceededonce reached.Any guidance on the preferred default and API/feature direction would be appreciated.