Skip to content

fix: honor byteOffset when deserializing L2 coordinate cache (Node Buffer) — v0.7.1#9

Merged
guidocerqueira merged 1 commit into
mainfrom
fix/coordinate-cache-buffer-alignment
Jun 15, 2026
Merged

fix: honor byteOffset when deserializing L2 coordinate cache (Node Buffer) — v0.7.1#9
guidocerqueira merged 1 commit into
mainfrom
fix/coordinate-cache-buffer-alignment

Conversation

@guidocerqueira

Copy link
Copy Markdown
Collaborator

Problem

Coordinate/time arrays served from the shared L2 coordinateCache (ManagedDataset.decodedArray) came back corrupted whenever the cache returned a Node Buffer.

bytesToFloat64 rebuilt the Float64Array via:

const copy = bytes.slice(0, usable);            // Buffer.slice → VIEW, not a copy
return new Float64Array(copy.buffer, 0, usable / 8); // reads offset 0 of the POOL

ioredis getBuffer carves Buffers from a shared pool, so the returned Buffer is a view with a non-zero byteOffset. Reading from offset 0 of the underlying ArrayBuffer (ignoring byteOffset) returns shifted/garbage values.

Downstream impact (observed in nautilus-api compare-models)

A garbled, non-ascending decoded time axis broke timeWindow's binary search → the requested window collapsed to the full range with a single repeated timestamp. Every series point rendered with the same timestamp. Same corruption silently affected lat/lon coordinate arrays (nearest-point resolution). Only manifested once the L2 cache was populated (first read = L2 miss = correct), which is why it looked like "just cache".

Fix

Copy the view's own byte range with ArrayBuffer.prototype.slice, honoring byteOffset:

const ab = bytes.buffer.slice(bytes.byteOffset, bytes.byteOffset + usable);
return new Float64Array(ab);

Robust for plain Uint8Array views and pooled Node Buffers alike.

Tests

  • Added a regression test in dataset-registry.test.ts that serves L2 through a cache returning a pooled Node Buffer (non-zero byteOffset). Fails on the old code ([0, 10, 20] shifted), passes with the fix.
  • Full suite: 335 passing; prepublishOnly chain (typecheck/lint/test/build/validate-exports/cjs/esm interop) green.

Release

Bumps to 0.7.1 + CHANGELOG entry. After merge, tag v0.7.1 to trigger the publish workflow.

…ffer)

bytesToFloat64 rebuilt the Float64Array via bytes.slice(0,n).buffer +
new Float64Array(buffer, 0, ...), reading from offset 0 of the underlying
ArrayBuffer. For a Node Buffer (ioredis getBuffer pools Buffers as views
with a non-zero byteOffset) this read the wrong bytes, corrupting any
coordinate/time array served from the shared coordinateCache. A garbled,
non-ascending time axis then collapsed downstream time-window lookups so
every series point folded onto one timestamp.

Copy the view's own byte range with ArrayBuffer.prototype.slice, honoring
byteOffset. Add a regression test exercising a pooled-Buffer round-trip.

Release v0.7.1.
@guidocerqueira guidocerqueira merged commit 28becdf into main Jun 15, 2026
2 checks passed
@brandao-andre brandao-andre deleted the fix/coordinate-cache-buffer-alignment branch June 15, 2026 21:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants