Skip to content

lib: refactor internal webidl converters#62979

Open
panva wants to merge 1 commit intonodejs:mainfrom
panva:webidl-pass
Open

lib: refactor internal webidl converters#62979
panva wants to merge 1 commit intonodejs:mainfrom
panva:webidl-pass

Conversation

@panva
Copy link
Copy Markdown
Member

@panva panva commented Apr 26, 2026

Rework lib/internal/webidl.js into a documented shared converter module that follows the Web IDL conversion algorithms more closely.

Improvements:

  • Add documented converters and helper factories for primitive values, dictionaries, enums, sequences, interfaces, required arguments, integers, Uint8Array, and BufferSource.
  • Move WebCrypto onto the shared converters, while keeping compatibility wrappers for its existing BufferSource and BigInteger behavior.
  • Use shared converters from Blob, Performance, Web Locks, and structured clone option handling.
  • Add benchmarks for ConvertToInt and WebCrypto Web IDL converter hot paths.
  • Add focused tests for core converters, WebCrypto converters, integer conversion, and buffer source behavior.

Fixes:

  • Make the shared BufferSource and Uint8Array converters reject resizable ArrayBuffer and growable SharedArrayBuffer backing stores unless explicitly allowed. WebCrypto preserves its legacy resizable backing-store behavior through compatibility wrappers until a semver-major follow-up can opt in to the stricter behavior.
  • Use Web IDL ToNumber and ToString behavior for BigInt, Symbol, and object primitive conversion.
  • Use exact BigInt modulo for 64-bit ConvertToInt wrapping and document the final Number approximation behavior.
  • Normalize mathematical modulo results to +0 where Web IDL requires it.
  • Process inherited dictionaries in least-derived to most-derived order, sorting members only within each dictionary level.
  • Use IteratorComplete truthiness for sequence conversion.
  • Cover detached buffers, resizable-backed views, growable-backed views, cross-realm buffer sources, mutation-after-call behavior, inherited dictionary member order, and sequence iterator completion behavior.

Benchmark(s)

@nodejs-github-bot
Copy link
Copy Markdown
Collaborator

nodejs-github-bot commented Apr 26, 2026

Review requested:

  • @nodejs/crypto
  • @nodejs/web-standards

@nodejs-github-bot nodejs-github-bot added lib / src Issues and PRs related to general changes in the lib or src directory. needs-ci PRs that need a full CI run. labels Apr 26, 2026
@panva panva added the web-standards Issues and PRs related to Web APIs label Apr 26, 2026
Comment thread lib/internal/webidl.js
Comment thread lib/internal/webidl.js Outdated
@codecov
Copy link
Copy Markdown

codecov Bot commented Apr 27, 2026

Codecov Report

❌ Patch coverage is 99.32886% with 9 lines in your changes missing coverage. Please review.
✅ Project coverage is 89.69%. Comparing base (21436f0) to head (0cfebb7).
⚠️ Report is 19 commits behind head on main.

Files with missing lines Patch % Lines
lib/internal/crypto/webidl.js 98.92% 5 Missing ⚠️
lib/internal/webidl.js 99.53% 4 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main   #62979      +/-   ##
==========================================
+ Coverage   89.66%   89.69%   +0.03%     
==========================================
  Files         706      707       +1     
  Lines      219391   219950     +559     
  Branches    42068    42111      +43     
==========================================
+ Hits       196712   197280     +568     
+ Misses      14578    14567      -11     
- Partials     8101     8103       +2     
Files with missing lines Coverage Δ
lib/internal/blob.js 99.44% <100.00%> (+<0.01%) ⬆️
lib/internal/locks.js 94.39% <100.00%> (ø)
lib/internal/perf/performance.js 93.54% <100.00%> (+0.05%) ⬆️
lib/internal/worker/js_transferable.js 98.56% <100.00%> (+0.01%) ⬆️
lib/internal/webidl.js 99.43% <99.53%> (+2.70%) ⬆️
lib/internal/crypto/webidl.js 99.33% <98.92%> (+0.95%) ⬆️

... and 42 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Comment thread lib/internal/crypto/webidl.js Outdated
Comment thread lib/internal/webidl.js Outdated
Comment thread lib/internal/webidl.js Outdated
Comment thread lib/internal/webidl.js Outdated
Comment thread lib/internal/webidl.js Outdated
Comment thread lib/internal/webidl.js Outdated
@panva
Copy link
Copy Markdown
Member Author

panva commented Apr 27, 2026

i'm nearing the end of what i can optimize on the hot paths, just one more fixup then i'll do a final benchmark

Comment thread lib/internal/webidl.js Outdated
Comment thread lib/internal/webidl.js Outdated
Comment thread lib/internal/crypto/webidl.js Outdated
@panva

This comment was marked as outdated.

Comment thread lib/internal/webidl.js Outdated
@panva
Copy link
Copy Markdown
Member Author

panva commented Apr 27, 2026

Benchmark CI: https://ci.nodejs.org/view/Node.js%20benchmark/job/benchmark-node-micro-benchmarks/1842

                                                                                         confidence improvement accuracy (*)     (**)    (***)
misc/webcrypto-webidl.js n=1000000 op='normalizeAlgorithm-dict'                                 ***     80.70 %       ±1.24%   ±1.66%   ±2.16%
misc/webcrypto-webidl.js n=1000000 op='normalizeAlgorithm-string'                               ***     57.96 %       ±0.77%   ±1.03%   ±1.36%
misc/webcrypto-webidl.js n=1000000 op='webidl-algorithm-identifier-object'                              -0.98 %       ±3.94%   ±5.27%   ±6.91%
misc/webcrypto-webidl.js n=1000000 op='webidl-algorithm-identifier-string'                              -5.16 %       ±5.97%   ±8.01%  ±10.55%
misc/webcrypto-webidl.js n=1000000 op='webidl-dict-enforce-range'                               ***     13.85 %       ±1.02%   ±1.37%   ±1.81%
misc/webcrypto-webidl.js n=1000000 op='webidl-dict-ensure-sha'                                  ***    170.12 %       ±1.38%   ±1.85%   ±2.45%
misc/webcrypto-webidl.js n=1000000 op='webidl-dict-null'                                        ***    -62.11 %       ±5.31%   ±7.13%   ±9.43%
misc/webcrypto-webidl.js n=1000000 op='webidl-dict'                                             ***     15.13 %       ±0.62%   ±0.83%   ±1.08%
misc/webidl-buffer-source.js n=1000000 input='arraybuffer'                                        *     -4.96 %       ±3.99%   ±5.32%   ±6.92%
misc/webidl-buffer-source.js n=1000000 input='bigint64array'                                             1.45 %       ±5.46%   ±7.27%   ±9.47%
misc/webidl-buffer-source.js n=1000000 input='biguint64array'                                            1.86 %       ±4.37%   ±5.83%   ±7.60%
misc/webidl-buffer-source.js n=1000000 input='buffer'                                                   -0.12 %       ±3.25%   ±4.32%   ±5.63%
misc/webidl-buffer-source.js n=1000000 input='dataview'                                         ***     -8.94 %       ±3.94%   ±5.24%   ±6.83%
misc/webidl-buffer-source.js n=1000000 input='float16array'                                              1.28 %       ±3.69%   ±4.92%   ±6.41%
misc/webidl-buffer-source.js n=1000000 input='float32array'                                             -1.30 %       ±3.71%   ±4.93%   ±6.42%
misc/webidl-buffer-source.js n=1000000 input='float64array'                                              1.09 %       ±5.30%   ±7.05%   ±9.17%
misc/webidl-buffer-source.js n=1000000 input='int16array'                                                0.21 %       ±5.04%   ±6.74%   ±8.86%
misc/webidl-buffer-source.js n=1000000 input='int32array'                                               -1.84 %       ±4.47%   ±5.95%   ±7.76%
misc/webidl-buffer-source.js n=1000000 input='int8array'                                                 1.56 %       ±3.78%   ±5.03%   ±6.55%
misc/webidl-buffer-source.js n=1000000 input='uint16array'                                               1.07 %       ±4.19%   ±5.59%   ±7.29%
misc/webidl-buffer-source.js n=1000000 input='uint32array'                                               0.09 %       ±5.04%   ±6.73%   ±8.79%
misc/webidl-buffer-source.js n=1000000 input='uint8array'                                               -1.34 %       ±5.15%   ±6.87%   ±8.98%
misc/webidl-buffer-source.js n=1000000 input='uint8clampedarray'                                        -2.39 %       ±3.98%   ±5.30%   ±6.91%
misc/webidl-convert-to-int.js n=1000000 input='clamp' converter='byte'                          ***   2364.33 %      ±37.30%  ±50.27%  ±66.73%
misc/webidl-convert-to-int.js n=1000000 input='clamp' converter='long long'                     ***    910.48 %      ±24.25%  ±32.68%  ±43.38%
misc/webidl-convert-to-int.js n=1000000 input='clamp' converter='octet'                         ***   1871.53 %      ±36.50%  ±49.19%  ±65.31%
misc/webidl-convert-to-int.js n=1000000 input='clamp' converter='unsigned long'                 ***    697.51 %      ±12.22%  ±16.47%  ±21.85%
misc/webidl-convert-to-int.js n=1000000 input='clamp' converter='unsigned short'                ***    680.33 %      ±13.68%  ±18.43%  ±24.47%
misc/webidl-convert-to-int.js n=1000000 input='enforce-range' converter='byte'                  ***    789.35 %      ±33.50%  ±45.15%  ±59.93%
misc/webidl-convert-to-int.js n=1000000 input='enforce-range' converter='long long'             ***    817.08 %      ±16.35%  ±22.03%  ±29.24%
misc/webidl-convert-to-int.js n=1000000 input='enforce-range' converter='octet'                 ***    267.04 %       ±5.89%   ±7.86%  ±10.27%
misc/webidl-convert-to-int.js n=1000000 input='enforce-range' converter='unsigned long'         ***    239.33 %       ±5.71%   ±7.63%   ±9.99%
misc/webidl-convert-to-int.js n=1000000 input='enforce-range' converter='unsigned short'        ***    228.15 %       ±8.52%  ±11.40%  ±14.96%
misc/webidl-convert-to-int.js n=1000000 input='fractional' converter='byte'                     ***   3303.13 %     ±155.26% ±209.25% ±277.80%
misc/webidl-convert-to-int.js n=1000000 input='fractional' converter='long long'                ***   3554.14 %     ±130.83% ±176.32% ±234.09%
misc/webidl-convert-to-int.js n=1000000 input='fractional' converter='octet'                    ***   2316.22 %      ±29.23%  ±39.39%  ±52.29%
misc/webidl-convert-to-int.js n=1000000 input='fractional' converter='unsigned long'            ***   2253.50 %      ±46.15%  ±62.20%  ±82.57%
misc/webidl-convert-to-int.js n=1000000 input='fractional' converter='unsigned short'           ***   2028.85 %      ±43.02%  ±57.98%  ±76.98%
misc/webidl-convert-to-int.js n=1000000 input='integer' converter='byte'                        ***   3904.02 %      ±26.77%  ±36.07%  ±47.89%
misc/webidl-convert-to-int.js n=1000000 input='integer' converter='long long'                   ***   4202.53 %      ±39.36%  ±53.04%  ±70.42%
misc/webidl-convert-to-int.js n=1000000 input='integer' converter='octet'                       ***   2569.17 %      ±50.27%  ±67.75%  ±89.95%
misc/webidl-convert-to-int.js n=1000000 input='integer' converter='unsigned long'               ***   2550.93 %      ±39.27%  ±52.93%  ±70.27%
misc/webidl-convert-to-int.js n=1000000 input='integer' converter='unsigned short'              ***   2447.31 %      ±33.04%  ±44.52%  ±59.11%
misc/webidl-convert-to-int.js n=1000000 input='object' converter='byte'                         ***    662.03 %      ±17.80%  ±23.99%  ±31.85%
misc/webidl-convert-to-int.js n=1000000 input='object' converter='long long'                    ***    668.98 %      ±14.18%  ±19.11%  ±25.37%
misc/webidl-convert-to-int.js n=1000000 input='object' converter='octet'                        ***    408.93 %       ±8.99%  ±12.11%  ±16.07%
misc/webidl-convert-to-int.js n=1000000 input='object' converter='unsigned long'                ***    403.11 %       ±7.30%   ±9.83%  ±13.04%
misc/webidl-convert-to-int.js n=1000000 input='object' converter='unsigned short'               ***    400.87 %       ±6.88%   ±9.27%  ±12.30%
misc/webidl-convert-to-int.js n=1000000 input='wrap' converter='byte'                           ***    736.95 %      ±29.07%  ±39.18%  ±52.01%
misc/webidl-convert-to-int.js n=1000000 input='wrap' converter='long long'                      ***    217.47 %       ±4.32%   ±5.82%   ±7.72%
misc/webidl-convert-to-int.js n=1000000 input='wrap' converter='octet'                          ***    481.05 %      ±13.79%  ±18.58%  ±24.66%
misc/webidl-convert-to-int.js n=1000000 input='wrap' converter='unsigned long'                  ***    473.76 %      ±20.60%  ±27.77%  ±36.86%
misc/webidl-convert-to-int.js n=1000000 input='wrap' converter='unsigned short'                 ***    491.65 %      ±27.24%  ±36.71%  ±48.74%

Be aware that when doing many comparisons the risk of a false-positive
result increases. In this case, there are 53 comparisons, you can thus
expect the following amount of false-positive results:
  2.65 false positives, when considering a   5% risk acceptance (*, **, ***),
  0.53 false positives, when considering a   1% risk acceptance (**, ***),
  0.05 false positives, when considering a 0.1% risk acceptance (***)

Rework lib/internal/webidl.js into a documented shared converter module
that follows the Web IDL conversion algorithms more closely.

Improvements:
- Add documented converters and helper factories for primitive values,
  dictionaries, enums, sequences, interfaces, required arguments,
  integers, `Uint8Array`, and `BufferSource`.
- Move WebCrypto onto the shared converters, while keeping compatibility
  wrappers for its existing `BufferSource` and `BigInteger` behavior.
- Use shared converters from Blob, Performance, Web Locks, and
  structured clone option handling.
- Add benchmarks for `ConvertToInt` and WebCrypto Web IDL converter hot
  paths.
- Add focused tests for core converters, WebCrypto converters, integer
  conversion, and buffer source behavior.

Fixes:
- Make the shared `BufferSource` and `Uint8Array` converters reject
  resizable `ArrayBuffer` and growable `SharedArrayBuffer` backing
  stores unless explicitly allowed. WebCrypto preserves its legacy
  resizable backing-store behavior through compatibility wrappers until
  a semver-major follow-up can opt in to the stricter behavior.
- Use Web IDL `ToNumber` and `ToString` behavior for BigInt, Symbol, and
  object primitive conversion.
- Use exact BigInt modulo for 64-bit `ConvertToInt` wrapping and
  document the final Number approximation behavior.
- Normalize mathematical modulo results to `+0` where Web IDL requires
  it.
- Process inherited dictionaries in least-derived to most-derived order,
  sorting members only within each dictionary level.
- Use `IteratorComplete` truthiness for sequence conversion.
- Cover detached buffers, resizable-backed views, growable-backed views,
  cross-realm buffer sources, mutation-after-call behavior, inherited
  dictionary member order, and sequence iterator completion behavior.

Signed-off-by: Filip Skokan <[email protected]>
@panva
Copy link
Copy Markdown
Member Author

panva commented Apr 27, 2026

(just a rebase, no changes)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

lib / src Issues and PRs related to general changes in the lib or src directory. needs-ci PRs that need a full CI run. web-standards Issues and PRs related to Web APIs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants