Skip to content

skip protected-header JSON parse when "zip" is absent#186

Merged
sergio-correia merged 1 commit into
latchset:masterfrom
hawk259:speed-up-zip
Jul 23, 2026
Merged

skip protected-header JSON parse when "zip" is absent#186
sergio-correia merged 1 commit into
latchset:masterfrom
hawk259:speed-up-zip

Conversation

@hawk259

@hawk259 hawk259 commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

Summary

handle_zip_enc() and zip_in_protected_header() sit on the JWE encrypt
hot path. For every call they base64url-decode the protected header and
run a full JSON parse (json_loadb) just to check for a zip key.

This gates the expensive JSON parse behind a cheap substring scan: decode the
header, scan the decoded bytes for the literal "zip", and only parse when
it's present. When zip is absent, we skip the parse and take the existing
no-zip path. A false-positive substring just falls through to the real parse,
so behaviour is unchanged in every case.

Why scan the decoded bytes, not the encoded string

An earlier version of this optimization scanned the base64url-encoded
protected string for "zip". That is incorrect: a zip key does not survive
base64url encoding as the literal substring "zip" — it lands on shifting
bit boundaries (e.g. {"...","zip":"DEF"} encodes with zip appearing as
emlw). Scanning the encoded form yields false negatives, so compression is
silently skipped on zip-enabled messages and the JWE fails to round-trip. This
version scans the decoded JSON text, where a zip key must appear verbatim.

Performance

zip_in_protected_header on a ~56-byte header, -O2, best of 3:

Header Before After Delta
zip absent ~4.5 µs ~3.2 µs ~30% faster (parse skipped)
zip present ~6.0 µs ~6.0 µs within noise (+~21 ns memmem)

The added cost on the (rare) zip-enabled path is a single memmem over the
decoded header (~21 ns); the win on the common no-zip path is the eliminated
json_loadb.

Testing

  • meson testall 26 tests pass, including api_jwe's zip
    compress/decompress round-trip (which the encoded-scan version failed with
    SIGABRT).
  • Verified the clean base passes the same suite, so no pre-existing failures.

Notes

  • The pre-check requires _GNU_SOURCE for memmem and <jose/cfg.h> for the
    jose_calloc/jose_free allocators (used to mirror jose_b64_dec_load's
    scrub-and-free of the decoded buffer).
  • prt is a borrowed reference from json_object_get() at the early-return
    points; it's cleared to NULL before returning so the json_auto_t cleanup
    doesn't spuriously json_decref it.

`handle_zip_enc()` and `zip_in_protected_header()` sit on the JWE encrypt
hot path. For every call they base64url-decode the `protected` header and
run a full JSON parse (`json_loadb`) just to check for a `zip` key.

This gates the expensive JSON parse behind a cheap substring scan: decode the
header, scan the **decoded** bytes for the literal `"zip"`, and only parse when
it's present. When `zip` is absent, we skip the parse and take the existing
no-zip path. A false-positive substring just falls through to the real parse,
so behaviour is unchanged in every case.

An earlier version of this optimization scanned the base64url-**encoded**
`protected` string for `"zip"`. That is incorrect: a `zip` key does not survive
base64url encoding as the literal substring `"zip"` — it lands on shifting
bit boundaries (e.g. `{"...","zip":"DEF"}` encodes with `zip` appearing as
`emlw`). Scanning the encoded form yields false negatives, so compression is
silently skipped on zip-enabled messages and the JWE fails to round-trip. This
version scans the decoded JSON text, where a `zip` key must appear verbatim.

`zip_in_protected_header` on a ~56-byte header, `-O2`, best of 3:

| Header      | Before   | After     | Delta            |
|-------------|----------|-----------|------------------|
| `zip` absent  | ~4.5 µs  | ~3.2 µs   | ~30% faster (parse skipped) |
| `zip` present | ~6.0 µs  | ~6.0 µs   | within noise (+~21 ns memmem) |

The added cost on the (rare) zip-enabled path is a single `memmem` over the
decoded header (~21 ns); the win on the common no-zip path is the eliminated
`json_loadb`.

- `meson test` — **all 26 tests pass**, including `api_jwe`'s zip
  compress/decompress round-trip (which the encoded-scan version failed with
  SIGABRT).
- Verified the clean base passes the same suite, so no pre-existing failures.

- The pre-check requires `_GNU_SOURCE` for `memmem` and `<jose/cfg.h>` for the
  `jose_calloc`/`jose_free` allocators (used to mirror `jose_b64_dec_load`'s
  scrub-and-free of the decoded buffer).
- `prt` is a borrowed reference from `json_object_get()` at the early-return
  points; it's cleared to `NULL` before returning so the `json_auto_t` cleanup
  doesn't spuriously `json_decref` it.

@simo5 simo5 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@sergio-correia sergio-correia left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, LGTM.

@sergio-correia
sergio-correia merged commit 41176df into latchset:master Jul 23, 2026
22 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants