Skip to content

fix(azure): switch to OkHttp transport to avoid Netty %2F/uppercase request-line rejection - #72

Merged
pallakartheekreddy merged 7 commits into
developfrom
fix/copyobject
Jun 4, 2026
Merged

fix(azure): switch to OkHttp transport to avoid Netty %2F/uppercase request-line rejection#72
pallakartheekreddy merged 7 commits into
developfrom
fix/copyobject

Conversation

@aimansharief

@aimansharief aimansharief commented Jun 3, 2026

Copy link
Copy Markdown

Summary

Azure SDK percent-encodes / in blob names as %2F in the HTTP request line, and Netty 4.1.108–4.1.130's HttpUtil.isEncodingSafeStartLineToken uses a buggy bitmask check (1L << c) that wraps modulo 64 and incorrectly rejects uppercase ASCII characters J (74), M (77), and backtick (96) as if they were LF / CR / SPACE. The combined effect breaks BlobClient.copyFromUrl (no-body PUT) for any blob whose name contains those characters — e.g. SCORM_API.js — with IllegalArgumentException: The URI contain illegal characters: ....

This PR switches the Azure BlobServiceClient from the default reactor-netty HTTP transport to OkHttp, which does not perform that validation. It also excludes azure-core-http-netty from the SDK classpath so consumers do not accidentally pick reactor-netty back up.

Root cause

Layer Behaviour
Azure SDK Java BlobClient.getBlobUrl() and the request builder URL-encode / in blob names as %2F in the request line
Reactor-netty (4.1.108–4.1.130) isEncodingSafeStartLineToken builds a bitmask with 1L << c. Java << on long wraps modulo 64, so 'M' = 77 maps to bit 13 (CR), 'J' = 74 maps to bit 10 (LF), '`' = 96 maps to bit 32 (SPACE). All three trigger rejection.
cloud-storage-sdk-azure (before this PR) destBlobClient.copyFromUrl(...) → no-body DefaultFullHttpRequest → Netty validates → IllegalArgumentException: The URI contain illegal characters

Production stack trace excerpt:

Caused by: java.lang.IllegalArgumentException: The URI contain illegal characters:
  /<container>/content%2Fscorm%2Fdo_xxx-version%2FSCORM_API.js
    at io.netty.handler.codec.http.HttpUtil.validateRequestLineTokens(HttpUtil.java:90)
    at io.netty.handler.codec.http.DefaultHttpRequest.<init>(DefaultHttpRequest.java:95)
    at reactor.netty.http.client.HttpClientOperations.newFullBodyMessage(...)
    at com.azure.storage.blob.specialized.BlobClientBase.copyFromUrl(BlobClientBase.java:704)
    at org.sunbird.cloud.storage.service.azure.AzureStorageService.copyObject(AzureStorageService.java:242)

The %2F portion is a red herring — it is valid percent-encoding. The actual char that trips the bitmask is the uppercase M in SCORM_API.js. Lowercase scorm_api.js (same content) passes because lowercase m (109) maps to bit 45, which is not in the forbidden mask. Streaming/chunked uploads also avoid the issue because they do not use DefaultFullHttpRequest.

Changes

  • Parent pom.xml: add azure.core.http.okhttp.version = 1.11.0 property and azure-core-http-okhttp dependencyManagement entry. Version 1.11.0 is intentional — it is compatible with azure-core 1.45.0 bundled by downstream consumers (e.g. asset-enrichment fat jar in knowledge-platform-jobs). Newer 1.11.20 calls BinaryData.writeTo(WritableByteChannel) (added in azure-core 1.49.0) and throws NoSuchMethodError at runtime.
  • cloud-storage-sdk-azure/pom.xml:
    • Add azure-core-http-okhttp dependency.
    • Exclude transitive azure-core-http-netty from azure-storage-blob and azure-identity so reactor-netty is not on the SDK classpath at all.
  • cloud-storage-sdk-azure/.../AzureStorageService.java: wire the OkHttp client on the BlobServiceClientBuilder:
    new BlobServiceClientBuilder()
        .endpoint(endpoint)
        .httpClient(new OkHttpAsyncHttpClientBuilder().build())
    copyObject itself is unchanged.
  • Bump SDK version 2.0.12.0.2 across parent + all modules.

Verification

Integration test against dev Azure account using the exact failing SCORM snapshot (content/scorm/do_2145843106720808961102-snapshot/) and SCORM_API.js filename:

Run HTTP transport Netty on classpath Filename Result
1 reactor-netty (default) 4.1.131 (no bitmask bug) SCORM_API.js PASS — netty fixed in 4.1.131, repro requires older
2 reactor-netty (default) 4.1.129 (bitmask bug) SCORM_API.js FAILURI contain illegal characters: ...%2F...SCORM_API.js (production bug reproduced)
3 reactor-netty (default) 4.1.129 (bitmask bug) scorm_api.js (lowercase) PASS — confirms the bug is uppercase-specific
4 OkHttp (this PR) none (excluded) SCORM_API.js PASS
5 OkHttp (this PR) 4.1.131 + reactor-netty added in test scope (simulates Flink user-jar with netty bundled) SCORM_API.js PASS — explicit .httpClient(okhttp) overrides default even when reactor-netty is present

All existing Azure integration tests (upload, list, download, signed URLs, copy, delete, search, getPaths) continue to pass on OkHttp.

Why OkHttp vs. other workarounds

Option Verdict
Decode dest URL + rebuild BlobClient via BlobClientBuilder.endpoint(decodedUrl) Does NOT work — SDK parses, splits container/blob, re-encodes blob path at request time
Bump netty to 4.1.131+ in consumers only Fragile — relies on every consumer aligning; some bundle older netty in fat jars (asset-enrichment)
Downgrade reactor-netty < 4.1.108 Reopens CVE-2024-29025 and related fixes; not acceptable
Replace Azure SDK calls with raw HTTP PUT + manual auth signing High complexity, separate path per auth type
Swap HTTP transport to OkHttp + exclude azure-core-http-netty (this PR) Single wiring change, isolates fix to SDK module, robust across consumer netty versions, removes netty from SDK classpath entirely

Test plan

  • All Azure integration tests pass (uploadAndList, uploadFolder, putAndGetData, downloadFile, getSignedReadUrl, getSignedWriteUrl, copyObject, deleteObject, searchObjectsByDate, getPaths, getObjectMetadata, getObjectWithPayload, etc.)
  • Bug reproduced under reactor-netty + Netty 4.1.129 with uppercase filename
  • Fix verified — OkHttp transport copies SCORM_API.js (uppercase M) successfully
  • Fix verified with reactor-netty also on classpath (simulates Flink user-jar)
  • Smoke test downstream knowledge-platform-jobs SCORM publish end-to-end on dev with cloud-store-sdk = 2.0.2
  • Smoke test ECML and HTML publish on dev — no regression

aimansharief and others added 5 commits June 2, 2026 19:03
…tion

Azure SDK percent-encodes '/' in blob names as '%2F' in the request line.
Reactor-netty's HttpUtil.validateRequestLineTokens rejects '%2F' in
DefaultFullHttpRequest, breaking single-shot REST ops (e.g. copyFromUrl) on
nested blob paths. Switching the Azure BlobServiceClient to the OkHttp HTTP
transport avoids that validation while preserving all other SDK behaviour.

Bump version to 2.0.2-beta.

Co-Authored-By: Claude Opus 4.7 <[email protected]>
fix: Use OkHttp transport for Azure to bypass reactor-netty %2F rejection
Exclude the transitive azure-core-http-netty from azure-storage-blob and
azure-identity so the SDK ships only the OkHttp HTTP transport. This removes
reactor-netty entirely from the SDK classpath, eliminating any future netty
validation regressions (e.g. the 4.1.129 bitmask bug where 1L << c wraps
modulo 64 and rejects uppercase ASCII chars J/M/backtick as LF/CR/SPACE).

Downgrade azure-core-http-okhttp 1.11.20 -> 1.11.0 to match the azure-core
1.45.0 bundled by downstream consumers (asset-enrichment fat jar in
knowledge-platform-jobs). 1.11.20 calls BinaryData.writeTo(WritableByteChannel)
which was added in azure-core 1.49.0 and triggers NoSuchMethodError at
runtime.

Co-Authored-By: Claude Opus 4.7 <[email protected]>
fix: Exclude azure-core-http-netty and align okhttp to azure-core 1.45.0
@aimansharief aimansharief changed the title Fix/copyobject fix(azure): switch to OkHttp transport to avoid Netty %2F/uppercase request-line rejection Jun 3, 2026

@pallakartheekreddy pallakartheekreddy left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall the fix is well-researched and the root cause analysis is excellent. A few inline notes below — two are must-fix before merge (import + pom comment), the rest are recommendations.

Comment thread pom.xml
Comment thread pom.xml
aimansharief and others added 2 commits June 3, 2026 16:39
Switch AzureStorageService to use the azure-core OkHttp async HTTP client (OkHttpAsyncHttpClientBuilder) to avoid reactor-netty's request-line validation bug that can reject percent-encoded/uppercase characters in blob names. Add and pin azure.core.http.okhttp.version to 1.11.0 in pom.xml with a comment noting newer okhttp bindings require azure-core >=1.49.0 while the build currently bundles azure-core 1.45.0.
fix: Use OkHttp transport for Azure SDK
@pallakartheekreddy
pallakartheekreddy merged commit 3ad4c2d into develop Jun 4, 2026
0 of 2 checks passed
@pallakartheekreddy
pallakartheekreddy deleted the fix/copyobject branch June 4, 2026 06:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants