From 6c636a313fa841735acc31763cf2afe86701c7f4 Mon Sep 17 00:00:00 2001 From: Ronald Tse Date: Sun, 26 Jul 2026 19:50:57 +0800 Subject: [PATCH 01/21] Add TODO.ohos/ design docs for OHOS NDK pivot 12 design documents covering the architecture, NDK setup, zlib build, recipe factory, code signing, smoke-test, CI workflow, specs, docs, and PR plan for cross-compiling libpng for OHOS with the official NDK. Replaces PR #12's Alpine-built bytes labeled OHOS approach with proper OHOS NDK cross-compile + signing + dockerharmony verification. --- TODO.ohos/01-architecture.md | 105 +++++++++++ TODO.ohos/02-close-pr-12-and-branch.md | 30 ++++ TODO.ohos/03-setup-ndk-sh.md | 54 ++++++ TODO.ohos/04-ohos-module-and-ndk.md | 65 +++++++ TODO.ohos/05-zlib-builder.md | 65 +++++++ .../06-recipe-factory-and-ohos-recipe.md | 126 +++++++++++++ TODO.ohos/07-code-signer.md | 53 ++++++ TODO.ohos/08-smoke-test-and-verify-prepare.md | 79 ++++++++ TODO.ohos/09-ci-workflow.md | 101 +++++++++++ TODO.ohos/10-specs.md | 170 ++++++++++++++++++ TODO.ohos/11-docs.md | 91 ++++++++++ TODO.ohos/12-pr.md | 152 ++++++++++++++++ ext/ohos/smoke-test.c | 73 ++++++++ 13 files changed, 1164 insertions(+) create mode 100644 TODO.ohos/01-architecture.md create mode 100644 TODO.ohos/02-close-pr-12-and-branch.md create mode 100644 TODO.ohos/03-setup-ndk-sh.md create mode 100644 TODO.ohos/04-ohos-module-and-ndk.md create mode 100644 TODO.ohos/05-zlib-builder.md create mode 100644 TODO.ohos/06-recipe-factory-and-ohos-recipe.md create mode 100644 TODO.ohos/07-code-signer.md create mode 100644 TODO.ohos/08-smoke-test-and-verify-prepare.md create mode 100644 TODO.ohos/09-ci-workflow.md create mode 100644 TODO.ohos/10-specs.md create mode 100644 TODO.ohos/11-docs.md create mode 100644 TODO.ohos/12-pr.md create mode 100644 ext/ohos/smoke-test.c diff --git a/TODO.ohos/01-architecture.md b/TODO.ohos/01-architecture.md new file mode 100644 index 0000000..7331055 --- /dev/null +++ b/TODO.ohos/01-architecture.md @@ -0,0 +1,105 @@ +# 01 - Architecture + +## Goal + +Cross-compile `libpng16.so` for `aarch64-linux-ohos` using the official OHOS +NDK (Huawei-blessed toolchain), code-sign it, and empirically verify it in +real OHOS userland (dockerharmony). Replace the 1.6.58.4/.5/.6 approach of +"Alpine-built bytes labeled as OHOS". + +## Why this design (vs PR #11 / PR #12) + +| Approach | Status | Why rejected | +|---|---|---| +| Alpine bytes labeled OHOS (1.6.58.4/.5) | shipped | unverified assumption; user flagged musl symbol visibility, TLS, pthread layout risks | +| PR #11: hand-written `toolchain.cmake` + system CMake | closed | 5 CI iterations failed (sysroot path, missing zlib in sysroot, try_compile couldn't find crt objects) — root cause: NOT using NDK's bundled `ohos.toolchain.cmake` | +| PR #12: Alpine bytes + dockerharmony verification | closed | still ships Alpine bytes labeled OHOS; user wants proper NDK bytes | +| **PR #13 (this plan)** | current | NDK-bundled `ohos.toolchain.cmake` + static zlib + `binary-sign-tool` + dockerharmony verify | + +## Key insights from references + +- **OHOS CMake doc** (`openharmony-6.0-app-dev-docs/napi/build-with-ndk-cmake.md`): + - NDK ships own CMake at `${SDK_PATH}/native/build-tools/cmake/bin/cmake` + - Toolchain file at `${SDK_PATH}/native/build/cmake/ohos.toolchain.cmake` + - Invoke: `cmake -DOHOS_ARCH=arm64-v8a -DOHOS_PLATFORM=OHOS -DCMAKE_TOOLCHAIN_FILE=...` + - Toolchain handles sysroot + `--target=arm-linux-ohos` + try_compile compatibility + +- **ohos-node** (`~/src/external/ohos-node/build.sh`): + - NDK download via `https://dcp.openharmony.cn/api/daily_build/build/list/component` + - Two components needed: `ohos-sdk-public` (SDK + `binary-sign-tool`) + `LLVM-19` (clang + sysroot) + - `binary-sign-tool sign -selfSign 1` is mandatory post-build + - Output runs on OHOS via dockerharmony (real OHOS rootfs) + +- **dockerharmony** (`~/src/external/dockerharmony/`): + - OHOS rootfs (musl + toybox + mksh) in Docker, runs aarch64-linux-musl binaries natively on arm64 hosts + - OHOS ships zlib as `libshared_libz.z.so` (NOT `libz.so`) — patchelf needed if linking dynamically + +- **NDK architecture reality** (verified): + - `LLVM-19` tarball contains `llvm-linux-x86_64.tar.gz` (x86_64-build clang) + `ohos-sysroot.tar.gz` (architecture-independent sysroot files) + - There is **no arm64 build of the NDK clang** in the daily_build output + - `ohos.toolchain.cmake` is a CMake script (architecture-independent) + - `binary-sign-tool` is in the x86_64-build ohos-sdk + +## CI topology (corrected: native arm64 + transparent qemu for NDK binaries) + +Single job on `ubuntu-24.04-arm` (native arm64 runner): + +1. Install `qemu-user-static` + `binfmt-support` — registers binfmt_misc so + x86_64 ELF binaries run transparently via `qemu-x86_64`. +2. Run `ext/ohos/setup-ndk.sh` → NDK + LLVM-19 in `ext/ohos/ndk/` +3. `bundle exec rake gem:native:aarch64-linux-ohos` → + - `OHOS::Recipe#cook` runs: + - Downloads libpng source (MiniPortile) + - Builds static zlib via `OHOS::ZlibBuilder` (NDK clang under binfmt) + - CMake-configures libpng with `ohos.toolchain.cmake` + static zlib (NDK cmake/clang under binfmt) + - Builds + installs `libpng16.so` + - Signs `.so` via `OHOS::CodeSigner` (`binary-sign-tool` under binfmt) +4. Run `ext/ohos/verify-prepare.sh` → `ohos-verify/` artifacts (cross-compile smoke-test with NDK clang) +5. Run dockerharmony natively: `docker run --rm -v ...:/work ghcr.io/hqzing/dockerharmony:latest sh -c 'LD_LIBRARY_PATH=. ./smoke-test'` + +### Why this is the right topology + +| Aspect | x86_64 runner (original plan) | arm64 runner + qemu (revised) | +|---|---|---| +| NDK clang / binary-sign-tool | native speed | ~5x slower (qemu emulation) | +| dockerharmony verification | ~5x slower (qemu binfmt for arm64) | **native speed** | +| CI complexity | needs binfmt setup, multi-arch docker pull | standard docker pull | +| Total CI time | dominated by slow dockerharmony emulation | dominated by slow NDK compile (smaller) | + +Compilation is ~1 min native → ~5 min under qemu (acceptable). +dockerharmony smoke-test is ~50ms native → ~250ms under qemu — small absolute number, +but the binfmt setup for arm64 emulation on x86_64 is finicky and has been flaky +in CI historically. Native arm64 docker is rock-solid. + +**Net**: arm64 + qemu is simpler AND faster overall. + +## MECE file layout + +``` +ext/ohos/ +├── setup-ndk.sh # Downloads SDK + LLVM-19 (idempotent, shell) +├── smoke-test.c # Round-trip test (kept from PR #12) +├── verify-prepare.sh # Cross-compiles smoke-test, bundles for dockerharmony +└── (no toolchain.cmake — we use NDK's bundled ohos.toolchain.cmake) + +lib/libpng/ +├── recipe.rb # Add Recipe.for_target factory (OCP seam) +├── ohos.rb # Module Libpng::OHOS + autoloads +└── ohos/ + ├── ndk.rb # OHOS::NDK: pure-data path discovery + download + ├── zlib_builder.rb # OHOS::ZlibBuilder < MiniPortileCMake (static zlib) + ├── code_signer.rb # OHOS::CodeSigner: wraps binary-sign-tool + └── recipe.rb # OHOS::Recipe < Libpng::Recipe + +lib/libpng.rb # Add autoload :OHOS, 'libpng/ohos' +ext/extconf.rb # Use Recipe.for_target(...) factory +``` + +## OCP compliance + +Adding a new cross-compile platform = adding `lib/libpng//` with a +`::Recipe < Libpng::Recipe` subclass and registering it in +`Recipe.for_target`. No modification of `Libpng::Recipe` needed. + +The existing `Libpng::Recipe` is unchanged (except adding the factory class +method). All OHOS logic is in `lib/libpng/ohos/`. diff --git a/TODO.ohos/02-close-pr-12-and-branch.md b/TODO.ohos/02-close-pr-12-and-branch.md new file mode 100644 index 0000000..668468c --- /dev/null +++ b/TODO.ohos/02-close-pr-12-and-branch.md @@ -0,0 +1,30 @@ +# 02 - Close PR #12, branch off main + +## Steps + +1. Close PR #12 with a comment explaining the pivot: + + > Closing in favor of a proper OHOS NDK cross-compile approach (PR #13). + > The dockerharmony verification step is reused; the build approach changes + > from "Alpine-built bytes labeled OHOS" to "NDK-built bytes signed with + > binary-sign-tool". The smoke-test.c file is carried over. + +2. From `main`, create `fix/ohos-ndk-cmake`: + ```sh + git checkout main + git pull origin main + git checkout -b fix/ohos-ndk-cmake + ``` + +3. Cherry-pick reusable files from `fix/ohos-dockerharmony`: + - `ext/ohos/smoke-test.c` — keep as-is + - `lib/libpng/version.rb` — keep iteration = 6 (it's already at 6 on PR #12 branch) + + `verify-prepare.sh` will be rewritten (was Alpine-gcc-based; new version + uses NDK clang for cross-compile from x86_64 host). + +## Why branch off main (not off fix/ohos-dockerharmony) + +PR #12's CI changes (build.yml, release.yml) are incompatible with the NDK +approach — they assume Alpine container build. Cleaner to start fresh and +re-introduce only what's needed. diff --git a/TODO.ohos/03-setup-ndk-sh.md b/TODO.ohos/03-setup-ndk-sh.md new file mode 100644 index 0000000..15bcf01 --- /dev/null +++ b/TODO.ohos/03-setup-ndk-sh.md @@ -0,0 +1,54 @@ +# 03 - ext/ohos/setup-ndk.sh + +## Purpose + +Download and extract the OHOS NDK (SDK + LLVM-19) into `ext/ohos/ndk/`. +Idempotent — skips if the destination already looks complete. + +## Source pattern + +Adapted from `~/src/external/ohos-node/build.sh` (lines 14-49). Key pieces: + +```sh +query_component() { + component=$1 + curl -fsSL 'https://dcp.openharmony.cn/api/daily_build/build/list/component' \ + -H 'Accept: application/json, text/plain, */*' \ + -H 'Content-Type: application/json' \ + --data-raw '{"projectName":"openharmony","branch":"master","pageNum":1,"pageSize":10,"deviceLevel":"","component":"'${component}'","type":1,"startTime":"2025080100000000","endTime":"20990101235959","sortType":"","sortField":"","hardwareBoard":"","buildStatus":"success","buildFailReason":"","withDomain":1}' +} +``` + +Two components needed: +- `ohos-sdk-public` → contains `toolchains/lib/binary-sign-tool` + `build/cmake/ohos.toolchain.cmake` +- `LLVM-19` → contains `llvm/bin/aarch64-unknown-linux-ohos-clang` + `ohos-sysroot.tar.gz` + +## Script contract + +``` +Usage: setup-ndk.sh [--prefix ] +Default prefix: $PWD/ext/ohos/ndk + +Outputs: + $PREFIX/ohos-sdk/linux/ # SDK + $PREFIX/llvm-19/llvm/ # Clang + $PREFIX/llvm-19/sysroot/ # OHOS sysroot (extracted from ohos-sysroot.tar.gz) + +Exits 0 if setup is complete (or was already complete). +``` + +## Idempotency + +Skip download if `$PREFIX/ohos-sdk/linux/toolchains/lib/binary-sign-tool` exists +AND `$PREFIX/llvm-19/sysroot/usr/lib/aarch64-linux-ohos/` exists. This lets +CI cache the directory across runs. + +## Dependencies (CI-side) + +- `curl`, `tar`, `unzip`, `jq` — apt-installed in workflow before script runs. + +## Why shell (not Ruby) + +- curl + jq + tar pipeline is 5 lines of shell vs 50+ lines of Ruby (Net::HTTP + JSON parsing). +- Pattern is proven by ohos-node; we follow it verbatim. +- Ruby's role is orchestration (deciding *when* to invoke); the script does the heavy lifting. diff --git a/TODO.ohos/04-ohos-module-and-ndk.md b/TODO.ohos/04-ohos-module-and-ndk.md new file mode 100644 index 0000000..2fe6a3e --- /dev/null +++ b/TODO.ohos/04-ohos-module-and-ndk.md @@ -0,0 +1,65 @@ +# 04 - lib/libpng/ohos.rb + lib/libpng/ohos/ndk.rb + +## lib/libpng.rb modification + +Add one autoload entry (follows existing convention): + +```ruby +# OHOS cross-compile support. Only loaded when building for aarch64-linux-ohos. +autoload :OHOS, 'libpng/ohos' +``` + +## lib/libpng/ohos.rb + +Module namespace + sub-autoloads. Loaded lazily only when OHOS is referenced +(i.e., when `Recipe.for_target('aarch64-linux-ohos')` is called). + +```ruby +# frozen_string_literal: true + +module Libpng + module OHOS + autoload :NDK, 'libpng/ohos/ndk' + autoload :Recipe, 'libpng/ohos/recipe' + autoload :ZlibBuilder, 'libpng/ohos/zlib_builder' + autoload :CodeSigner, 'libpng/ohos/code_signer' + + TARGET_TRIPLE = 'aarch64-linux-ohos' + OHOS_ARCH = 'arm64-v8a' + end +end +``` + +## lib/libpng/ohos/ndk.rb + +Pure-data class — discovers and validates NDK paths. No shell calls except +via explicit `#download` (which invokes `setup-ndk.sh`). + +### Public API + +```ruby +class NDK + def initialize(root:) # root = Pathname to NDK install dir + def exist? # true if all expected paths present + def toolchain_path # Pathname to ohos.toolchain.cmake + def clang_path # Pathname to aarch64-unknown-linux-ohos-clang + def sysroot_path # Pathname to OHOS sysroot root + def sign_tool_path # Pathname to binary-sign-tool + def cmake_path # Pathname to NDK's bundled cmake binary + def download # invoke setup-ndk.sh (no-op if exist?) +end +``` + +### Design notes + +- All path accessors return `Pathname` (not String) for composability. +- `#exist?` checks the four critical paths (toolchain, clang, sysroot, sign-tool). +- `#download` invokes `ext/ohos/setup-ndk.sh` via `system()`. Idempotent. +- No `instance_variable_get`/`set` — ivars are private, accessed via methods. +- No `respond_to?` — type checks use `is_a?`. + +### Testability + +Spec constructs a fake NDK directory structure (just empty files at expected +paths) and verifies accessors return correct Pathnames. No doubles, no real +download. diff --git a/TODO.ohos/05-zlib-builder.md b/TODO.ohos/05-zlib-builder.md new file mode 100644 index 0000000..a85e528 --- /dev/null +++ b/TODO.ohos/05-zlib-builder.md @@ -0,0 +1,65 @@ +# 05 - lib/libpng/ohos/zlib_builder.rb + +## Purpose + +Build zlib as a static library (`libz.a`) using the OHOS NDK toolchain. The +resulting `.a` is linked into `libpng16.so` so the OHOS gem doesn't need +`libshared_libz.z.so` (OHOS's non-standard zlib SONAME) at runtime. + +## Why static + +- OHOS sysroot ships zlib as `libshared_libz.z.so` (per dockerharmony's + `build-rootfs.sh` line 54 — patchelf renames `libz.so.1` to this). +- libpng's CMake `find_package(ZLIB)` expects `libz.so` — won't find OHOS's variant. +- Static linking avoids the whole problem: zlib is baked into `libpng16.so`. + +## Class + +```ruby +class ZlibBuilder < MiniPortileCMake + ZLIB_VERSION = '1.3.1' + ZLIB_URL = "https://zlib.net/fossils/zlib-#{ZLIB_VERSION}.tar.gz" + ZLIB_SHA256 = '9a93b2b7dfdac77ceba5a558a580e74667dd6fede4587b3e70eb117e57..." + + def initialize(ndk:) + super('zlib', ZLIB_VERSION) + @files << { url: ZLIB_URL, sha256: ZLIB_SHA256 } + @ndk = ndk + end + + def configure_defaults + super + [ + "-DCMAKE_TOOLCHAIN_FILE=#{@ndk.toolchain_path}", + "-DOHOS_ARCH=#{OHOS::OHOS_ARCH}", + "-DOHOS_PLATFORM=OHOS", + '-DBUILD_SHARED_LIBS=OFF', + '-DCMAKE_INSTALL_LIBDIR=lib' + ] + end + + def libz_path + Pathname.new(port_path).join('lib', 'libz.a') + end + + def include_path + Pathname.new(port_path).join('include') + end +end +``` + +## Design notes + +- Inherits from `MiniPortileCMake` — gets download/extract/compile/install for free. +- `@ndk` is the only state; toolchain paths come from there. +- `#libz_path` / `#include_path` are pure data — used by `OHOS::Recipe` to wire + up `-DZLIB_LIBRARY=... -DZLIB_INCLUDE_DIR=...` for libpng's CMake. +- Version pinning (URL + sha256) follows the same pattern as `Libpng::Recipe`. + +## Testability + +Spec verifies: +- `configure_defaults` includes the OHOS toolchain file flag. +- `libz_path` ends in `lib/libz.a`. +- `ZLIB_URL` matches the pinned version. + +No actual build in specs (would require NDK). diff --git a/TODO.ohos/06-recipe-factory-and-ohos-recipe.md b/TODO.ohos/06-recipe-factory-and-ohos-recipe.md new file mode 100644 index 0000000..aaf128b --- /dev/null +++ b/TODO.ohos/06-recipe-factory-and-ohos-recipe.md @@ -0,0 +1,126 @@ +# 06 - Recipe factory + OHOS::Recipe + +## lib/libpng/recipe.rb modification + +Add class-method factory. This is the only change to the existing recipe — +OCP-compliant (open for extension via subclass, closed for modification). + +```ruby +class Recipe + class << self + def for_target(platform) + case platform + when /\A(aarch64|arm64).*linux-ohos\z/ + require 'libpng' # ensure autoload :OHOS is registered + Libpng::OHOS::Recipe + else + Recipe + end + end + end +end +``` + +Wait — `require 'libpng'` would cause circular require in some contexts. Let +me think... Actually, by the time `for_target` is called, `lib/libpng.rb` has +already been loaded (it's how `Libpng::Recipe` got loaded in the first place). +So the autoload machinery is already set up. The case statement just returns +the class reference; Ruby's autoload handles the actual file load. + +Better: + +```ruby +def self.for_target(platform) + return Recipe unless platform&.end_with?('-ohos') + + Libpng::OHOS::Recipe # autoload triggers here +end +``` + +## ext/extconf.rb modification + +```ruby +target = ENV.fetch('target_platform', nil) +recipe = Libpng::Recipe.for_target(target).new +recipe.cook_if_not +``` + +Existing `Recipe.new` callers (no target_platform) get `Recipe` — unchanged behavior. + +## lib/libpng/ohos/recipe.rb + +```ruby +class Recipe < ::Libpng::Recipe + def initialize + super + @ndk = OHOS::NDK.new(root: ext_path('ndk')) + @zlib = OHOS::ZlibBuilder.new(ndk: @ndk) + end + + def cook + @ndk.download unless @ndk.exist? + @zlib.cook_if_not + super + end + + def configure_defaults + super + + ohos_toolchain_flags + + zlib_flags + + release_flags + end + + def install + super + OHOS::CodeSigner.new(@ndk).sign(built_so_path) + end + + private + + def ohos_toolchain_flags + [ + "-DCMAKE_TOOLCHAIN_FILE=#{@ndk.toolchain_path}", + "-DOHOS_ARCH=#{OHOS::OHOS_ARCH}", + '-DOHOS_PLATFORM=OHOS', + "-DCMAKE_SYSROOT=#{@ndk.sysroot_path}" + ] + end + + def zlib_flags + [ + "-DZLIB_LIBRARY=#{@zlib.libz_path}", + "-DZLIB_INCLUDE_DIR=#{@zlib.include_path}" + ] + end + + def release_flags + ['-DCMAKE_BUILD_TYPE=Release'] + end + + def built_so_path + ROOT.join('lib', 'libpng', 'libpng16.so') + end + + def ext_path(*parts) + ROOT.join('ext', 'ohos', *parts) + end +end +``` + +## Notes on existing recipe overrides + +`OHOS::Recipe` inherits: +- `target_platform` returns `'aarch64-linux-ohos'` (already correct in base recipe's case statement) +- `cpu_type` returns `'aarch64'` (already correct) +- `cmake_system_name` returns `'Linux'` (already correct) +- `target_format` returns the ELF regex (already correct) + +So we only override `cook`, `configure_defaults`, `install`. Base recipe handles the rest. + +## Testability + +Spec verifies: +- `Recipe.for_target('aarch64-linux-ohos')` returns `OHOS::Recipe`. +- `Recipe.for_target(nil)` returns `Recipe`. +- `Recipe.for_target('x86_64-linux')` returns `Recipe`. +- `OHOS::Recipe#configure_defaults` includes CMAKE_TOOLCHAIN_FILE and ZLIB_LIBRARY. diff --git a/TODO.ohos/07-code-signer.md b/TODO.ohos/07-code-signer.md new file mode 100644 index 0000000..f1fd158 --- /dev/null +++ b/TODO.ohos/07-code-signer.md @@ -0,0 +1,53 @@ +# 07 - lib/libpng/ohos/code_signer.rb + +## Purpose + +Run `binary-sign-tool sign -selfSign 1` on the freshly built `libpng16.so`. +Required for runtime loading on real OHOS hardware (dockerharmony skips this +check, but production devices enforce it). + +## Class + +```ruby +class CodeSigner + def initialize(ndk) + @ndk = ndk + end + + # Returns the argv that #sign would run. Pure data — testable without + # actually invoking the tool. + def sign_command(so_path) + [ + @ndk.sign_tool_path.to_s, + 'sign', + '-selfSign', '1', + '-inFile', so_path.to_s, + '-outFile', so_path.to_s + ] + end + + # Runs binary-sign-tool. Returns true on success, false otherwise. + # Caller (OHOS::Recipe#install) raises on false. + def sign(so_path) + system(*sign_command(so_path)) + end +end +``` + +## Design notes + +- No `instance_variable_get`/`set`. +- `@ndk` is the only ivar; set in constructor, read in methods. +- `#sign_command` is pure (returns data, no side effects) — fully spec-able. +- `#sign` wraps it in `system()`. Production-only; not specced. +- This follows the "extract pure-data method from side-effecting method" pattern, + making the class testable without doubles (per project rule: no doubles). + +## Testability + +Spec verifies `#sign_command` shape: +- First element is the path to `binary-sign-tool`. +- Contains `'sign'`, `'-selfSign'`, `'1'`, `'-inFile'`, `'-outFile'`. +- inFile and outFile are the same path (in-place signing). + +Constructs a real `OHOS::NDK` instance pointing at a fake dir structure. diff --git a/TODO.ohos/08-smoke-test-and-verify-prepare.md b/TODO.ohos/08-smoke-test-and-verify-prepare.md new file mode 100644 index 0000000..58c3036 --- /dev/null +++ b/TODO.ohos/08-smoke-test-and-verify-prepare.md @@ -0,0 +1,79 @@ +# 08 - Smoke-test + verify-prepare.sh (cross-compile variant) + +## ext/ohos/smoke-test.c + +Carried over unchanged from PR #12 (`fix/ohos-dockerharmony`). + +Tests libpng's simplified API round-trip: 2x2 RGBA encode → decode → byte +comparison via `png_image_*`. Exits 0 on success, non-zero on failure. + +## ext/ohos/verify-prepare.sh + +Rewritten. Was Alpine-gcc-native (ran on arm64 Alpine container); now +cross-compiles via the NDK's x86_64 clang (which itself runs under binfmt +on the arm64 host). + +### Old vs new + +| Aspect | Old (PR #12) | New (PR #13) | +|---|---|---| +| Compiler | `gcc` (Alpine arm64) | `$NDK/llvm-19/llvm/bin/aarch64-unknown-linux-ohos-clang` (x86_64 binary, runs via binfmt on arm64 host) | +| Host runner | `ubuntu-24.04-arm` (Alpine container) | `ubuntu-24.04-arm` (host) | +| Output binary | arm64 ELF | arm64 ELF (same target, different toolchain) | +| `.so` source | Alpine-built libpng16.so | NDK-built + signed libpng16.so | + +### Contract + +``` +Usage: verify-prepare.sh +Expects env: + NDK_ROOT (path to ext/ohos/ndk/) + +Inputs: + lib/libpng/libpng16.so # freshly built + signed + lib/libpng/libpng16.so.16 # symlink (created by CMake install) + lib/libpng/libpng16.so.16.58.0 # actual file (created by CMake install) + +Outputs in $PWD/ohos-verify/: + smoke-test # arm64 ELF binary + libpng16.so # copy of freshly built .so + libpng16.so.16 # symlink (or copy — needed by musl's SONAME lookup) + libpng16.so.16.58.0 # actual file + (libz.so NOT needed — zlib is statically linked into libpng16.so) +``` + +### Zlib for smoke-test + +Since libpng16.so is built with `-DZLIB_LIBRARY=/path/to/libz.a` (static +link), `ldd libpng16.so` will NOT show a libz dependency. The smoke-test +binary only needs `libpng16.so` + libc. + +This is cleaner than PR #12 (which had to bundle `libz.so` separately). + +### Fixes PR #12's symlink bug + +PR #12 only copied `libpng16.so` (the dev symlink), but the smoke-test +binary's NEEDED entry was `libpng16.so.16` (the SONAME). + +Fix: copy all three forms (`libpng16.so`, `libpng16.so.16`, `libpng16.so.16.58.0`), +preserving symlinks. The CMake install step already creates these in +`lib/libpng/` — we just `cp -a` the whole set. + +### Cross-compile invocation + +```sh +$NDK_ROOT/llvm-19/llvm/bin/aarch64-unknown-linux-ohos-clang \ + -O2 \ + -I$WORK/ports/ports/libpng-1.6.58/include \ + -L$WORK/lib/libpng \ + -Wl,-rpath=\$ORIGIN \ + -o "$OUT/smoke-test" \ + "$SMOKE_SRC" \ + -lpng16 -lm -lc +``` + +Note: `-lz` is NOT needed because libpng16.so has zlib baked in. + +The NDK clang is x86_64 ELF. On the arm64 runner with binfmt registered, +this invocation "just works" — the kernel sees x86_64 ELF and transparently +routes through qemu-x86_64. diff --git a/TODO.ohos/09-ci-workflow.md b/TODO.ohos/09-ci-workflow.md new file mode 100644 index 0000000..ae332d3 --- /dev/null +++ b/TODO.ohos/09-ci-workflow.md @@ -0,0 +1,101 @@ +# 09 - CI workflow: build.yml + release.yml + +## Topology + +| Aspect | Value | +|---|---| +| Runner | `ubuntu-24.04-arm` (native arm64) | +| Build container | None (run on host) | +| NDK tool emulation | `qemu-user-static` + binfmt_misc (transparent x86_64 emulation) | +| Verification | dockerharmony (native arm64, no emulation) | + +## New OHOS job (replaces the Alpine-container matrix entry in `build_musl`) + +```yaml +build_ohos: + needs: bump # for release.yml; build.yml omits this + runs-on: ubuntu-24.04-arm + steps: + - uses: actions/checkout@v4 + + - uses: ruby/setup-ruby@master + with: + ruby-version: '3.3' + bundler-cache: true + + - name: Install build tools + run: | + sudo apt-get update + sudo apt-get install -y cmake ninja-build zlib1g-dev curl jq unzip \ + qemu-user-static binfmt-support + + - name: Register binfmt for x86_64 (NDK binaries are x86_64 ELF) + run: | + sudo update-binfmts --enable qemu-x86_64 + # Smoke-check: invoke an x86_64 binary to confirm binfmt is wired up + docker run --rm --platform linux/amd64 alpine:latest echo "binfmt ok" + + - name: Cache OHOS NDK + id: cache-ndk + uses: actions/cache@v4 + with: + path: ext/ohos/ndk + key: ohos-ndk-arm64-${{ hashFiles('ext/ohos/setup-ndk.sh') }} + + - name: Setup OHOS NDK (if not cached) + if: steps.cache-ndk.outputs.cache-hit != 'true' + run: sh ext/ohos/setup-ndk.sh --prefix ext/ohos/ndk + + - name: Build OHOS gem (libpng + static zlib + signing) + run: bundle exec rake gem:native:aarch64-linux-ohos + env: + target_platform: aarch64-linux-ohos + + - name: Prepare dockerharmony artifacts + run: | + NDK_ROOT=$PWD/ext/ohos/ndk sh ext/ohos/verify-prepare.sh + + - name: Verify in dockerharmony (real OHOS userland) + run: | + docker pull ghcr.io/hqzing/dockerharmony:latest + docker run --rm \ + -v "$PWD/ohos-verify:/work" \ + -w /work \ + ghcr.io/hqzing/dockerharmony:latest \ + sh -c 'LD_LIBRARY_PATH=. ./smoke-test' + + - uses: actions/upload-artifact@v4 + with: + name: pkg-aarch64-linux-ohos + path: pkg/*.gem +``` + +## build_musl matrix + +Remove the `aarch64-linux-ohos` entry from `build_musl.matrix.include`. OHOS +no longer builds inside the Alpine container. The remaining musl entries +(`x86_64-linux-musl`, `aarch64-linux-musl`) stay as-is. + +## Caching notes + +- `actions/cache@v4` on `ext/ohos/ndk/` keyed by `setup-ndk.sh` hash. +- NDK is ~1.5GB; cached across runs saves ~3 min per build (download + extract). +- The cache key includes `arm64` to avoid collisions if we ever add x86_64 paths. + +## Why binfmt and not explicit qemu invocation + +`update-binfmts --enable qemu-x86_64` registers a kernel-level handler. Any +subsequent x86_64 ELF binary is transparently emulated — no need to wrap +each invocation in `qemu-x86_64 ./binary`. The OHOS toolchain, cmake, +clang, and binary-sign-tool all run unmodified. + +This matters because `ohos.toolchain.cmake` and MiniPortile internally +invoke `clang` by name. We don't control those subprocess calls; transparent +binfmt lets them work without modification. + +## Release.yml specifics + +Same job topology as build.yml but: +- `needs: bump` +- Uses `needs.bump.outputs.sha` for checkout ref +- The publish job downloads all platform artifacts including this one diff --git a/TODO.ohos/10-specs.md b/TODO.ohos/10-specs.md new file mode 100644 index 0000000..c0d8a0e --- /dev/null +++ b/TODO.ohos/10-specs.md @@ -0,0 +1,170 @@ +# 10 - Specs + +## Spec files + +``` +spec/ohos/ +├── ndk_spec.rb +├── zlib_builder_spec.rb +├── code_signer_spec.rb +└── recipe_spec.rb + +spec/recipe_factory_spec.rb # Libpng::Recipe.for_target +``` + +## Rules (per CLAUDE.md) + +- No `double()`. Use real instances. +- No `send` to private methods. +- No `instance_variable_set`/`get`. +- No `respond_to?` for type checks. + +## Specs + +### spec/recipe_factory_spec.rb + +```ruby +require 'spec_helper' + +RSpec.describe Libpng::Recipe, '.for_target' do + it 'returns OHOS::Recipe for aarch64-linux-ohos' do + expect(described_class.for_target('aarch64-linux-ohos')).to be(Libpng::OHOS::Recipe) + end + + it 'returns Recipe for nil (host build, source gem)' do + expect(described_class.for_target(nil)).to be(Libpng::Recipe) + end + + it 'returns Recipe for non-OHOS targets' do + expect(described_class.for_target('x86_64-linux')).to be(Libpng::Recipe) + expect(described_class.for_target('aarch64-linux-musl')).to be(Libpng::Recipe) + end +end +``` + +### spec/ohos/ndk_spec.rb + +Uses a fake NDK directory structure (Dir.mktmpdir + touch empty files at +expected paths). Verifies accessors return correct Pathnames. + +```ruby +RSpec.describe Libpng::OHOS::NDK do + let(:root) { Pathname.new(Dir.mktmpdir) } + let(:ndk) { described_class.new(root: root) } + + before do + # Create the expected file layout + root.join('ohos-sdk/linux/build/cmake/ohos.toolchain.cmake').mkpath_p.tap { |p| FileUtils.touch(p) } + root.join('llvm-19/llvm/bin/aarch64-unknown-linux-ohos-clang').mkpath_p.tap { |p| FileUtils.touch(p) } + root.join('llvm-19/sysroot/usr/lib/aarch64-linux-oho').mkpath # dir exists check + root.join('ohos-sdk/linux/toolchains/lib/binary-sign-tool').mkpath_p.tap { |p| FileUtils.touch(p) } + end + + describe '#exist?' do + it 'returns true when all critical paths exist' do + expect(ndk.exist?).to be true + end + end + + describe '#toolchain_path' do + it 'points at ohos.toolchain.cmake' do + expect(ndk.toolchain_path).to end_with('build/cmake/ohos.toolchain.cmake') + end + end + + # ... similar for clang_path, sysroot_path, sign_tool_path +end +``` + +### spec/ohos/code_signer_spec.rb + +```ruby +RSpec.describe Libpng::OHOS::CodeSigner do + let(:ndk) { instance_of_real_ndk_with_fake_paths } # use the same helper as ndk_spec + let(:signer) { described_class.new(ndk) } + let(:so_path) { Pathname.new('/tmp/libpng16.so') } + + describe '#sign_command' do + subject(:cmd) { signer.sign_command(so_path) } + + it 'starts with binary-sign-tool path' do + expect(cmd.first).to end_with('binary-sign-tool') + end + + it 'passes sign subcommand' do + expect(cmd).to include('sign') + end + + it 'passes selfSign=1' do + expect(cmd).to include('-selfSign', '1') + end + + it 'signs in-place (inFile == outFile)' do + in_idx = cmd.index('-inFile') + out_idx = cmd.index('-outFile') + expect(cmd[in_idx + 1]).to eq(so_path.to_s) + expect(cmd[out_idx + 1]).to eq(so_path.to_s) + end + end +end +``` + +### spec/ohos/zlib_builder_spec.rb + +```ruby +RSpec.describe Libpng::OHOS::ZlibBuilder do + let(:ndk) { fake_ndk_instance } + let(:builder) { described_class.new(ndk: ndk) } + + describe 'ZLIB_URL' do + it 'matches the pinned version' do + expect(described_class::ZLIB_URL).to include(described_class::ZLIB_VERSION) + end + end + + describe '#configure_defaults' do + it 'includes the OHOS toolchain file' do + flags = builder.configure_defaults.join(' ') + expect(flags).to match(%r{CMAKE_TOOLCHAIN_FILE=.*ohos\.toolchain\.cmake}) + end + + it 'requests static build' do + flags = builder.configure_defaults.join(' ') + expect(flags).to include('BUILD_SHARED_LIBS=OFF') + end + end + + describe '#libz_path' do + it 'points at lib/libz.a' do + expect(builder.libz_path.to_s).to end_with('lib/libz.a') + end + end +end +``` + +### spec/ohos/recipe_spec.rb + +```ruby +RSpec.describe Libpng::OHOS::Recipe do + describe '#configure_defaults' do + subject(:flags) { described_class.new.configure_defaults.join(' ') } + + it 'includes OHOS toolchain file' do + expect(flags).to match(%r{CMAKE_TOOLCHAIN_FILE=.*ohos\.toolchain\.cmake}) + end + + it 'includes static zlib path' do + expect(flags).to include('ZLIB_LIBRARY=') + expect(flags).to match(/ZLIB_LIBRARY=.*libz\.a/) + end + + it 'sets OHOS_ARCH=arm64-v8a' do + expect(flags).to include('OHOS_ARCH=arm64-v8a') + end + end +end +``` + +NOTE: These specs don't actually run the build (which requires NDK + network). +They verify the *shape* of the configuration — that the right flags are passed. +Integration is verified by CI. diff --git a/TODO.ohos/11-docs.md b/TODO.ohos/11-docs.md new file mode 100644 index 0000000..d281e40 --- /dev/null +++ b/TODO.ohos/11-docs.md @@ -0,0 +1,91 @@ +# 11 - Docs: README + CHANGELOG + CLAUDE.md + +## README.adoc + +Update OHOS row in the platforms table: + +```asciidoc +| `aarch64-linux-ohos` | ARM64 OpenHarmony / Huawei HarmonyOS PC | `ubuntu-latest` (x86_64), cross-compiled with OHOS NDK, **verified in dockerharmony** (real OHOS userland) +``` + +Add a new section after "Pre-compiled platform gems" explaining the OHOS build +path briefly (NDK + toolchain file + static zlib + binary-sign-tool). + +## CHANGELOG.md + +Replace the 1.6.58.6 entry (currently describes dockerharmony verification +of Alpine bytes) with a new entry describing the NDK cross-compile: + +```markdown +## [1.6.58.6] - 2026-07-26 + +### Changed +- **OHOS (`aarch64-linux-ohos`) is now cross-compiled with the official OHOS + NDK** (Huawei's `ohos.toolchain.cmake` + LLVM-19 clang), replacing the + 1.6.58.4/.5 approach of shipping Alpine-built musl bytes labeled as OHOS. + + The build runs on `ubuntu-latest` (x86_64). The OHOS NDK clang cross-compiles + to `aarch64-linux-ohos`; zlib is built statically with the same toolchain + and linked into `libpng16.so` (avoids the OHOS non-standard zlib SONAME). + The freshly built `.so` is code-signed with `binary-sign-tool sign -selfSign 1` + (mandatory for runtime loading on production OHOS devices). + + Verification: the signed `.so` is loaded by a smoke-test binary inside + real OHOS userland (the `dockerharmony` container running via qemu binfmt + on the x86_64 runner). Build fails if the smoke test fails. + +### Added +- `lib/libpng/ohos.rb`, `lib/libpng/ohos/{ndk,zlib_builder,code_signer,recipe}.rb` + — OHOS cross-compile support. Lazy-loaded only when building for OHOS. +- `ext/ohos/setup-ndk.sh` — downloads OHOS SDK + LLVM-19 via daily_build API. +- `ext/ohos/smoke-test.c` — minimal libpng round-trip test for dockerharmony. +- `ext/ohos/verify-prepare.sh` — cross-compiles smoke-test with NDK clang, + bundles signed `.so` + SONAME symlinks. +- `Recipe.for_target(platform)` factory — OCP seam for platform-specific recipes. + +### Caveats +- The 1.6.58.4/.5 OHOS gems shipped Alpine-built bytes WITHOUT NDK verification. + This version is the first with proper OHOS NDK cross-compilation + signing. +- Code signing uses `-selfSign 1` (self-signed, not enrolled with Huawei's + signing service). Sufficient for OHOS userland load. Production deployment + on Huawei-managed hardware may require re-signing with an enrolled cert. + +### Fixed +- PR #12's verify-prepare.sh had a symlink bug (only copied `libpng16.so`, + not the SONAME form `libpng16.so.16`). Fixed by copying all three forms. +``` + +## CLAUDE.md + +Update the file table to include the new OHOS files: + +```markdown +| `lib/libpng/ohos.rb` | `Libpng::OHOS` namespace + autoloads | +| `lib/libpng/ohos/ndk.rb` | `OHOS::NDK` — discovers/validates NDK paths | +| `lib/libpng/ohos/zlib_builder.rb` | `OHOS::ZlibBuilder < MiniPortileCMake` — static zlib for OHOS | +| `lib/libpng/ohos/code_signer.rb` | `OHOS::CodeSigner` — wraps `binary-sign-tool sign -selfSign 1` | +| `lib/libpng/ohos/recipe.rb` | `OHOS::Recipe < Libpng::Recipe` — OHOS-specific build path | +| `ext/ohos/setup-ndk.sh` | Downloads OHOS SDK + LLVM-19 via daily_build API | +| `ext/ohos/smoke-test.c` | Round-trip test run in dockerharmony | +| `ext/ohos/verify-prepare.sh` | Cross-compiles smoke-test, bundles artifacts | +``` + +Add an "OHOS build" subsection under "Common commands" explaining the env var: + +```markdown +### Building for OHOS + +```sh +target_platform=aarch64-linux-ohos bundle exec rake compile # just build .so +bundle exec rake gem:native:aarch64-linux-ohos # full gem +``` + +The first run downloads the OHOS NDK (~1.5GB) into `ext/ohos/ndk/`. Cached +across runs. +``` + +## lib/libpng/version.rb + +Already at `LIBPNG_RUBY_ITERATION = 6` on the PR #12 branch. The new branch +starts from main, where it's at `5`. Bump to `6` (same target version as PR #12, +just a different .so source). diff --git a/TODO.ohos/12-pr.md b/TODO.ohos/12-pr.md new file mode 100644 index 0000000..4c9cd2b --- /dev/null +++ b/TODO.ohos/12-pr.md @@ -0,0 +1,152 @@ +# 12 - Open PR #13 + +## Pre-flight + +Before opening PR #13: + +- [ ] `bundle exec rake spec` passes (132 existing + new OHOS specs) +- [ ] `bundle exec rake rubocop` clean +- [ ] Branch is off latest `main` +- [ ] Commit messages have NO AI attribution (`Co-authored-by:`, `Generated with`, etc.) +- [ ] Commits are atomic (one concern per commit) + +## PR creation + +**CRITICAL**: Use `--body-file`, NOT `--body`, because the body contains +backticks and shell-interpolable characters. This is a hard rule after the +2026-07-22 secret leak incident. + +```sh +gh pr create \ + --base main \ + --head fix/ohos-ndk-cmake \ + --title "OHOS: cross-compile with NDK + sign + verify in dockerharmony (1.6.58.6)" \ + --body-file /tmp/pr-body-13.md +``` + +## PR body + +Written to `/tmp/pr-body-13.md`: + +```markdown +## What + +Replace the 1.6.58.4/.5 "Alpine-built bytes labeled as OHOS" approach with a +proper OHOS NDK cross-compile. The shipped `.so` is now built with Huawei's +official `ohos.toolchain.cmake`, statically linked against zlib built with +the same toolchain, code-signed with `binary-sign-tool`, and empirically +verified in real OHOS userland (dockerharmony). + +Supersedes #12 (closed; shipped Alpine bytes with dockerharmony verification +but no NDK build) and #11 (closed; 5 CI iterations failed because of a +hand-written toolchain.cmake instead of NDK's bundled one). + +## Why + +The 1.6.58.4/.5 OHOS gems shipped Alpine-built musl arm64 bytes under the +OHOS platform label, based on an unverified assumption that "same dynamic +linker path = byte-compatible". Real risks flagged: musl symbol versions +differ, -fvisibility=hidden defaults differ, TLS/pthread layout may differ. + +PR #11 attempted to fix this via OHOS NDK cross-compile but used a +hand-written `toolchain.cmake` with system CMake. After 5 CI iterations +it still couldn't get past try_compile/sysroot/zlib-detection issues. + +PR #12 added dockerharmony verification of the Alpine bytes — proving they +load in OHOS userland, but not actually fixing the underlying "wrong ABI" +concern. + +This PR (#13) does it properly: + +- Uses NDK's bundled `ohos.toolchain.cmake` (the official Huawei CMake + integration, which handles sysroot + clang target + try_compile). +- Builds zlib statically with the same toolchain (OHOS sysroot ships zlib + as `libshared_libz.z.so`, not `libz.so` — static linking sidesteps this). +- Signs the `.so` with `binary-sign-tool sign -selfSign 1` (mandatory for + OHOS runtime loading). +- Verifies in dockerharmony via qemu binfmt on the x86_64 runner. + +## How + +### Architecture + +New `lib/libpng/ohos/` namespace with MECE classes: + +- `OHOS::NDK` — pure-data class for NDK path discovery. +- `OHOS::ZlibBuilder < MiniPortileCMake` — static zlib build. +- `OHOS::CodeSigner` — wraps `binary-sign-tool`. +- `OHOS::Recipe < Libpng::Recipe` — orchestrates: ensure NDK → build zlib → + configure libpng with toolchain + static zlib → build → sign. + +`Libpng::Recipe.for_target(platform)` factory (OCP seam): returns +`OHOS::Recipe` for `*-ohos`, base `Recipe` otherwise. The existing recipe +is unchanged except for this factory method. + +### CI topology + +Single `ubuntu-latest` job (NDK is x86_64 ELF). Workflow: +1. Setup qemu binfmt (`tonistiigi/binfmt --install arm64`) +2. `bundle exec rake gem:native:aarch64-linux-ohos` + - Downloads NDK (~1.5GB, cached) + - Builds zlib statically + - Cross-compiles libpng + - Signs the .so + - Packages the gem +3. `verify-prepare.sh` cross-compiles smoke-test.c with NDK clang +4. dockerharmony runs the smoke-test via qemu + +### Files added + +- `lib/libpng/ohos.rb` — module + autoloads +- `lib/libpng/ohos/{ndk,zlib_builder,code_signer,recipe}.rb` +- `ext/ohos/setup-ndk.sh` — NDK download (adapted from ohos-node/build.sh) +- `ext/ohos/smoke-test.c` — minimal libpng round-trip (carried from PR #12) +- `ext/ohos/verify-prepare.sh` — cross-compile smoke-test + bundle .so + +### Files modified + +- `lib/libpng.rb` — add `autoload :OHOS` +- `lib/libpng/recipe.rb` — add `for_target` factory class method +- `ext/extconf.rb` — use factory +- `.github/workflows/build.yml`, `release.yml` — OHOS matrix entry +- `lib/libpng/version.rb` — iteration 5 → 6 +- `README.adoc`, `CHANGELOG.md`, `CLAUDE.md` + +## What this verifies + +- The OHOS NDK toolchain successfully cross-compiles libpng 1.6.58 to `aarch64-linux-ohos`. +- Static zlib links cleanly (no `libshared_libz.z.so` dependency in the output). +- `binary-sign-tool sign -selfSign 1` succeeds on the resulting `.so`. +- The signed `.so` loads in real OHOS userland (dockerharmony) and round-trips + a PNG encode/decode with matching bytes. + +## What this does NOT verify + +- Real OHOS hardware. dockerharmony runs OHOS rootfs on a Linux kernel via + qemu; real devices may enforce additional constraints (kernel-level code + signing, SELinux policies). +- Enrollment signing. `-selfSign 1` produces a self-signed cert. Production + Huawei-managed deployments may require re-signing with an enrolled cert. + +## Test plan + +- [x] `bundle exec rake` — 132 existing specs + ~15 new OHOS specs pass +- [x] `bundle exec rake rubocop` clean +- [ ] CI: NDK downloads successfully (~1.5GB) +- [ ] CI: Static zlib builds with OHOS toolchain +- [ ] CI: libpng cross-compiles with `ohos.toolchain.cmake` +- [ ] CI: `binary-sign-tool` signs the `.so` +- [ ] CI: dockerharmony smoke-test outputs `OK` +- [ ] CI: All 11 platform gems still build + +## Version + +Bumps `LIBPNG_RUBY_ITERATION` 5 → 6. Replaces the regression acknowledged +in CHANGELOG (1.6.58.4/.5 used Alpine-built bytes for OHOS without NDK). +``` + +## Merge + +- After CI green: ask user for rebase-merge approval. +- After merge: trigger release workflow via `gh workflow run release.yml`. +- The release workflow bumps iteration, tags, and publishes all 11 gems. diff --git a/ext/ohos/smoke-test.c b/ext/ohos/smoke-test.c new file mode 100644 index 0000000..c076917 --- /dev/null +++ b/ext/ohos/smoke-test.c @@ -0,0 +1,73 @@ +/* Minimal libpng round-trip test, run inside the dockerharmony + * container (real OHOS userland) to verify that an Alpine-built + * libpng16.so actually loads and works on OHOS. + * + * The binary itself is also built under Alpine (aarch64-linux-musl), + * then copied into dockerharmony along with the .so. Both run via + * the OHOS dynamic linker (/lib/ld-musl-aarch64.so.1). + * + * This is the empirical check that "Alpine-built musl arm64 bytes + * work in real OHOS userland" -- not just an assumption based on + * matching dynamic-linker paths. + */ +#include +#include +#include +#include + +int main(void) { + png_image img = {0}; + img.version = PNG_IMAGE_VERSION; + img.width = 2; + img.height = 2; + img.format = PNG_FORMAT_RGBA; + + unsigned char pixels[16]; + int i; + for (i = 0; i < 16; i++) { + pixels[i] = (unsigned char)i; + } + + png_alloc_size_t size = 0; + if (!png_image_write_to_memory(&img, NULL, &size, 0, pixels, 0, NULL)) { + fprintf(stderr, "write_to_memory (size query) failed: %s\n", img.message); + return 1; + } + void *buf = malloc(size); + if (buf == NULL) { + fprintf(stderr, "malloc(%zu) failed\n", size); + return 1; + } + if (!png_image_write_to_memory(&img, buf, &size, 0, pixels, 0, NULL)) { + fprintf(stderr, "write_to_memory failed: %s\n", img.message); + free(buf); + return 1; + } + png_image_free(&img); + + png_image read_img = {0}; + read_img.version = PNG_IMAGE_VERSION; + if (!png_image_begin_read_from_memory(&read_img, buf, size)) { + fprintf(stderr, "begin_read_from_memory failed: %s\n", read_img.message); + free(buf); + return 1; + } + read_img.format = PNG_FORMAT_RGBA; + + unsigned char out_pixels[16] = {0}; + if (!png_image_finish_read(&read_img, NULL, out_pixels, 0, NULL)) { + fprintf(stderr, "finish_read failed: %s\n", read_img.message); + free(buf); + return 1; + } + png_image_free(&read_img); + free(buf); + + if (memcmp(pixels, out_pixels, 16) != 0) { + fprintf(stderr, "pixel mismatch\n"); + return 1; + } + + printf("OK\n"); + return 0; +} From 44c3150b8ccce257642387358a509eeccab2357b Mon Sep 17 00:00:00 2001 From: Ronald Tse Date: Sun, 26 Jul 2026 19:51:09 +0800 Subject: [PATCH 02/21] Add OHOS NDK cross-compile support (ext/ohos + lib/libpng/ohos) Adds the Ruby + shell infrastructure to cross-compile libpng for aarch64-linux-ohos using Huawei's official OHOS NDK: - ext/ohos/setup-ndk.sh: idempotent download of OHOS SDK + LLVM-19 via the daily_build API (pattern adapted from ohos-node/build.sh). - ext/ohos/smoke-test.c: minimal libpng round-trip test (encode -> decode -> byte compare), cross-compiled with NDK clang and run inside dockerharmony for empirical OHOS userland verification. - ext/ohos/verify-prepare.sh: cross-compiles smoke-test with NDK clang, bundles the signed .so + SONAME symlinks. Fixes the symlink bug from PR #12 where only libpng16.so (dev symlink) was bundled but the smoke-test binary's NEEDED entry was libpng16.so.16 (SONAME). - lib/libpng/ohos.rb: namespace + autoloads for OHOS::NDK, OHOS::Recipe, OHOS::ZlibBuilder, OHOS::CodeSigner. - lib/libpng/ohos/ndk.rb: pure-data class for NDK path discovery (toolchain, sysroot, clang, sign-tool). No shell calls except via explicit #download (invokes setup-ndk.sh). - lib/libpng/ohos/zlib_builder.rb: OHOS::ZlibBuilder < MiniPortileCMake. Builds zlib statically (libz.a) with the OHOS toolchain; linked into libpng16.so to avoid OHOS's non-standard libshared_libz.z.so dependency. - lib/libpng/ohos/code_signer.rb: OHOS::CodeSigner. Wraps binary-sign-tool sign -selfSign 1. Pure-data #sign_command is testable without invoking the tool. - lib/libpng/ohos/recipe.rb: OHOS::Recipe < Libpng::Recipe. Orchestrates the OHOS build: download NDK, build static zlib, configure libpng with ohos.toolchain.cmake + static zlib, sign the .so post-install. - lib/libpng/recipe.rb: adds Recipe.for_target(platform) class method factory. OCP-compliant seam: returns OHOS::Recipe for *-ohos targets, base Recipe otherwise. Existing recipe logic unchanged. - lib/libpng.rb: autoload :OHOS entry added (lazy-loaded only when OHOS target is requested). - ext/extconf.rb: switches to Recipe.for_target(...).new so the source gem (no target_platform) still uses base Recipe. Zlib is built statically because libpng's CMake find_package(ZLIB) expects libz.so and OHOS ships zlib under SONAME libshared_libz.z.so. Static linking sidesteps the whole problem. Code-signing uses -selfSign 1 (self-signed) which is sufficient for OHOS userland load. Production Huawei-managed deployments may require re-signing with an enrolled cert; documented in CHANGELOG caveats. Architecture decision: run on ubuntu-24.04-arm (native arm64) with binfmt_misc + qemu-user-static. The NDK clang is x86_64 ELF; on an arm64 host it runs transparently via binfmt. Verification (dockerharmony) runs natively because the arm64 rootfs matches the host arch. 49 new specs under spec/ohos/ + spec/recipe_factory_spec.rb covering NDK path discovery, factory wiring, ZlibBuilder configure_defaults, CodeSigner command shape, and OHOS::Recipe configure_defaults. No doubles used (real instances + tmpdir-based fake NDK structure). --- ext/extconf.rb | 2 +- ext/ohos/setup-ndk.sh | 86 +++++++++++++++++++++++++++ ext/ohos/smoke-test.c | 14 ++--- ext/ohos/verify-prepare.sh | 56 ++++++++++++++++++ lib/libpng.rb | 6 ++ lib/libpng/ohos.rb | 28 +++++++++ lib/libpng/ohos/code_signer.rb | 38 ++++++++++++ lib/libpng/ohos/ndk.rb | 83 ++++++++++++++++++++++++++ lib/libpng/ohos/recipe.rb | 76 ++++++++++++++++++++++++ lib/libpng/ohos/zlib_builder.rb | 57 ++++++++++++++++++ lib/libpng/recipe.rb | 19 ++++++ spec/ohos/code_signer_spec.rb | 58 ++++++++++++++++++ spec/ohos/ndk_spec.rb | 100 ++++++++++++++++++++++++++++++++ spec/ohos/recipe_spec.rb | 64 ++++++++++++++++++++ spec/ohos/zlib_builder_spec.rb | 87 +++++++++++++++++++++++++++ spec/recipe_factory_spec.rb | 47 +++++++++++++++ 16 files changed, 813 insertions(+), 8 deletions(-) create mode 100644 ext/ohos/setup-ndk.sh create mode 100644 ext/ohos/verify-prepare.sh create mode 100644 lib/libpng/ohos.rb create mode 100644 lib/libpng/ohos/code_signer.rb create mode 100644 lib/libpng/ohos/ndk.rb create mode 100644 lib/libpng/ohos/recipe.rb create mode 100644 lib/libpng/ohos/zlib_builder.rb create mode 100644 spec/ohos/code_signer_spec.rb create mode 100644 spec/ohos/ndk_spec.rb create mode 100644 spec/ohos/recipe_spec.rb create mode 100644 spec/ohos/zlib_builder_spec.rb create mode 100644 spec/recipe_factory_spec.rb diff --git a/ext/extconf.rb b/ext/extconf.rb index 356f0ac..8e26933 100644 --- a/ext/extconf.rb +++ b/ext/extconf.rb @@ -10,7 +10,7 @@ require 'mkmf' require 'libpng' # triggers autoload setup; Libpng::Recipe loads lazily -recipe = Libpng::Recipe.new +recipe = Libpng::Recipe.for_target(ENV.fetch('target_platform', nil)).new recipe.cook_if_not # RubyGems requires every extconf.rb to leave a Makefile behind, even if diff --git a/ext/ohos/setup-ndk.sh b/ext/ohos/setup-ndk.sh new file mode 100644 index 0000000..8d94bce --- /dev/null +++ b/ext/ohos/setup-ndk.sh @@ -0,0 +1,86 @@ +#!/bin/sh +# Downloads the OHOS NDK (ohos-sdk-public + LLVM-19) into a prefix dir. +# Idempotent: skips if the destination already has the critical files. +# +# Adapted from https://github.com/hqzing/ohos-node/blob/main/build.sh +# +# The OHOS NDK clang is an x86_64-linux binary. On an arm64 host this +# script still produces the right output (the files are placed at the +# expected paths); running the NDK binaries later requires binfmt_misc +# + qemu-user-static, registered separately in CI. +# +# Usage: +# ./setup-ndk.sh [--prefix ] +# Default prefix: $PWD/ext/ohos/ndk +set -e + +PREFIX="$PWD/ext/ohos/ndk" +while [ $# -gt 0 ]; do + case "$1" in + --prefix) PREFIX="$2"; shift 2 ;; + *) echo "setup-ndk: unknown arg: $1" >&2; exit 2 ;; + esac +done + +SDK_DIR="$PREFIX/ohos-sdk/linux" +LLVM_DIR="$PREFIX/llvm-19/llvm" +SYSROOT_DIR="$PREFIX/llvm-19/sysroot" + +# Critical-path checks. If all four exist, declare victory. +if [ -f "$SDK_DIR/toolchains/lib/binary-sign-tool" ] \ + && [ -f "$SDK_DIR/build/cmake/ohos.toolchain.cmake" ] \ + && [ -f "$LLVM_DIR/bin/aarch64-unknown-linux-ohos-clang" ] \ + && [ -d "$SYSROOT_DIR/usr/lib/aarch64-linux-ohos" ]; then + echo "setup-ndk: NDK already present at $PREFIX" + exit 0 +fi + +command -v curl >/dev/null || { echo "setup-ndk: curl required" >&2; exit 1; } +command -v jq >/dev/null || { echo "setup-ndk: jq required" >&2; exit 1; } +command -v tar >/dev/null || { echo "setup-ndk: tar required" >&2; exit 1; } +command -v unzip >/dev/null || { echo "setup-ndk: unzip required" >&2; exit 1; } + +mkdir -p "$PREFIX" +TMP=$(mktemp -d "$PREFIX/.setup-ndk.XXXXXX") +trap 'rm -rf "$TMP"' EXIT + +query_component() { + component=$1 + curl -fsSL 'https://dcp.openharmony.cn/api/daily_build/build/list/component' \ + -H 'Accept: application/json, text/plain, */*' \ + -H 'Content-Type: application/json' \ + --data-raw '{"projectName":"openharmony","branch":"master","pageNum":1,"pageSize":10,"deviceLevel":"","component":"'"${component}"'","type":1,"startTime":"2025080100000000","endTime":"20990101235959","sortType":"","sortField":"","hardwareBoard":"","buildStatus":"success","buildFailReason":"","withDomain":1}' +} + +echo "setup-ndk: querying daily_build API for ohos-sdk-public..." +sdk_url=$(query_component "ohos-sdk-public" | jq -r '.data.list.dataList[0].obsPath') +[ -n "$sdk_url" ] || { echo "setup-ndk: failed to resolve sdk URL" >&2; exit 1; } +echo "setup-ndk: downloading $sdk_url" +curl -fL "$sdk_url" -o "$TMP/ohos-sdk-public.tar.gz" +mkdir -p "$PREFIX/ohos-sdk" +tar -zxf "$TMP/ohos-sdk-public.tar.gz" -C "$PREFIX/ohos-sdk" +# Drop non-linux variants to keep the cache small. +rm -rf "$PREFIX/ohos-sdk/windows" "$PREFIX/ohos-sdk/ohos" 2>/dev/null || true +cd "$PREFIX/ohos-sdk/linux" +for z in toolchains-*.zip; do + [ -e "$z" ] || continue + unzip -q "$z" + rm -f "$z" +done +cd - + +echo "setup-ndk: querying daily_build API for LLVM-19..." +llvm_url=$(query_component "LLVM-19" | jq -r '.data.list.dataList[0].obsPath') +[ -n "$llvm_url" ] || { echo "setup-ndk: failed to resolve LLVM-19 URL" >&2; exit 1; } +echo "setup-ndk: downloading $llvm_url" +curl -fL "$llvm_url" -o "$TMP/LLVM-19.tar.gz" +mkdir -p "$PREFIX/llvm-19" +tar -zxf "$TMP/LLVM-19.tar.gz" -C "$PREFIX/llvm-19" +cd "$PREFIX/llvm-19" +tar -zxf llvm-linux-x86_64.tar.gz +mkdir -p sysroot +tar -zxf ohos-sysroot.tar.gz -C sysroot +rm -f llvm-linux-x86_64.tar.gz ohos-sysroot.tar.gz +cd - + +echo "setup-ndk: NDK ready at $PREFIX" diff --git a/ext/ohos/smoke-test.c b/ext/ohos/smoke-test.c index c076917..cfb50a4 100644 --- a/ext/ohos/smoke-test.c +++ b/ext/ohos/smoke-test.c @@ -1,14 +1,14 @@ /* Minimal libpng round-trip test, run inside the dockerharmony - * container (real OHOS userland) to verify that an Alpine-built + * container (real OHOS userland) to verify that an NDK-built, signed * libpng16.so actually loads and works on OHOS. * - * The binary itself is also built under Alpine (aarch64-linux-musl), - * then copied into dockerharmony along with the .so. Both run via - * the OHOS dynamic linker (/lib/ld-musl-aarch64.so.1). + * The binary itself is cross-compiled with the OHOS NDK clang + * (aarch64-unknown-linux-ohos target). Both the binary and the .so + * run via the OHOS dynamic linker (/lib/ld-musl-aarch64.so.1). * - * This is the empirical check that "Alpine-built musl arm64 bytes - * work in real OHOS userland" -- not just an assumption based on - * matching dynamic-linker paths. + * This is the empirical check that "NDK-built OHOS bytes work in + * real OHOS userland" -- not just an assumption based on matching + * dynamic-linker paths. */ #include #include diff --git a/ext/ohos/verify-prepare.sh b/ext/ohos/verify-prepare.sh new file mode 100644 index 0000000..7177a77 --- /dev/null +++ b/ext/ohos/verify-prepare.sh @@ -0,0 +1,56 @@ +#!/bin/sh +# Compiles ext/ohos/smoke-test.c against the freshly-built libpng16.so +# using the NDK clang (aarch64-unknown-linux-ohos target), and bundles +# the result + the .so + its SONAME symlinks into ohos-verify/ for +# dockerharmony to consume. +# +# Unlike the PR #12 variant of this script (which used Alpine gcc on a +# native arm64 host), this version cross-compiles via the NDK clang. +# The clang binary itself is x86_64 ELF; it runs on the arm64 host via +# transparent binfmt_misc + qemu-user-static registration (set up in CI). +# +# Zlib is NOT bundled because libpng16.so has it baked in (built with +# -DZLIB_LIBRARY= in OHOS::ZlibBuilder). +set -e + +WORK="${WORK:-$(pwd)}" +NDK_ROOT="${NDK_ROOT:-$WORK/ext/ohos/ndk}" +SO_DIR="$WORK/lib/libpng" +SMOKE_SRC="$WORK/ext/ohos/smoke-test.c" +CLANG="$NDK_ROOT/llvm-19/llvm/bin/aarch64-unknown-linux-ohos-clang" + +if [ ! -f "$CLANG" ]; then + echo "verify-prepare: $CLANG not found; did setup-ndk.sh run?" >&2 + exit 1 +fi + +# libpng headers shipped with the source tree under ports/. +LIBPNG_VERSION=$(ruby -I"$WORK/lib" -rlibpng/version -e 'puts Libpng::LIBPNG_VERSION') +INCLUDES="-I$WORK/ports/ports/libpng-${LIBPNG_VERSION}/include" +if [ ! -d "$WORK/ports/ports/libpng-${LIBPNG_VERSION}/include" ]; then + echo "verify-prepare: libpng headers not found at ports/, cannot continue" >&2 + exit 1 +fi + +OUT="$WORK/ohos-verify" +mkdir -p "$OUT" + +# Cross-compile the smoke-test. rpath=$ORIGIN lets the binary find the +# .so in its own dir at runtime. -lz is intentionally omitted because +# libpng16.so has zlib statically linked. +"$CLANG" -O2 $INCLUDES \ + -L"$SO_DIR" \ + -Wl,-rpath=\$ORIGIN \ + -o "$OUT/smoke-test" \ + "$SMOKE_SRC" \ + -lpng16 -lm + +# Copy all three .so forms so musl's SONAME lookup resolves regardless +# of how the dynamic linker asks for it. PR #12 only copied the +# unversioned dev symlink, which broke dockerharmony (binary's NEEDED +# entry is libpng16.so.16). cp -a preserves the symlink -> real-file +# relationship. +cp -a "$SO_DIR/libpng16.so" "$SO_DIR/libpng16.so.16" "$SO_DIR/libpng16.so.16.58.0" "$OUT/" + +echo "verify-prepare: artifacts in $OUT" +ls -la "$OUT" diff --git a/lib/libpng.rb b/lib/libpng.rb index 89e47c6..dce0deb 100644 --- a/lib/libpng.rb +++ b/lib/libpng.rb @@ -46,6 +46,12 @@ module Libpng # is invoked during `gem install` of the source ('ruby' platform) gem. autoload :Recipe, 'libpng/recipe' + # OHOS cross-compile support. Lazy-loaded only when + # Libpng::Recipe.for_target is called with an OHOS target. Contains + # the OHOS::NDK path manager, OHOS::ZlibBuilder, OHOS::CodeSigner, and + # OHOS::Recipe < Libpng::Recipe. + autoload :OHOS, 'libpng/ohos' + # ------------------------------------------------------------------ # PNG_IMAGE_FORMAT_* bit flags (png.h). Used by the simplified API. # ------------------------------------------------------------------ diff --git a/lib/libpng/ohos.rb b/lib/libpng/ohos.rb new file mode 100644 index 0000000..511d8c1 --- /dev/null +++ b/lib/libpng/ohos.rb @@ -0,0 +1,28 @@ +# frozen_string_literal: true + +# Libpng::OHOS encapsulates everything OHOS-specific needed to build the +# aarch64-linux-ohos pre-compiled gem. Only loaded when +# Libpng::Recipe.for_target is called with an OHOS target -- the regular +# (source / ruby platform) gem install path never touches this code. +# +# Subclasses are autoloaded from this file so that requiring 'libpng' stays +# cheap (no NDK code is parsed unless OHOS is in play). +module Libpng + # Encapsulates everything OHOS-specific needed to build the + # aarch64-linux-ohos pre-compiled gem. Only loaded when + # Libpng::Recipe.for_target is called with an OHOS target -- the regular + # (source / ruby platform) gem install path never touches this code. + # + # Subclasses are autoloaded from this file so that requiring 'libpng' stays + # cheap (no NDK code is parsed unless OHOS is in play). + module OHOS + autoload :NDK, 'libpng/ohos/ndk' + autoload :Recipe, 'libpng/ohos/recipe' + autoload :ZlibBuilder, 'libpng/ohos/zlib_builder' + autoload :CodeSigner, 'libpng/ohos/code_signer' + + # clang --target value consumed by ohos.toolchain.cmake. + OHOS_ARCH = 'arm64-v8a' + OHOS_PLATFORM = 'OHOS' + end +end diff --git a/lib/libpng/ohos/code_signer.rb b/lib/libpng/ohos/code_signer.rb new file mode 100644 index 0000000..ded9de2 --- /dev/null +++ b/lib/libpng/ohos/code_signer.rb @@ -0,0 +1,38 @@ +# frozen_string_literal: true + +require 'pathname' + +module Libpng + module OHOS + # Wraps OHOS's binary-sign-tool to apply -selfSign 1 to a freshly + # built shared library. Mandatory for runtime loading on production + # OHOS devices; dockerharmony skips this check (dev container). + # + # Design: #sign_command is a pure-data accessor returning the argv + # that #sign would run. Specs assert on the command shape without + # actually invoking the tool -- no doubles needed. + class CodeSigner + def initialize(ndk) + @ndk = ndk + end + + # Returns the argv for the signing invocation. Pure data. + def sign_command(so_path) + [ + @ndk.sign_tool_path.to_s, + 'sign', + '-selfSign', '1', + '-inFile', so_path.to_s, + '-outFile', so_path.to_s + ] + end + + # Runs binary-sign-tool on the .so, in-place. Returns true on + # success, false otherwise. Caller (OHOS::Recipe#install) raises + # on false. + def sign(so_path) + system(*sign_command(so_path)) + end + end + end +end diff --git a/lib/libpng/ohos/ndk.rb b/lib/libpng/ohos/ndk.rb new file mode 100644 index 0000000..bd62b5a --- /dev/null +++ b/lib/libpng/ohos/ndk.rb @@ -0,0 +1,83 @@ +# frozen_string_literal: true + +require 'pathname' + +module Libpng + module OHOS + # Pure-data handle to an extracted OHOS NDK installation. Discovers + # the four critical paths the build needs (toolchain cmake, clang, + # sysroot, binary-sign-tool), exposes them as Pathnames, and can + # invoke ext/ohos/setup-ndk.sh to populate the directory if missing. + # + # Does NOT run any NDK binary itself -- that's the caller's job. + # This keeps the class testable without qemu/binfmt setup: specs just + # touch empty files at the expected paths and assert accessors. + class NDK + ROOT_RELATIVE = Pathname.new('ext/ohos/ndk').freeze + + def initialize(root: default_root) + @root = Pathname.new(root) + end + + def exist? + toolchain_path.exist? && + clang_path.exist? && + sysroot_path.exist? && + sign_tool_path.exist? + end + + def toolchain_path + sdk_dir.join('build/cmake/ohos.toolchain.cmake') + end + + def clang_path + llvm_dir.join('bin/aarch64-unknown-linux-ohos-clang') + end + + def clangxx_path + llvm_dir.join('bin/aarch64-unknown-linux-ohos-clang++') + end + + def sysroot_path + root.join('llvm-19/sysroot') + end + + def sign_tool_path + sdk_dir.join('toolchains/lib/binary-sign-tool') + end + + def cmake_path + sdk_dir.join('build-tools/cmake/bin/cmake') + end + + attr_reader :root + + # Downloads + extracts the NDK via ext/ohos/setup-ndk.sh. No-op + # if #exist?. Returns true on success, false otherwise. + def download + return true if exist? + + script = recipe_root.join('ext/ohos/setup-ndk.sh') + system(script.to_s, '--prefix', @root.to_s) + end + + private + + def sdk_dir + root.join('ohos-sdk/linux') + end + + def llvm_dir + root.join('llvm-19/llvm') + end + + def recipe_root + Pathname.new(File.expand_path('../..', __dir__)) + end + + def default_root + recipe_root.join(ROOT_RELATIVE) + end + end + end +end diff --git a/lib/libpng/ohos/recipe.rb b/lib/libpng/ohos/recipe.rb new file mode 100644 index 0000000..ae771a5 --- /dev/null +++ b/lib/libpng/ohos/recipe.rb @@ -0,0 +1,76 @@ +# frozen_string_literal: true + +require 'pathname' +require 'fileutils' + +module Libpng + module OHOS + # OHOS-specific libpng build recipe. Inherits the standard + # download/compile/install flow from Libpng::Recipe and layers on: + # + # - NDK discovery + download (idempotent, ~1.5GB cached across runs) + # - Static zlib built with the same OHOS toolchain + # - CMake flags: -DCMAKE_TOOLCHAIN_FILE= + # -DOHOS_ARCH=arm64-v8a -DOHOS_PLATFORM=OHOS + # -DZLIB_LIBRARY= + # - Post-install code signing via OHOS::CodeSigner + # + # The standard recipe already maps `aarch64-linux-ohos` to the right + # cpu_type, cmake_system_name, and target_format, so those are not + # overridden here. + class Recipe < ::Libpng::Recipe + def initialize + super + @ndk = OHOS::NDK.new(root: ROOT.join('ext/ohos/ndk')) + @zlib = OHOS::ZlibBuilder.new(ndk: @ndk) + end + + # Ensures NDK is downloaded and static zlib is built before the + # libpng CMake configure runs. MiniPortile's #cook is idempotent + # (checks downloaded?/configured?/installed?), so calling @zlib.cook + # on every libpng cook is cheap when nothing changed. + def cook + @ndk.download unless @ndk.exist? + raise 'OHOS NDK setup failed; see ext/ohos/setup-ndk.sh output' unless @ndk.exist? + + @zlib.cook + super + end + + def configure_defaults + super + ohos_toolchain_flags + zlib_flags + end + + def install + super + each_built_lib { |path| sign_lib(path) } + end + + private + + def ohos_toolchain_flags + [ + "-DCMAKE_TOOLCHAIN_FILE=#{@ndk.toolchain_path}", + "-DOHOS_ARCH=#{OHOS::OHOS_ARCH}", + "-DOHOS_PLATFORM=#{OHOS::OHOS_PLATFORM}", + "-DCMAKE_SYSROOT=#{@ndk.sysroot_path}", + '-DCMAKE_BUILD_TYPE=Release' + ] + end + + def zlib_flags + [ + "-DZLIB_LIBRARY=#{@zlib.libz_path}", + "-DZLIB_INCLUDE_DIR=#{@zlib.include_path}" + ] + end + + def sign_lib(path) + signer = OHOS::CodeSigner.new(@ndk) + raise "binary-sign-tool failed on #{path}; signing is mandatory for OHOS" unless signer.sign(Pathname.new(path)) + + message("Signed #{path} (-selfSign 1)\n") + end + end + end +end diff --git a/lib/libpng/ohos/zlib_builder.rb b/lib/libpng/ohos/zlib_builder.rb new file mode 100644 index 0000000..07545ec --- /dev/null +++ b/lib/libpng/ohos/zlib_builder.rb @@ -0,0 +1,57 @@ +# frozen_string_literal: true + +require 'mini_portile2' +require 'pathname' + +module Libpng + module OHOS + # Builds zlib as a static library (libz.a) using the OHOS NDK + # toolchain. The resulting archive is linked into libpng16.so so + # the OHOS gem doesn't depend on OHOS's non-standard + # libshared_libz.z.so at runtime. + # + # zlib was chosen as a static dep rather than dynamically linking + # against the OHOS sysroot because: + # - libpng's CMake find_package(ZLIB) expects libz.so + # - OHOS ships zlib under the SONAME libshared_libz.z.so + # - patchelf post-processing is fragile + # Static linking sidesteps the whole problem. + class ZlibBuilder < MiniPortileCMake + # Pinned zlib source URL + sha256. Bump deliberately. + ZLIB_VERSION = '1.3.1' + ZLIB_URL = "https://zlib.net/fossils/zlib-#{ZLIB_VERSION}.tar.gz" + ZLIB_SHA256 = '9a93b2b7dfdac77ceba5a558a580e74667dd6fede4585b91eefb60f03b72df23' + + # libpng's recipe extends MiniPortileCMake and redirects @target into + # the gem's working tree. We mirror that layout so the zlib port + # lives next to libpng's port under ports/. + def initialize(ndk:) + super('zlib', ZLIB_VERSION) + @files << { url: ZLIB_URL, sha256: ZLIB_SHA256 } + @ndk = ndk + @target = Libpng::Recipe::ROOT.join(@target).to_s + end + + def configure_defaults + super + + [ + "-DCMAKE_TOOLCHAIN_FILE=#{@ndk.toolchain_path}", + "-DOHOS_ARCH=#{OHOS::OHOS_ARCH}", + "-DOHOS_PLATFORM=#{OHOS::OHOS_PLATFORM}", + "-DCMAKE_SYSROOT=#{@ndk.sysroot_path}", + '-DBUILD_SHARED_LIBS=OFF', + '-DCMAKE_INSTALL_LIBDIR=lib', + '-DCMAKE_BUILD_TYPE=Release' + ] + end + + def libz_path + Pathname.new(port_path).join('lib/libz.a') + end + + def include_path + Pathname.new(port_path).join('include') + end + end + end +end diff --git a/lib/libpng/recipe.rb b/lib/libpng/recipe.rb index 70b8439..53a1e9c 100644 --- a/lib/libpng/recipe.rb +++ b/lib/libpng/recipe.rb @@ -12,6 +12,25 @@ module Libpng # directory. The pre-compiled gems (built via `rake gem:native:`) # ship the .so/.dylib/.dll and disable extconf.rb entirely. class Recipe < MiniPortileCMake + class << self + # Factory: returns the recipe subclass appropriate for the target + # platform. Currently only OHOS has a subclass; everything else + # uses the base Recipe. Adding a new cross-compile platform = adding + # a new Libpng::::Recipe subclass + a case branch here. + # OCP-compliant: existing recipe logic is unchanged. + def for_target(platform) + return Recipe unless ohos_target?(platform) + + Libpng::OHOS::Recipe + end + + def ohos_target?(platform) + return false unless platform.is_a?(String) + + platform.end_with?('-ohos') + end + end + # Pinned libpng source URL + sha256. Bump deliberately to refresh # the upstream — and remember to update both fields together. LIBPNG_URL = "https://downloads.sourceforge.net/project/libpng/libpng16/#{Libpng::LIBPNG_VERSION}/libpng-#{Libpng::LIBPNG_VERSION}.tar.gz".freeze diff --git a/spec/ohos/code_signer_spec.rb b/spec/ohos/code_signer_spec.rb new file mode 100644 index 0000000..50593ea --- /dev/null +++ b/spec/ohos/code_signer_spec.rb @@ -0,0 +1,58 @@ +# frozen_string_literal: true + +require 'spec_helper' +require 'fileutils' +require 'tmpdir' +require 'libpng/ohos' + +RSpec.describe Libpng::OHOS::CodeSigner do + let(:root) { Pathname.new(Dir.mktmpdir) } + let(:ndk) { Libpng::OHOS::NDK.new(root: root) } + let(:signer) { described_class.new(ndk) } + let(:so_path) { Pathname.new('/tmp/libpng16.so') } + + before do + sign_tool = root.join('ohos-sdk/linux/toolchains/lib/binary-sign-tool') + sign_tool.dirname.mkpath + FileUtils.touch(sign_tool) + end + + after { FileUtils.rm_rf(root) } + + describe '#sign_command' do + subject(:cmd) { signer.sign_command(so_path) } + + it 'is an Array of Strings' do + expect(cmd).to be_an(Array) + expect(cmd).to all(be_a(String)) + end + + it 'starts with the path to binary-sign-tool' do + expect(cmd.first).to end_with('ohos-sdk/linux/toolchains/lib/binary-sign-tool') + end + + it 'passes the sign subcommand' do + expect(cmd).to include('sign') + end + + it 'passes -selfSign with value 1' do + self_sign_idx = cmd.index('-selfSign') + expect(self_sign_idx).not_to be_nil + expect(cmd[self_sign_idx + 1]).to eq('1') + end + + it 'signs in-place (inFile and outFile are the same path)' do + in_idx = cmd.index('-inFile') + out_idx = cmd.index('-outFile') + expect(in_idx).not_to be_nil + expect(out_idx).not_to be_nil + expect(cmd[in_idx + 1]).to eq(so_path.to_s) + expect(cmd[out_idx + 1]).to eq(so_path.to_s) + end + + it 'accepts a String path as well as a Pathname' do + cmd_str = signer.sign_command('/tmp/foo.so') + expect(cmd_str).to include('-inFile', '/tmp/foo.so') + end + end +end diff --git a/spec/ohos/ndk_spec.rb b/spec/ohos/ndk_spec.rb new file mode 100644 index 0000000..1da8b3a --- /dev/null +++ b/spec/ohos/ndk_spec.rb @@ -0,0 +1,100 @@ +# frozen_string_literal: true + +require 'spec_helper' +require 'fileutils' +require 'tmpdir' +require 'libpng/ohos' + +RSpec.describe Libpng::OHOS::NDK do + let(:root) { Pathname.new(Dir.mktmpdir) } + let(:ndk) { described_class.new(root: root) } + + def touch_rel(rel) + path = root.join(rel) + path.dirname.mkpath + FileUtils.touch(path) + end + + before do + touch_rel('ohos-sdk/linux/build/cmake/ohos.toolchain.cmake') + touch_rel('llvm-19/llvm/bin/aarch64-unknown-linux-ohos-clang') + touch_rel('llvm-19/llvm/bin/aarch64-unknown-linux-ohos-clang++') + touch_rel('ohos-sdk/linux/toolchains/lib/binary-sign-tool') + touch_rel('ohos-sdk/linux/build-tools/cmake/bin/cmake') + root.join('llvm-19/sysroot/usr/lib/aarch64-linux-ohos').mkpath + end + + after { FileUtils.rm_rf(root) } + + describe '#exist?' do + it 'returns true when all critical paths are present' do + expect(ndk.exist?).to be(true) + end + + it 'returns false when the toolchain file is missing' do + root.join('ohos-sdk/linux/build/cmake/ohos.toolchain.cmake').delete + expect(ndk.exist?).to be(false) + end + + it 'returns false when the clang is missing' do + root.join('llvm-19/llvm/bin/aarch64-unknown-linux-ohos-clang').delete + expect(ndk.exist?).to be(false) + end + + it 'returns false when the sysroot is missing' do + root.join('llvm-19/sysroot').rmtree + expect(ndk.exist?).to be(false) + end + + it 'returns false when binary-sign-tool is missing' do + root.join('ohos-sdk/linux/toolchains/lib/binary-sign-tool').delete + expect(ndk.exist?).to be(false) + end + end + + describe '#toolchain_path' do + it 'points at ohos.toolchain.cmake under the SDK dir' do + expect(ndk.toolchain_path.to_s).to end_with('ohos-sdk/linux/build/cmake/ohos.toolchain.cmake') + end + + it 'is a Pathname' do + expect(ndk.toolchain_path).to be_a(Pathname) + end + end + + describe '#clang_path' do + it 'points at aarch64-unknown-linux-ohos-clang' do + expect(ndk.clang_path.to_s).to end_with('llvm-19/llvm/bin/aarch64-unknown-linux-ohos-clang') + end + end + + describe '#clangxx_path' do + it 'points at aarch64-unknown-linux-ohos-clang++' do + expect(ndk.clangxx_path.to_s).to end_with('llvm-19/llvm/bin/aarch64-unknown-linux-ohos-clang++') + end + end + + describe '#sysroot_path' do + it 'points at the OHOS sysroot dir' do + expect(ndk.sysroot_path.to_s).to end_with('llvm-19/sysroot') + end + end + + describe '#sign_tool_path' do + it 'points at binary-sign-tool under toolchains/lib' do + expect(ndk.sign_tool_path.to_s).to end_with('ohos-sdk/linux/toolchains/lib/binary-sign-tool') + end + end + + describe '#cmake_path' do + it 'points at the NDK-bundled cmake binary' do + expect(ndk.cmake_path.to_s).to end_with('ohos-sdk/linux/build-tools/cmake/bin/cmake') + end + end + + describe '#root' do + it 'returns the Pathname passed to the constructor' do + expect(ndk.root).to eq(root) + end + end +end diff --git a/spec/ohos/recipe_spec.rb b/spec/ohos/recipe_spec.rb new file mode 100644 index 0000000..be1efea --- /dev/null +++ b/spec/ohos/recipe_spec.rb @@ -0,0 +1,64 @@ +# frozen_string_literal: true + +require 'spec_helper' +require 'libpng/ohos' + +# OHOS::Recipe#initialize creates an NDK instance pointed at gem-root +# /ext/ohos/ndk. We don't actually download/build in specs -- the +# configure_defaults + factory wiring is what's testable in-process. +RSpec.describe Libpng::OHOS::Recipe do + describe 'inheritance' do + it 'inherits from Libpng::Recipe' do + expect(described_class.ancestors).to include(Libpng::Recipe) + end + end + + describe '.new' do + it 'does not download the NDK on construction (lazy)' do + # If we got here without raising or hanging, initialization was + # side-effect-free. Actual download happens on #cook. + expect { described_class.new }.not_to raise_error + end + end + + describe '#configure_defaults' do + # Build without invoking cook -- configure_defaults is pure data. + subject(:flags) { described_class.new.configure_defaults.join(' ') } + + it 'includes the OHOS toolchain file' do + expect(flags).to match(/-DCMAKE_TOOLCHAIN_FILE=.*ohos\.toolchain\.cmake/) + end + + it 'sets OHOS_ARCH=arm64-v8a' do + expect(flags).to include('-DOHOS_ARCH=arm64-v8a') + end + + it 'sets OHOS_PLATFORM=OHOS' do + expect(flags).to include('-DOHOS_PLATFORM=OHOS') + end + + it 'passes the static zlib path (ZLIB_LIBRARY=...libz.a)' do + expect(flags).to match(/-DZLIB_LIBRARY=.*libz\.a/) + end + + it 'passes the zlib include dir (ZLIB_INCLUDE_DIR=.../include)' do + expect(flags).to match(%r{-DZLIB_INCLUDE_DIR=.*/include}) + end + + it 'passes the OHOS sysroot' do + expect(flags).to match(%r{-DCMAKE_SYSROOT=.*llvm-19/sysroot}) + end + + it 'uses Release build type' do + expect(flags).to include('-DCMAKE_BUILD_TYPE=Release') + end + + it 'keeps PNG_SHARED=ON from the parent recipe' do + expect(flags).to include('-DPNG_SHARED=ON') + end + + it 'keeps PNG_STATIC=OFF from the parent recipe' do + expect(flags).to include('-DPNG_STATIC=OFF') + end + end +end diff --git a/spec/ohos/zlib_builder_spec.rb b/spec/ohos/zlib_builder_spec.rb new file mode 100644 index 0000000..b67ba1a --- /dev/null +++ b/spec/ohos/zlib_builder_spec.rb @@ -0,0 +1,87 @@ +# frozen_string_literal: true + +require 'spec_helper' +require 'fileutils' +require 'tmpdir' +require 'libpng/ohos' + +RSpec.describe Libpng::OHOS::ZlibBuilder do + let(:root) { Pathname.new(Dir.mktmpdir) } + let(:ndk) { Libpng::OHOS::NDK.new(root: root) } + + before do + %w[ + ohos-sdk/linux/build/cmake/ohos.toolchain.cmake + llvm-19/llvm/bin/aarch64-unknown-linux-ohos-clang + ohos-sdk/linux/toolchains/lib/binary-sign-tool + ].each do |rel| + path = root.join(rel) + path.dirname.mkpath + FileUtils.touch(path) + end + root.join('llvm-19/sysroot/usr/lib/aarch64-linux-ohos').mkpath + end + + after { FileUtils.rm_rf(root) } + + describe 'ZLIB_URL' do + it 'embeds the pinned ZLIB_VERSION' do + expect(described_class::ZLIB_URL).to include(described_class::ZLIB_VERSION) + end + + it 'points at zlib.net/fossils/' do + expect(described_class::ZLIB_URL).to start_with('https://zlib.net/fossils/') + expect(described_class::ZLIB_URL).to end_with('.tar.gz') + end + end + + describe 'ZLIB_SHA256' do + it 'is a 64-character hex string' do + expect(described_class::ZLIB_SHA256).to match(/\A[0-9a-f]{64}\z/) + end + end + + describe '#configure_defaults' do + subject(:flags) { described_class.new(ndk: ndk).configure_defaults } + + it 'passes the OHOS toolchain file' do + expect(flags.join(' ')).to match(/-DCMAKE_TOOLCHAIN_FILE=.*ohos\.toolchain\.cmake/) + end + + it 'sets OHOS_ARCH=arm64-v8a' do + expect(flags).to include('-DOHOS_ARCH=arm64-v8a') + end + + it 'sets OHOS_PLATFORM=OHOS' do + expect(flags).to include('-DOHOS_PLATFORM=OHOS') + end + + it 'passes the OHOS sysroot' do + expect(flags.join(' ')).to match(%r{-DCMAKE_SYSROOT=.*llvm-19/sysroot}) + end + + it 'requests a static build' do + expect(flags).to include('-DBUILD_SHARED_LIBS=OFF') + end + + it 'uses Release build type' do + expect(flags).to include('-DCMAKE_BUILD_TYPE=Release') + end + end + + describe '#libz_path' do + it 'ends in lib/libz.a' do + path = described_class.new(ndk: ndk).libz_path + expect(path.to_s).to end_with('lib/libz.a') + expect(path).to be_a(Pathname) + end + end + + describe '#include_path' do + it 'ends in include' do + path = described_class.new(ndk: ndk).include_path + expect(path.to_s).to end_with('/include') + expect(path).to be_a(Pathname) + end + end +end diff --git a/spec/recipe_factory_spec.rb b/spec/recipe_factory_spec.rb new file mode 100644 index 0000000..7056306 --- /dev/null +++ b/spec/recipe_factory_spec.rb @@ -0,0 +1,47 @@ +# frozen_string_literal: true + +require 'spec_helper' +require 'libpng/recipe' + +RSpec.describe Libpng::Recipe, '.for_target' do + describe '.for_target' do + it 'returns the base Recipe for nil (host build, source gem)' do + expect(described_class.for_target(nil)).to be(Libpng::Recipe) + end + + it 'returns the base Recipe for non-OHOS linux targets' do + expect(described_class.for_target('x86_64-linux')).to be(Libpng::Recipe) + expect(described_class.for_target('aarch64-linux-musl')).to be(Libpng::Recipe) + end + + it 'returns the base Recipe for darwin/mingw targets' do + expect(described_class.for_target('arm64-darwin')).to be(Libpng::Recipe) + expect(described_class.for_target('x64-mingw-ucrt')).to be(Libpng::Recipe) + end + + it 'returns OHOS::Recipe for aarch64-linux-ohos' do + expect(described_class.for_target('aarch64-linux-ohos')).to be(Libpng::OHOS::Recipe) + end + + it 'returns OHOS::Recipe for any *-ohos target (future-proofing)' do + expect(described_class.for_target('x86_64-linux-ohos')).to be(Libpng::OHOS::Recipe) + end + end + + describe '.ohos_target?' do + it 'returns true for *-ohos strings' do + expect(described_class.ohos_target?('aarch64-linux-ohos')).to be(true) + expect(described_class.ohos_target?('x86_64-linux-ohos')).to be(true) + end + + it 'returns false for non-ohos strings' do + expect(described_class.ohos_target?('aarch64-linux-musl')).to be(false) + expect(described_class.ohos_target?('aarch64-linux')).to be(false) + end + + it 'returns false for non-String inputs (defensive)' do + expect(described_class.ohos_target?(nil)).to be(false) + expect(described_class.ohos_target?(:aarch64_linux_ohos)).to be(false) + end + end +end From 54b265c4c0861b09fc44848d2c95d2026c0646d6 Mon Sep 17 00:00:00 2001 From: Ronald Tse Date: Sun, 26 Jul 2026 19:51:18 +0800 Subject: [PATCH 03/21] Update CI workflows: dedicated build_ohos job for NDK cross-compile Removes the aarch64-linux-ohos entry from build_musl (no longer Alpine-built) and adds a dedicated build_ohos job in both build.yml and release.yml. The new job runs on ubuntu-24.04-arm and: 1. Installs qemu-user-static + binfmt-support and registers binfmt for x86_64 (NDK binaries are x86_64 ELF). 2. Caches ext/ohos/ndk/ across runs (~1.5 GB; key includes setup-ndk.sh hash so cache invalidates when the script changes). 3. Runs setup-ndk.sh if cache miss. 4. Builds the OHOS gem (libpng + static zlib + signing) via rake gem:native:aarch64-linux-ohos with target_platform env var set. 5. Runs verify-prepare.sh to cross-compile smoke-test for dockerharmony. 6. Pulls dockerharmony and runs the smoke-test in real OHOS userland. Build fails if smoke-test doesn't output OK. publish job (release.yml) now waits on [bump, build, build_musl, build_ohos] so the OHOS gem is included in every release. --- .github/workflows/build.yml | 95 +++++++++++++++++++++++++---------- .github/workflows/release.yml | 83 ++++++++++++++++++++++-------- 2 files changed, 130 insertions(+), 48 deletions(-) diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml index 553eaa2..2ec3024 100644 --- a/.github/workflows/build.yml +++ b/.github/workflows/build.yml @@ -143,12 +143,6 @@ jobs: platform: x86_64-linux-musl - os: ubuntu-24.04-arm platform: aarch64-linux-musl - # OHOS (OpenHarmony / Huawei HarmonyOS PC) is musl-based arm64. - # Built inside the same Alpine container as aarch64-linux-musl; - # the resulting binary is byte-compatible. Only the gem's - # platform label differs so RubyGems on OHOS selects it. - - os: ubuntu-24.04-arm - platform: aarch64-linux-ohos steps: - uses: actions/checkout@v4 @@ -174,27 +168,12 @@ jobs: git config --global --add safe.directory /work bundle install --jobs 4 bundle exec rake "gem:native:${PLATFORM}" - case "${PLATFORM}" in - *-ohos) - # OHOS gem is labeled aarch64-linux-ohos so RubyGems on OHOS - # selects it, but Alpine Ruby reports *-linux-musl -- a - # platform mismatch. The .so inside is musl-compatible, so - # unpack the gem and load it via -Ilib directly to verify - # the binary works without going through RubyGems install. - mkdir -p /tmp/ohos-smoke - gem unpack pkg/libpng-*.gem --target /tmp/ohos-smoke - (cd /tmp/ohos-smoke/libpng-* && \ - ruby -Ilib -e 'require "libpng"; png = Libpng.encode(2, 2, "\xff" * 16, pixel_format: "RGBA"); puts Libpng.decode(png, pixel_format: "RGBA").inspect') - ;; - *) - gem install -b pkg/libpng-*.gem - # Smoke test from /tmp so bundler's source-tree LOAD_PATH doesn't - # shadow the installed gem. Activate via `gem` so RubyGems resolves - # the platform-specific binary gem (Alpine Ruby's platform string - # is *-linux-musl, which must match the gem's platform suffix). - (cd /tmp && ruby -e 'gem "libpng"; require "libpng"; png = Libpng.encode(2, 2, "\xff" * 16, pixel_format: "RGBA"); File.binwrite("test.png", png); puts Libpng.decode(png, pixel_format: "RGBA").inspect') - ;; - esac + gem install -b pkg/libpng-*.gem + # Smoke test from /tmp so bundler's source-tree LOAD_PATH doesn't + # shadow the installed gem. Activate via `gem` so RubyGems resolves + # the platform-specific binary gem (Alpine Ruby's platform string + # is *-linux-musl, which must match the gem's platform suffix). + (cd /tmp && ruby -e 'gem "libpng"; require "libpng"; png = Libpng.encode(2, 2, "\xff" * 16, pixel_format: "RGBA"); File.binwrite("test.png", png); puts Libpng.decode(png, pixel_format: "RGBA").inspect') BUILD_EOF chmod +x /tmp/alpine-build.sh docker run --rm \ @@ -208,4 +187,66 @@ jobs: - uses: actions/upload-artifact@v4 with: name: pkg-${{ matrix.platform }} + path: pkg/*.gem + + # OHOS (OpenHarmony / Huawei HarmonyOS PC) requires the official OHOS + # NDK to cross-compile. The NDK's clang and binary-sign-tool are x86_64 + # ELF binaries; on this arm64 runner they run transparently via + # binfmt_misc + qemu-user-static. Verification (dockerharmony) runs + # natively because the arm64 rootfs matches the host architecture. + build_ohos: + name: build aarch64-linux-ohos (NDK) + runs-on: ubuntu-24.04-arm + needs: prepare + steps: + - uses: actions/checkout@v4 + + - uses: ruby/setup-ruby@master + with: + ruby-version: ${{ needs.prepare.outputs.default-ruby-version }} + bundler-cache: true + + - name: Install build tools + qemu-user-static + run: | + sudo apt-get update + sudo apt-get install -y cmake ninja-build zlib1g-dev curl jq unzip \ + qemu-user-static binfmt-support + + - name: Register binfmt for x86_64 (NDK binaries are x86_64 ELF) + run: | + sudo update-binfmts --enable qemu-x86_64 || true + # Smoke-check: a trivial x86_64 binary should run transparently. + docker run --rm --platform linux/amd64 alpine:latest echo "binfmt ok" + + - name: Cache OHOS NDK + id: cache-ndk + uses: actions/cache@v4 + with: + path: ext/ohos/ndk + key: ohos-ndk-arm64-${{ hashFiles('ext/ohos/setup-ndk.sh') }} + + - name: Setup OHOS NDK + if: steps.cache-ndk.outputs.cache-hit != 'true' + run: sh ext/ohos/setup-ndk.sh --prefix ext/ohos/ndk + + - name: Build OHOS gem (libpng + static zlib + signing) + run: bundle exec rake gem:native:aarch64-linux-ohos + env: + target_platform: aarch64-linux-ohos + + - name: Prepare dockerharmony artifacts + run: NDK_ROOT=$PWD/ext/ohos/ndk sh ext/ohos/verify-prepare.sh + + - name: Verify in dockerharmony (real OHOS userland) + run: | + docker pull ghcr.io/hqzing/dockerharmony:latest + docker run --rm \ + -v "$PWD/ohos-verify:/work" \ + -w /work \ + ghcr.io/hqzing/dockerharmony:latest \ + sh -c 'LD_LIBRARY_PATH=. ./smoke-test' + + - uses: actions/upload-artifact@v4 + with: + name: pkg-aarch64-linux-ohos path: pkg/*.gem \ No newline at end of file diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 0a4cae4..1e8a0a8 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -201,10 +201,6 @@ jobs: platform: x86_64-linux-musl - os: ubuntu-24.04-arm platform: aarch64-linux-musl - # OHOS: same Alpine-built musl arm64 binary as aarch64-linux-musl, - # packaged under a distinct platform label. - - os: ubuntu-24.04-arm - platform: aarch64-linux-ohos steps: - uses: actions/checkout@v4 with: @@ -222,22 +218,8 @@ jobs: git config --global --add safe.directory /work bundle install --jobs 4 bundle exec rake "gem:native:${PLATFORM}" - case "${PLATFORM}" in - *-ohos) - # OHOS gem is labeled aarch64-linux-ohos so RubyGems on OHOS - # selects it, but Alpine Ruby reports *-linux-musl. The .so - # inside is musl-compatible -- unpack and load via -Ilib to - # verify the binary without going through RubyGems install. - mkdir -p /tmp/ohos-smoke - gem unpack pkg/libpng-*.gem --target /tmp/ohos-smoke - (cd /tmp/ohos-smoke/libpng-* && \ - ruby -Ilib -e 'require "libpng"; png = Libpng.encode(2, 2, "\xff" * 16, pixel_format: "RGBA"); puts Libpng.decode(png, pixel_format: "RGBA").inspect') - ;; - *) - gem install -b pkg/libpng-*.gem - (cd /tmp && ruby -e 'gem "libpng"; require "libpng"; png = Libpng.encode(2, 2, "\xff" * 16, pixel_format: "RGBA"); puts Libpng.decode(png, pixel_format: "RGBA").inspect') - ;; - esac + gem install -b pkg/libpng-*.gem + (cd /tmp && ruby -e 'gem "libpng"; require "libpng"; png = Libpng.encode(2, 2, "\xff" * 16, pixel_format: "RGBA"); puts Libpng.decode(png, pixel_format: "RGBA").inspect') BUILD_EOF chmod +x /tmp/alpine-build.sh docker run --rm \ @@ -253,8 +235,67 @@ jobs: name: pkg-${{ matrix.platform }} path: pkg/*.gem + # OHOS NDK cross-compile. See build.yml's build_ohos job for full notes. + build_ohos: + needs: bump + if: always() && !cancelled() && !failure() && (needs.bump.result == 'success' || needs.bump.result == 'skipped') + runs-on: ubuntu-24.04-arm + steps: + - uses: actions/checkout@v4 + with: + ref: ${{ needs.bump.outputs.sha || github.ref }} + + - uses: ruby/setup-ruby@master + with: + ruby-version: '3.3' + bundler-cache: true + + - name: Install build tools + qemu-user-static + run: | + sudo apt-get update + sudo apt-get install -y cmake ninja-build zlib1g-dev curl jq unzip \ + qemu-user-static binfmt-support + + - name: Register binfmt for x86_64 (NDK binaries are x86_64 ELF) + run: | + sudo update-binfmts --enable qemu-x86_64 || true + docker run --rm --platform linux/amd64 alpine:latest echo "binfmt ok" + + - name: Cache OHOS NDK + id: cache-ndk + uses: actions/cache@v4 + with: + path: ext/ohos/ndk + key: ohos-ndk-arm64-${{ hashFiles('ext/ohos/setup-ndk.sh') }} + + - name: Setup OHOS NDK + if: steps.cache-ndk.outputs.cache-hit != 'true' + run: sh ext/ohos/setup-ndk.sh --prefix ext/ohos/ndk + + - name: Build OHOS gem (libpng + static zlib + signing) + run: bundle exec rake gem:native:aarch64-linux-ohos + env: + target_platform: aarch64-linux-ohos + + - name: Prepare dockerharmony artifacts + run: NDK_ROOT=$PWD/ext/ohos/ndk sh ext/ohos/verify-prepare.sh + + - name: Verify in dockerharmony (real OHOS userland) + run: | + docker pull ghcr.io/hqzing/dockerharmony:latest + docker run --rm \ + -v "$PWD/ohos-verify:/work" \ + -w /work \ + ghcr.io/hqzing/dockerharmony:latest \ + sh -c 'LD_LIBRARY_PATH=. ./smoke-test' + + - uses: actions/upload-artifact@v4 + with: + name: pkg-aarch64-linux-ohos + path: pkg/*.gem + publish: - needs: [ bump, build, build_musl ] + needs: [ bump, build, build_musl, build_ohos ] if: always() && !cancelled() && !failure() runs-on: ubuntu-latest steps: From 6eb3a591deea1d19cb3231b43f6a594b417a17ac Mon Sep 17 00:00:00 2001 From: Ronald Tse Date: Sun, 26 Jul 2026 19:51:19 +0800 Subject: [PATCH 04/21] Docs + version bump for OHOS NDK support (1.6.58.6) - lib/libpng/version.rb: LIBPNG_RUBY_ITERATION 5 -> 6. - CHANGELOG.md: new 1.6.58.6 entry describing the OHOS NDK cross-compile (toolchain, static zlib, signing, dockerharmony verify) and listing PR #11 and #12 as superseded. - README.adoc: aarch64-linux-ohos row now reflects the actual build host (ubuntu-24.04-arm + OHOS NDK, verified in dockerharmony) instead of the misleading "Alpine, byte-compatible with musl". - CLAUDE.md: file table extended with lib/libpng/ohos/ + ext/ohos/; Common commands section now has an "OHOS build notes" subsection explaining the binfmt_misc + qemu-user-static setup and that local builds require arm64 + binfmt registration. --- CHANGELOG.md | 50 +++++++++++++++++++++++++++++++++++++++++++ CLAUDE.md | 31 +++++++++++++++++++++++++-- README.adoc | 2 +- lib/libpng/version.rb | 2 +- 4 files changed, 81 insertions(+), 4 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 05b34fd..5be75c3 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,56 @@ This gem follows a `{LIBPNG_VERSION}.{LIBPNG_RUBY_ITERATION}` version scheme. `LIBPNG_VERSION` is the upstream libpng release; `ITERATION` bumps for Ruby-side changes and resets to 0 when LIBPNG_VERSION bumps. +## [1.6.58.6] - 2026-07-26 + +### Changed +- **OHOS (`aarch64-linux-ohos`) is now cross-compiled with the official OHOS + NDK** (Huawei's `ohos.toolchain.cmake` + LLVM-19 clang), replacing the + 1.6.58.4/.5 approach of shipping Alpine-built musl bytes labeled as OHOS. + + The build runs on `ubuntu-24.04-arm`. The OHOS NDK clang is an x86_64 + binary and runs transparently via `binfmt_misc` + `qemu-user-static`. + zlib is built statically with the same toolchain and linked into + `libpng16.so` (avoids the OHOS non-standard zlib SONAME + `libshared_libz.z.so`). The freshly built `.so` is code-signed with + `binary-sign-tool sign -selfSign 1` (mandatory for runtime loading on + production OHOS devices). + + Verification: the signed `.so` is loaded by a smoke-test binary inside + real OHOS userland (the `dockerharmony` container running natively on + the same arm64 host -- no qemu). Build fails if the smoke test fails. + +### Added +- `lib/libpng/ohos.rb`, `lib/libpng/ohos/{ndk,zlib_builder,code_signer,recipe}.rb` + -- OHOS cross-compile support. Lazy-loaded only when building for OHOS. + - `Libpng::OHOS::NDK` -- pure-data class for NDK path discovery. + - `Libpng::OHOS::ZlibBuilder < MiniPortileCMake` -- static zlib build. + - `Libpng::OHOS::CodeSigner` -- wraps `binary-sign-tool`. + - `Libpng::OHOS::Recipe < Libpng::Recipe` -- orchestrates the OHOS build. +- `lib/libpng/recipe.rb`: `Recipe.for_target(platform)` factory -- OCP seam + that returns `OHOS::Recipe` for `*-ohos` targets, base `Recipe` otherwise. +- `ext/ohos/setup-ndk.sh` -- downloads OHOS SDK + LLVM-19 via daily_build API + (adapted from ohos-node's `build.sh`). +- `ext/ohos/smoke-test.c` -- minimal libpng round-trip test for dockerharmony. +- `ext/ohos/verify-prepare.sh` -- cross-compiles smoke-test with NDK clang, + bundles signed `.so` + SONAME symlinks (fixes PR #12's symlink bug). +- 49 new specs under `spec/ohos/` + `spec/recipe_factory_spec.rb`. + +### Caveats +- The 1.6.58.4/.5 OHOS gems shipped Alpine-built bytes without NDK + verification. This version is the first with proper OHOS NDK + cross-compilation + signing. +- Code signing uses `-selfSign 1` (self-signed). Sufficient for OHOS + userland load. Production deployment on Huawei-managed hardware may + require re-signing with an enrolled cert. + +### Superseded +- PR #11 (closed) -- hand-written `toolchain.cmake` + system CMake. Replaced + by NDK's bundled `ohos.toolchain.cmake`. +- PR #12 (closed) -- dockerharmony verification of Alpine bytes. The + verification step is reused here; the Alpine-built `.so` is replaced + with the NDK-built, signed one. + ## [1.6.58.5] - 2026-07-26 ### Added diff --git a/CLAUDE.md b/CLAUDE.md index 6837fd4..ae01aed 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -29,8 +29,16 @@ loaded via `autoload` from `lib/libpng.rb`. **Never use `require_relative` | `lib/libpng/standard_decoder.rb` | `Libpng::StandardDecoder` (libpng standard read API; explicit transform control) | | `lib/libpng/metadata_writer.rb` | `Libpng::MetadataWriter` (validates + writes text/gAMA/sRGB/cHRM/iCCP/pHYs onto a png_ptr/info_ptr pair) | | `lib/libpng/text_writer.rb` | `Libpng::TextWriter` + `Libpng::TextEntry` (builds png_text struct array, calls `png_set_text`) | -| `lib/libpng/recipe.rb` | `Libpng::Recipe < MiniPortileCMake` (builds libpng from source for the source gem) | -| `ext/extconf.rb` | Gem extension entry. Triggers `Libpng::Recipe` autoload via `require 'libpng'`, then emits a dummy Makefile | +| `lib/libpng/recipe.rb` | `Libpng::Recipe < MiniPortileCMake` (builds libpng from source for the source gem). Has `.for_target(platform)` factory -- OCP seam that returns `OHOS::Recipe` for `*-ohos`, base `Recipe` otherwise | +| `lib/libpng/ohos.rb` | `Libpng::OHOS` namespace + autoloads. Lazy-loaded only when `Recipe.for_target` is called with an OHOS target | +| `lib/libpng/ohos/ndk.rb` | `OHOS::NDK` -- pure-data class for NDK path discovery (toolchain/sysroot/clang/sign-tool). Idempotent `#download` via `setup-ndk.sh` | +| `lib/libpng/ohos/zlib_builder.rb` | `OHOS::ZlibBuilder < MiniPortileCMake` -- builds `libz.a` statically with the OHOS toolchain | +| `lib/libpng/ohos/code_signer.rb` | `OHOS::CodeSigner` -- wraps `binary-sign-tool sign -selfSign 1`. Pure-data `#sign_command` is testable without invoking the tool | +| `lib/libpng/ohos/recipe.rb` | `OHOS::Recipe < Libpng::Recipe` -- orchestrates NDK setup, zlib build, libpng cross-compile with `ohos.toolchain.cmake`, and post-install signing | +| `ext/extconf.rb` | Gem extension entry. Calls `Libpng::Recipe.for_target(ENV['target_platform']).new`, then emits a dummy Makefile | +| `ext/ohos/setup-ndk.sh` | Downloads OHOS SDK + LLVM-19 via OpenHarmony daily_build API (adapted from ohos-node/build.sh). Idempotent | +| `ext/ohos/smoke-test.c` | Minimal libpng round-trip test, cross-compiled with NDK clang and run inside dockerharmony | +| `ext/ohos/verify-prepare.sh` | Cross-compiles smoke-test, bundles signed `.so` + SONAME symlinks for dockerharmony | ### Public API @@ -74,6 +82,25 @@ bundle exec rake gem:native:any # source gem (compiles on install) Platform gem tasks: `x64-mingw32`, `x64-mingw-ucrt`, `aarch64-mingw-ucrt`, `x86_64-linux`, `x86_64-linux-musl`, `aarch64-linux`, `aarch64-linux-musl`, `aarch64-linux-ohos`, `x86_64-darwin`, `arm64-darwin`. +### OHOS (`aarch64-linux-ohos`) build notes + +OHOS uses a dedicated `build_ohos` CI job (see `.github/workflows/build.yml`) +that runs on `ubuntu-24.04-arm`. The NDK's clang and `binary-sign-tool` are +x86_64 ELF and run via `binfmt_misc` + `qemu-user-static`. The job: + +1. Downloads OHOS SDK + LLVM-19 via `ext/ohos/setup-ndk.sh` (~1.5 GB, cached + across runs by `actions/cache@v4`). +2. Runs `rake gem:native:aarch64-linux-ohos`, which delegates to + `Libpng::OHOS::Recipe` (sets up NDK, builds static zlib, cross-compiles + libpng with `ohos.toolchain.cmake`, signs the `.so`). +3. Cross-compiles `ext/ohos/smoke-test.c` with NDK clang via + `verify-prepare.sh`, then runs it inside the `dockerharmony` container + (real OHOS userland) — build fails if smoke-test doesn't output `OK`. + +Local dev: `target_platform=aarch64-linux-ohos bundle exec rake compile` +will only succeed on an arm64 host with `qemu-user-static` registered, +because the NDK clang is x86_64 ELF. CI handles this automatically. + ## Release process Releases are tag-triggered via `.github/workflows/release.yml`. Trigger with: diff --git a/README.adoc b/README.adoc index 70ff588..de5d080 100644 --- a/README.adoc +++ b/README.adoc @@ -44,7 +44,7 @@ for the current set): | `x86_64-linux-musl` | x86_64 Linux (musl, e.g. Alpine) | `ruby:-alpine` on x86_64 | `aarch64-linux` | ARM64 Linux (glibc) | `ubuntu-24.04-arm` (native) | `aarch64-linux-musl` | ARM64 Linux (musl) | `ruby:-alpine` on arm64 -| `aarch64-linux-ohos` | ARM64 OpenHarmony / Huawei HarmonyOS PC | `ruby:-alpine` on arm64 (byte-compatible with `aarch64-linux-musl`) +| `aarch64-linux-ohos` | ARM64 OpenHarmony / Huawei HarmonyOS PC | `ubuntu-24.04-arm` + OHOS NDK (cross-compiled, signed, **verified in dockerharmony**) | `x64-mingw32` | x64 Windows, RubyInstaller < 3.0 (MSVCRT) | `windows-latest` | `x64-mingw-ucrt` | x64 Windows, RubyInstaller >= 3.0 (UCRT) | `windows-latest` | `aarch64-mingw-ucrt` | ARM64 Windows (Ruby >= 3.4) | `windows-11-arm` (native) diff --git a/lib/libpng/version.rb b/lib/libpng/version.rb index 8f5ac20..a0f61c0 100644 --- a/lib/libpng/version.rb +++ b/lib/libpng/version.rb @@ -11,6 +11,6 @@ module Libpng # new libpng release. The iteration resets to 0 each time # LIBPNG_VERSION bumps. LIBPNG_VERSION = '1.6.58' - LIBPNG_RUBY_ITERATION = 5 + LIBPNG_RUBY_ITERATION = 6 VERSION = "#{LIBPNG_VERSION}.#{LIBPNG_RUBY_ITERATION}" end From c0263fc59b6c71108c6da478eb5b51583d61b84c Mon Sep 17 00:00:00 2001 From: Ronald Tse Date: Sun, 26 Jul 2026 20:06:58 +0800 Subject: [PATCH 05/21] Fix setup-ndk.sh: SDK tarball already contains ohos-sdk/ at top level Run 1 failed: "cd: can't cd to ext/ohos/ndk/ohos-sdk/linux". The ohos-sdk-public tarball extracts to ./ohos-sdk/{linux,windows,ohos}/ at the top level (per ohos-node/build.sh). Extracting into $PREFIX/ohos-sdk created a nested $PREFIX/ohos-sdk/ohos-sdk/linux/. Fix: extract straight into $PREFIX so the path lands at $PREFIX/ohos-sdk/linux/ as expected. Same pattern for LLVM-19. --- ext/ohos/setup-ndk.sh | 13 ++++++++++--- 1 file changed, 10 insertions(+), 3 deletions(-) diff --git a/ext/ohos/setup-ndk.sh b/ext/ohos/setup-ndk.sh index 8d94bce..f7e181d 100644 --- a/ext/ohos/setup-ndk.sh +++ b/ext/ohos/setup-ndk.sh @@ -57,8 +57,11 @@ sdk_url=$(query_component "ohos-sdk-public" | jq -r '.data.list.dataList[0].obsP [ -n "$sdk_url" ] || { echo "setup-ndk: failed to resolve sdk URL" >&2; exit 1; } echo "setup-ndk: downloading $sdk_url" curl -fL "$sdk_url" -o "$TMP/ohos-sdk-public.tar.gz" -mkdir -p "$PREFIX/ohos-sdk" -tar -zxf "$TMP/ohos-sdk-public.tar.gz" -C "$PREFIX/ohos-sdk" +# The SDK tarball already contains ohos-sdk/{linux,windows,ohos}/ at the +# top level. Extract straight into $PREFIX so $PREFIX/ohos-sdk/linux/ ends +# up at the expected path (extracting into $PREFIX/ohos-sdk would create +# $PREFIX/ohos-sdk/ohos-sdk/linux/ -- which broke CI run 1). +tar -zxf "$TMP/ohos-sdk-public.tar.gz" -C "$PREFIX" # Drop non-linux variants to keep the cache small. rm -rf "$PREFIX/ohos-sdk/windows" "$PREFIX/ohos-sdk/ohos" 2>/dev/null || true cd "$PREFIX/ohos-sdk/linux" @@ -74,13 +77,17 @@ llvm_url=$(query_component "LLVM-19" | jq -r '.data.list.dataList[0].obsPath') [ -n "$llvm_url" ] || { echo "setup-ndk: failed to resolve LLVM-19 URL" >&2; exit 1; } echo "setup-ndk: downloading $llvm_url" curl -fL "$llvm_url" -o "$TMP/LLVM-19.tar.gz" +# LLVM-19 tarball extracts to ./llvm-linux-x86_64.tar.gz + ./ohos-sysroot.tar.gz +# at the top level. Extract straight into $PREFIX/llvm-19/ so the inner +# tarballs land where the next step expects them. mkdir -p "$PREFIX/llvm-19" tar -zxf "$TMP/LLVM-19.tar.gz" -C "$PREFIX/llvm-19" cd "$PREFIX/llvm-19" tar -zxf llvm-linux-x86_64.tar.gz +rm -rf llvm-linux-x86_64.tar.gz mkdir -p sysroot tar -zxf ohos-sysroot.tar.gz -C sysroot -rm -f llvm-linux-x86_64.tar.gz ohos-sysroot.tar.gz +rm -rf ohos-sysroot.tar.gz cd - echo "setup-ndk: NDK ready at $PREFIX" From 320991fcf59b773da0dcf9f69832f20f941b23e2 Mon Sep 17 00:00:00 2001 From: Ronald Tse Date: Sun, 26 Jul 2026 20:12:02 +0800 Subject: [PATCH 06/21] Add diagnostics to setup-ndk.sh to find actual NDK paths Run 2 failed because OHOS::NDK#exist? returned false even though setup-ndk.sh reported "NDK ready". Likely my hardcoded paths (ohos-sdk/linux/{build/cmake/ohos.toolchain.cmake, toolchains/lib/binary-sign-tool}) don't match the actual extracted layout. The OHOS CMake doc suggests paths include a native/ prefix (SDK_PATH/native/build/cmake/ohos.toolchain.cmake). Add a find + ls at the end of setup-ndk.sh to print the actual locations of ohos.toolchain.cmake, binary-sign-tool, and clang. Once we see the real layout we can fix the NDK class paths. --- ext/ohos/setup-ndk.sh | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/ext/ohos/setup-ndk.sh b/ext/ohos/setup-ndk.sh index f7e181d..fd9512f 100644 --- a/ext/ohos/setup-ndk.sh +++ b/ext/ohos/setup-ndk.sh @@ -91,3 +91,7 @@ rm -rf ohos-sysroot.tar.gz cd - echo "setup-ndk: NDK ready at $PREFIX" +echo "setup-ndk: locating critical files..." +find "$PREFIX" -maxdepth 8 \( -name 'ohos.toolchain.cmake' -o -name 'binary-sign-tool' -o -name 'aarch64-unknown-linux-ohos-clang' \) -print | head -20 +echo "setup-ndk: top-level dir listing:" +ls "$PREFIX" From 1765b95ac4f98363de665d0764aa68ff1809f999 Mon Sep 17 00:00:00 2001 From: Ronald Tse Date: Sun, 26 Jul 2026 20:21:12 +0800 Subject: [PATCH 07/21] Fix NDK paths + extract all SDK zips + richer diagnostics Run 3 find output revealed ohos.toolchain.cmake is NOT in the daily_build ohos-sdk-public tarball (only binary-sign-tool + clang were found). Two possible causes: 1. The SDK ships native-*.zip separately and ohos-node only unzipped toolchains-*.zip. Fix: change the unzip loop from 'toolchains-*.zip' to '*.zip' so any native-*.zip gets extracted too. 2. The toolchain file lives under ohos-sdk/linux/native/build/cmake/ (per the OHOS CMake doc, which uses SDK_PATH/native/build/cmake/). Fix: update OHOS::NDK#toolchain_path and #cmake_path to include the native/ prefix. Both fixes are pushed together. If native/ doesn't appear after this, the find diagnostics will show what's actually there so we can adapt (fallback: write our own minimal toolchain.cmake or use system clang with --target=aarch64-linux-ohos). Specs updated to match the new native/ paths. --- ext/ohos/setup-ndk.sh | 14 ++++++++++---- lib/libpng/ohos/ndk.rb | 4 ++-- spec/ohos/ndk_spec.rb | 10 +++++----- spec/ohos/zlib_builder_spec.rb | 2 +- 4 files changed, 18 insertions(+), 12 deletions(-) diff --git a/ext/ohos/setup-ndk.sh b/ext/ohos/setup-ndk.sh index fd9512f..3933645 100644 --- a/ext/ohos/setup-ndk.sh +++ b/ext/ohos/setup-ndk.sh @@ -65,7 +65,11 @@ tar -zxf "$TMP/ohos-sdk-public.tar.gz" -C "$PREFIX" # Drop non-linux variants to keep the cache small. rm -rf "$PREFIX/ohos-sdk/windows" "$PREFIX/ohos-sdk/ohos" 2>/dev/null || true cd "$PREFIX/ohos-sdk/linux" -for z in toolchains-*.zip; do +# The SDK ships multiple zips: toolchains-*.zip (binary-sign-tool etc.) +# and native-*.zip (ohos.toolchain.cmake, build-tools/cmake). Unzip them +# all so the NDK class can find both toolchains/lib/binary-sign-tool and +# native/build/cmake/ohos.toolchain.cmake. +for z in *.zip; do [ -e "$z" ] || continue unzip -q "$z" rm -f "$z" @@ -92,6 +96,8 @@ cd - echo "setup-ndk: NDK ready at $PREFIX" echo "setup-ndk: locating critical files..." -find "$PREFIX" -maxdepth 8 \( -name 'ohos.toolchain.cmake' -o -name 'binary-sign-tool' -o -name 'aarch64-unknown-linux-ohos-clang' \) -print | head -20 -echo "setup-ndk: top-level dir listing:" -ls "$PREFIX" +find "$PREFIX" -maxdepth 8 \( -name 'ohos.toolchain.cmake' -o -name 'binary-sign-tool' -o -name 'aarch64-unknown-linux-ohos-clang' -o -name '*.cmake' \) -print | head -30 +echo "setup-ndk: ohos-sdk/linux/ listing:" +ls "$PREFIX/ohos-sdk/linux" 2>&1 || true +echo "setup-ndk: any 'native' dir?" +find "$PREFIX/ohos-sdk" -type d -name native 2>&1 | head -5 diff --git a/lib/libpng/ohos/ndk.rb b/lib/libpng/ohos/ndk.rb index bd62b5a..da10528 100644 --- a/lib/libpng/ohos/ndk.rb +++ b/lib/libpng/ohos/ndk.rb @@ -27,7 +27,7 @@ def exist? end def toolchain_path - sdk_dir.join('build/cmake/ohos.toolchain.cmake') + sdk_dir.join('native/build/cmake/ohos.toolchain.cmake') end def clang_path @@ -47,7 +47,7 @@ def sign_tool_path end def cmake_path - sdk_dir.join('build-tools/cmake/bin/cmake') + sdk_dir.join('native/build-tools/cmake/bin/cmake') end attr_reader :root diff --git a/spec/ohos/ndk_spec.rb b/spec/ohos/ndk_spec.rb index 1da8b3a..4293cc4 100644 --- a/spec/ohos/ndk_spec.rb +++ b/spec/ohos/ndk_spec.rb @@ -16,11 +16,11 @@ def touch_rel(rel) end before do - touch_rel('ohos-sdk/linux/build/cmake/ohos.toolchain.cmake') + touch_rel('ohos-sdk/linux/native/build/cmake/ohos.toolchain.cmake') touch_rel('llvm-19/llvm/bin/aarch64-unknown-linux-ohos-clang') touch_rel('llvm-19/llvm/bin/aarch64-unknown-linux-ohos-clang++') touch_rel('ohos-sdk/linux/toolchains/lib/binary-sign-tool') - touch_rel('ohos-sdk/linux/build-tools/cmake/bin/cmake') + touch_rel('ohos-sdk/linux/native/build-tools/cmake/bin/cmake') root.join('llvm-19/sysroot/usr/lib/aarch64-linux-ohos').mkpath end @@ -32,7 +32,7 @@ def touch_rel(rel) end it 'returns false when the toolchain file is missing' do - root.join('ohos-sdk/linux/build/cmake/ohos.toolchain.cmake').delete + root.join('ohos-sdk/linux/native/build/cmake/ohos.toolchain.cmake').delete expect(ndk.exist?).to be(false) end @@ -54,7 +54,7 @@ def touch_rel(rel) describe '#toolchain_path' do it 'points at ohos.toolchain.cmake under the SDK dir' do - expect(ndk.toolchain_path.to_s).to end_with('ohos-sdk/linux/build/cmake/ohos.toolchain.cmake') + expect(ndk.toolchain_path.to_s).to end_with('ohos-sdk/linux/native/build/cmake/ohos.toolchain.cmake') end it 'is a Pathname' do @@ -88,7 +88,7 @@ def touch_rel(rel) describe '#cmake_path' do it 'points at the NDK-bundled cmake binary' do - expect(ndk.cmake_path.to_s).to end_with('ohos-sdk/linux/build-tools/cmake/bin/cmake') + expect(ndk.cmake_path.to_s).to end_with('ohos-sdk/linux/native/build-tools/cmake/bin/cmake') end end diff --git a/spec/ohos/zlib_builder_spec.rb b/spec/ohos/zlib_builder_spec.rb index b67ba1a..a4d6ef2 100644 --- a/spec/ohos/zlib_builder_spec.rb +++ b/spec/ohos/zlib_builder_spec.rb @@ -11,7 +11,7 @@ before do %w[ - ohos-sdk/linux/build/cmake/ohos.toolchain.cmake + ohos-sdk/linux/native/build/cmake/ohos.toolchain.cmake llvm-19/llvm/bin/aarch64-unknown-linux-ohos-clang ohos-sdk/linux/toolchains/lib/binary-sign-tool ].each do |rel| From b9c2a88658dd4580cd143ee0b21a2361c0e4e6d5 Mon Sep 17 00:00:00 2001 From: Ronald Tse Date: Sun, 26 Jul 2026 20:26:46 +0800 Subject: [PATCH 08/21] Register binfmt with fix-binary flag for x86_64 NDK binaries Run 4 failed at the very first try_compile with: x86_64-binfmt-P: Could not open '/lib64/ld-linux-x86-64.so.2' Root cause: 'update-binfmts --enable qemu-x86_64' registers qemu as the handler for x86_64 ELF but WITHOUT the fix-binary (F) flag. Without F, when the kernel tries to exec an x86_64 DYNAMIC binary, it tries to use the host's /lib64/ld-linux-x86-64.so.2 interpreter (which doesn't exist on arm64 Ubuntu) instead of the guest's interpreter embedded by qemu. Switch to 'docker run --privileged --rm tonistiigi/binfmt --install amd64' which registers binfmt_misc with the F flag. This embeds qemu into the kernel's binfmt handlers at registration time, so dynamic x86_64 binaries run transparently on the arm64 host. Also strengthen the smoke check: compile + run a static x86_64 hello-world binary INSIDE an amd64 docker container, then run it on the host file system. If host binfmt is broken, the binary won't run. Drop qemu-user-static + binfmt-support from apt install since tonistiigi/binfmt installs its own qemu binaries. --- .github/workflows/build.yml | 26 +++++++++++++++++++------- .github/workflows/release.yml | 13 ++++++++----- 2 files changed, 27 insertions(+), 12 deletions(-) diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml index 2ec3024..454f1d0 100644 --- a/.github/workflows/build.yml +++ b/.github/workflows/build.yml @@ -209,14 +209,26 @@ jobs: - name: Install build tools + qemu-user-static run: | sudo apt-get update - sudo apt-get install -y cmake ninja-build zlib1g-dev curl jq unzip \ - qemu-user-static binfmt-support - - - name: Register binfmt for x86_64 (NDK binaries are x86_64 ELF) + sudo apt-get install -y cmake ninja-build zlib1g-dev curl jq unzip + + - name: Register binfmt for x86_64 with fix-binary flag + # The NDK's clang + binary-sign-tool are x86_64 ELF. They run on the + # arm64 host via binfmt_misc + qemu-user-static. The fix-binary (F) + # flag is required so the kernel can run x86_64 DYNAMIC binaries + # (which need their own /lib64/ld-linux-x86-64.so.2 interpreter, + # not the host's). update-binfmts doesn't set F; tonistiigi/binfmt + # does. Without F, clang fails with "Could not open + # /lib64/ld-linux-x86-64.so.2". run: | - sudo update-binfmts --enable qemu-x86_64 || true - # Smoke-check: a trivial x86_64 binary should run transparently. - docker run --rm --platform linux/amd64 alpine:latest echo "binfmt ok" + docker run --privileged --rm tonistiigi/binfmt --install amd64 + # Smoke check: run an x86_64 binary DIRECTLY on the host (not in + # docker) to confirm binfmt_misc + F flag is wired up. The docker + # container has its own binfmt handling; the host needs the kernel + # registration so CMake's direct clang invocation works. + docker run --rm --platform linux/amd64 alpine:latest echo "docker binfmt ok" + printf '#include \nint main(){puts("host binfmt ok");return 0;}\n' > /tmp/hello.c + docker run --rm --platform linux/amd64 -v /tmp:/tmp alpine:latest \ + sh -c 'apk add --no-cache gcc >/dev/null 2>&1 && gcc -static /tmp/hello.c -o /tmp/hello && /tmp/hello' - name: Cache OHOS NDK id: cache-ndk diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 1e8a0a8..91a8620 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -253,13 +253,16 @@ jobs: - name: Install build tools + qemu-user-static run: | sudo apt-get update - sudo apt-get install -y cmake ninja-build zlib1g-dev curl jq unzip \ - qemu-user-static binfmt-support + sudo apt-get install -y cmake ninja-build zlib1g-dev curl jq unzip - - name: Register binfmt for x86_64 (NDK binaries are x86_64 ELF) + - name: Register binfmt for x86_64 with fix-binary flag + # See build.yml's build_ohos job for full notes on why F flag matters. run: | - sudo update-binfmts --enable qemu-x86_64 || true - docker run --rm --platform linux/amd64 alpine:latest echo "binfmt ok" + docker run --privileged --rm tonistiigi/binfmt --install amd64 + docker run --rm --platform linux/amd64 alpine:latest echo "docker binfmt ok" + printf '#include \nint main(){puts("host binfmt ok");return 0;}\n' > /tmp/hello.c + docker run --rm --platform linux/amd64 -v /tmp:/tmp alpine:latest \ + sh -c 'apk add --no-cache gcc >/dev/null 2>&1 && gcc -static /tmp/hello.c -o /tmp/hello && /tmp/hello' - name: Cache OHOS NDK id: cache-ndk From 93b55883c683b78a68b7432ca888ab79366a4948 Mon Sep 17 00:00:00 2001 From: Ronald Tse Date: Sun, 26 Jul 2026 20:31:39 +0800 Subject: [PATCH 09/21] Drop broken binfmt smoke check (Alpine gcc lacks stdio.h) The smoke check compiled a hello-world inside an alpine:amd64 container to verify host binfmt was wired up. But apk's gcc package on Alpine doesn't pull in libc-dev, so stdio.h is missing and the smoke check itself failed. Drop the smoke check entirely -- the NDK clang invocation in the next step is the real test. If binfmt is broken, the build fails clearly with the same x86_64-binfmt-P error we saw in run 4. --- .github/workflows/build.yml | 11 +---------- .github/workflows/release.yml | 7 +------ 2 files changed, 2 insertions(+), 16 deletions(-) diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml index 454f1d0..df48779 100644 --- a/.github/workflows/build.yml +++ b/.github/workflows/build.yml @@ -219,16 +219,7 @@ jobs: # not the host's). update-binfmts doesn't set F; tonistiigi/binfmt # does. Without F, clang fails with "Could not open # /lib64/ld-linux-x86-64.so.2". - run: | - docker run --privileged --rm tonistiigi/binfmt --install amd64 - # Smoke check: run an x86_64 binary DIRECTLY on the host (not in - # docker) to confirm binfmt_misc + F flag is wired up. The docker - # container has its own binfmt handling; the host needs the kernel - # registration so CMake's direct clang invocation works. - docker run --rm --platform linux/amd64 alpine:latest echo "docker binfmt ok" - printf '#include \nint main(){puts("host binfmt ok");return 0;}\n' > /tmp/hello.c - docker run --rm --platform linux/amd64 -v /tmp:/tmp alpine:latest \ - sh -c 'apk add --no-cache gcc >/dev/null 2>&1 && gcc -static /tmp/hello.c -o /tmp/hello && /tmp/hello' + run: docker run --privileged --rm tonistiigi/binfmt --install amd64 - name: Cache OHOS NDK id: cache-ndk diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 91a8620..2a54d45 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -257,12 +257,7 @@ jobs: - name: Register binfmt for x86_64 with fix-binary flag # See build.yml's build_ohos job for full notes on why F flag matters. - run: | - docker run --privileged --rm tonistiigi/binfmt --install amd64 - docker run --rm --platform linux/amd64 alpine:latest echo "docker binfmt ok" - printf '#include \nint main(){puts("host binfmt ok");return 0;}\n' > /tmp/hello.c - docker run --rm --platform linux/amd64 -v /tmp:/tmp alpine:latest \ - sh -c 'apk add --no-cache gcc >/dev/null 2>&1 && gcc -static /tmp/hello.c -o /tmp/hello && /tmp/hello' + run: docker run --privileged --rm tonistiigi/binfmt --install amd64 - name: Cache OHOS NDK id: cache-ndk From 2de6dbf05a33b4bdd3ebec4c56085f6edd887904 Mon Sep 17 00:00:00 2001 From: Ronald Tse Date: Sun, 26 Jul 2026 20:36:56 +0800 Subject: [PATCH 10/21] Provide x86_64 loader for qemu-user-static via libc6-amd64-cross Run 6 still failed with 'qemu-x86_64: Could not open /lib64/ld-linux-x86-64.so.2' even after switching to tonistiigi/binfmt. The F flag registers qemu as the handler, but qemu itself still needs to find the x86_64 dynamic loader to exec dynamic binaries. tonistiigi/binfmt doesn't install an x86_64 rootfs. Install libc6-amd64-cross (which provides the loader at /usr/x86_64-linux-gnu/lib/ld-linux-x86-64.so.2) and symlink it to /lib64/ld-linux-x86-64.so.2 (the path qemu looks for). --- .github/workflows/build.yml | 25 ++++++++++++++++--------- .github/workflows/release.yml | 9 ++++++--- 2 files changed, 22 insertions(+), 12 deletions(-) diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml index df48779..65a2f66 100644 --- a/.github/workflows/build.yml +++ b/.github/workflows/build.yml @@ -206,19 +206,26 @@ jobs: ruby-version: ${{ needs.prepare.outputs.default-ruby-version }} bundler-cache: true - - name: Install build tools + qemu-user-static + - name: Install build tools + x86_64 cross-libc (for NDK clang under qemu) run: | sudo apt-get update - sudo apt-get install -y cmake ninja-build zlib1g-dev curl jq unzip + sudo apt-get install -y cmake ninja-build zlib1g-dev curl jq unzip \ + libc6-amd64-cross + # The NDK's clang + binary-sign-tool are x86_64 ELF. On the arm64 + # host they run via qemu-user-static, which needs to find the x86_64 + # dynamic loader at /lib64/ld-linux-x86-64.so.2. libc6-amd64-cross + # installs it at /usr/x86_64-linux-gnu/lib/ld-linux-x86-64.so.2; + # symlink it into place so qemu-user-static can resolve it. + sudo mkdir -p /lib64 + sudo ln -sf /usr/x86_64-linux-gnu/lib/ld-linux-x86-64.so.2 /lib64/ld-linux-x86-64.so.2 - name: Register binfmt for x86_64 with fix-binary flag - # The NDK's clang + binary-sign-tool are x86_64 ELF. They run on the - # arm64 host via binfmt_misc + qemu-user-static. The fix-binary (F) - # flag is required so the kernel can run x86_64 DYNAMIC binaries - # (which need their own /lib64/ld-linux-x86-64.so.2 interpreter, - # not the host's). update-binfmts doesn't set F; tonistiigi/binfmt - # does. Without F, clang fails with "Could not open - # /lib64/ld-linux-x86-64.so.2". + # The F flag embeds the qemu binary into binfmt_misc at registration + # time so dynamic x86_64 ELF binaries can run transparently on the + # arm64 host. Without F, the kernel tries to use the host's + # /lib64/ld-linux-x86-64.so.2 (which doesn't exist on arm64 Ubuntu). + # Combined with the libc6-amd64-cross symlink above, this lets the + # NDK clang exec without "Could not open /lib64/ld-linux-x86-64.so.2". run: docker run --privileged --rm tonistiigi/binfmt --install amd64 - name: Cache OHOS NDK diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 2a54d45..9d8500d 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -250,13 +250,16 @@ jobs: ruby-version: '3.3' bundler-cache: true - - name: Install build tools + qemu-user-static + - name: Install build tools + x86_64 cross-libc (for NDK clang under qemu) run: | sudo apt-get update - sudo apt-get install -y cmake ninja-build zlib1g-dev curl jq unzip + sudo apt-get install -y cmake ninja-build zlib1g-dev curl jq unzip \ + libc6-amd64-cross + sudo mkdir -p /lib64 + sudo ln -sf /usr/x86_64-linux-gnu/lib/ld-linux-x86-64.so.2 /lib64/ld-linux-x86-64.so.2 - name: Register binfmt for x86_64 with fix-binary flag - # See build.yml's build_ohos job for full notes on why F flag matters. + # See build.yml's build_ohos job for full notes. run: docker run --privileged --rm tonistiigi/binfmt --install amd64 - name: Cache OHOS NDK From 415d1a7466c137ec470a74616c689ed66f7e0c26 Mon Sep 17 00:00:00 2001 From: Ronald Tse Date: Sun, 26 Jul 2026 20:42:41 +0800 Subject: [PATCH 11/21] Switch to Debian multiarch for full x86_64 userland under qemu Run 7 got past the loader error but then hit 'cannot open shared object file: libpthread.so.0'. libc6-amd64-cross only ships libc -- not the libstdc++/libpthread/libgcc_s the NDK clang needs. Debian multiarch installs the full x86_64 userland alongside arm64: dpkg --add-architecture amd64 + apt install libc6:amd64 libstdc++6:amd64 libgcc-s1:amd64. The packages install at /lib/x86_64-linux-gnu/ with the loader symlinked at /lib64/ld-linux-x86-64.so.2. --- .github/workflows/build.yml | 35 ++++++++++++++++++++--------------- .github/workflows/release.yml | 10 +++++----- 2 files changed, 25 insertions(+), 20 deletions(-) diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml index 65a2f66..a8b9ca8 100644 --- a/.github/workflows/build.yml +++ b/.github/workflows/build.yml @@ -206,26 +206,31 @@ jobs: ruby-version: ${{ needs.prepare.outputs.default-ruby-version }} bundler-cache: true - - name: Install build tools + x86_64 cross-libc (for NDK clang under qemu) + - name: Install build tools + x86_64 multiarch userland (for NDK clang under qemu) run: | sudo apt-get update - sudo apt-get install -y cmake ninja-build zlib1g-dev curl jq unzip \ - libc6-amd64-cross + sudo apt-get install -y cmake ninja-build zlib1g-dev curl jq unzip + # The NDK's clang + binary-sign-tool are x86_64 ELF. On the arm64 - # host they run via qemu-user-static, which needs to find the x86_64 - # dynamic loader at /lib64/ld-linux-x86-64.so.2. libc6-amd64-cross - # installs it at /usr/x86_64-linux-gnu/lib/ld-linux-x86-64.so.2; - # symlink it into place so qemu-user-static can resolve it. - sudo mkdir -p /lib64 - sudo ln -sf /usr/x86_64-linux-gnu/lib/ld-linux-x86-64.so.2 /lib64/ld-linux-x86-64.so.2 + # host they run via qemu-user-static, which needs the full x86_64 + # userland (loader, libc, libstdc++, libpthread, libgcc_s) -- not + # just the dynamic loader. + # + # Debian multiarch installs these alongside the arm64 versions: + # libc:amd64 lives at /lib/x86_64-linux-gnu/, and the loader is + # symlinked at /lib64/ld-linux-x86-64.so.2 by the package. This is + # more complete than libc6-amd64-cross (which only ships libc and + # not libstdc++/libpthread). + sudo dpkg --add-architecture amd64 + sudo apt-get update + sudo apt-get install -y libc6:amd64 libstdc++6:amd64 libgcc-s1:amd64 - name: Register binfmt for x86_64 with fix-binary flag - # The F flag embeds the qemu binary into binfmt_misc at registration - # time so dynamic x86_64 ELF binaries can run transparently on the - # arm64 host. Without F, the kernel tries to use the host's - # /lib64/ld-linux-x86-64.so.2 (which doesn't exist on arm64 Ubuntu). - # Combined with the libc6-amd64-cross symlink above, this lets the - # NDK clang exec without "Could not open /lib64/ld-linux-x86-64.so.2". + # F flag embeds qemu into binfmt_misc at registration time so dynamic + # x86_64 ELF binaries run transparently on the arm64 host. Combined + # with the multiarch userland above, this lets the NDK clang exec + # without "Could not open /lib64/ld-linux-x86-64.so.2" or + # "cannot open shared object file: libpthread.so.0". run: docker run --privileged --rm tonistiigi/binfmt --install amd64 - name: Cache OHOS NDK diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 9d8500d..75dd910 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -250,13 +250,13 @@ jobs: ruby-version: '3.3' bundler-cache: true - - name: Install build tools + x86_64 cross-libc (for NDK clang under qemu) + - name: Install build tools + x86_64 multiarch userland (for NDK clang under qemu) run: | sudo apt-get update - sudo apt-get install -y cmake ninja-build zlib1g-dev curl jq unzip \ - libc6-amd64-cross - sudo mkdir -p /lib64 - sudo ln -sf /usr/x86_64-linux-gnu/lib/ld-linux-x86-64.so.2 /lib64/ld-linux-x86-64.so.2 + sudo apt-get install -y cmake ninja-build zlib1g-dev curl jq unzip + sudo dpkg --add-architecture amd64 + sudo apt-get update + sudo apt-get install -y libc6:amd64 libstdc++6:amd64 libgcc-s1:amd64 - name: Register binfmt for x86_64 with fix-binary flag # See build.yml's build_ohos job for full notes. From 5651340fe1c2b25f679f90b980d8bd6e16dce2f8 Mon Sep 17 00:00:00 2001 From: Ronald Tse Date: Sun, 26 Jul 2026 20:48:43 +0800 Subject: [PATCH 12/21] Run OHOS build inside amd64 docker container (simpler than multiarch) Runs 5-8 tried to make the arm64 host run x86_64 NDK binaries directly via qemu-user-static. Each fix revealed another missing piece: - update-binfmts lacks F flag -> tonistiigi/binfmt - qemu needs x86_64 loader -> libc6-amd64-cross symlink - clang needs libpthread/libstdc++ -> Debian multiarch - ports.ubuntu.com has no binary-amd64 -> need to add archive.ubuntu.com source Each step added complexity. Stepping back: docker's binfmt_misc registration is rock-solid (run 5 confirmed docker emulation works on this runner). Move the entire build INSIDE an amd64 docker container. Docker handles the amd64 emulation as a whole, with no host-side library juggling. New topology: - Runner: ubuntu-24.04-arm (arm64 native, satisfies the 'OHOS runs on ARM' framing) - Build: docker run --platform linux/amd64 ruby:3.3 -> NDK clang runs natively x86_64 inside the container - Verify: docker run ghcr.io/hqzing/dockerharmony -> arm64 native (no emulation) Trade-off: ~3-5x slowdown for the build phase (qemu emulates the entire container), but reliability >> speed here. The build phase takes ~1-2 min natively, so ~5-10 min under emulation is acceptable. NDK cache key drops the 'arm64' suffix since the cached NDK bytes are architecture-independent (just downloaded + extracted). --- .github/workflows/build.yml | 87 +++++++++++++++++++---------------- .github/workflows/release.yml | 47 +++++++++++-------- 2 files changed, 77 insertions(+), 57 deletions(-) diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml index a8b9ca8..bd6feb4 100644 --- a/.github/workflows/build.yml +++ b/.github/workflows/build.yml @@ -201,36 +201,18 @@ jobs: steps: - uses: actions/checkout@v4 - - uses: ruby/setup-ruby@master - with: - ruby-version: ${{ needs.prepare.outputs.default-ruby-version }} - bundler-cache: true - - - name: Install build tools + x86_64 multiarch userland (for NDK clang under qemu) + - name: Install host-side tools + # These are used for: NDK setup (curl/jq/unzip), artifact prep + # (file), and dockerharmony verification (docker is preinstalled). run: | sudo apt-get update - sudo apt-get install -y cmake ninja-build zlib1g-dev curl jq unzip - - # The NDK's clang + binary-sign-tool are x86_64 ELF. On the arm64 - # host they run via qemu-user-static, which needs the full x86_64 - # userland (loader, libc, libstdc++, libpthread, libgcc_s) -- not - # just the dynamic loader. - # - # Debian multiarch installs these alongside the arm64 versions: - # libc:amd64 lives at /lib/x86_64-linux-gnu/, and the loader is - # symlinked at /lib64/ld-linux-x86-64.so.2 by the package. This is - # more complete than libc6-amd64-cross (which only ships libc and - # not libstdc++/libpthread). - sudo dpkg --add-architecture amd64 - sudo apt-get update - sudo apt-get install -y libc6:amd64 libstdc++6:amd64 libgcc-s1:amd64 - - - name: Register binfmt for x86_64 with fix-binary flag - # F flag embeds qemu into binfmt_misc at registration time so dynamic - # x86_64 ELF binaries run transparently on the arm64 host. Combined - # with the multiarch userland above, this lets the NDK clang exec - # without "Could not open /lib64/ld-linux-x86-64.so.2" or - # "cannot open shared object file: libpthread.so.0". + sudo apt-get install -y curl jq unzip file + + - name: Register binfmt for amd64 docker emulation + # The OHOS NDK is x86_64-only. We run the entire build inside an + # amd64 docker container on this arm64 host -- docker's binfmt_misc + # registration handles the emulation transparently inside the + # container, with no library juggling on the host side. run: docker run --privileged --rm tonistiigi/binfmt --install amd64 - name: Cache OHOS NDK @@ -238,21 +220,48 @@ jobs: uses: actions/cache@v4 with: path: ext/ohos/ndk - key: ohos-ndk-arm64-${{ hashFiles('ext/ohos/setup-ndk.sh') }} + key: ohos-ndk-${{ hashFiles('ext/ohos/setup-ndk.sh') }} - - name: Setup OHOS NDK + - name: Setup OHOS NDK (host-side; only curl/jq/unzip needed) if: steps.cache-ndk.outputs.cache-hit != 'true' run: sh ext/ohos/setup-ndk.sh --prefix ext/ohos/ndk - - name: Build OHOS gem (libpng + static zlib + signing) - run: bundle exec rake gem:native:aarch64-linux-ohos - env: - target_platform: aarch64-linux-ohos - - - name: Prepare dockerharmony artifacts - run: NDK_ROOT=$PWD/ext/ohos/ndk sh ext/ohos/verify-prepare.sh - - - name: Verify in dockerharmony (real OHOS userland) + - name: Build OHOS gem in amd64 container + # The entire build (libpng + static zlib + signing) runs inside an + # amd64 docker container so the NDK's x86_64 clang + binary-sign-tool + # run natively (no qemu at the process level; docker's binfmt handles + # the container emulation as a whole). + run: | + docker run --rm --platform linux/amd64 \ + -v "$PWD:/work" -w /work \ + -e target_platform=aarch64-linux-ohos \ + ruby:3.3 \ + sh -c ' + set -e + apt-get update + apt-get install -y --no-install-recommends cmake ninja-build zlib1g-dev + bundle install --jobs 4 + bundle exec rake gem:native:aarch64-linux-ohos + # Confirm the .so was produced and is arm64 ELF (sanity). + file lib/libpng/libpng16.so + ' + + - name: Prepare dockerharmony artifacts (host-side) + # verify-prepare.sh cross-compiles smoke-test.c using the NDK clang. + # We run it inside the same amd64 container for the same reason. + run: | + docker run --rm --platform linux/amd64 \ + -v "$PWD:/work" -w /work \ + -e NDK_ROOT=/work/ext/ohos/ndk \ + ruby:3.3 \ + sh -c ' + set -e + apt-get update + apt-get install -y --no-install-recommends file + sh ext/ohos/verify-prepare.sh + ' + + - name: Verify in dockerharmony (real OHOS userland, arm64-native) run: | docker pull ghcr.io/hqzing/dockerharmony:latest docker run --rm \ diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 75dd910..4aed14b 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -245,21 +245,12 @@ jobs: with: ref: ${{ needs.bump.outputs.sha || github.ref }} - - uses: ruby/setup-ruby@master - with: - ruby-version: '3.3' - bundler-cache: true - - - name: Install build tools + x86_64 multiarch userland (for NDK clang under qemu) + - name: Install host-side tools run: | sudo apt-get update - sudo apt-get install -y cmake ninja-build zlib1g-dev curl jq unzip - sudo dpkg --add-architecture amd64 - sudo apt-get update - sudo apt-get install -y libc6:amd64 libstdc++6:amd64 libgcc-s1:amd64 + sudo apt-get install -y curl jq unzip file - - name: Register binfmt for x86_64 with fix-binary flag - # See build.yml's build_ohos job for full notes. + - name: Register binfmt for amd64 docker emulation run: docker run --privileged --rm tonistiigi/binfmt --install amd64 - name: Cache OHOS NDK @@ -267,19 +258,39 @@ jobs: uses: actions/cache@v4 with: path: ext/ohos/ndk - key: ohos-ndk-arm64-${{ hashFiles('ext/ohos/setup-ndk.sh') }} + key: ohos-ndk-${{ hashFiles('ext/ohos/setup-ndk.sh') }} - name: Setup OHOS NDK if: steps.cache-ndk.outputs.cache-hit != 'true' run: sh ext/ohos/setup-ndk.sh --prefix ext/ohos/ndk - - name: Build OHOS gem (libpng + static zlib + signing) - run: bundle exec rake gem:native:aarch64-linux-ohos - env: - target_platform: aarch64-linux-ohos + - name: Build OHOS gem in amd64 container + run: | + docker run --rm --platform linux/amd64 \ + -v "$PWD:/work" -w /work \ + -e target_platform=aarch64-linux-ohos \ + ruby:3.3 \ + sh -c ' + set -e + apt-get update + apt-get install -y --no-install-recommends cmake ninja-build zlib1g-dev + bundle install --jobs 4 + bundle exec rake gem:native:aarch64-linux-ohos + file lib/libpng/libpng16.so + ' - name: Prepare dockerharmony artifacts - run: NDK_ROOT=$PWD/ext/ohos/ndk sh ext/ohos/verify-prepare.sh + run: | + docker run --rm --platform linux/amd64 \ + -v "$PWD:/work" -w /work \ + -e NDK_ROOT=/work/ext/ohos/ndk \ + ruby:3.3 \ + sh -c ' + set -e + apt-get update + apt-get install -y --no-install-recommends file + sh ext/ohos/verify-prepare.sh + ' - name: Verify in dockerharmony (real OHOS userland) run: | From 1a32b0277914150616bc94f2c7e1349cb129c35d Mon Sep 17 00:00:00 2001 From: Ronald Tse Date: Sun, 26 Jul 2026 21:00:56 +0800 Subject: [PATCH 13/21] Symlink LLVM-19 sysroot over SDK's partial sysroot Run 9 got the toolchain + clang working but failed at: ohos-sdk/linux/native/sysroot/usr/include/limits.h:6: fatal error: 'bits/alltypes.h' file not found The SDK bundles a partial sysroot at ohos-sdk/linux/native/sysroot/ (missing arch-specific musl headers). The LLVM-19 tarball we also download ships ohos-sysroot.tar.gz with the complete sysroot. Replace the SDK's partial sysroot with a symlink to the LLVM-19 sysroot so ohos.toolchain.cmake (which resolves CMAKE_SYSROOT relative to its own location) picks up the complete headers. Add diagnostics to confirm the symlink took and that alltypes.h is now reachable from both sysroot paths. --- ext/ohos/setup-ndk.sh | 15 ++++++++++++--- 1 file changed, 12 insertions(+), 3 deletions(-) diff --git a/ext/ohos/setup-ndk.sh b/ext/ohos/setup-ndk.sh index 3933645..2a1f18c 100644 --- a/ext/ohos/setup-ndk.sh +++ b/ext/ohos/setup-ndk.sh @@ -95,9 +95,18 @@ rm -rf ohos-sysroot.tar.gz cd - echo "setup-ndk: NDK ready at $PREFIX" +# The SDK bundles a partial sysroot at ohos-sdk/linux/native/sysroot/ +# (missing arch-specific musl headers like bits/alltypes.h). The LLVM-19 +# tarball ships a more complete sysroot. Replace the SDK's partial sysroot +# with a symlink to the LLVM-19 one so ohos.toolchain.cmake (which resolves +# CMAKE_SYSROOT relative to its own location) picks up the complete headers. +if [ -d "$PREFIX/llvm-19/sysroot/usr/include" ] && [ -d "$PREFIX/ohos-sdk/linux/native/sysroot" ]; then + rm -rf "$PREFIX/ohos-sdk/linux/native/sysroot" + ln -s "$PREFIX/llvm-19/sysroot" "$PREFIX/ohos-sdk/linux/native/sysroot" +fi echo "setup-ndk: locating critical files..." -find "$PREFIX" -maxdepth 8 \( -name 'ohos.toolchain.cmake' -o -name 'binary-sign-tool' -o -name 'aarch64-unknown-linux-ohos-clang' -o -name '*.cmake' \) -print | head -30 +find "$PREFIX" -maxdepth 8 \( -name 'ohos.toolchain.cmake' -o -name 'binary-sign-tool' -o -name 'aarch64-unknown-linux-ohos-clang' -o -name 'alltypes.h' \) -print | head -30 echo "setup-ndk: ohos-sdk/linux/ listing:" ls "$PREFIX/ohos-sdk/linux" 2>&1 || true -echo "setup-ndk: any 'native' dir?" -find "$PREFIX/ohos-sdk" -type d -name native 2>&1 | head -5 +echo "setup-ndk: llvm-19/sysroot/usr/include/ listing:" +ls "$PREFIX/llvm-19/sysroot/usr/include" 2>&1 | head -20 || true From f16fcf26e485413fd4f4edd1c1843e50ee7d1266 Mon Sep 17 00:00:00 2001 From: Ronald Tse Date: Sun, 26 Jul 2026 21:13:02 +0800 Subject: [PATCH 14/21] Fix LLVM-19 sysroot extraction: tarball already has sysroot/ at top Run 10 diagnostic showed llvm-19/sysroot/usr/include/ doesn't exist. My 'mkdir sysroot && tar -C sysroot -zxf ohos-sysroot.tar.gz' created a nested llvm-19/sysroot/sysroot/usr/include/ instead. ohos-sysroot.tar.gz already contains sysroot/ at the top level (per ohos-node/build.sh which extracts with no -C). Drop the mkdir + -C and extract directly into llvm-19/. --- ext/ohos/setup-ndk.sh | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/ext/ohos/setup-ndk.sh b/ext/ohos/setup-ndk.sh index 2a1f18c..2090441 100644 --- a/ext/ohos/setup-ndk.sh +++ b/ext/ohos/setup-ndk.sh @@ -89,8 +89,11 @@ tar -zxf "$TMP/LLVM-19.tar.gz" -C "$PREFIX/llvm-19" cd "$PREFIX/llvm-19" tar -zxf llvm-linux-x86_64.tar.gz rm -rf llvm-linux-x86_64.tar.gz -mkdir -p sysroot -tar -zxf ohos-sysroot.tar.gz -C sysroot +# ohos-sysroot.tar.gz already contains sysroot/ at the top level (per +# ohos-node/build.sh). Extract directly into $PREFIX/llvm-19/ -- NOT into +# $PREFIX/llvm-19/sysroot/ (that would create a nested sysroot/sysroot/ +# and break the SDK-sysroot symlink below). +tar -zxf ohos-sysroot.tar.gz rm -rf ohos-sysroot.tar.gz cd - From c857f1af67d698833ee22a22fd4e2cc1b1367828 Mon Sep 17 00:00:00 2001 From: Ronald Tse Date: Sun, 26 Jul 2026 21:18:19 +0800 Subject: [PATCH 15/21] Add curl retry logic for OHOS CDN downloads Run 11 failed at the SDK download step with: curl: (35) Recv failure: Connection reset by peer cidownload.openharmony.cn sometimes resets connections mid-transfer (observed on runs 1-10 too, but always succeeded on retry). Add --retry 5 --retry-delay 10 --retry-all-errors to both the API query and the tarball downloads so transient network issues don't fail the build. --- ext/ohos/setup-ndk.sh | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/ext/ohos/setup-ndk.sh b/ext/ohos/setup-ndk.sh index 2090441..b7a82ea 100644 --- a/ext/ohos/setup-ndk.sh +++ b/ext/ohos/setup-ndk.sh @@ -46,17 +46,23 @@ trap 'rm -rf "$TMP"' EXIT query_component() { component=$1 - curl -fsSL 'https://dcp.openharmony.cn/api/daily_build/build/list/component' \ + curl --retry 5 --retry-delay 5 --retry-all-errors -fsSL \ + 'https://dcp.openharmony.cn/api/daily_build/build/list/component' \ -H 'Accept: application/json, text/plain, */*' \ -H 'Content-Type: application/json' \ --data-raw '{"projectName":"openharmony","branch":"master","pageNum":1,"pageSize":10,"deviceLevel":"","component":"'"${component}"'","type":1,"startTime":"2025080100000000","endTime":"20990101235959","sortType":"","sortField":"","hardwareBoard":"","buildStatus":"success","buildFailReason":"","withDomain":1}' } +# Curl download helper with retries (the OHOS CDN sometimes resets connections). +dl() { + curl --retry 5 --retry-delay 10 --retry-all-errors -fL "$1" -o "$2" +} + echo "setup-ndk: querying daily_build API for ohos-sdk-public..." sdk_url=$(query_component "ohos-sdk-public" | jq -r '.data.list.dataList[0].obsPath') [ -n "$sdk_url" ] || { echo "setup-ndk: failed to resolve sdk URL" >&2; exit 1; } echo "setup-ndk: downloading $sdk_url" -curl -fL "$sdk_url" -o "$TMP/ohos-sdk-public.tar.gz" +dl "$sdk_url" "$TMP/ohos-sdk-public.tar.gz" # The SDK tarball already contains ohos-sdk/{linux,windows,ohos}/ at the # top level. Extract straight into $PREFIX so $PREFIX/ohos-sdk/linux/ ends # up at the expected path (extracting into $PREFIX/ohos-sdk would create @@ -80,7 +86,7 @@ echo "setup-ndk: querying daily_build API for LLVM-19..." llvm_url=$(query_component "LLVM-19" | jq -r '.data.list.dataList[0].obsPath') [ -n "$llvm_url" ] || { echo "setup-ndk: failed to resolve LLVM-19 URL" >&2; exit 1; } echo "setup-ndk: downloading $llvm_url" -curl -fL "$llvm_url" -o "$TMP/LLVM-19.tar.gz" +dl "$llvm_url" "$TMP/LLVM-19.tar.gz" # LLVM-19 tarball extracts to ./llvm-linux-x86_64.tar.gz + ./ohos-sysroot.tar.gz # at the top level. Extract straight into $PREFIX/llvm-19/ so the inner # tarballs land where the next step expects them. From d733557ccdaba9524440e8109bcbbf9749ad8d61 Mon Sep 17 00:00:00 2001 From: Ronald Tse Date: Sun, 26 Jul 2026 21:36:12 +0800 Subject: [PATCH 16/21] Dump actual llvm-19 layout to find where sysroot landed Run 12 still failed with 'bits/alltypes.h not found' AND diagnostic showed llvm-19/sysroot/usr/include/ doesn't exist. My fix (drop the mkdir + -C sysroot) didn't work, which means ohos-sysroot.tar.gz has yet another top-level layout. Add diagnostics: - ls llvm-19/ (top-level) - find llvm-19/ -maxdepth 2 -type d (the dir tree) - find anywhere for alltypes.h or 'bits' dir - tar -tzf on LLVM-19.tar.gz to see what the inner sysroot tarball contains This will tell us the actual layout so we can extract correctly. --- ext/ohos/setup-ndk.sh | 14 ++++++++++---- 1 file changed, 10 insertions(+), 4 deletions(-) diff --git a/ext/ohos/setup-ndk.sh b/ext/ohos/setup-ndk.sh index b7a82ea..5f424c9 100644 --- a/ext/ohos/setup-ndk.sh +++ b/ext/ohos/setup-ndk.sh @@ -104,6 +104,14 @@ rm -rf ohos-sysroot.tar.gz cd - echo "setup-ndk: NDK ready at $PREFIX" +echo "setup-ndk: llvm-19/ top-level contents:" +ls "$PREFIX/llvm-19" 2>&1 +echo "setup-ndk: llvm-19/ tree (2 levels):" +find "$PREFIX/llvm-19" -maxdepth 2 -type d 2>&1 | head -20 +echo "setup-ndk: any 'bits' or 'alltypes.h' anywhere?" +find "$PREFIX" -name 'alltypes.h' -o -type d -name 'bits' 2>&1 | head -10 +echo "setup-ndk: ohos-sysroot.tar.gz contents (top-level):" +tar -tzf "$TMP/LLVM-19.tar.gz" 2>&1 | grep -i sysroot | head -5 # The SDK bundles a partial sysroot at ohos-sdk/linux/native/sysroot/ # (missing arch-specific musl headers like bits/alltypes.h). The LLVM-19 # tarball ships a more complete sysroot. Replace the SDK's partial sysroot @@ -112,10 +120,8 @@ echo "setup-ndk: NDK ready at $PREFIX" if [ -d "$PREFIX/llvm-19/sysroot/usr/include" ] && [ -d "$PREFIX/ohos-sdk/linux/native/sysroot" ]; then rm -rf "$PREFIX/ohos-sdk/linux/native/sysroot" ln -s "$PREFIX/llvm-19/sysroot" "$PREFIX/ohos-sdk/linux/native/sysroot" +else + echo "setup-ndk: WARNING - cannot find LLVM-19 sysroot; SDK sysroot will be incomplete" fi echo "setup-ndk: locating critical files..." find "$PREFIX" -maxdepth 8 \( -name 'ohos.toolchain.cmake' -o -name 'binary-sign-tool' -o -name 'aarch64-unknown-linux-ohos-clang' -o -name 'alltypes.h' \) -print | head -30 -echo "setup-ndk: ohos-sdk/linux/ listing:" -ls "$PREFIX/ohos-sdk/linux" 2>&1 || true -echo "setup-ndk: llvm-19/sysroot/usr/include/ listing:" -ls "$PREFIX/llvm-19/sysroot/usr/include" 2>&1 | head -20 || true From d689c8def43fae216937a8346a593696f253bd1e Mon Sep 17 00:00:00 2001 From: Ronald Tse Date: Sun, 26 Jul 2026 21:48:15 +0800 Subject: [PATCH 17/21] Symlink to per-arch sysroot (aarch64-linux-ohos), not multiarch root Run 13 diagnostics revealed the layout mismatch: - SDK sysroot at ohos-sdk/linux/native/sysroot/usr/include/ uses MULTIARCH layout: usr/include//bits/alltypes.h - LLVM-19 sysroot at llvm-19/sysroot//usr/include/ uses PER-ARCH layout: /usr/include/bits/alltypes.h The OHOS toolchain expects per-arch layout (one sysroot per arch with plain usr/include/bits/ inside). My earlier symlink pointed at the LLVM-19 multiarch root, which still didn't have usr/include/bits/ at the expected path. Fix: point the symlink at llvm-19/sysroot/aarch64-linux-ohos/ specifically (per-arch subdir). Then ohos-sdk/linux/native/sysroot/usr/include/bits/ alltypes.h resolves correctly to the LLVM-19 per-arch sysroot. --- ext/ohos/setup-ndk.sh | 30 +++++++++++++----------------- 1 file changed, 13 insertions(+), 17 deletions(-) diff --git a/ext/ohos/setup-ndk.sh b/ext/ohos/setup-ndk.sh index 5f424c9..ca75e45 100644 --- a/ext/ohos/setup-ndk.sh +++ b/ext/ohos/setup-ndk.sh @@ -104,24 +104,20 @@ rm -rf ohos-sysroot.tar.gz cd - echo "setup-ndk: NDK ready at $PREFIX" -echo "setup-ndk: llvm-19/ top-level contents:" -ls "$PREFIX/llvm-19" 2>&1 -echo "setup-ndk: llvm-19/ tree (2 levels):" -find "$PREFIX/llvm-19" -maxdepth 2 -type d 2>&1 | head -20 -echo "setup-ndk: any 'bits' or 'alltypes.h' anywhere?" -find "$PREFIX" -name 'alltypes.h' -o -type d -name 'bits' 2>&1 | head -10 -echo "setup-ndk: ohos-sysroot.tar.gz contents (top-level):" -tar -tzf "$TMP/LLVM-19.tar.gz" 2>&1 | grep -i sysroot | head -5 -# The SDK bundles a partial sysroot at ohos-sdk/linux/native/sysroot/ -# (missing arch-specific musl headers like bits/alltypes.h). The LLVM-19 -# tarball ships a more complete sysroot. Replace the SDK's partial sysroot -# with a symlink to the LLVM-19 one so ohos.toolchain.cmake (which resolves -# CMAKE_SYSROOT relative to its own location) picks up the complete headers. -if [ -d "$PREFIX/llvm-19/sysroot/usr/include" ] && [ -d "$PREFIX/ohos-sdk/linux/native/sysroot" ]; then +# The OHOS toolchain (ohos.toolchain.cmake) resolves CMAKE_SYSROOT relative +# to its own location: ohos-sdk/linux/native/sysroot/. That sysroot uses the +# MULTIARCH layout (usr/include/aarch64-linux-ohos/bits/...), but the +# toolchain expects the PER-ARCH layout (sysroot/usr/include/bits/...). +# +# The LLVM-19 tarball ships per-arch sysroots at llvm-19/sysroot//. +# Symlink the SDK's multiarch sysroot to the per-arch aarch64 sysroot so +# the toolchain finds bits/alltypes.h and friends at the expected paths. +ARCH_SYSROOT="$PREFIX/llvm-19/sysroot/aarch64-linux-ohos" +if [ -d "$ARCH_SYSROOT/usr/include" ] && [ -d "$PREFIX/ohos-sdk/linux/native/sysroot" ]; then rm -rf "$PREFIX/ohos-sdk/linux/native/sysroot" - ln -s "$PREFIX/llvm-19/sysroot" "$PREFIX/ohos-sdk/linux/native/sysroot" + ln -s "$ARCH_SYSROOT" "$PREFIX/ohos-sdk/linux/native/sysroot" else - echo "setup-ndk: WARNING - cannot find LLVM-19 sysroot; SDK sysroot will be incomplete" + echo "setup-ndk: WARNING - $ARCH_SYSROOT missing; SDK sysroot will be incomplete" fi echo "setup-ndk: locating critical files..." -find "$PREFIX" -maxdepth 8 \( -name 'ohos.toolchain.cmake' -o -name 'binary-sign-tool' -o -name 'aarch64-unknown-linux-ohos-clang' -o -name 'alltypes.h' \) -print | head -30 +find "$PREFIX" -maxdepth 9 \( -name 'ohos.toolchain.cmake' -o -name 'binary-sign-tool' -o -name 'aarch64-unknown-linux-ohos-clang' -o -name 'alltypes.h' \) -print | head -20 From 5e421cb490297d40a8f4a56877347d627bdadc0b Mon Sep 17 00:00:00 2001 From: Ronald Tse Date: Sun, 26 Jul 2026 21:58:26 +0800 Subject: [PATCH 18/21] Add diagnostics: check CRT objects + libs in both sysroots Run 14 got past header errors but linker fails: ld.lld: cannot open Scrt1.o ld.lld: cannot open crti.o ld.lld: unable to find library -lm ld.lld: unable to find library -lc Possible cause: my symlink to llvm-19/sysroot/aarch64-linux-ohos/ points at a sysroot that has headers but no libraries. The original SDK sysroot (multiarch) had the libraries. Add diagnostics to see where Scrt1.o, crti.o, libc.so, libm.so actually live. If they're only in the multiarch SDK sysroot, we may need to merge both sysroots instead of replacing one with the other. --- ext/ohos/setup-ndk.sh | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/ext/ohos/setup-ndk.sh b/ext/ohos/setup-ndk.sh index ca75e45..54f82f2 100644 --- a/ext/ohos/setup-ndk.sh +++ b/ext/ohos/setup-ndk.sh @@ -121,3 +121,10 @@ else fi echo "setup-ndk: locating critical files..." find "$PREFIX" -maxdepth 9 \( -name 'ohos.toolchain.cmake' -o -name 'binary-sign-tool' -o -name 'aarch64-unknown-linux-ohos-clang' -o -name 'alltypes.h' \) -print | head -20 +echo "setup-ndk: CRT objects + libs in per-arch sysroot:" +find "$PREFIX/llvm-19/sysroot/aarch64-linux-ohos" -name 'Scrt1.o' -o -name 'crti.o' -o -name 'libc.so*' -o -name 'libm.so*' 2>&1 | head -10 +echo "setup-ndk: CRT objects + libs in original SDK sysroot (multiarch):" +find "$PREFIX/ohos-sdk/linux/native/sysroot" -maxdepth 7 -name 'Scrt1.o' -o -name 'crti.o' -o -name 'libc.so*' -o -name 'libm.so*' 2>&1 | head -10 +echo "setup-ndk: ohos-sdk/linux/native/sysroot resolves to:" +readlink "$PREFIX/ohos-sdk/linux/native/sysroot" 2>&1 || echo "(not a symlink)" +ls -la "$PREFIX/ohos-sdk/linux/native/sysroot/usr/lib" 2>&1 | head -20 From 8b7e0a3c6e0134595f72bf6fffa18b91bed75145 Mon Sep 17 00:00:00 2001 From: Ronald Tse Date: Sun, 26 Jul 2026 22:15:35 +0800 Subject: [PATCH 19/21] Resolve PREFIX to absolute path so ln -s creates valid symlinks Run 15 diagnostic showed the symlink wasn't being created correctly: setup-ndk: ohos-sdk/linux/native/sysroot resolves to: ls: cannot access 'ext/ohos/ndk/ohos-sdk/linux/native/sysroot/usr/lib' Root cause: $PREFIX was passed as 'ext/ohos/ndk' (relative) via the workflow's --prefix arg. ln -s interprets relative targets relative to the SYMLINK's location, not the script's cwd. So the symlink at ext/ohos/ndk/ohos-sdk/linux/native/sysroot pointed at ext/ohos/ndk/llvm-19/sysroot/aarch64-linux-ohos -- which from the symlink's dir resolves to ext/ohos/ndk/ohos-sdk/linux/native/ext/ohos/ndk/llvm-19/... (dangling). Fix: resolve PREFIX to an absolute path with $(cd "$PREFIX" && pwd) right after parsing args. Now ln -s writes an absolute target and the symlink resolves correctly regardless of cwd. --- ext/ohos/setup-ndk.sh | 14 ++++++++------ 1 file changed, 8 insertions(+), 6 deletions(-) diff --git a/ext/ohos/setup-ndk.sh b/ext/ohos/setup-ndk.sh index 54f82f2..c65317e 100644 --- a/ext/ohos/setup-ndk.sh +++ b/ext/ohos/setup-ndk.sh @@ -22,6 +22,11 @@ while [ $# -gt 0 ]; do esac done +# Resolve to absolute path. Relative paths break `ln -s` below: symlink +# targets are interpreted relative to the SYMLINK's location, not the +# script's cwd. A relative $PREFIX would create a dangling symlink. +PREFIX="$(cd "$PREFIX" && pwd)" + SDK_DIR="$PREFIX/ohos-sdk/linux" LLVM_DIR="$PREFIX/llvm-19/llvm" SYSROOT_DIR="$PREFIX/llvm-19/sysroot" @@ -121,10 +126,7 @@ else fi echo "setup-ndk: locating critical files..." find "$PREFIX" -maxdepth 9 \( -name 'ohos.toolchain.cmake' -o -name 'binary-sign-tool' -o -name 'aarch64-unknown-linux-ohos-clang' -o -name 'alltypes.h' \) -print | head -20 -echo "setup-ndk: CRT objects + libs in per-arch sysroot:" -find "$PREFIX/llvm-19/sysroot/aarch64-linux-ohos" -name 'Scrt1.o' -o -name 'crti.o' -o -name 'libc.so*' -o -name 'libm.so*' 2>&1 | head -10 -echo "setup-ndk: CRT objects + libs in original SDK sysroot (multiarch):" -find "$PREFIX/ohos-sdk/linux/native/sysroot" -maxdepth 7 -name 'Scrt1.o' -o -name 'crti.o' -o -name 'libc.so*' -o -name 'libm.so*' 2>&1 | head -10 -echo "setup-ndk: ohos-sdk/linux/native/sysroot resolves to:" +echo "setup-ndk: SDK sysroot is symlink to:" readlink "$PREFIX/ohos-sdk/linux/native/sysroot" 2>&1 || echo "(not a symlink)" -ls -la "$PREFIX/ohos-sdk/linux/native/sysroot/usr/lib" 2>&1 | head -20 +echo "setup-ndk: SDK sysroot/usr/lib/ contents:" +ls "$PREFIX/ohos-sdk/linux/native/sysroot/usr/lib" 2>&1 | head -10 From 744dbd29e22a7e0fae4b8f484e263d5b38755074 Mon Sep 17 00:00:00 2001 From: Ronald Tse Date: Sun, 26 Jul 2026 22:21:03 +0800 Subject: [PATCH 20/21] mkdir PREFIX before resolving to absolute path Run 16 failed at 'cd: can't cd to ext/ohos/ndk' on the new absolute-path resolution line. The dir doesn't exist yet at that point -- it was being created later in the script. Move mkdir earlier. --- ext/ohos/setup-ndk.sh | 1 + 1 file changed, 1 insertion(+) diff --git a/ext/ohos/setup-ndk.sh b/ext/ohos/setup-ndk.sh index c65317e..5cd01f8 100644 --- a/ext/ohos/setup-ndk.sh +++ b/ext/ohos/setup-ndk.sh @@ -25,6 +25,7 @@ done # Resolve to absolute path. Relative paths break `ln -s` below: symlink # targets are interpreted relative to the SYMLINK's location, not the # script's cwd. A relative $PREFIX would create a dangling symlink. +mkdir -p "$PREFIX" PREFIX="$(cd "$PREFIX" && pwd)" SDK_DIR="$PREFIX/ohos-sdk/linux" From d7eb4aee756ff8359571c0532825f8618900daa0 Mon Sep 17 00:00:00 2001 From: Ronald Tse Date: Sun, 26 Jul 2026 22:31:57 +0800 Subject: [PATCH 21/21] Use relative symlink target so it resolves in docker container too Run 17 diagnostic confirmed the symlink works on the host: SDK sysroot is symlink to: /home/runner/work/libpng-ruby/libpng-ruby/ext/ohos/ndk/llvm-19/sysroot/aarch64-linux-ohos But the BUILD runs inside a docker container that mounts the repo at /work/, not /home/runner/.... The absolute symlink target dangles inside the container, so clang's --sysroot path appears empty and ld.lld can't find Scrt1.o, libc.so, etc. Fix: use a relative symlink target (../../../llvm-19/sysroot/aarch64- linux-ohos). The relative path from ohos-sdk/linux/native/ to llvm-19/sysroot/aarch64-linux-ohos is the same regardless of where the repo is mounted. --- ext/ohos/setup-ndk.sh | 13 ++++++++++--- 1 file changed, 10 insertions(+), 3 deletions(-) diff --git a/ext/ohos/setup-ndk.sh b/ext/ohos/setup-ndk.sh index 5cd01f8..b5f57b3 100644 --- a/ext/ohos/setup-ndk.sh +++ b/ext/ohos/setup-ndk.sh @@ -118,10 +118,17 @@ echo "setup-ndk: NDK ready at $PREFIX" # The LLVM-19 tarball ships per-arch sysroots at llvm-19/sysroot//. # Symlink the SDK's multiarch sysroot to the per-arch aarch64 sysroot so # the toolchain finds bits/alltypes.h and friends at the expected paths. +# +# IMPORTANT: use a RELATIVE symlink target. The build itself runs inside +# a docker container that mounts $PREFIX at a different absolute path +# (/work/...) than the host (/home/runner/...). An absolute target would +# dangle inside the container; a relative target resolves correctly in +# both contexts. ARCH_SYSROOT="$PREFIX/llvm-19/sysroot/aarch64-linux-ohos" -if [ -d "$ARCH_SYSROOT/usr/include" ] && [ -d "$PREFIX/ohos-sdk/linux/native/sysroot" ]; then - rm -rf "$PREFIX/ohos-sdk/linux/native/sysroot" - ln -s "$ARCH_SYSROOT" "$PREFIX/ohos-sdk/linux/native/sysroot" +NATIVE_DIR="$PREFIX/ohos-sdk/linux/native" +if [ -d "$ARCH_SYSROOT/usr/include" ] && [ -d "$NATIVE_DIR/sysroot" ]; then + rm -rf "$NATIVE_DIR/sysroot" + ln -s "../../../llvm-19/sysroot/aarch64-linux-ohos" "$NATIVE_DIR/sysroot" else echo "setup-ndk: WARNING - $ARCH_SYSROOT missing; SDK sysroot will be incomplete" fi