Skip to content

OHOS: verify Alpine-built .so in dockerharmony (1.6.58.6) - #12

Closed
ronaldtse wants to merge 1 commit into
mainfrom
fix/ohos-dockerharmony
Closed

OHOS: verify Alpine-built .so in dockerharmony (1.6.58.6)#12
ronaldtse wants to merge 1 commit into
mainfrom
fix/ohos-dockerharmony

Conversation

@ronaldtse

Copy link
Copy Markdown
Contributor

What

Replace the unverified "Alpine-built bytes labeled as OHOS" approach with dockerharmony-based empirical verification. Simpler than PR #11's OHOS NDK cross-compile (which hit repeated issues), and provides actual evidence the bytes work in OHOS userland.

Why

The 1.6.58.4/.5 OHOS gem shipped Alpine-built musl arm64 bytes under the OHOS platform label, based on an unverified assumption that "same dynamic linker path = byte-compatible". PR #11 tried to fix this via OHOS NDK cross-compile but hit repeated issues across 5 CI iterations:

  • sysroot path was llvm-19/sysroot/aarch64-linux-ohos/usr/lib, not llvm-19/ohos-sysroot/usr/lib/...
  • OHOS clang didn't auto-detect multiarch library paths
  • zlib not shipped as separate libz.so in the OHOS sysroot
  • ls | head triggered pipefail exit

Each fix revealed a new issue. The complexity tax of cross-compilation for libpng (which uses CMake and requires zlib as an external dependency) isn't paying for itself.

dockerharmony (https://github.com/hqzing/dockerharmony) provides real OHOS userland in a Docker container. Alpine-built aarch64-linux-musl binaries run natively in it. We can use this to empirically verify our Alpine-built .so actually loads and works in OHOS — without any cross-compile machinery.

How

Files added

  • ext/ohos/smoke-test.c — minimal libpng round-trip test (2x2 RGBA encode → decode → byte comparison via the simplified API).
  • ext/ohos/verify-prepare.sh — runs inside the Alpine container (where we already have gcc + libpng headers). Compiles smoke-test.c against the freshly built libpng16.so with rpath=$ORIGIN, copies the .so + libz.so next to the binary, outputs to $PWD/ohos-verify/.

CI changes (build.yml + release.yml)

The aarch64-linux-ohos entry in build_musl now does two things inside the Alpine container:

  1. The existing gem build + Alpine-Ruby smoke test (unchanged).
  2. NEW: runs verify-prepare.sh to produce the dockerharmony artifacts.

A new step Verify in dockerharmony (OHOS userland) runs after the Alpine container:

  • Gated on matrix.platform == 'aarch64-linux-ohos'.
  • Pulls ghcr.io/hqzing/dockerharmony:latest (cached on the arm64 runner).
  • Runs LD_LIBRARY_PATH=. ./smoke-test inside the container, with the artifacts dir mounted.
  • Build fails if the smoke test fails.

The dockerharmony image is real OHOS mini rootfs (musl + toybox + mksh), running on the arm64 runner natively. Not qemu emulation. If the smoke test passes there, the bytes actually work in OHOS userland.

What this verifies

  • ✅ The OHOS dynamic linker (/lib/ld-musl-aarch64.so.1) loads our Alpine-built .so without errors.
  • ✅ libpng's simplified API round-trips encode→decode with matching pixel bytes.
  • ✅ All of this happens inside real OHOS userland, not an emulated sysroot.

What this does NOT verify

  • Real OHOS hardware. dockerharmony runs the OHOS rootfs on a Linux kernel; real devices may have additional constraints (SELinux policies, code signing enforcement at the kernel level).
  • Code signing. OHOS requires .so files to be code-signed for runtime loading on production devices. dockerharmony skips this check (it's a dev container). Users on real OHOS hardware may need to sign the .so via binary-sign-tool sign -selfSign 1 themselves. Documented in CHANGELOG. A future release may add automatic signing if a hardware user reports breakage.

Why not the ohos-node approach (OHOS NDK + bundled zlib)?

Considered and rejected (for now). ohos-node's pattern works for Node.js because Node bundles its own zlib + uses gyp configure that handles cross-compile cleanly. libpng uses CMake (more demanding on toolchain config) and requires zlib as an external find_package(ZLIB) — the OHOS sysroot doesn't ship libz.so as a separate file, so we'd need to also build zlib from source with the NDK. That's ~250 lines of code, a new zlib source URL+sha256 to maintain, and the result still needs verification (which would be the same dockerharmony check).

If a real OHOS hardware user reports signing or symbol issues, we can revisit the NDK approach as a follow-up — with the lessons learned from PR #11 and a bundled zlib build step.

Test plan

  • bundle exec rake — 132 specs pass, rubocop clean (no source/test changes)
  • CI: Alpine build for aarch64-linux-ohos succeeds, produces ohos-verify/ artifacts.
  • CI: dockerharmony container pulls successfully on ubuntu-24.04-arm.
  • CI: smoke-test outputs OK inside dockerharmony.
  • CI: all 11 platform gems still build (10 native + source ruby).

Version

Bumps LIBPNG_RUBY_ITERATION 5 → 6. Replaces the regression acknowledged in CHANGELOG (1.6.58.4/.5 used Alpine-built bytes for OHOS without verification).

…58.6)

The 1.6.58.4/.5 aarch64-linux-ohos gem shipped Alpine-built musl arm64
bytes under the OHOS platform label, based on an unverified assumption
that 'same dynamic linker path = byte-compatible'. PR #11 tried to
fix this via OHOS NDK cross-compile but hit repeated issues (sysroot
layout, missing zlib, pipefail on diagnostics) -- 5 CI iterations
without a green build.

This change takes the simpler path: keep the Alpine-built bytes (they
SHOULD work), but add empirical verification via the dockerharmony
container (real OHOS userland). If the .so loads and round-trips
encode/decode in dockerharmony, we have evidence it works in OHOS,
not just an assumption.

Files added:
- ext/ohos/smoke-test.c -- minimal libpng round-trip (2x2 RGBA
  encode -> decode -> byte comparison via simplified API).
- ext/ohos/verify-prepare.sh -- compiles smoke-test.c inside the
  Alpine container (where gcc + libpng headers live), bundles the
  .so + smoke binary + libz.so for dockerharmony to consume.

CI changes (build.yml + release.yml):
- aarch64-linux-ohos entry in build_musl now ALSO runs verify-prepare.sh
  inside Alpine, producing /work/ohos-verify/{smoke-test,libpng16.so,libz.so}.
- New 'Verify in dockerharmony (OHOS userland)' step (gated on
  platform == aarch64-linux-ohos) runs the smoke-test binary inside
  the ghcr.io/hqzing/dockerharmony:latest container. Build fails if
  the smoke test fails.

What this gets us:
- Real OHOS userland test (musl + toybox + mksh rootfs), not qemu
  emulation of an OHOS sysroot.
- ~200 lines simpler than the PR #11 OHOS NDK approach (no
  setup-toolchain.sh, no toolchain.cmake, no OHOS-specific recipe.rb
  code, no zlib detection, no sysroot path detection).
- Empirical CI evidence the bytes work, replacing the assumption.

What this does NOT verify:
- Real OHOS hardware (would need device or full OHOS VM).
- Code signing (dockerharmony skips signing checks; users on real
  OHOS hardware may need to sign the .so via binary-sign-tool
  themselves). Documented in CHANGELOG.

Reference: https://github.com/hqzing/dockerharmony

Bumps LIBPNG_RUBY_ITERATION to 6.
@ronaldtse

Copy link
Copy Markdown
Contributor Author

Closing in favor of PR #13 (proper OHOS NDK cross-compile). The dockerharmony verification step is reused; the build approach changes from 'Alpine-built bytes labeled OHOS' to 'NDK-built bytes signed with binary-sign-tool'. smoke-test.c is carried over. See TODO.ohos/ in the new branch for the architecture.

@ronaldtse ronaldtse closed this Jul 26, 2026
ronaldtse added a commit that referenced this pull request Jul 26, 2026
12 design documents covering the architecture, NDK setup, zlib build,
recipe factory, code signing, smoke-test, CI workflow, specs, docs,
and PR plan for cross-compiling libpng for OHOS with the official NDK.

Replaces PR #12's Alpine-built bytes labeled OHOS approach with proper
OHOS NDK cross-compile + signing + dockerharmony verification.
ronaldtse added a commit that referenced this pull request Jul 26, 2026
Adds the Ruby + shell infrastructure to cross-compile libpng for
aarch64-linux-ohos using Huawei's official OHOS NDK:

- ext/ohos/setup-ndk.sh: idempotent download of OHOS SDK + LLVM-19
  via the daily_build API (pattern adapted from ohos-node/build.sh).
- ext/ohos/smoke-test.c: minimal libpng round-trip test (encode ->
  decode -> byte compare), cross-compiled with NDK clang and run
  inside dockerharmony for empirical OHOS userland verification.
- ext/ohos/verify-prepare.sh: cross-compiles smoke-test with NDK
  clang, bundles the signed .so + SONAME symlinks. Fixes the symlink
  bug from PR #12 where only libpng16.so (dev symlink) was bundled
  but the smoke-test binary's NEEDED entry was libpng16.so.16 (SONAME).

- lib/libpng/ohos.rb: namespace + autoloads for OHOS::NDK,
  OHOS::Recipe, OHOS::ZlibBuilder, OHOS::CodeSigner.
- lib/libpng/ohos/ndk.rb: pure-data class for NDK path discovery
  (toolchain, sysroot, clang, sign-tool). No shell calls except via
  explicit #download (invokes setup-ndk.sh).
- lib/libpng/ohos/zlib_builder.rb: OHOS::ZlibBuilder < MiniPortileCMake.
  Builds zlib statically (libz.a) with the OHOS toolchain; linked into
  libpng16.so to avoid OHOS's non-standard libshared_libz.z.so dependency.
- lib/libpng/ohos/code_signer.rb: OHOS::CodeSigner. Wraps
  binary-sign-tool sign -selfSign 1. Pure-data #sign_command is
  testable without invoking the tool.
- lib/libpng/ohos/recipe.rb: OHOS::Recipe < Libpng::Recipe. Orchestrates
  the OHOS build: download NDK, build static zlib, configure libpng
  with ohos.toolchain.cmake + static zlib, sign the .so post-install.

- lib/libpng/recipe.rb: adds Recipe.for_target(platform) class method
  factory. OCP-compliant seam: returns OHOS::Recipe for *-ohos targets,
  base Recipe otherwise. Existing recipe logic unchanged.
- lib/libpng.rb: autoload :OHOS entry added (lazy-loaded only when
  OHOS target is requested).
- ext/extconf.rb: switches to Recipe.for_target(...).new so the
  source gem (no target_platform) still uses base Recipe.

Zlib is built statically because libpng's CMake find_package(ZLIB)
expects libz.so and OHOS ships zlib under SONAME libshared_libz.z.so.
Static linking sidesteps the whole problem.

Code-signing uses -selfSign 1 (self-signed) which is sufficient for
OHOS userland load. Production Huawei-managed deployments may require
re-signing with an enrolled cert; documented in CHANGELOG caveats.

Architecture decision: run on ubuntu-24.04-arm (native arm64) with
binfmt_misc + qemu-user-static. The NDK clang is x86_64 ELF; on an
arm64 host it runs transparently via binfmt. Verification (dockerharmony)
runs natively because the arm64 rootfs matches the host arch.

49 new specs under spec/ohos/ + spec/recipe_factory_spec.rb covering
NDK path discovery, factory wiring, ZlibBuilder configure_defaults,
CodeSigner command shape, and OHOS::Recipe configure_defaults.
No doubles used (real instances + tmpdir-based fake NDK structure).
ronaldtse added a commit that referenced this pull request Jul 26, 2026
- lib/libpng/version.rb: LIBPNG_RUBY_ITERATION 5 -> 6.
- CHANGELOG.md: new 1.6.58.6 entry describing the OHOS NDK cross-compile
  (toolchain, static zlib, signing, dockerharmony verify) and listing
  PR #11 and #12 as superseded.
- README.adoc: aarch64-linux-ohos row now reflects the actual build host
  (ubuntu-24.04-arm + OHOS NDK, verified in dockerharmony) instead of
  the misleading "Alpine, byte-compatible with musl".
- CLAUDE.md: file table extended with lib/libpng/ohos/ + ext/ohos/;
  Common commands section now has an "OHOS build notes" subsection
  explaining the binfmt_misc + qemu-user-static setup and that local
  builds require arm64 + binfmt registration.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant