sneak preview: uv, Click, Rich, tui; flavors

[rpardini]

sneak preview: uv, Click, Rich, tui; flavors; Secure Boot?

• 🌳 captain: fix .isEnabledFor(logging.DEBUG) calls
• 🌴 captain: force type console in init.py
• 🍃 captain: catch required arg and loosen type (WiP: other way around.)
• 🍀 captain: util: Rich rule's separating run() output
• 🐸 captain: better logging; in-docker: prefix
• 🌿 captain: add .editorconfig matching ruff
  • .editorconfig for shell scripts and shell templates
• 🌱 docker/mkosi: mount a Docker Volume at /work, so mkosi can cache the tools tree across invocations
• 🌵 util: use a Panel to log the full subprocess command line
• 🌳 captain: introduce Config build_kernel boolean
  • this will allow using a generic/distro kernel instead of building one
• 🌴 WiP: use linux-image-generic distro kernel
  • hack until it uses the mkosi-supplied kernel for everything
  • artifacts: look for mkosi-supplied vmlinuz first
• 🍃 gha: skip all kernel stuff
  • self-built kernel from source will come back at a later stage, as a .deb package
• 🍀 gha: single matrix except for combined amd64/arm64 image at the end
  • just do everything at once
• 🐸 mkosi.conf: just use tiny-initramfs instead of blocking the others (dracut/initramfs-tools)
• 🌿 mkosi.finalize: export DTBs for arches that have them
  • also: show info about modules and full rootfs
• 🌱 artifacts: collect dtb mkosi.output folder into out/
• 🌵 publish: publish DTBs in OCI image as a directory
• 🌳 kernel: drop all kernel-build related code
  • later to be reborn as standard apt package
• 🌴 mkosi.finalize: drop kernel sound/media/net-wireless modules
  • mkosi.finalize: clean up logging a bit
  • mkosi.postinst: debugs
• 🍃 gha: introduce FLAVOR_ID (nee KERNEL_VERSION) gha: fix
• 🍀 release: include cfg.kernel_version as suffix to all tags
• 🐸 gha: bump external actions so GHA stops complaining about Node 24
• 🌿 gha: pass DEFAULT_FLAVOR_ID as KERNEL_VERSION for publish-combined gha: pass DEFAULT_FLAVOR_ID as KERNEL_VERSION for publish-combined - retry
• 🌱 captain/gha: KERNEL_VERSION (et al) is now FLAVOR_ID
  • --flavor-id
• 🌵 captain: cleanups / doc updates WiP (drop kernel build-related)
• 🌳 captain: DEFAULT_FLAVOR_ID = "trixie-full"
• 🌴 flavors: introduce flavors but logging sucks and no dataclass better, logging still sucks for Full -> Common much better in-package this sucks less cosmetics some static and template rendering and hardcoded cleanup kinda-works kinda works
• 🍃 gha: add trixie-rockchip64 flavor
• 🍀 stages: show what tools_mode is running
• 🐸 gha: pass --arch to build.py build
• 🌿 common_debian: add 01nopty with Dpkg::Use-Pty "0"; to mkosi sandbox tree
• 🌱 captain: add support for mkosi skeleton tree
  • gha: still trying to fix dpkg/apt output in gha
  • this went nowhere, but hindsight is 20/20
• 🌵 common_debian: common bash header with logging, sprinkle some dust on postinst and finalize
• 🌳 mkosi.conf: output a JSON manifest
• 🌴 mkosi.conf: force tools tree to be Debian Trixie
  • mkosi defaults to 'testing'
• 🍃 mkosi.conf: use sandbox tree also for tools tree
• 🍀 flavor: ensure flavor supports requested architecture
• 🐸 flavors: cleanup a bit
• 🌿 flavors/gha: introduce meson64 flavor
• 🌱 docker: never interactive, never a terminal
• 🌵 cli/flavor: pass Flavor down to build and initramfs cmds; introduce BaseFlavor::has_iso()
  • so only the fully UEFI/ACPI flavor gets ISOs by default
  • they don't make sense for DeviceTree flavors
• 🌳 gha/docker: set and pass down FORCE_COLOR=1
• 🌴 captain: don't shorten out logging record name
• 🍃 buildah: take env BUILDAH_INSECURE=1 for --tls-verify=false
  • so one can use an HTTP local registry:2 for testing pushes
• 🍀 release: pass flavor down to release, avoid releasing iso for !has_iso
• 🐸 cli: new cli under click_cli
  • add to pyproject.toml, thus uv run captain ...
• 🌿 click: add tools command
  • using captain.cli._stages
• 🌱 click: some sugar, list available flavors via reflection
• 🌵 docker: build_builder(): more info
• 🌳 click: kill old tools cli (keep stage) click: kill old cli completely
• 🌴 click: add iso command back
  • as it is called internally via docker
• 🍃 gha: switch to click cli
• 🍀 gha: release-publish
• 🐸 captain: rework obtain_builder
• 🌿 gha: separate build-dockerfile arch-based matrix job
• 🌱 captain: adapt to options moved to common
• 🌵 captain: simplify Rich logging and trace handler captain: simplify Rich logging and trace handler; let's not have two click packages
• 🌳 gha: force tools to run native
• 🌴 docker/Dockerfile: consolidate single Dockerfile; split and balance layers to optimize for parallel pulls
• 🍃 docker: rework re-launching inside docker & docker envs
• 🍀 docker/mkosi: get rid of tools tree; back to Debian's trixie system Python + pip for mkosi
  • mkosi re-launches itself and ignores uv's env without a tools tree
  • uv instead does use the system Python if it matches constraints
  • build in /work in Dockerfile, allows to reuse .venv
  • don't mount a volume in /work, instead use the one provided by the Dockerfile
• 🐸 captain: honor FORCE_COLOR=1 for internal logging
• 🌿 flavors: tighten semantics, refactor
  • refactor add_static_dir() into BaseFlavor
  • make has_iso() abstractmethod at BaseFlavor level
  • make kernel_packages() abstractmethod at common-debian level
• 🌱 flavors: introduce common-acpi, move acpi/impi stuff there
  • so it's common-debian > common-acpi > trixie-full
  • in the future: common-debian > common-acpi > trixie-mainline
  • common-acpi implements has_iso() as True
• 🌵 click: introduce CliContext and reuse common options via @click.Group
  • CliContext is a Config factory, taking both common and specific options
  • actually implement --verbose, thus new default logging level is INFO
• 🌳 captain: make Rich richer
  • show Locals
  • show all frames in traces
  • use Highlights for FORCE_COLOR=1
• 🌴 captain: introduce Trogon(/Textual) to auto-create a TUI from Click
  • See https://github.com/Textualize/trogon and https://github.com/Textualize/textual
  • this cashes out on the Click / Rich investment
• 🍃 gha: use envvars, not --options
  • reduce suffering while I juggle Click
• 🍀 captain: even Richer Rich
  • uv: split Config::verbose_uv from --verbose/Logging.DEBUG; use --quiet otherwise
  • use shutil.get_terminal_size to obtain and pass-down-Docker COLUMNS
  • nicer logging format, use a whale emoji for in-Docker logs
  • show Rich Table with Docker environment vars if --verbose
  • show Rich Panel and Rich Syntax for util.run() if --verbose
• 🐸 gha: don't upload .iso as part of initramfs artifact
  • we've a separate artifact for .iso
• 🌿 flavor/gha: add trixie-rockchip64-vendor and trixie-armbian-rpi flavors
  • also build them
  • trixie-armbian-rpi strays from standard naming as I think we might need to build our own kernel for RPi to achieve small-enough-for-eeprom-netboot eventually
• 🌱 captain: tink-agent-setup Wants/After time-set.target
  • so time is set to something more reasonable, otherwise pulling fails with date-related TLS issues
• 🌵 captain: move systemd enablement/disablement from postinst to systemd preset files
  • doing this in postinst has no effect, as mkosi applies presets after postinst
  • it could be done in finalize, but that would un-do the presets applied by mkosi
  • so instead do it all in two preset files:
    • one for generic OS-related enablement and disablement (20-captainos-base)
    • one for tink-agent related stuff (10-captainos-tink)
• 🌳 captain: loosen systemd-networkd-wait-online to accept any interface and 15s timeout
  • otherwise hangs and delays on machines with multiple interfaces
• 🌴 captain: tink-agent-setup Wants/After time-sync.target
  • time-set.target is not enough for machines without an RTC and working battery
  • time-sync.target requires timesyncd to be fully sync'ed
  • so time is set to something more reasonable, otherwise pulling fails with date-related TLS issues
• 🍃 bring back "some" kernel stuff
• 🍀 6.18.y.arm64: defconfig
• 🐸 6.18.y.arm64: defconfig: CONFIG_MODULE_COMPRESS_ZSTD=y
• 🌿 util: show run()'s env vars in a Rich Table
• 🌱 docker: rework run_in_builder() with both command_and_args and extra_docker_args
  • also introduce cfg.verbose_docker for setting env CAPTAIN_VERBOSE=1 when relaunching
• 🌵 captain/cli: --verbose sets root level as well as current logger to DEBUG
• 🌳 cli: re-introduce shell subcommand
  • which runs bash interactively under Docker
  • using same envs / volumes / mounts as regular run_in_builder()
• 🌴 kernel: rework with kernel's own make bindeb-pkg into a .deb output
  • kernel packaging code already knows how to:
    • strip modules
    • zstd modules
    • handle Image name differences
    • handle depmod
    • produce a .deb package
  • but we gotta go 2-deep as .deb's are output in parent directory by kbuild
  • Dockerfile: add layers with deps:
    • rust compiler layer (inevitable after 7.0.y)
    • debhelper + etc needed for kernel deb packaging
• 🍃 gha: reintroduce kernel build jobs
  • using the Docker builder, so depend on that
  • abuse GHA caches and artifacts
  • the whole matrix will wait on kernel builds
    • but only a few will actually download and use the artifact
• 🍀 flavor: introduce BaseFlavor::pre_mkosi_stage()
  • so flavors can assert their requirements are built/met
• 🐸 mkosi: show Rich.Syntax'ed mkosi.conf before running mkosi
  • ... since we already paid the price of Pygments...
• 🌿 captain: allow building amd64 kernel on arm64
  • specify a CROSS_COMPILE for x86_64
• 🌱 common_debian: introduce package_directories() for mkosi's PackageDirectories=
  • this allows flavors to add .deb's in directories as a local apt repo
  • .deb's can then be installed by name
  • mkosi handles the repo creation and usage directly (via apt-utils)
• 🌵 builder: Dockerfile: add cross-arch libssl-dev and native apt-utils
  • cross libssl-dev needed for make bindeb-pkg to work cross-arch
  • apt-utils needed for mkosi's PackageDirectories=
• 🌳 cli: expose stages (nee _stages) module
  • this skews package protection to allow flavors to reuse code
  • also expose function build_kernel_stage()
• 🌴 flavors: introduce trixie-slim flavor, using captainos kernel
  • this one uses the captain-build-from-source kernel .deb
  • let's violate some package boundaries:
    • cli acrobatics: delegates to kernel build stage before initramfs build
    • impls package_directories() by calling kernel
• 🍃 gha: split matrix to add kernel job dependency for trixie-slim jobs
  • add gha builds for trixie-slim flavors (using captainos-kernel)
  • we've two kinds of build-captainos jobs:
    • ones that depend on kernel-build (trixie-slim)
    • ones that don't (all the others)
  • use YAML anchor trick to avoid duplicating 95% of code (envs: and steps: are defined only once)
  • matrix and needs (dependencies) are the only difference
  • artifact download is common but conditional on matrix values
• 🍀 gha: make trixie-slim the default flavor
  • so it is the one included in the combined images
  • this does not change the CLI defaults, ofc
• 🐸 gha: don't push OCI images for pull request workflows
  • (Docker) builder will end up being built multiple times for PR workflows, but such is life
    • it's also too huge for GHA artifact or GHA cache
  • don't push any captainos artifacts either (simple or combined)
    • also: only push those when building main branch