Stop embedding a boot script in the EFI binaries

.github/workflows/build.yml

@@ -69,7 +69,7 @@ jobs:
         # One tarball, not 85 loose assets. Release assets cannot contain
         # directories, so publishing them individually would mean mangling
         # every path into a flat name and teaching the installer to unmangle
-        # it. The tree has structure -- i386-efi/, arm64-efi/, autoexec/ --
+        # it. The tree has structure -- i386-efi/, arm64-efi/, 10secdelay/ --
         # and a tarball preserves it for free. A single checksum then covers
         # every file, rather than 85 chances to get one verification wrong.
         run: |

README.md

@@ -17,7 +17,8 @@ The FOG installer downloads these binaries from a release, checksums them, and d
 ```
 src/            BIOS build   — config headers + embedded boot scripts
 src-efi/        EFI build    — same, with EFI-specific feature selection
-autoexec.ipxe   boot script for the EMBED-less EFI binaries
+                               (config headers only; EFI embeds no script)
+autoexec.ipxe   the boot script every EFI binary downloads and runs
 buildipxe.sh    the build
 secureboot/     pinned upstream signed binaries — republished, not built
 ```
@@ -41,11 +42,10 @@ Both arguments optional; defaults are the FOG CA if one is present and `./output
 The output tree mirrors FOG's `packages/tftp/` layout exactly, so it can be copied over a TFTP root unchanged:
 
 ```
-output/                 BIOS + x86_64 EFI, boot script embedded
-output/i386-efi/        32-bit EFI
-output/arm64-efi/       arm64 EFI
-output/10secdelay/      as above, with a 10 second pre-DHCP sleep
-output/autoexec/        EFI with NO embedded script — reads autoexec.ipxe
+output/                 BIOS (script embedded) + x86_64 EFI + autoexec.ipxe
+output/i386-efi/        32-bit EFI + autoexec.ipxe
+output/arm64-efi/       arm64 EFI + autoexec.ipxe
+output/10secdelay/      BIOS only, with a 10 second pre-DHCP sleep
 ```
 
 Requirements: `git`, `make`, `gcc`, `binutils`, `perl`, `liblzma`, `mtools`, `xorriso`, and `gcc-aarch64-linux-gnu` for the arm64 binaries.
@@ -58,13 +58,31 @@ The upstream clone is pinned to a tag. Bumping it is a deliberate act — FOG sp
 IPXEVER=v2.0.1 ./buildipxe.sh
 ```
 
-### The `autoexec/` binaries
-
-Built without `EMBED=`, so they read their boot script from `autoexec.ipxe` on the TFTP server instead of having it compiled in. Changing the boot logic stops requiring a rebuild — which is also why there is no `10secdelay` equivalent of them, the delay being a two-line edit.
-
-They are **opt-in**: point your DHCP `filename` at `autoexec/…`. A binary that finds no `autoexec.ipxe` falls through to plain `netboot()` and loses FOG's multi-NIC and proxyDHCP handling, so they must not be served to clients until the file is in place.
-
-This is also the only variant that can work under UEFI Secure Boot, since an embedded script is not permitted in a Secure Boot build. See the [Secure Boot how-to](https://docs.fogproject.org/kb/how-tos/secure-boot-signing/).
+### Boot scripts: EFI reads `autoexec.ipxe`, BIOS embeds one
+
+No EFI binary here is built with `EMBED=`. Each one downloads `autoexec.ipxe`
+from the directory it was itself loaded from — falling back to the TFTP root —
+and executes it, because with nothing compiled in `first_image()` finds no
+image ahead of it. Changing the boot logic is therefore editing one text file,
+with no toolchain and no rebuild. It is also the only shape that works under
+UEFI Secure Boot, since an embedded script is not permitted in a Secure Boot
+build. See the [Secure Boot how-to](https://docs.fogproject.org/kb/how-tos/secure-boot-signing/).
+
+A copy of `autoexec.ipxe` ships in every directory holding an EFI binary. FOG's
+installer hard-links them so there is exactly one script however many paths
+reach it.
+
+**Do not put an `EMBED=` binary in a directory an EMBED-less one can fall back
+to.** An embedded binary still *downloads* `autoexec.ipxe` — `efi_probe()`
+registers it before any driver is connected — but never runs it, so nothing
+unregisters it and `initrd_load_all()` concatenates it into the ramdisk ahead
+of `init.xz`. The kernel then panics on the missing compression magic. This is
+why `EMBED=` is gone from every EFI target rather than most of them.
+
+Legacy BIOS is the exception and keeps `EMBED=`: it has no
+`efi_autoexec_load()`, so there is no downloaded script for it to read. That is
+also why `10secdelay/` still exists and now holds BIOS files only — on EFI the
+delay is an installer option that inserts a `sleep` into `autoexec.ipxe`.
 
 ## Secure Boot
 

autoexec.ipxe

@@ -1,18 +1,27 @@
 #!ipxe
-# Boot script for the EMBED-less EFI binaries in packages/tftp/autoexec/.
+# The boot script for every EFI binary FOG ships.
 #
-# iPXE 2.0.0's efi_autoexec_load() downloads "autoexec.ipxe" from the current
-# working URI -- i.e. the TFTP directory the .efi binary itself was loaded from
-# -- before any driver is connected, and runs it because first_image() then
-# finds no embedded image ahead of it. That is the whole point of this variant:
-# the boot logic lives in a text file the server can rewrite, instead of being
-# compiled into every binary. Changing the sleep below no longer needs a
-# rebuild, which is why there is no autoexec equivalent of 10secdelay/.
+# iPXE's efi_autoexec_load() downloads "autoexec.ipxe" from the current working
+# URI -- the TFTP directory the .efi binary itself was loaded from -- before any
+# driver is connected, and runs it because first_image() finds no embedded image
+# ahead of it. None of FOG's EFI binaries embed one, deliberately, so this file
+# is not a variant any more: it is how they boot. A site changes its boot logic
+# by editing this file, with no toolchain and no rebuild.
 #
-# Keep this in step with src/ipxe/src/ipxescript -- the binaries that embed
-# that script and the ones that fetch this one must behave identically, or
-# which file a site is using becomes a variable in every bug report. Refs
-# GH-957.
+# The installer hard-links one copy into every directory holding an EFI binary
+# -- the TFTP root, i386-efi/, arm64-efi/ and the secureboot/ tree -- so there
+# is exactly one script however many paths reach it. efi_autoexec_network()
+# looks in the binary's own directory first and only then falls back to
+# /autoexec.ipxe, so the per-directory copies just save a failed request.
+#
+# The 10-second STP/powersave delay is an installer option (--boot-delay) that
+# inserts a sleep below, not a second set of binaries. Legacy BIOS still needs
+# its own build for that -- 10secdelay/ -- because BIOS has no
+# efi_autoexec_load() and so has no script to edit.
+#
+# Keep this in step with src/ipxescript, the script the BIOS binaries embed:
+# the two must behave identically, or which platform a site booted becomes a
+# variable in every bug report. Refs GH-957.
 echo Checking net0 for DHCP...
 isset ${net0/mac} && ifopen net0 && dhcp net0 || goto dhcpnet1
 echo Received DHCP answer on interface net0 && goto proxycheck

buildipxe.sh

@@ -84,7 +84,7 @@ apply_fog_patches() {
 
 # The output tree is emitted in exactly fogproject's packages/tftp layout, so
 # the installer can copy it over its tftpdir unchanged.
-mkdir -p "$BASE" ${OUTDIR}/{10secdelay/{i386-efi,arm64-efi},i386-efi,arm64-efi,autoexec/{i386-efi,arm64-efi}}
+mkdir -p "$BASE" ${OUTDIR}/{10secdelay,i386-efi,arm64-efi}
 
 if [[ -d ${BASE}/ipxe ]]; then
   cd ${BASE}/ipxe
@@ -160,8 +160,6 @@ apply_fog_patches ${BASE}/ipxe-efi
 
 # Overlay this repository's headers and boot scripts onto the clone.
 echo "Copy (overwrite) iPXE headers and scripts..."
-cp ${FOGDIR}/src-efi/ipxescript .
-cp ${FOGDIR}/src-efi/ipxescript10sec .
 cp ${FOGDIR}/src-efi/config/general.h config/
 cp ${FOGDIR}/src-efi/config/settings.h config/
 cp ${FOGDIR}/src-efi/config/console.h config/
@@ -172,60 +170,59 @@ cp ${FOGDIR}/src-efi/config/console.h config/
 mkdir -p config/local
 cp ${FOGDIR}/src-efi/config/local/usb.h config/local/
 
-# Build the files
-make -j$(nproc) EMBED=ipxescript bin-{i386,x86_64}-efi/{snp{,only},ipxe,intel,realtek}.efi ${BUILDOPTS}
+# Build the EFI binaries. One pass, no EMBED.
+#
+# WHY NO EMBED HERE
+#
+# With no script compiled in, first_image() finds nothing at INIT_LATE, so
+# efi_probe()'s efi_autoexec_load() gets to register autoexec.ipxe and ipxe()
+# executes that instead. Three things follow, and they are the whole reason
+# this is the only EFI build now:
+#
+#   1. A site can change its boot logic without a toolchain. The script is a
+#      file on the TFTP server, not bytes inside 15 binaries.
+#
+#   2. It is the only shape that works under Secure Boot: efi_autoexec.c is
+#      FILE_SECBOOT ( PERMITTED ), an embedded script is not.
+#
+#   3. It is the only shape that can safely share a directory with
+#      autoexec.ipxe. An EMBED-marked binary still DOWNLOADS the script --
+#      efi_probe() registers it unconditionally -- but never executes it,
+#      because first_image() returns the embedded one ahead of it. Nothing
+#      unregisters it, so initrd_load_all() concatenates it into the ramdisk
+#      ahead of init.xz and the kernel panics on the missing compression
+#      magic. An EMBED-less binary EXECUTES the script, and image_exec()
+#      unregisters it for the duration, so it is gone by the time boot runs.
+#
+# (3) is why the previous layout had to keep the EMBED-less binaries in a
+# separate autoexec/ directory and delete any autoexec.ipxe from the TFTP
+# root: efi_autoexec_network() falls back to /autoexec.ipxe when the
+# binary's own directory has none, so ONE embedded EFI binary anywhere in the
+# tree was enough to poison every client that fell back to the root. Removing
+# EMBED from every EFI target removes that constraint, which is what lets the
+# autoexec/ duplicate tree go away and autoexec.ipxe become the normal
+# mechanism rather than an opt-in.
+#
+# The 10-second delay variant goes with it. It differed from the default by
+# two lines -- an echo and a sleep -- which is now an edit the installer makes
+# to autoexec.ipxe rather than a second copy of every binary. 10secdelay/
+# keeps its BIOS files, which genuinely do need a separate build because BIOS
+# has no efi_autoexec_load() and therefore no script to edit.
+make -j$(nproc) bin-{i386,x86_64}-efi/{snp{,only},ipxe,intel,realtek}.efi ${BUILDOPTS}
 [[ $? -eq 0 ]] || exit 80
 
-# Apply USB configuration for ARM64 build
-make -j$(nproc) CROSS_COMPILE=aarch64-linux-gnu- ARCH=arm64 EMBED=ipxescript bin-arm64-efi/{snp{,only},ipxe,intel,realtek}.efi ${BUILDOPTS}
+make -j$(nproc) CROSS_COMPILE=aarch64-linux-gnu- ARCH=arm64 bin-arm64-efi/{snp{,only},ipxe,intel,realtek}.efi ${BUILDOPTS}
 [[ $? -eq 0 ]] || exit 82
 
 # Collect into the output tree
 cp bin-arm64-efi/{snp{,only},ipxe,intel,realtek}.efi ${OUTDIR}/arm64-efi/
 cp bin-i386-efi/{snp{,only},ipxe,intel,realtek}.efi ${OUTDIR}/i386-efi/
 cp bin-x86_64-efi/{snp{,only},ipxe,intel,realtek}.efi ${OUTDIR}/
 
-# Build with 10 second delay
-make -j$(nproc) EMBED=ipxescript10sec bin-{i386,x86_64}-efi/{snp{,only},ipxe,intel,realtek}.efi ${BUILDOPTS}
-[[ $? -eq 0 ]] || exit 91
-
-make -j$(nproc) CROSS_COMPILE=aarch64-linux-gnu- ARCH=arm64 EMBED=ipxescript10sec bin-arm64-efi/{snp{,only},ipxe,intel,realtek}.efi ${BUILDOPTS}
-[[ $? -eq 0 ]] || exit 93
-
-# Collect into the output tree
-cp bin-arm64-efi/{snp{,only},ipxe,intel,realtek}.efi ${OUTDIR}/10secdelay/arm64-efi/
-cp bin-i386-efi/{snp{,only},ipxe,intel,realtek}.efi ${OUTDIR}/10secdelay/i386-efi/
-cp bin-x86_64-efi/{snp{,only},ipxe,intel,realtek}.efi ${OUTDIR}/10secdelay/
-
-# Build the EMBED-less EFI variant.
-#
-# With no embedded script, first_image() finds nothing at INIT_LATE, so
-# efi_probe()'s efi_autoexec_load() gets to register autoexec.ipxe and ipxe()
-# executes that instead. The script is then a file on the TFTP server rather
-# than something compiled in, so a site can change its boot logic without a
-# toolchain. This is also the only build that can work under Secure Boot, since
-# efi_autoexec.c is FILE_SECBOOT ( PERMITTED ) while an embedded script is not.
-#
-# Shipped alongside the embedded binaries rather than replacing them: an
-# existing server has no autoexec.ipxe in its TFTP root, and a binary that
-# finds none falls through to plain netboot(), losing FOG's multi-NIC and
-# proxyDHCP handling. Opting in is a DHCP filename change. Refs GH-957.
-#
-# There is deliberately no 10secdelay counterpart -- with the script on disk,
-# the delay is a two-line edit to autoexec.ipxe.
-make -j$(nproc) bin-{i386,x86_64}-efi/{snp{,only},ipxe,intel,realtek}.efi ${BUILDOPTS}
-[[ $? -eq 0 ]] || exit 95
-
-make -j$(nproc) CROSS_COMPILE=aarch64-linux-gnu- ARCH=arm64 bin-arm64-efi/{snp{,only},ipxe,intel,realtek}.efi ${BUILDOPTS}
-[[ $? -eq 0 ]] || exit 97
-
-cp bin-arm64-efi/{snp{,only},ipxe,intel,realtek}.efi ${OUTDIR}/autoexec/arm64-efi/
-cp bin-i386-efi/{snp{,only},ipxe,intel,realtek}.efi ${OUTDIR}/autoexec/i386-efi/
-cp bin-x86_64-efi/{snp{,only},ipxe,intel,realtek}.efi ${OUTDIR}/autoexec/
-
-# One copy per directory: efi_autoexec_network() asks for autoexec.ipxe
-# relative to the binary's own URI first and only then retries at the TFTP
-# root, so a per-directory copy saves a failed request on every boot.
-for d in autoexec autoexec/i386-efi autoexec/arm64-efi; do
+# One copy per directory holding an EFI binary. efi_autoexec_network() asks for
+# autoexec.ipxe relative to the binary's own URI first and only then retries at
+# the TFTP root, so a per-directory copy saves a failed request on every boot.
+# The installer hard-links these together afterwards so they cannot drift.
+for d in . i386-efi arm64-efi; do
   cp ${FOGDIR}/autoexec.ipxe ${OUTDIR}/$d/
 done

tools/check-linked-objects.sh

@@ -75,11 +75,13 @@ BASELINEDIR="${FOGDIR}/tools/linked-objects"
 # separate headers (src/config vs src-efi/config), and arm64 is a separate
 # compile of the EFI tree whose object set differs by architecture.
 #
-# buildipxe.sh builds the EFI targets three times over (EMBED=ipxescript, then
-# ipxescript10sec, then EMBED-less for autoexec), so these maps are from its
-# last pass. That is fine and stable: EMBED changes which script bytes land in
-# embedded.o, not which objects the linker pulls in, and config.o references
-# obj_embedded either way.
+# The EFI targets are built once, EMBED-less -- see the long note in
+# buildipxe.sh. The BIOS targets are still built twice (EMBED=ipxescript, then
+# ipxescript10sec) so undionly.kkpxe's map is from that second pass. That is
+# fine and stable: EMBED changes which script bytes land in embedded.o, not
+# which objects the linker pulls in, and config.o references obj_embedded
+# either way -- which is also why dropping EMBED from the EFI builds did not
+# move these baselines.
 TARGETS=(
     "undionly.kkpxe:${BASE}/ipxe/src/bin/undionly.kkpxe.tmp.map"
     "x86_64-efi-snponly.efi:${BASE}/ipxe-efi/src/bin-x86_64-efi/snponly.efi.tmp.map"