firecracker: serve fork mem-file via per-template uffd page server#219
Draft
sjmiller609 wants to merge 3 commits into
Draft
firecracker: serve fork mem-file via per-template uffd page server#219sjmiller609 wants to merge 3 commits into
sjmiller609 wants to merge 3 commits into
Conversation
Wires firecracker's snapshot restore through a userfaultfd-backed page server when the fork descends from a template. Each template gets one uffd.Server lazily started on the first fork and torn down once the last fork is released, so non-template forks (the symlink-only path) are unaffected. Adds an E2E test that boots a firecracker source, standbys it, promotes it to a template, forks against the template, and asserts: fork reaches Running, mem-file is a symlink to the template's, refcount bumps to 1, the per-template uffd tracker registers the fork, and on fork delete the refcount drops and the tracker detaches. Also adds unit coverage for the uffd tracker lifecycle (lazy start, multi-fork share, last-release teardown, closeAll, empty-acquire rejection).
Unix domain socket paths cap at 108 bytes (sun_path). Putting the per- fork socket under <DataDir>/templates/<25-char-cuid>/uffd/<25-char- cuid>.uffd blew that limit on CI runners where t.TempDir() returns long prefixes, surfacing as "bind: invalid argument" in TestFirecrackerForkFromTemplate. Sockets are ephemeral, so anchoring them at /tmp/h-uffd/<templateID>/ keeps the path well under the limit regardless of how deep DataDir is. The tracker now also rm -rfs the per-template socket dir on the last release so we don't leak stale entries.
Firecracker enables snapshot base reuse, which renames the post-restore snapshot dir from snapshot-latest to snapshot-base. The hardlink survives the rename (same inode), so the test just needs the right path.
sjmiller609
force-pushed
the
hypeship/uffd-prefetch-hotpages
branch
from
21bdfb5 to
579a72e
Compare
sjmiller609
force-pushed
the
hypeship/uffd-firecracker-wiring
branch
from
030b69e to
0c36cb3
Compare
sjmiller609
force-pushed
the
hypeship/uffd-prefetch-hotpages
branch
from
579a72e to
df782fa
Compare
sjmiller609
force-pushed
the
hypeship/uffd-firecracker-wiring
branch
from
0c36cb3 to
cc003a7
Compare
sjmiller609
force-pushed
the
hypeship/uffd-prefetch-hotpages
branch
from
df782fa to
f141f3c
Compare
sjmiller609
force-pushed
the
hypeship/uffd-firecracker-wiring
branch
from
cc003a7 to
49a5f5b
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
uffd.Serverinstead of mmaping the mem-file directly.uffd.Serverper template on the first fork acquire and tears it down once the last fork is released. Non-template forks (the symlink-only path from PR fork: share template mem-file via symlink for firecracker fan-out #214) are unaffected.UffdSocketPaththroughForkPrepareRequest→ firecrackerrestoreMetadata→snapshotLoadParams.MemBackend({ backend_type: "Uffd", backend_path: <UDS> }), withmem_file_pathcleared when a backend is set.DeleteInstancefor forks (best-effort warn on error).Test plan
go vet ./...cleango build ./...cleango test ./lib/instances/... -run TestUffdTracker -count=1TestFirecrackerForkFromTemplateboots a firecracker source, standbys → promotes → forks → asserts:ForkCountbumps to 1ForkCountand detaches from uffdconfig_test.goupdated for uffd backend assertionsStacked on #218.
🤖 Generated with Claude Code