wclayer: fix sparse file logical size - #2926
Open
Dawei Wei (rzlink) wants to merge 1 commit into
Open
Conversation
Derive a sparse file's logical size from sparse block streams when BackupData has no inline data. Add regression coverage for normal and sparse files. Signed-off-by: Dawei Wei <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Prevent Windows container layer export from writing an all-hole sparse file as a zero-byte tar entry, which truncates the file's contents on export.
A sparse file can have a nonzero logical size without allocated data—for example, a 1 MiB file containing only zeroes. In this case the first
BackupDatastream reports size zero even though the file's logical EOF is 1 MiB.findBackupStreamSizepreviously used thatBackupDatasize directly. The resulting zero size was passed to the tar writer, so the exported entry was treated as empty and the file was restored truncated.Keep the existing fast path for non-sparse files and sparse files with inline data. For a sparse
BackupDatastream with no inline data, derive the logical size fromBackupSparseBlockrecords instead: the terminal zero-length block'soffset marks logical EOF, and allocated blocks satisfy
offset+size <= logical size, so the maximum end position is the logical size.Add regression coverage for regular files and sparse files with empty, all-hole, single-range, and multi-range layouts.
Context
Reported downstream in Docker/Moby as moby/moby#53464: building or committing a Windows image whose layer contains an NTFS sparse file fails.
That failure has two independent causes in two libraries:
0, truncating its contents on export.BackupSparseBlockrecords withunknown stream ID 9.The two fixes are complementary. This change touches only
findBackupStreamSizeand uses existing go-winio APIs, so it builds, tests, and merges independently of go-winio#375. Once that PR lands and is tagged, a follow-up will bump the go-winio dependency here so a tagged hcsshim release carries the complete end-to-end fix.Testing
go test ./internal/wclayer