file_scan: handle sparse files with a trailing hole (#374) - #411
Open
martinus wants to merge 1 commit into
Open
Conversation
A file whose data ends before EOF (data followed by a hole) has a last mapped extent that ends before filesize; FIEMAP never reports the hole. process_extents() then broke in two ways once file_off entered the hole: - dummy = ext_end_off - filesize underflowed (size_t) because the last extent ends *before* filesize, so `file_off + dummy == ext_end_off` never held and the last real extent was never stored; - the next get_extent() returned NULL, which printed "unable to get extent" and returned 1, so the caller declared the file "changed" and abandoned it - the file was never hashed or deduped. Only subtract the past-EOF overshoot when the extent actually runs past filesize, and treat a NULL from get_extent() as "past the last extent = trailing hole, stop cleanly" rather than an error. Fixes: markfasheh#374 Co-Authored-By: Claude Opus 4.8 <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this fixes
A file whose data ends before EOF (data followed by a hole up to end-of-file) has a last mapped extent that ends before
filesize; FIEMAP never reports the trailing hole.process_extents()then broke in two ways oncefile_offreached the hole:dummy = ext_end_off - filesizeunderflowed (size_t), because the last extent ends beforefilesize, sofile_off + dummy == ext_end_offnever held and the last real extent was never stored;get_extent()returnedNULL, which printedprocess_extents: unable to get extentand returned 1, so the caller declared the file "changed" and abandoned it.What happens without the fix
Any file with a trailing hole is silently never hashed or deduped — it's skipped every run.
The fix
Only subtract the past-EOF overshoot when the extent actually runs past
filesize, and treat aNULLfromget_extent()as "past the last extent = trailing hole, stop cleanly" rather than an error. Verified that trailing-, middle- and leading-hole files all scan and dedupe with data preserved byte-for-byte.(Originally filed as broken compressed-FS handling, but compression maps fine on current kernels — logical offsets/lengths with the ENCODED flag — the real trigger is the trailing hole.)
Fixes: #374