Skip to content

file_scan: handle sparse files with a trailing hole (#374) - #411

Open
martinus wants to merge 1 commit into
markfasheh:masterfrom
martinus:backport/sparse-trailing-hole
Open

file_scan: handle sparse files with a trailing hole (#374)#411
martinus wants to merge 1 commit into
markfasheh:masterfrom
martinus:backport/sparse-trailing-hole

Conversation

@martinus

Copy link
Copy Markdown

What this fixes

A file whose data ends before EOF (data followed by a hole up to end-of-file) has a last mapped extent that ends before filesize; FIEMAP never reports the trailing hole. process_extents() then broke in two ways once file_off reached the hole:

  • dummy = ext_end_off - filesize underflowed (size_t), because the last extent ends before filesize, so file_off + dummy == ext_end_off never held and the last real extent was never stored;
  • the next get_extent() returned NULL, which printed process_extents: unable to get extent and returned 1, so the caller declared the file "changed" and abandoned it.

What happens without the fix

Any file with a trailing hole is silently never hashed or deduped — it's skipped every run.

The fix

Only subtract the past-EOF overshoot when the extent actually runs past filesize, and treat a NULL from get_extent() as "past the last extent = trailing hole, stop cleanly" rather than an error. Verified that trailing-, middle- and leading-hole files all scan and dedupe with data preserved byte-for-byte.

(Originally filed as broken compressed-FS handling, but compression maps fine on current kernels — logical offsets/lengths with the ENCODED flag — the real trigger is the trailing hole.)

Fixes: #374

A file whose data ends before EOF (data followed by a hole) has a last
mapped extent that ends before filesize; FIEMAP never reports the hole.
process_extents() then broke in two ways once file_off entered the hole:

- dummy = ext_end_off - filesize underflowed (size_t) because the last
  extent ends *before* filesize, so `file_off + dummy == ext_end_off`
  never held and the last real extent was never stored;
- the next get_extent() returned NULL, which printed "unable to get
  extent" and returned 1, so the caller declared the file "changed" and
  abandoned it - the file was never hashed or deduped.

Only subtract the past-EOF overshoot when the extent actually runs past
filesize, and treat a NULL from get_extent() as "past the last extent =
trailing hole, stop cleanly" rather than an error.

Fixes: markfasheh#374
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Extent handling for compressed filesystems is broken

1 participant