process_dedupes() in dedupe.c advances by info->bytes_deduped and re-queues the request whenever the ioctl succeeded and req_total has not yet reached orig_len:
req->req_loff += info->bytes_deduped;
req->req_total += info->bytes_deduped;
if (info->status || req->req_total >= ctxt->orig_len) {
... completed
} else {
/* put us back on the queued list for another go around */If a successful FIDEDUPERANGE reports zero bytes deduped, that re-queues a request with identical parameters and dedupe_extents() never terminates.
duperemove does not hit this today, and I have not seen it hang. set_aligned_same_length() rounds src_length down to fs_blocksize, and I traced a -dr run and a --dedupe-options=partial -b 4096 run over a mixed corpus: every request was block-aligned, or was a whole-file dedupe whose range reaches EOF. So this is defence in depth, not a live bug report.
It is worth guarding anyway, because the input is real rather than hypothetical.
The kernel side
vfs_dedupe_file_range() reports the requested length rather than the amount actually deduplicated. generic_remap_check_len() silently rounds an unaligned length down to a block multiple when the destination range is not at EOF, and the loop then records len where deduped holds the real figure.
That means a sub-block request away from EOF currently comes back as "success, N bytes" having deduplicated nothing at all. On btrfs, stock v7.2-rc5:
requested=2048 bytes_deduped=2048 status=0
requested=1 bytes_deduped=1 status=0
The kernel reports deduplicating one byte, on a filesystem with 4096-byte granularity.
This was fixed once — 4a57a8400075 — and reverted the next day in b926f2adb044 because generic/517 asserts the buggy figure as its expected output. Matthias Goergens has a revival of that fix in the works.
With the fix applied (same tree, same corpus, btrfs in a VM), those become:
requested=2048 bytes_deduped=0 status=0
requested=1 bytes_deduped=0 status=0
and a plain rmlint --dedupe small.bin big.bin, where big begins with small's content, elicits it in ordinary use on its second call:
src_length=100000 => bytes_deduped=98304, status=0
src_offset=98304, len=1696 => bytes_deduped=0, status=0
rmlint survives because it happens to guard against zero. duperemove would re-queue that request forever.
Why fix it here regardless
Even once the kernel change lands, duperemove will keep running on kernels that do not have it — every LTS since 4.19 over-reports. A one-condition guard makes duperemove correct on both old and new kernels, and means it is not the reason the kernel side stays reverted.
PR follows.
process_dedupes()indedupe.cadvances byinfo->bytes_dedupedand re-queues the request whenever the ioctl succeeded andreq_totalhas not yet reachedorig_len:If a successful
FIDEDUPERANGEreports zero bytes deduped, that re-queues a request with identical parameters anddedupe_extents()never terminates.duperemove does not hit this today, and I have not seen it hang.
set_aligned_same_length()roundssrc_lengthdown tofs_blocksize, and I traced a-drrun and a--dedupe-options=partial -b 4096run over a mixed corpus: every request was block-aligned, or was a whole-file dedupe whose range reaches EOF. So this is defence in depth, not a live bug report.It is worth guarding anyway, because the input is real rather than hypothetical.
The kernel side
vfs_dedupe_file_range()reports the requested length rather than the amount actually deduplicated.generic_remap_check_len()silently rounds an unaligned length down to a block multiple when the destination range is not at EOF, and the loop then recordslenwherededupedholds the real figure.That means a sub-block request away from EOF currently comes back as "success, N bytes" having deduplicated nothing at all. On btrfs, stock v7.2-rc5:
The kernel reports deduplicating one byte, on a filesystem with 4096-byte granularity.
This was fixed once —
4a57a8400075— and reverted the next day inb926f2adb044becausegeneric/517asserts the buggy figure as its expected output. Matthias Goergens has a revival of that fix in the works.With the fix applied (same tree, same corpus, btrfs in a VM), those become:
and a plain
rmlint --dedupe small.bin big.bin, wherebigbegins withsmall's content, elicits it in ordinary use on its second call:rmlint survives because it happens to guard against zero. duperemove would re-queue that request forever.
Why fix it here regardless
Even once the kernel change lands, duperemove will keep running on kernels that do not have it — every LTS since 4.19 over-reports. A one-condition guard makes duperemove correct on both old and new kernels, and means it is not the reason the kernel side stays reverted.
PR follows.