block: add BIO_COMPLETE_IN_TASK for task-context completion#739
Open
blktests-ci[bot] wants to merge 8 commits intolinus-master_basefrom
Open
block: add BIO_COMPLETE_IN_TASK for task-context completion#739blktests-ci[bot] wants to merge 8 commits intolinus-master_basefrom
blktests-ci[bot] wants to merge 8 commits intolinus-master_basefrom
Conversation
Author
|
Upstream branch: d60bc14 |
blktests-ci
Bot
force-pushed
the
linus-master_base
branch
from
6b4d829 to
ceec5ed
Compare
Author
|
Upstream branch: b4e0758 |
blktests-ci
Bot
force-pushed
the
series/1079338=>linus-master
branch
from
aac4ed3 to
04b8b2f
Compare
blktests-ci
Bot
force-pushed
the
linus-master_base
branch
from
ceec5ed to
3b54e52
Compare
Author
|
Upstream branch: 6596a02 |
blktests-ci
Bot
force-pushed
the
series/1079338=>linus-master
branch
from
04b8b2f to
8aee906
Compare
blktests-ci
Bot
force-pushed
the
linus-master_base
branch
from
3b54e52 to
6a0b974
Compare
Author
|
Upstream branch: 507bd4b |
blktests-ci
Bot
force-pushed
the
series/1079338=>linus-master
branch
from
8aee906 to
906cfab
Compare
blktests-ci
Bot
force-pushed
the
linus-master_base
branch
from
6a0b974 to
59ca59b
Compare
Author
|
Upstream branch: dd6c438 |
blktests-ci
Bot
force-pushed
the
series/1079338=>linus-master
branch
from
906cfab to
93ee5b4
Compare
blktests-ci
Bot
force-pushed
the
linus-master_base
branch
2 times, most recently
from
94f0438 to
857ada9
Compare
Author
|
Upstream branch: dd6c438 |
blktests-ci
Bot
force-pushed
the
series/1079338=>linus-master
branch
from
93ee5b4 to
b77541d
Compare
blktests-ci
Bot
force-pushed
the
linus-master_base
branch
from
857ada9 to
482ce5b
Compare
Author
|
Upstream branch: dca922e |
blktests-ci
Bot
force-pushed
the
series/1079338=>linus-master
branch
from
b77541d to
4611e24
Compare
blktests-ci
Bot
force-pushed
the
linus-master_base
branch
from
482ce5b to
5a9f7c7
Compare
Some bio completion handlers need to run in task context but bio_endio() can be called from IRQ context (e.g. buffer_head writeback). Add a BIO_COMPLETE_IN_TASK flag that bio submitters can set to request task-context completion of their bi_end_io callback. When bio_endio() sees this flag and is running in non-task context, it queues the bio to a per-cpu lockless list and schedules a delayed work item to call bi_end_io() from task context. The delayed work uses a 1-jiffie delay to allow batches of completions to accumulate before processing. A CPU hotplug dead callback drains any remaining bios from the departing CPU's batch. This will be used to enable RWF_DONTCACHE for block devices, and could be used for other subsystems like fscrypt that need task-context bio completion. Suggested-by: Matthew Wilcox <[email protected]> Signed-off-by: Tal Zussman <[email protected]>
Set BIO_COMPLETE_IN_TASK on iomap writeback bios when a dropbehind folio is added. This ensures that bi_end_io runs in task context, where folio_end_dropbehind() can safely invalidate folios. With the bio layer now handling task-context deferral generically, IOMAP_IOEND_DONTCACHE is no longer needed, as XFS no longer needs to route DONTCACHE ioends through its completion workqueue. Remove the flag and its NOMERGE entry. Without the NOMERGE, regular I/Os that get merged with a dropbehind folio will also have their completion deferred to task context. Signed-off-by: Tal Zussman <[email protected]>
Block device buffered reads and writes already pass through filemap_read() and iomap_file_buffered_write() respectively, both of which handle IOCB_DONTCACHE. Enable RWF_DONTCACHE for block device files by setting FOP_DONTCACHE in def_blk_fops. For CONFIG_BUFFER_HEAD=y paths, add block_write_begin_iocb() which threads the kiocb through so that buffer_head-based I/O can use DONTCACHE behavior. The existing block_write_begin() is preserved as a wrapper that passes a NULL iocb. Set BIO_COMPLETE_IN_TASK in submit_bh_wbc() when the folio has dropbehind so that buffer_head writeback completions get deferred to task context. CONFIG_BUFFER_HEAD=n paths are handled by the previously added iomap BIO_COMPLETE_IN_TASK support. This support is useful for databases that operate on raw block devices, among other userspace applications. Signed-off-by: Tal Zussman <[email protected]>
Replace the bio-flag based interface with an explicit bio_complete_in_task() API. The advantage is that this can also be called from inside the ->bi_end_io callback and thus dynamically. This will be important to use it for fserror reporting. Signed-off-by: Christoph Hellwig <[email protected]>
As described in commit c99fab6, some block drivers might call into ->bi_end_io from non-preemptible context. Copy and past the logic from that commit, although having a core helper for it would be nicer. Signed-off-by: Christoph Hellwig <[email protected]>
Replace out own hand-crafted complete in task context scheme with the generic block code. Signed-off-by: Christoph Hellwig <[email protected]>
Replace out own hand-crafted complete in task context scheme with the generic block code. Signed-off-by: Christoph Hellwig <[email protected]>
Commit 3fffb58 ("erofs: add per-cpu threads for decompression as an option") explains why workqueue aren't great for low-latency completion handling. Switch to a per-cpu kthread to handle it instead. This code is based on the erofs code in the above commit, but further simplified by directly using a kthread instead of a kthread_work. Signed-off-by: Christoph Hellwig <[email protected]>
Author
|
Upstream branch: e75a43c |
blktests-ci
Bot
force-pushed
the
series/1079338=>linus-master
branch
from
4611e24 to
b5542ab
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Pull request for series with
subject: block: add BIO_COMPLETE_IN_TASK for task-context completion
version: 1
url: https://patchwork.kernel.org/project/linux-block/list/?series=1079338