Conversation
…/kernel/git/kvmarm/kvmarm into HEAD KVM/arm64 fixes for 6.16, take #2 - Rework of system register accessors for system registers that are directly writen to memory, so that sanitisation of the in-memory value happens at the correct time (after the read, or before the write). For convenience, RMW-style accessors are also provided. - Multiple fixes for the so-called "arch-timer-edge-cases' selftest, which was always broken.
As-per the SBI specification, an SBI remote fence operation applies to the entire address space if either: 1) start_addr and size are both 0 2) size is equal to 2^XLEN-1 >From the above, only #1 is checked by SBI SFENCE calls so fix the size parameter check in SBI SFENCE calls to cover #2 as well. Fixes: 13acfec ("RISC-V: KVM: Add remote HFENCE functions based on VCPU requests") Reviewed-by: Atish Patra <[email protected]> Signed-off-by: Anup Patel <[email protected]> Link: https://lore.kernel.org/r/[email protected] Signed-off-by: Anup Patel <[email protected]>
This patch fixes an issue seen in a large-scale deployment under heavy
incoming pkts where the aRFS flow wrongly matches a flow and reprograms the
NIC with wrong settings. That mis-steering causes RX-path latency spikes
and noisy neighbor effects when many connections collide on the same
hash (some of our production servers have 20-30K connections).
set_rps_cpu() calls ndo_rx_flow_steer() with flow_id that is calculated by
hashing the skb sized by the per rx-queue table size. This results in
multiple connections (even across different rx-queues) getting the same
hash value. The driver steer function modifies the wrong flow to use this
rx-queue, e.g.: Flow#1 is first added:
Flow#1: <ip1, port1, ip2, port2>, Hash 'h', q#10
Later when a new flow needs to be added:
Flow#2: <ip3, port3, ip4, port4>, Hash 'h', q#20
The driver finds the hash 'h' from Flow#1 and updates it to use q#20. This
results in both flows getting un-optimized - packets for Flow#1 goes to
q#20, and then reprogrammed back to q#10 later and so on; and Flow #2
programming is never done as Flow#1 is matched first for all misses. Many
flows may wrongly share the same hash and reprogram rules of the original
flow each with their own q#.
Tested on two 144-core servers with 16K netperf sessions for 180s. Netperf
clients are pinned to cores 0-71 sequentially (so that wrong packets on q#s
72-143 can be measured). IRQs are set 1:1 for queues -> CPUs, enable XPS,
enable aRFS (global value is 144 * rps_flow_cnt).
Test notes about results from ice_rx_flow_steer():
---------------------------------------------------
1. "Skip:" counter increments here:
if (fltr_info->q_index == rxq_idx ||
arfs_entry->fltr_state != ICE_ARFS_ACTIVE)
goto out;
2. "Add:" counter increments here:
ret = arfs_entry->fltr_info.fltr_id;
INIT_HLIST_NODE(&arfs_entry->list_entry);
3. "Update:" counter increments here:
/* update the queue to forward to on an already existing flow */
Runtime comparison: original code vs with the patch for different
rps_flow_cnt values.
+-------------------------------+--------------+--------------+
| rps_flow_cnt | 512 | 2048 |
+-------------------------------+--------------+--------------+
| Ratio of Pkts on Good:Bad q's | 214 vs 822K | 1.1M vs 980K |
| Avoid wrong aRFS programming | 0 vs 310K | 0 vs 30K |
| CPU User | 216 vs 183 | 216 vs 206 |
| CPU System | 1441 vs 1171 | 1447 vs 1320 |
| CPU Softirq | 1245 vs 920 | 1238 vs 961 |
| CPU Total | 29 vs 22.7 | 29 vs 24.9 |
| aRFS Update | 533K vs 59 | 521K vs 32 |
| aRFS Skip | 82M vs 77M | 7.2M vs 4.5M |
+-------------------------------+--------------+--------------+
A separate TCP_STREAM and TCP_RR with 1,4,8,16,64,128,256,512 connections
showed no performance degradation.
Some points on the patch/aRFS behavior:
1. Enabling full tuple matching ensures flows are always correctly matched,
even with smaller hash sizes.
2. 5-6% drop in CPU utilization as the packets arrive at the correct CPUs
and fewer calls to driver for programming on misses.
3. Larger hash tables reduces mis-steering due to more unique flow hashes,
but still has clashes. However, with larger per-device rps_flow_cnt, old
flows take more time to expire and new aRFS flows cannot be added if h/w
limits are reached (rps_may_expire_flow() succeeds when 10*rps_flow_cnt
pkts have been processed by this cpu that are not part of the flow).
Fixes: 28bf267 ("ice: Implement aRFS")
Signed-off-by: Krishna Kumar <[email protected]>
Reviewed-by: Simon Horman <[email protected]>
Tested-by: Rinitha S <[email protected]> (A Contingent worker at Intel)
Signed-off-by: Tony Nguyen <[email protected]>
syzkaller reported a null-ptr-deref in sock_omalloc() while allocating a CALIPSO option. [0] The NULL is of struct sock, which was fetched by sk_to_full_sk() in calipso_req_setattr(). Since commit a1a5344 ("tcp: avoid two atomic ops for syncookies"), reqsk->rsk_listener could be NULL when SYN Cookie is returned to its client, as hinted by the leading SYN Cookie log. Here are 3 options to fix the bug: 1) Return 0 in calipso_req_setattr() 2) Return an error in calipso_req_setattr() 3) Alaways set rsk_listener 1) is no go as it bypasses LSM, but 2) effectively disables SYN Cookie for CALIPSO. 3) is also no go as there have been many efforts to reduce atomic ops and make TCP robust against DDoS. See also commit 3b24d85 ("tcp/dccp: do not touch listener sk_refcnt under synflood"). As of the blamed commit, SYN Cookie already did not need refcounting, and no one has stumbled on the bug for 9 years, so no CALIPSO user will care about SYN Cookie. Let's return an error in calipso_req_setattr() and calipso_req_delattr() in the SYN Cookie case. This can be reproduced by [1] on Fedora and now connect() of nc times out. [0]: TCP: request_sock_TCPv6: Possible SYN flooding on port [::]:20002. Sending cookies. Oops: general protection fault, probably for non-canonical address 0xdffffc0000000006: 0000 [#1] PREEMPT SMP KASAN NOPTI KASAN: null-ptr-deref in range [0x0000000000000030-0x0000000000000037] CPU: 3 UID: 0 PID: 12262 Comm: syz.1.2611 Not tainted 6.14.0 #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.16.3-0-ga6ed6b701f0a-prebuilt.qemu.org 04/01/2014 RIP: 0010:read_pnet include/net/net_namespace.h:406 [inline] RIP: 0010:sock_net include/net/sock.h:655 [inline] RIP: 0010:sock_kmalloc+0x35/0x170 net/core/sock.c:2806 Code: 89 d5 41 54 55 89 f5 53 48 89 fb e8 25 e3 c6 fd e8 f0 91 e3 00 48 8d 7b 30 48 b8 00 00 00 00 00 fc ff df 48 89 fa 48 c1 ea 03 <80> 3c 02 00 0f 85 26 01 00 00 48 b8 00 00 00 00 00 fc ff df 4c 8b RSP: 0018:ffff88811af89038 EFLAGS: 00010216 RAX: dffffc0000000000 RBX: 0000000000000000 RCX: ffff888105266400 RDX: 0000000000000006 RSI: ffff88800c890000 RDI: 0000000000000030 RBP: 0000000000000050 R08: 0000000000000000 R09: ffff88810526640e R10: ffffed1020a4cc81 R11: ffff88810526640f R12: 0000000000000000 R13: 0000000000000820 R14: ffff888105266400 R15: 0000000000000050 FS: 00007f0653a07640(0000) GS:ffff88811af80000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 00007f863ba096f4 CR3: 00000000163c0005 CR4: 0000000000770ef0 PKRU: 80000000 Call Trace: <IRQ> ipv6_renew_options+0x279/0x950 net/ipv6/exthdrs.c:1288 calipso_req_setattr+0x181/0x340 net/ipv6/calipso.c:1204 calipso_req_setattr+0x56/0x80 net/netlabel/netlabel_calipso.c:597 netlbl_req_setattr+0x18a/0x440 net/netlabel/netlabel_kapi.c:1249 selinux_netlbl_inet_conn_request+0x1fb/0x320 security/selinux/netlabel.c:342 selinux_inet_conn_request+0x1eb/0x2c0 security/selinux/hooks.c:5551 security_inet_conn_request+0x50/0xa0 security/security.c:4945 tcp_v6_route_req+0x22c/0x550 net/ipv6/tcp_ipv6.c:825 tcp_conn_request+0xec8/0x2b70 net/ipv4/tcp_input.c:7275 tcp_v6_conn_request+0x1e3/0x440 net/ipv6/tcp_ipv6.c:1328 tcp_rcv_state_process+0xafa/0x52b0 net/ipv4/tcp_input.c:6781 tcp_v6_do_rcv+0x8a6/0x1a40 net/ipv6/tcp_ipv6.c:1667 tcp_v6_rcv+0x505e/0x5b50 net/ipv6/tcp_ipv6.c:1904 ip6_protocol_deliver_rcu+0x17c/0x1da0 net/ipv6/ip6_input.c:436 ip6_input_finish+0x103/0x180 net/ipv6/ip6_input.c:480 NF_HOOK include/linux/netfilter.h:314 [inline] NF_HOOK include/linux/netfilter.h:308 [inline] ip6_input+0x13c/0x6b0 net/ipv6/ip6_input.c:491 dst_input include/net/dst.h:469 [inline] ip6_rcv_finish net/ipv6/ip6_input.c:79 [inline] ip6_rcv_finish+0xb6/0x490 net/ipv6/ip6_input.c:69 NF_HOOK include/linux/netfilter.h:314 [inline] NF_HOOK include/linux/netfilter.h:308 [inline] ipv6_rcv+0xf9/0x490 net/ipv6/ip6_input.c:309 __netif_receive_skb_one_core+0x12e/0x1f0 net/core/dev.c:5896 __netif_receive_skb+0x1d/0x170 net/core/dev.c:6009 process_backlog+0x41e/0x13b0 net/core/dev.c:6357 __napi_poll+0xbd/0x710 net/core/dev.c:7191 napi_poll net/core/dev.c:7260 [inline] net_rx_action+0x9de/0xde0 net/core/dev.c:7382 handle_softirqs+0x19a/0x770 kernel/softirq.c:561 do_softirq.part.0+0x36/0x70 kernel/softirq.c:462 </IRQ> <TASK> do_softirq arch/x86/include/asm/preempt.h:26 [inline] __local_bh_enable_ip+0xf1/0x110 kernel/softirq.c:389 local_bh_enable include/linux/bottom_half.h:33 [inline] rcu_read_unlock_bh include/linux/rcupdate.h:919 [inline] __dev_queue_xmit+0xc2a/0x3c40 net/core/dev.c:4679 dev_queue_xmit include/linux/netdevice.h:3313 [inline] neigh_hh_output include/net/neighbour.h:523 [inline] neigh_output include/net/neighbour.h:537 [inline] ip6_finish_output2+0xd69/0x1f80 net/ipv6/ip6_output.c:141 __ip6_finish_output net/ipv6/ip6_output.c:215 [inline] ip6_finish_output+0x5dc/0xd60 net/ipv6/ip6_output.c:226 NF_HOOK_COND include/linux/netfilter.h:303 [inline] ip6_output+0x24b/0x8d0 net/ipv6/ip6_output.c:247 dst_output include/net/dst.h:459 [inline] NF_HOOK include/linux/netfilter.h:314 [inline] NF_HOOK include/linux/netfilter.h:308 [inline] ip6_xmit+0xbbc/0x20d0 net/ipv6/ip6_output.c:366 inet6_csk_xmit+0x39a/0x720 net/ipv6/inet6_connection_sock.c:135 __tcp_transmit_skb+0x1a7b/0x3b40 net/ipv4/tcp_output.c:1471 tcp_transmit_skb net/ipv4/tcp_output.c:1489 [inline] tcp_send_syn_data net/ipv4/tcp_output.c:4059 [inline] tcp_connect+0x1c0c/0x4510 net/ipv4/tcp_output.c:4148 tcp_v6_connect+0x156c/0x2080 net/ipv6/tcp_ipv6.c:333 __inet_stream_connect+0x3a7/0xed0 net/ipv4/af_inet.c:677 tcp_sendmsg_fastopen+0x3e2/0x710 net/ipv4/tcp.c:1039 tcp_sendmsg_locked+0x1e82/0x3570 net/ipv4/tcp.c:1091 tcp_sendmsg+0x2f/0x50 net/ipv4/tcp.c:1358 inet6_sendmsg+0xb9/0x150 net/ipv6/af_inet6.c:659 sock_sendmsg_nosec net/socket.c:718 [inline] __sock_sendmsg+0xf4/0x2a0 net/socket.c:733 __sys_sendto+0x29a/0x390 net/socket.c:2187 __do_sys_sendto net/socket.c:2194 [inline] __se_sys_sendto net/socket.c:2190 [inline] __x64_sys_sendto+0xe1/0x1c0 net/socket.c:2190 do_syscall_x64 arch/x86/entry/common.c:52 [inline] do_syscall_64+0xc3/0x1d0 arch/x86/entry/common.c:83 entry_SYSCALL_64_after_hwframe+0x77/0x7f RIP: 0033:0x7f06553c47ed Code: 02 b8 ff ff ff ff c3 66 0f 1f 44 00 00 f3 0f 1e fa 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 a8 ff ff ff f7 d8 64 89 01 48 RSP: 002b:00007f0653a06fc8 EFLAGS: 00000246 ORIG_RAX: 000000000000002c RAX: ffffffffffffffda RBX: 00007f0655605fa0 RCX: 00007f06553c47ed RDX: 0000000000000000 RSI: 0000000000000000 RDI: 000000000000000b RBP: 00007f065545db38 R08: 0000200000000140 R09: 000000000000001c R10: f7384d4ea84b01bd R11: 0000000000000246 R12: 0000000000000000 R13: 00007f0655605fac R14: 00007f0655606038 R15: 00007f06539e7000 </TASK> Modules linked in: [1]: dnf install -y selinux-policy-targeted policycoreutils netlabel_tools procps-ng nmap-ncat mount -t selinuxfs none /sys/fs/selinux load_policy netlabelctl calipso add pass doi:1 netlabelctl map del default netlabelctl map add default address:::1 protocol:calipso,1 sysctl net.ipv4.tcp_syncookies=2 nc -l ::1 80 & nc ::1 80 Fixes: e1adea9 ("calipso: Allow request sockets to be relabelled by the lsm.") Reported-by: syzkaller <[email protected]> Reported-by: John Cheung <[email protected]> Closes: https://lore.kernel.org/netdev/CAP=Rh=MvfhrGADy+-WJiftV2_WzMH4VEhEFmeT28qY+4yxNu4w@mail.gmail.com/ Signed-off-by: Kuniyuki Iwashima <[email protected]> Acked-by: Paul Moore <[email protected]> Link: https://patch.msgid.link/[email protected] Signed-off-by: Jakub Kicinski <[email protected]>
The issue arises when kzalloc() is invoked while holding umem_mutex or
any other lock acquired under umem_mutex. This is problematic because
kzalloc() can trigger fs_reclaim_aqcuire(), which may, in turn, invoke
mmu_notifier_invalidate_range_start(). This function can lead to
mlx5_ib_invalidate_range(), which attempts to acquire umem_mutex again,
resulting in a deadlock.
The problematic flow:
CPU0 | CPU1
---------------------------------------|------------------------------------------------
mlx5_ib_dereg_mr() |
→ revoke_mr() |
→ mutex_lock(&umem_odp->umem_mutex) |
| mlx5_mkey_cache_init()
| → mutex_lock(&dev->cache.rb_lock)
| → mlx5r_cache_create_ent_locked()
| → kzalloc(GFP_KERNEL)
| → fs_reclaim()
| → mmu_notifier_invalidate_range_start()
| → mlx5_ib_invalidate_range()
| → mutex_lock(&umem_odp->umem_mutex)
→ cache_ent_find_and_store() |
→ mutex_lock(&dev->cache.rb_lock) |
Additionally, when kzalloc() is called from within
cache_ent_find_and_store(), we encounter the same deadlock due to
re-acquisition of umem_mutex.
Solve by releasing umem_mutex in dereg_mr() after umr_revoke_mr()
and before acquiring rb_lock. This ensures that we don't hold
umem_mutex while performing memory allocations that could trigger
the reclaim path.
This change prevents the deadlock by ensuring proper lock ordering and
avoiding holding locks during memory allocation operations that could
trigger the reclaim path.
The following lockdep warning demonstrates the deadlock:
python3/20557 is trying to acquire lock:
ffff888387542128 (&umem_odp->umem_mutex){+.+.}-{4:4}, at:
mlx5_ib_invalidate_range+0x5b/0x550 [mlx5_ib]
but task is already holding lock:
ffffffff82f6b840 (mmu_notifier_invalidate_range_start){+.+.}-{0:0}, at:
unmap_vmas+0x7b/0x1a0
which lock already depends on the new lock.
the existing dependency chain (in reverse order) is:
-> #3 (mmu_notifier_invalidate_range_start){+.+.}-{0:0}:
fs_reclaim_acquire+0x60/0xd0
mem_cgroup_css_alloc+0x6f/0x9b0
cgroup_init_subsys+0xa4/0x240
cgroup_init+0x1c8/0x510
start_kernel+0x747/0x760
x86_64_start_reservations+0x25/0x30
x86_64_start_kernel+0x73/0x80
common_startup_64+0x129/0x138
-> #2 (fs_reclaim){+.+.}-{0:0}:
fs_reclaim_acquire+0x91/0xd0
__kmalloc_cache_noprof+0x4d/0x4c0
mlx5r_cache_create_ent_locked+0x75/0x620 [mlx5_ib]
mlx5_mkey_cache_init+0x186/0x360 [mlx5_ib]
mlx5_ib_stage_post_ib_reg_umr_init+0x3c/0x60 [mlx5_ib]
__mlx5_ib_add+0x4b/0x190 [mlx5_ib]
mlx5r_probe+0xd9/0x320 [mlx5_ib]
auxiliary_bus_probe+0x42/0x70
really_probe+0xdb/0x360
__driver_probe_device+0x8f/0x130
driver_probe_device+0x1f/0xb0
__driver_attach+0xd4/0x1f0
bus_for_each_dev+0x79/0xd0
bus_add_driver+0xf0/0x200
driver_register+0x6e/0xc0
__auxiliary_driver_register+0x6a/0xc0
do_one_initcall+0x5e/0x390
do_init_module+0x88/0x240
init_module_from_file+0x85/0xc0
idempotent_init_module+0x104/0x300
__x64_sys_finit_module+0x68/0xc0
do_syscall_64+0x6d/0x140
entry_SYSCALL_64_after_hwframe+0x4b/0x53
-> #1 (&dev->cache.rb_lock){+.+.}-{4:4}:
__mutex_lock+0x98/0xf10
__mlx5_ib_dereg_mr+0x6f2/0x890 [mlx5_ib]
mlx5_ib_dereg_mr+0x21/0x110 [mlx5_ib]
ib_dereg_mr_user+0x85/0x1f0 [ib_core]
uverbs_free_mr+0x19/0x30 [ib_uverbs]
destroy_hw_idr_uobject+0x21/0x80 [ib_uverbs]
uverbs_destroy_uobject+0x60/0x3d0 [ib_uverbs]
uobj_destroy+0x57/0xa0 [ib_uverbs]
ib_uverbs_cmd_verbs+0x4d5/0x1210 [ib_uverbs]
ib_uverbs_ioctl+0x129/0x230 [ib_uverbs]
__x64_sys_ioctl+0x596/0xaa0
do_syscall_64+0x6d/0x140
entry_SYSCALL_64_after_hwframe+0x4b/0x53
-> #0 (&umem_odp->umem_mutex){+.+.}-{4:4}:
__lock_acquire+0x1826/0x2f00
lock_acquire+0xd3/0x2e0
__mutex_lock+0x98/0xf10
mlx5_ib_invalidate_range+0x5b/0x550 [mlx5_ib]
__mmu_notifier_invalidate_range_start+0x18e/0x1f0
unmap_vmas+0x182/0x1a0
exit_mmap+0xf3/0x4a0
mmput+0x3a/0x100
do_exit+0x2b9/0xa90
do_group_exit+0x32/0xa0
get_signal+0xc32/0xcb0
arch_do_signal_or_restart+0x29/0x1d0
syscall_exit_to_user_mode+0x105/0x1d0
do_syscall_64+0x79/0x140
entry_SYSCALL_64_after_hwframe+0x4b/0x53
Chain exists of:
&dev->cache.rb_lock --> mmu_notifier_invalidate_range_start -->
&umem_odp->umem_mutex
Possible unsafe locking scenario:
CPU0 CPU1
---- ----
lock(&umem_odp->umem_mutex);
lock(mmu_notifier_invalidate_range_start);
lock(&umem_odp->umem_mutex);
lock(&dev->cache.rb_lock);
*** DEADLOCK ***
Fixes: abb604a ("RDMA/mlx5: Fix a race for an ODP MR which leads to CQE with error")
Signed-off-by: Or Har-Toov <[email protected]>
Reviewed-by: Michael Guralnik <[email protected]>
Link: https://patch.msgid.link/3c8f225a8a9fade647d19b014df1172544643e4a.1750061612.git.leon@kernel.org
Signed-off-by: Leon Romanovsky <[email protected]>
The WARN_ON_ONCE is introduced on truncate_folio_batch_exceptionals() to capture whether the filesystem has removed all DAX entries or not. And the fix has been applied on the filesystem xfs and ext4 by the commit 0e2f80a ("fs/dax: ensure all pages are idle prior to filesystem unmount"). Apply the missed fix on filesystem fuse to fix the runtime warning: [ 2.011450] ------------[ cut here ]------------ [ 2.011873] WARNING: CPU: 0 PID: 145 at mm/truncate.c:89 truncate_folio_batch_exceptionals+0x272/0x2b0 [ 2.012468] Modules linked in: [ 2.012718] CPU: 0 UID: 1000 PID: 145 Comm: weston Not tainted 6.16.0-rc2-WSL2-STABLE #2 PREEMPT(undef) [ 2.013292] RIP: 0010:truncate_folio_batch_exceptionals+0x272/0x2b0 [ 2.013704] Code: 48 63 d0 41 29 c5 48 8d 1c d5 00 00 00 00 4e 8d 6c 2a 01 49 c1 e5 03 eb 09 48 83 c3 08 49 39 dd 74 83 41 f6 44 1c 08 01 74 ef <0f> 0b 49 8b 34 1e 48 89 ef e8 10 a2 17 00 eb df 48 8b 7d 00 e8 35 [ 2.014845] RSP: 0018:ffffa47ec33f3b10 EFLAGS: 00010202 [ 2.015279] RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000000000000 [ 2.015884] RDX: 0000000000000000 RSI: ffffa47ec33f3ca0 RDI: ffff98aa44f3fa80 [ 2.016377] RBP: ffff98aa44f3fbf0 R08: ffffa47ec33f3ba8 R09: 0000000000000000 [ 2.016942] R10: 0000000000000001 R11: 0000000000000000 R12: ffffa47ec33f3ca0 [ 2.017437] R13: 0000000000000008 R14: ffffa47ec33f3ba8 R15: 0000000000000000 [ 2.017972] FS: 000079ce006afa40(0000) GS:ffff98aade441000(0000) knlGS:0000000000000000 [ 2.018510] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2.018987] CR2: 000079ce03e74000 CR3: 000000010784f006 CR4: 0000000000372eb0 [ 2.019518] Call Trace: [ 2.019729] <TASK> [ 2.019901] truncate_inode_pages_range+0xd8/0x400 [ 2.020280] ? timerqueue_add+0x66/0xb0 [ 2.020574] ? get_nohz_timer_target+0x2a/0x140 [ 2.020904] ? timerqueue_add+0x66/0xb0 [ 2.021231] ? timerqueue_del+0x2e/0x50 [ 2.021646] ? __remove_hrtimer+0x39/0x90 [ 2.022017] ? srso_alias_untrain_ret+0x1/0x10 [ 2.022497] ? psi_group_change+0x136/0x350 [ 2.023046] ? _raw_spin_unlock+0xe/0x30 [ 2.023514] ? finish_task_switch.isra.0+0x8d/0x280 [ 2.024068] ? __schedule+0x532/0xbd0 [ 2.024551] fuse_evict_inode+0x29/0x190 [ 2.025131] evict+0x100/0x270 [ 2.025641] ? _atomic_dec_and_lock+0x39/0x50 [ 2.026316] ? __pfx_generic_delete_inode+0x10/0x10 [ 2.026843] __dentry_kill+0x71/0x180 [ 2.027335] dput+0xeb/0x1b0 [ 2.027725] __fput+0x136/0x2b0 [ 2.028054] __x64_sys_close+0x3d/0x80 [ 2.028469] do_syscall_64+0x6d/0x1b0 [ 2.028832] ? clear_bhb_loop+0x30/0x80 [ 2.029182] ? clear_bhb_loop+0x30/0x80 [ 2.029533] ? clear_bhb_loop+0x30/0x80 [ 2.029902] entry_SYSCALL_64_after_hwframe+0x76/0x7e [ 2.030423] RIP: 0033:0x79ce03d0d067 [ 2.030820] Code: b8 ff ff ff ff e9 3e ff ff ff 66 0f 1f 84 00 00 00 00 00 f3 0f 1e fa 64 8b 04 25 18 00 00 00 85 c0 75 10 b8 03 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 41 c3 48 83 ec 18 89 7c 24 0c e8 c3 a7 f8 ff [ 2.032354] RSP: 002b:00007ffef0498948 EFLAGS: 00000246 ORIG_RAX: 0000000000000003 [ 2.032939] RAX: ffffffffffffffda RBX: 00007ffef0498960 RCX: 000079ce03d0d067 [ 2.033612] RDX: 0000000000000003 RSI: 0000000000001000 RDI: 000000000000000d [ 2.034289] RBP: 00007ffef0498a30 R08: 000000000000000d R09: 0000000000000000 [ 2.034944] R10: 00007ffef0498978 R11: 0000000000000246 R12: 0000000000000001 [ 2.035610] R13: 00007ffef0498960 R14: 000079ce03e09ce0 R15: 0000000000000003 [ 2.036301] </TASK> [ 2.036532] ---[ end trace 0000000000000000 ]--- Link: https://lkml.kernel.org/r/[email protected] Fixes: bde708f ("fs/dax: always remove DAX page-cache entries when breaking layouts") Signed-off-by: Haiyue Wang <[email protected]> Cc: Alistair Popple <[email protected]> Cc: Dan Williams <[email protected]> Cc: Miklos Szeredi <[email protected]> Cc: <[email protected]> Signed-off-by: Andrew Morton <[email protected]>
Fix cifs_signal_cifsd_for_reconnect() to take the correct lock order
and prevent the following deadlock from happening
======================================================
WARNING: possible circular locking dependency detected
6.16.0-rc3-build2+ #1301 Tainted: G S W
------------------------------------------------------
cifsd/6055 is trying to acquire lock:
ffff88810ad56038 (&tcp_ses->srv_lock){+.+.}-{3:3}, at: cifs_signal_cifsd_for_reconnect+0x134/0x200
but task is already holding lock:
ffff888119c64330 (&ret_buf->chan_lock){+.+.}-{3:3}, at: cifs_signal_cifsd_for_reconnect+0xcf/0x200
which lock already depends on the new lock.
the existing dependency chain (in reverse order) is:
-> #2 (&ret_buf->chan_lock){+.+.}-{3:3}:
validate_chain+0x1cf/0x270
__lock_acquire+0x60e/0x780
lock_acquire.part.0+0xb4/0x1f0
_raw_spin_lock+0x2f/0x40
cifs_setup_session+0x81/0x4b0
cifs_get_smb_ses+0x771/0x900
cifs_mount_get_session+0x7e/0x170
cifs_mount+0x92/0x2d0
cifs_smb3_do_mount+0x161/0x460
smb3_get_tree+0x55/0x90
vfs_get_tree+0x46/0x180
do_new_mount+0x1b0/0x2e0
path_mount+0x6ee/0x740
do_mount+0x98/0xe0
__do_sys_mount+0x148/0x180
do_syscall_64+0xa4/0x260
entry_SYSCALL_64_after_hwframe+0x76/0x7e
-> #1 (&ret_buf->ses_lock){+.+.}-{3:3}:
validate_chain+0x1cf/0x270
__lock_acquire+0x60e/0x780
lock_acquire.part.0+0xb4/0x1f0
_raw_spin_lock+0x2f/0x40
cifs_match_super+0x101/0x320
sget+0xab/0x270
cifs_smb3_do_mount+0x1e0/0x460
smb3_get_tree+0x55/0x90
vfs_get_tree+0x46/0x180
do_new_mount+0x1b0/0x2e0
path_mount+0x6ee/0x740
do_mount+0x98/0xe0
__do_sys_mount+0x148/0x180
do_syscall_64+0xa4/0x260
entry_SYSCALL_64_after_hwframe+0x76/0x7e
-> #0 (&tcp_ses->srv_lock){+.+.}-{3:3}:
check_noncircular+0x95/0xc0
check_prev_add+0x115/0x2f0
validate_chain+0x1cf/0x270
__lock_acquire+0x60e/0x780
lock_acquire.part.0+0xb4/0x1f0
_raw_spin_lock+0x2f/0x40
cifs_signal_cifsd_for_reconnect+0x134/0x200
__cifs_reconnect+0x8f/0x500
cifs_handle_standard+0x112/0x280
cifs_demultiplex_thread+0x64d/0xbc0
kthread+0x2f7/0x310
ret_from_fork+0x2a/0x230
ret_from_fork_asm+0x1a/0x30
other info that might help us debug this:
Chain exists of:
&tcp_ses->srv_lock --> &ret_buf->ses_lock --> &ret_buf->chan_lock
Possible unsafe locking scenario:
CPU0 CPU1
---- ----
lock(&ret_buf->chan_lock);
lock(&ret_buf->ses_lock);
lock(&ret_buf->chan_lock);
lock(&tcp_ses->srv_lock);
*** DEADLOCK ***
3 locks held by cifsd/6055:
#0: ffffffff857de398 (&cifs_tcp_ses_lock){+.+.}-{3:3}, at: cifs_signal_cifsd_for_reconnect+0x7b/0x200
#1: ffff888119c64060 (&ret_buf->ses_lock){+.+.}-{3:3}, at: cifs_signal_cifsd_for_reconnect+0x9c/0x200
#2: ffff888119c64330 (&ret_buf->chan_lock){+.+.}-{3:3}, at: cifs_signal_cifsd_for_reconnect+0xcf/0x200
Cc: [email protected]
Reported-by: David Howells <[email protected]>
Fixes: d7d7a66 ("cifs: avoid use of global locks for high contention data")
Reviewed-by: David Howells <[email protected]>
Tested-by: David Howells <[email protected]>
Signed-off-by: Paulo Alcantara (Red Hat) <[email protected]>
Signed-off-by: David Howells <[email protected]>
Signed-off-by: Steve French <[email protected]>
When I run the NVME over TCP test in virtme-ng, I get the following "suspicious RCU usage" warning in nvme_mpath_add_sysfs_link(): ''' [ 5.024557][ T44] nvmet: Created nvm controller 1 for subsystem nqn.2025-06.org.nvmexpress.mptcp for NQN nqn.2014-08.org.nvmexpress:uuid:f7f6b5e0-ff97-4894-98ac-c85309e0bc77. [ 5.027401][ T183] nvme nvme0: creating 2 I/O queues. [ 5.029017][ T183] nvme nvme0: mapped 2/0/0 default/read/poll queues. [ 5.032587][ T183] nvme nvme0: new ctrl: NQN "nqn.2025-06.org.nvmexpress.mptcp", addr 127.0.0.1:4420, hostnqn: nqn.2014-08.org.nvmexpress:uuid:f7f6b5e0-ff97-4894-98ac-c85309e0bc77 [ 5.042214][ T25] [ 5.042440][ T25] ============================= [ 5.042579][ T25] WARNING: suspicious RCU usage [ 5.042705][ T25] 6.16.0-rc3+ #23 Not tainted [ 5.042812][ T25] ----------------------------- [ 5.042934][ T25] drivers/nvme/host/multipath.c:1203 RCU-list traversed in non-reader section!! [ 5.043111][ T25] [ 5.043111][ T25] other info that might help us debug this: [ 5.043111][ T25] [ 5.043341][ T25] [ 5.043341][ T25] rcu_scheduler_active = 2, debug_locks = 1 [ 5.043502][ T25] 3 locks held by kworker/u9:0/25: [ 5.043615][ T25] #0: ffff888008730948 ((wq_completion)async){+.+.}-{0:0}, at: process_one_work+0x7ed/0x1350 [ 5.043830][ T25] #1: ffffc900001afd40 ((work_completion)(&entry->work)){+.+.}-{0:0}, at: process_one_work+0xcf3/0x1350 [ 5.044084][ T25] #2: ffff888013ee0020 (&head->srcu){.+.+}-{0:0}, at: nvme_mpath_add_sysfs_link.part.0+0xb4/0x3a0 [ 5.044300][ T25] [ 5.044300][ T25] stack backtrace: [ 5.044439][ T25] CPU: 0 UID: 0 PID: 25 Comm: kworker/u9:0 Not tainted 6.16.0-rc3+ #23 PREEMPT(full) [ 5.044441][ T25] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 [ 5.044442][ T25] Workqueue: async async_run_entry_fn [ 5.044445][ T25] Call Trace: [ 5.044446][ T25] <TASK> [ 5.044449][ T25] dump_stack_lvl+0x6f/0xb0 [ 5.044453][ T25] lockdep_rcu_suspicious.cold+0x4f/0xb1 [ 5.044457][ T25] nvme_mpath_add_sysfs_link.part.0+0x2fb/0x3a0 [ 5.044459][ T25] ? queue_work_on+0x90/0xf0 [ 5.044461][ T25] ? lockdep_hardirqs_on+0x78/0x110 [ 5.044466][ T25] nvme_mpath_set_live+0x1e9/0x4f0 [ 5.044470][ T25] nvme_mpath_add_disk+0x240/0x2f0 [ 5.044472][ T25] ? __pfx_nvme_mpath_add_disk+0x10/0x10 [ 5.044475][ T25] ? add_disk_fwnode+0x361/0x580 [ 5.044480][ T25] nvme_alloc_ns+0x81c/0x17c0 [ 5.044483][ T25] ? kasan_quarantine_put+0x104/0x240 [ 5.044487][ T25] ? __pfx_nvme_alloc_ns+0x10/0x10 [ 5.044495][ T25] ? __pfx_nvme_find_get_ns+0x10/0x10 [ 5.044496][ T25] ? rcu_read_lock_any_held+0x45/0xa0 [ 5.044498][ T25] ? validate_chain+0x232/0x4f0 [ 5.044503][ T25] nvme_scan_ns+0x4c8/0x810 [ 5.044506][ T25] ? __pfx_nvme_scan_ns+0x10/0x10 [ 5.044508][ T25] ? find_held_lock+0x2b/0x80 [ 5.044512][ T25] ? ktime_get+0x16d/0x220 [ 5.044517][ T25] ? kvm_clock_get_cycles+0x18/0x30 [ 5.044520][ T25] ? __pfx_nvme_scan_ns_async+0x10/0x10 [ 5.044522][ T25] async_run_entry_fn+0x97/0x560 [ 5.044523][ T25] ? rcu_is_watching+0x12/0xc0 [ 5.044526][ T25] process_one_work+0xd3c/0x1350 [ 5.044532][ T25] ? __pfx_process_one_work+0x10/0x10 [ 5.044536][ T25] ? assign_work+0x16c/0x240 [ 5.044539][ T25] worker_thread+0x4da/0xd50 [ 5.044545][ T25] ? __pfx_worker_thread+0x10/0x10 [ 5.044546][ T25] kthread+0x356/0x5c0 [ 5.044548][ T25] ? __pfx_kthread+0x10/0x10 [ 5.044549][ T25] ? ret_from_fork+0x1b/0x2e0 [ 5.044552][ T25] ? __lock_release.isra.0+0x5d/0x180 [ 5.044553][ T25] ? ret_from_fork+0x1b/0x2e0 [ 5.044555][ T25] ? rcu_is_watching+0x12/0xc0 [ 5.044557][ T25] ? __pfx_kthread+0x10/0x10 [ 5.044559][ T25] ret_from_fork+0x218/0x2e0 [ 5.044561][ T25] ? __pfx_kthread+0x10/0x10 [ 5.044562][ T25] ret_from_fork_asm+0x1a/0x30 [ 5.044570][ T25] </TASK> ''' This patch uses sleepable RCU version of helper list_for_each_entry_srcu() instead of list_for_each_entry_rcu() to fix it. Fixes: 4dbd2b2 ("nvme-multipath: Add visibility for round-robin io-policy") Signed-off-by: Geliang Tang <[email protected]> Reviewed-by: Keith Busch <[email protected]> Reviewed-by: Hannes Reinecke <[email protected]> Reviewed-by: Nilay Shroff <[email protected]> Signed-off-by: Christoph Hellwig <[email protected]>
With VIRTCHNL2_CAP_MACFILTER enabled, the following warning is generated
on module load:
[ 324.701677] BUG: sleeping function called from invalid context at kernel/locking/mutex.c:578
[ 324.701684] in_atomic(): 1, irqs_disabled(): 0, non_block: 0, pid: 1582, name: NetworkManager
[ 324.701689] preempt_count: 201, expected: 0
[ 324.701693] RCU nest depth: 0, expected: 0
[ 324.701697] 2 locks held by NetworkManager/1582:
[ 324.701702] #0: ffffffff9f7be770 (rtnl_mutex){....}-{3:3}, at: rtnl_newlink+0x791/0x21e0
[ 324.701730] #1: ff1100216c380368 (_xmit_ETHER){....}-{2:2}, at: __dev_open+0x3f0/0x870
[ 324.701749] Preemption disabled at:
[ 324.701752] [<ffffffff9cd23b9d>] __dev_open+0x3dd/0x870
[ 324.701765] CPU: 30 UID: 0 PID: 1582 Comm: NetworkManager Not tainted 6.15.0-rc5+ #2 PREEMPT(voluntary)
[ 324.701771] Hardware name: Intel Corporation M50FCP2SBSTD/M50FCP2SBSTD, BIOS SE5C741.86B.01.01.0001.2211140926 11/14/2022
[ 324.701774] Call Trace:
[ 324.701777] <TASK>
[ 324.701779] dump_stack_lvl+0x5d/0x80
[ 324.701788] ? __dev_open+0x3dd/0x870
[ 324.701793] __might_resched.cold+0x1ef/0x23d
<..>
[ 324.701818] __mutex_lock+0x113/0x1b80
<..>
[ 324.701917] idpf_ctlq_clean_sq+0xad/0x4b0 [idpf]
[ 324.701935] ? kasan_save_track+0x14/0x30
[ 324.701941] idpf_mb_clean+0x143/0x380 [idpf]
<..>
[ 324.701991] idpf_send_mb_msg+0x111/0x720 [idpf]
[ 324.702009] idpf_vc_xn_exec+0x4cc/0x990 [idpf]
[ 324.702021] ? rcu_is_watching+0x12/0xc0
[ 324.702035] idpf_add_del_mac_filters+0x3ed/0xb50 [idpf]
<..>
[ 324.702122] __hw_addr_sync_dev+0x1cf/0x300
[ 324.702126] ? find_held_lock+0x32/0x90
[ 324.702134] idpf_set_rx_mode+0x317/0x390 [idpf]
[ 324.702152] __dev_open+0x3f8/0x870
[ 324.702159] ? __pfx___dev_open+0x10/0x10
[ 324.702174] __dev_change_flags+0x443/0x650
<..>
[ 324.702208] netif_change_flags+0x80/0x160
[ 324.702218] do_setlink.isra.0+0x16a0/0x3960
<..>
[ 324.702349] rtnl_newlink+0x12fd/0x21e0
The sequence is as follows:
rtnl_newlink()->
__dev_change_flags()->
__dev_open()->
dev_set_rx_mode() - > # disables BH and grabs "dev->addr_list_lock"
idpf_set_rx_mode() -> # proceed only if VIRTCHNL2_CAP_MACFILTER is ON
__dev_uc_sync() ->
idpf_add_mac_filter ->
idpf_add_del_mac_filters ->
idpf_send_mb_msg() ->
idpf_mb_clean() ->
idpf_ctlq_clean_sq() # mutex_lock(cq_lock)
Fix by converting cq_lock to a spinlock. All operations under the new
lock are safe except freeing the DMA memory, which may use vunmap(). Fix
by requesting a contiguous physical memory for the DMA mapping.
Fixes: a251eee ("idpf: add SRIOV support and other ndo_ops")
Reviewed-by: Aleksandr Loktionov <[email protected]>
Signed-off-by: Ahmed Zaki <[email protected]>
Reviewed-by: Simon Horman <[email protected]>
Tested-by: Samuel Salin <[email protected]>
Signed-off-by: Tony Nguyen <[email protected]>
nbd grabs device lock nbd->config_lock for updating nr_hw_queues, this ways cause the following lock dependency: -> #2 (&disk->open_mutex){+.+.}-{4:4}: lock_acquire kernel/locking/lockdep.c:5871 [inline] lock_acquire+0x1ac/0x448 kernel/locking/lockdep.c:5828 __mutex_lock_common kernel/locking/mutex.c:602 [inline] __mutex_lock+0x166/0x1292 kernel/locking/mutex.c:747 mutex_lock_nested+0x14/0x1c kernel/locking/mutex.c:799 __del_gendisk+0x132/0xac6 block/genhd.c:706 del_gendisk+0xf6/0x19a block/genhd.c:819 nbd_dev_remove+0x3c/0xf2 drivers/block/nbd.c:268 nbd_dev_remove_work+0x1c/0x26 drivers/block/nbd.c:284 process_one_work+0x96a/0x1f32 kernel/workqueue.c:3238 process_scheduled_works kernel/workqueue.c:3321 [inline] worker_thread+0x5ce/0xde8 kernel/workqueue.c:3402 kthread+0x39c/0x7d4 kernel/kthread.c:464 ret_from_fork_kernel+0x2a/0xbb2 arch/riscv/kernel/process.c:214 ret_from_fork_kernel_asm+0x16/0x18 arch/riscv/kernel/entry.S:327 -> #1 (&set->update_nr_hwq_lock){++++}-{4:4}: lock_acquire kernel/locking/lockdep.c:5871 [inline] lock_acquire+0x1ac/0x448 kernel/locking/lockdep.c:5828 down_write+0x9c/0x19a kernel/locking/rwsem.c:1577 blk_mq_update_nr_hw_queues+0x3e/0xb86 block/blk-mq.c:5041 nbd_start_device+0x140/0xb2c drivers/block/nbd.c:1476 nbd_genl_connect+0xae0/0x1b24 drivers/block/nbd.c:2201 genl_family_rcv_msg_doit+0x206/0x2e6 net/netlink/genetlink.c:1115 genl_family_rcv_msg net/netlink/genetlink.c:1195 [inline] genl_rcv_msg+0x514/0x78e net/netlink/genetlink.c:1210 netlink_rcv_skb+0x206/0x3be net/netlink/af_netlink.c:2534 genl_rcv+0x36/0x4c net/netlink/genetlink.c:1219 netlink_unicast_kernel net/netlink/af_netlink.c:1313 [inline] netlink_unicast+0x4f0/0x82c net/netlink/af_netlink.c:1339 netlink_sendmsg+0x85e/0xdd6 net/netlink/af_netlink.c:1883 sock_sendmsg_nosec net/socket.c:712 [inline] __sock_sendmsg+0xcc/0x160 net/socket.c:727 ____sys_sendmsg+0x63e/0x79c net/socket.c:2566 ___sys_sendmsg+0x144/0x1e6 net/socket.c:2620 __sys_sendmsg+0x188/0x246 net/socket.c:2652 __do_sys_sendmsg net/socket.c:2657 [inline] __se_sys_sendmsg net/socket.c:2655 [inline] __riscv_sys_sendmsg+0x70/0xa2 net/socket.c:2655 syscall_handler+0x94/0x118 arch/riscv/include/asm/syscall.h:112 do_trap_ecall_u+0x396/0x530 arch/riscv/kernel/traps.c:341 handle_exception+0x146/0x152 arch/riscv/kernel/entry.S:197 -> #0 (&nbd->config_lock){+.+.}-{4:4}: check_noncircular+0x132/0x146 kernel/locking/lockdep.c:2178 check_prev_add kernel/locking/lockdep.c:3168 [inline] check_prevs_add kernel/locking/lockdep.c:3287 [inline] validate_chain kernel/locking/lockdep.c:3911 [inline] __lock_acquire+0x12b2/0x24ea kernel/locking/lockdep.c:5240 lock_acquire kernel/locking/lockdep.c:5871 [inline] lock_acquire+0x1ac/0x448 kernel/locking/lockdep.c:5828 __mutex_lock_common kernel/locking/mutex.c:602 [inline] __mutex_lock+0x166/0x1292 kernel/locking/mutex.c:747 mutex_lock_nested+0x14/0x1c kernel/locking/mutex.c:799 refcount_dec_and_mutex_lock+0x60/0xd8 lib/refcount.c:118 nbd_config_put+0x3a/0x610 drivers/block/nbd.c:1423 nbd_release+0x94/0x15c drivers/block/nbd.c:1735 blkdev_put_whole+0xac/0xee block/bdev.c:721 bdev_release+0x3fe/0x600 block/bdev.c:1144 blkdev_release+0x1a/0x26 block/fops.c:684 __fput+0x382/0xa8c fs/file_table.c:465 ____fput+0x1c/0x26 fs/file_table.c:493 task_work_run+0x16a/0x25e kernel/task_work.c:227 resume_user_mode_work include/linux/resume_user_mode.h:50 [inline] exit_to_user_mode_loop+0x118/0x134 kernel/entry/common.c:114 exit_to_user_mode_prepare include/linux/entry-common.h:330 [inline] syscall_exit_to_user_mode_work include/linux/entry-common.h:414 [inline] syscall_exit_to_user_mode include/linux/entry-common.h:449 [inline] do_trap_ecall_u+0x3f0/0x530 arch/riscv/kernel/traps.c:355 handle_exception+0x146/0x152 arch/riscv/kernel/entry.S:197 Also it isn't necessary to require nbd->config_lock, because blk_mq_update_nr_hw_queues() does grab tagset lock for sync everything. Fixes the issue by releasing ->config_lock & retry in case of concurrent updating nr_hw_queues. Fixes: 98e68f6 ("block: prevent adding/deleting disk during updating nr_hw_queues") Reported-by: [email protected] Closes: https://lore.kernel.org/all/[email protected] Reviewed-by: Yu Kuai <[email protected]> Cc: Nilay Shroff <[email protected]> Signed-off-by: Ming Lei <[email protected]> Reviewed-by: Nilay Shroff <[email protected]> Link: https://lore.kernel.org/r/[email protected] Signed-off-by: Jens Axboe <[email protected]>
3d460ec to
ac8b12e
Compare
|
branch: linus-master_test |
a53bf31 to
230a8dd
Compare
nbd grabs device lock nbd->config_lock for updating nr_hw_queues, this ways cause the following lock dependency: -> #2 (&disk->open_mutex){+.+.}-{4:4}: lock_acquire kernel/locking/lockdep.c:5871 [inline] lock_acquire+0x1ac/0x448 kernel/locking/lockdep.c:5828 __mutex_lock_common kernel/locking/mutex.c:602 [inline] __mutex_lock+0x166/0x1292 kernel/locking/mutex.c:747 mutex_lock_nested+0x14/0x1c kernel/locking/mutex.c:799 __del_gendisk+0x132/0xac6 block/genhd.c:706 del_gendisk+0xf6/0x19a block/genhd.c:819 nbd_dev_remove+0x3c/0xf2 drivers/block/nbd.c:268 nbd_dev_remove_work+0x1c/0x26 drivers/block/nbd.c:284 process_one_work+0x96a/0x1f32 kernel/workqueue.c:3238 process_scheduled_works kernel/workqueue.c:3321 [inline] worker_thread+0x5ce/0xde8 kernel/workqueue.c:3402 kthread+0x39c/0x7d4 kernel/kthread.c:464 ret_from_fork_kernel+0x2a/0xbb2 arch/riscv/kernel/process.c:214 ret_from_fork_kernel_asm+0x16/0x18 arch/riscv/kernel/entry.S:327 -> #1 (&set->update_nr_hwq_lock){++++}-{4:4}: lock_acquire kernel/locking/lockdep.c:5871 [inline] lock_acquire+0x1ac/0x448 kernel/locking/lockdep.c:5828 down_write+0x9c/0x19a kernel/locking/rwsem.c:1577 blk_mq_update_nr_hw_queues+0x3e/0xb86 block/blk-mq.c:5041 nbd_start_device+0x140/0xb2c drivers/block/nbd.c:1476 nbd_genl_connect+0xae0/0x1b24 drivers/block/nbd.c:2201 genl_family_rcv_msg_doit+0x206/0x2e6 net/netlink/genetlink.c:1115 genl_family_rcv_msg net/netlink/genetlink.c:1195 [inline] genl_rcv_msg+0x514/0x78e net/netlink/genetlink.c:1210 netlink_rcv_skb+0x206/0x3be net/netlink/af_netlink.c:2534 genl_rcv+0x36/0x4c net/netlink/genetlink.c:1219 netlink_unicast_kernel net/netlink/af_netlink.c:1313 [inline] netlink_unicast+0x4f0/0x82c net/netlink/af_netlink.c:1339 netlink_sendmsg+0x85e/0xdd6 net/netlink/af_netlink.c:1883 sock_sendmsg_nosec net/socket.c:712 [inline] __sock_sendmsg+0xcc/0x160 net/socket.c:727 ____sys_sendmsg+0x63e/0x79c net/socket.c:2566 ___sys_sendmsg+0x144/0x1e6 net/socket.c:2620 __sys_sendmsg+0x188/0x246 net/socket.c:2652 __do_sys_sendmsg net/socket.c:2657 [inline] __se_sys_sendmsg net/socket.c:2655 [inline] __riscv_sys_sendmsg+0x70/0xa2 net/socket.c:2655 syscall_handler+0x94/0x118 arch/riscv/include/asm/syscall.h:112 do_trap_ecall_u+0x396/0x530 arch/riscv/kernel/traps.c:341 handle_exception+0x146/0x152 arch/riscv/kernel/entry.S:197 -> #0 (&nbd->config_lock){+.+.}-{4:4}: check_noncircular+0x132/0x146 kernel/locking/lockdep.c:2178 check_prev_add kernel/locking/lockdep.c:3168 [inline] check_prevs_add kernel/locking/lockdep.c:3287 [inline] validate_chain kernel/locking/lockdep.c:3911 [inline] __lock_acquire+0x12b2/0x24ea kernel/locking/lockdep.c:5240 lock_acquire kernel/locking/lockdep.c:5871 [inline] lock_acquire+0x1ac/0x448 kernel/locking/lockdep.c:5828 __mutex_lock_common kernel/locking/mutex.c:602 [inline] __mutex_lock+0x166/0x1292 kernel/locking/mutex.c:747 mutex_lock_nested+0x14/0x1c kernel/locking/mutex.c:799 refcount_dec_and_mutex_lock+0x60/0xd8 lib/refcount.c:118 nbd_config_put+0x3a/0x610 drivers/block/nbd.c:1423 nbd_release+0x94/0x15c drivers/block/nbd.c:1735 blkdev_put_whole+0xac/0xee block/bdev.c:721 bdev_release+0x3fe/0x600 block/bdev.c:1144 blkdev_release+0x1a/0x26 block/fops.c:684 __fput+0x382/0xa8c fs/file_table.c:465 ____fput+0x1c/0x26 fs/file_table.c:493 task_work_run+0x16a/0x25e kernel/task_work.c:227 resume_user_mode_work include/linux/resume_user_mode.h:50 [inline] exit_to_user_mode_loop+0x118/0x134 kernel/entry/common.c:114 exit_to_user_mode_prepare include/linux/entry-common.h:330 [inline] syscall_exit_to_user_mode_work include/linux/entry-common.h:414 [inline] syscall_exit_to_user_mode include/linux/entry-common.h:449 [inline] do_trap_ecall_u+0x3f0/0x530 arch/riscv/kernel/traps.c:355 handle_exception+0x146/0x152 arch/riscv/kernel/entry.S:197 Also it isn't necessary to require nbd->config_lock, because blk_mq_update_nr_hw_queues() does grab tagset lock for sync everything. Fixes the issue by releasing ->config_lock & retry in case of concurrent updating nr_hw_queues. Fixes: 98e68f6 ("block: prevent adding/deleting disk during updating nr_hw_queues") Reported-by: [email protected] Closes: https://lore.kernel.org/all/[email protected] Reviewed-by: Yu Kuai <[email protected]> Cc: Nilay Shroff <[email protected]> Signed-off-by: Ming Lei <[email protected]> Reviewed-by: Nilay Shroff <[email protected]>
…-flight
Reject migration of SEV{-ES} state if either the source or destination VM
is actively creating a vCPU, i.e. if kvm_vm_ioctl_create_vcpu() is in the
section between incrementing created_vcpus and online_vcpus. The bulk of
vCPU creation runs _outside_ of kvm->lock to allow creating multiple vCPUs
in parallel, and so sev_info.es_active can get toggled from false=>true in
the destination VM after (or during) svm_vcpu_create(), resulting in an
SEV{-ES} VM effectively having a non-SEV{-ES} vCPU.
The issue manifests most visibly as a crash when trying to free a vCPU's
NULL VMSA page in an SEV-ES VM, but any number of things can go wrong.
BUG: unable to handle page fault for address: ffffebde00000000
#PF: supervisor read access in kernel mode
#PF: error_code(0x0000) - not-present page
PGD 0 P4D 0
Oops: Oops: 0000 [#1] SMP KASAN NOPTI
CPU: 227 UID: 0 PID: 64063 Comm: syz.5.60023 Tainted: G U O 6.15.0-smp-DEV #2 NONE
Tainted: [U]=USER, [O]=OOT_MODULE
Hardware name: Google, Inc. Arcadia_IT_80/Arcadia_IT_80, BIOS 12.52.0-0 10/28/2024
RIP: 0010:constant_test_bit arch/x86/include/asm/bitops.h:206 [inline]
RIP: 0010:arch_test_bit arch/x86/include/asm/bitops.h:238 [inline]
RIP: 0010:_test_bit include/asm-generic/bitops/instrumented-non-atomic.h:142 [inline]
RIP: 0010:PageHead include/linux/page-flags.h:866 [inline]
RIP: 0010:___free_pages+0x3e/0x120 mm/page_alloc.c:5067
Code: <49> f7 06 40 00 00 00 75 05 45 31 ff eb 0c 66 90 4c 89 f0 4c 39 f0
RSP: 0018:ffff8984551978d0 EFLAGS: 00010246
RAX: 0000777f80000001 RBX: 0000000000000000 RCX: ffffffff918aeb98
RDX: 0000000000000000 RSI: 0000000000000008 RDI: ffffebde00000000
RBP: 0000000000000000 R08: ffffebde00000007 R09: 1ffffd7bc0000000
R10: dffffc0000000000 R11: fffff97bc0000001 R12: dffffc0000000000
R13: ffff8983e19751a8 R14: ffffebde00000000 R15: 1ffffd7bc0000000
FS: 0000000000000000(0000) GS:ffff89ee661d3000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: ffffebde00000000 CR3: 000000793ceaa000 CR4: 0000000000350ef0
DR0: 0000000000000000 DR1: 0000000000000b5f DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Call Trace:
<TASK>
sev_free_vcpu+0x413/0x630 arch/x86/kvm/svm/sev.c:3169
svm_vcpu_free+0x13a/0x2a0 arch/x86/kvm/svm/svm.c:1515
kvm_arch_vcpu_destroy+0x6a/0x1d0 arch/x86/kvm/x86.c:12396
kvm_vcpu_destroy virt/kvm/kvm_main.c:470 [inline]
kvm_destroy_vcpus+0xd1/0x300 virt/kvm/kvm_main.c:490
kvm_arch_destroy_vm+0x636/0x820 arch/x86/kvm/x86.c:12895
kvm_put_kvm+0xb8e/0xfb0 virt/kvm/kvm_main.c:1310
kvm_vm_release+0x48/0x60 virt/kvm/kvm_main.c:1369
__fput+0x3e4/0x9e0 fs/file_table.c:465
task_work_run+0x1a9/0x220 kernel/task_work.c:227
exit_task_work include/linux/task_work.h:40 [inline]
do_exit+0x7f0/0x25b0 kernel/exit.c:953
do_group_exit+0x203/0x2d0 kernel/exit.c:1102
get_signal+0x1357/0x1480 kernel/signal.c:3034
arch_do_signal_or_restart+0x40/0x690 arch/x86/kernel/signal.c:337
exit_to_user_mode_loop kernel/entry/common.c:111 [inline]
exit_to_user_mode_prepare include/linux/entry-common.h:329 [inline]
__syscall_exit_to_user_mode_work kernel/entry/common.c:207 [inline]
syscall_exit_to_user_mode+0x67/0xb0 kernel/entry/common.c:218
do_syscall_64+0x7c/0x150 arch/x86/entry/syscall_64.c:100
entry_SYSCALL_64_after_hwframe+0x76/0x7e
RIP: 0033:0x7f87a898e969
</TASK>
Modules linked in: gq(O)
gsmi: Log Shutdown Reason 0x03
CR2: ffffebde00000000
---[ end trace 0000000000000000 ]---
Deliberately don't check for a NULL VMSA when freeing the vCPU, as crashing
the host is likely desirable due to the VMSA being consumed by hardware.
E.g. if KVM manages to allow VMRUN on the vCPU, hardware may read/write a
bogus VMSA page. Accessing PFN 0 is "fine"-ish now that it's sequestered
away thanks to L1TF, but panicking in this scenario is preferable to
potentially running with corrupted state.
Reported-by: Alexander Potapenko <[email protected]>
Tested-by: Alexander Potapenko <[email protected]>
Fixes: 0b020f5 ("KVM: SEV: Add support for SEV-ES intra host migration")
Fixes: b566393 ("KVM: SEV: Add support for SEV intra host migration")
Cc: [email protected]
Cc: James Houghton <[email protected]>
Cc: Peter Gonda <[email protected]>
Reviewed-by: Liam Merwick <[email protected]>
Tested-by: Liam Merwick <[email protected]>
Reviewed-by: James Houghton <[email protected]>
Link: https://lore.kernel.org/r/[email protected]
Signed-off-by: Sean Christopherson <[email protected]>
ac8b12e to
9a69d4f
Compare
|
branch: linus-master_test |
230a8dd to
bea61c2
Compare
nbd grabs device lock nbd->config_lock for updating nr_hw_queues, this ways cause the following lock dependency: -> #2 (&disk->open_mutex){+.+.}-{4:4}: lock_acquire kernel/locking/lockdep.c:5871 [inline] lock_acquire+0x1ac/0x448 kernel/locking/lockdep.c:5828 __mutex_lock_common kernel/locking/mutex.c:602 [inline] __mutex_lock+0x166/0x1292 kernel/locking/mutex.c:747 mutex_lock_nested+0x14/0x1c kernel/locking/mutex.c:799 __del_gendisk+0x132/0xac6 block/genhd.c:706 del_gendisk+0xf6/0x19a block/genhd.c:819 nbd_dev_remove+0x3c/0xf2 drivers/block/nbd.c:268 nbd_dev_remove_work+0x1c/0x26 drivers/block/nbd.c:284 process_one_work+0x96a/0x1f32 kernel/workqueue.c:3238 process_scheduled_works kernel/workqueue.c:3321 [inline] worker_thread+0x5ce/0xde8 kernel/workqueue.c:3402 kthread+0x39c/0x7d4 kernel/kthread.c:464 ret_from_fork_kernel+0x2a/0xbb2 arch/riscv/kernel/process.c:214 ret_from_fork_kernel_asm+0x16/0x18 arch/riscv/kernel/entry.S:327 -> #1 (&set->update_nr_hwq_lock){++++}-{4:4}: lock_acquire kernel/locking/lockdep.c:5871 [inline] lock_acquire+0x1ac/0x448 kernel/locking/lockdep.c:5828 down_write+0x9c/0x19a kernel/locking/rwsem.c:1577 blk_mq_update_nr_hw_queues+0x3e/0xb86 block/blk-mq.c:5041 nbd_start_device+0x140/0xb2c drivers/block/nbd.c:1476 nbd_genl_connect+0xae0/0x1b24 drivers/block/nbd.c:2201 genl_family_rcv_msg_doit+0x206/0x2e6 net/netlink/genetlink.c:1115 genl_family_rcv_msg net/netlink/genetlink.c:1195 [inline] genl_rcv_msg+0x514/0x78e net/netlink/genetlink.c:1210 netlink_rcv_skb+0x206/0x3be net/netlink/af_netlink.c:2534 genl_rcv+0x36/0x4c net/netlink/genetlink.c:1219 netlink_unicast_kernel net/netlink/af_netlink.c:1313 [inline] netlink_unicast+0x4f0/0x82c net/netlink/af_netlink.c:1339 netlink_sendmsg+0x85e/0xdd6 net/netlink/af_netlink.c:1883 sock_sendmsg_nosec net/socket.c:712 [inline] __sock_sendmsg+0xcc/0x160 net/socket.c:727 ____sys_sendmsg+0x63e/0x79c net/socket.c:2566 ___sys_sendmsg+0x144/0x1e6 net/socket.c:2620 __sys_sendmsg+0x188/0x246 net/socket.c:2652 __do_sys_sendmsg net/socket.c:2657 [inline] __se_sys_sendmsg net/socket.c:2655 [inline] __riscv_sys_sendmsg+0x70/0xa2 net/socket.c:2655 syscall_handler+0x94/0x118 arch/riscv/include/asm/syscall.h:112 do_trap_ecall_u+0x396/0x530 arch/riscv/kernel/traps.c:341 handle_exception+0x146/0x152 arch/riscv/kernel/entry.S:197 -> #0 (&nbd->config_lock){+.+.}-{4:4}: check_noncircular+0x132/0x146 kernel/locking/lockdep.c:2178 check_prev_add kernel/locking/lockdep.c:3168 [inline] check_prevs_add kernel/locking/lockdep.c:3287 [inline] validate_chain kernel/locking/lockdep.c:3911 [inline] __lock_acquire+0x12b2/0x24ea kernel/locking/lockdep.c:5240 lock_acquire kernel/locking/lockdep.c:5871 [inline] lock_acquire+0x1ac/0x448 kernel/locking/lockdep.c:5828 __mutex_lock_common kernel/locking/mutex.c:602 [inline] __mutex_lock+0x166/0x1292 kernel/locking/mutex.c:747 mutex_lock_nested+0x14/0x1c kernel/locking/mutex.c:799 refcount_dec_and_mutex_lock+0x60/0xd8 lib/refcount.c:118 nbd_config_put+0x3a/0x610 drivers/block/nbd.c:1423 nbd_release+0x94/0x15c drivers/block/nbd.c:1735 blkdev_put_whole+0xac/0xee block/bdev.c:721 bdev_release+0x3fe/0x600 block/bdev.c:1144 blkdev_release+0x1a/0x26 block/fops.c:684 __fput+0x382/0xa8c fs/file_table.c:465 ____fput+0x1c/0x26 fs/file_table.c:493 task_work_run+0x16a/0x25e kernel/task_work.c:227 resume_user_mode_work include/linux/resume_user_mode.h:50 [inline] exit_to_user_mode_loop+0x118/0x134 kernel/entry/common.c:114 exit_to_user_mode_prepare include/linux/entry-common.h:330 [inline] syscall_exit_to_user_mode_work include/linux/entry-common.h:414 [inline] syscall_exit_to_user_mode include/linux/entry-common.h:449 [inline] do_trap_ecall_u+0x3f0/0x530 arch/riscv/kernel/traps.c:355 handle_exception+0x146/0x152 arch/riscv/kernel/entry.S:197 Also it isn't necessary to require nbd->config_lock, because blk_mq_update_nr_hw_queues() does grab tagset lock for sync everything. Fixes the issue by releasing ->config_lock & retry in case of concurrent updating nr_hw_queues. Fixes: 98e68f6 ("block: prevent adding/deleting disk during updating nr_hw_queues") Reported-by: [email protected] Closes: https://lore.kernel.org/all/[email protected] Reviewed-by: Yu Kuai <[email protected]> Cc: Nilay Shroff <[email protected]> Signed-off-by: Ming Lei <[email protected]> Reviewed-by: Nilay Shroff <[email protected]>
9a69d4f to
e311dd9
Compare
|
branch: linus-master_test |
bea61c2 to
970dc06
Compare
nbd grabs device lock nbd->config_lock for updating nr_hw_queues, this ways cause the following lock dependency: -> #2 (&disk->open_mutex){+.+.}-{4:4}: lock_acquire kernel/locking/lockdep.c:5871 [inline] lock_acquire+0x1ac/0x448 kernel/locking/lockdep.c:5828 __mutex_lock_common kernel/locking/mutex.c:602 [inline] __mutex_lock+0x166/0x1292 kernel/locking/mutex.c:747 mutex_lock_nested+0x14/0x1c kernel/locking/mutex.c:799 __del_gendisk+0x132/0xac6 block/genhd.c:706 del_gendisk+0xf6/0x19a block/genhd.c:819 nbd_dev_remove+0x3c/0xf2 drivers/block/nbd.c:268 nbd_dev_remove_work+0x1c/0x26 drivers/block/nbd.c:284 process_one_work+0x96a/0x1f32 kernel/workqueue.c:3238 process_scheduled_works kernel/workqueue.c:3321 [inline] worker_thread+0x5ce/0xde8 kernel/workqueue.c:3402 kthread+0x39c/0x7d4 kernel/kthread.c:464 ret_from_fork_kernel+0x2a/0xbb2 arch/riscv/kernel/process.c:214 ret_from_fork_kernel_asm+0x16/0x18 arch/riscv/kernel/entry.S:327 -> #1 (&set->update_nr_hwq_lock){++++}-{4:4}: lock_acquire kernel/locking/lockdep.c:5871 [inline] lock_acquire+0x1ac/0x448 kernel/locking/lockdep.c:5828 down_write+0x9c/0x19a kernel/locking/rwsem.c:1577 blk_mq_update_nr_hw_queues+0x3e/0xb86 block/blk-mq.c:5041 nbd_start_device+0x140/0xb2c drivers/block/nbd.c:1476 nbd_genl_connect+0xae0/0x1b24 drivers/block/nbd.c:2201 genl_family_rcv_msg_doit+0x206/0x2e6 net/netlink/genetlink.c:1115 genl_family_rcv_msg net/netlink/genetlink.c:1195 [inline] genl_rcv_msg+0x514/0x78e net/netlink/genetlink.c:1210 netlink_rcv_skb+0x206/0x3be net/netlink/af_netlink.c:2534 genl_rcv+0x36/0x4c net/netlink/genetlink.c:1219 netlink_unicast_kernel net/netlink/af_netlink.c:1313 [inline] netlink_unicast+0x4f0/0x82c net/netlink/af_netlink.c:1339 netlink_sendmsg+0x85e/0xdd6 net/netlink/af_netlink.c:1883 sock_sendmsg_nosec net/socket.c:712 [inline] __sock_sendmsg+0xcc/0x160 net/socket.c:727 ____sys_sendmsg+0x63e/0x79c net/socket.c:2566 ___sys_sendmsg+0x144/0x1e6 net/socket.c:2620 __sys_sendmsg+0x188/0x246 net/socket.c:2652 __do_sys_sendmsg net/socket.c:2657 [inline] __se_sys_sendmsg net/socket.c:2655 [inline] __riscv_sys_sendmsg+0x70/0xa2 net/socket.c:2655 syscall_handler+0x94/0x118 arch/riscv/include/asm/syscall.h:112 do_trap_ecall_u+0x396/0x530 arch/riscv/kernel/traps.c:341 handle_exception+0x146/0x152 arch/riscv/kernel/entry.S:197 -> #0 (&nbd->config_lock){+.+.}-{4:4}: check_noncircular+0x132/0x146 kernel/locking/lockdep.c:2178 check_prev_add kernel/locking/lockdep.c:3168 [inline] check_prevs_add kernel/locking/lockdep.c:3287 [inline] validate_chain kernel/locking/lockdep.c:3911 [inline] __lock_acquire+0x12b2/0x24ea kernel/locking/lockdep.c:5240 lock_acquire kernel/locking/lockdep.c:5871 [inline] lock_acquire+0x1ac/0x448 kernel/locking/lockdep.c:5828 __mutex_lock_common kernel/locking/mutex.c:602 [inline] __mutex_lock+0x166/0x1292 kernel/locking/mutex.c:747 mutex_lock_nested+0x14/0x1c kernel/locking/mutex.c:799 refcount_dec_and_mutex_lock+0x60/0xd8 lib/refcount.c:118 nbd_config_put+0x3a/0x610 drivers/block/nbd.c:1423 nbd_release+0x94/0x15c drivers/block/nbd.c:1735 blkdev_put_whole+0xac/0xee block/bdev.c:721 bdev_release+0x3fe/0x600 block/bdev.c:1144 blkdev_release+0x1a/0x26 block/fops.c:684 __fput+0x382/0xa8c fs/file_table.c:465 ____fput+0x1c/0x26 fs/file_table.c:493 task_work_run+0x16a/0x25e kernel/task_work.c:227 resume_user_mode_work include/linux/resume_user_mode.h:50 [inline] exit_to_user_mode_loop+0x118/0x134 kernel/entry/common.c:114 exit_to_user_mode_prepare include/linux/entry-common.h:330 [inline] syscall_exit_to_user_mode_work include/linux/entry-common.h:414 [inline] syscall_exit_to_user_mode include/linux/entry-common.h:449 [inline] do_trap_ecall_u+0x3f0/0x530 arch/riscv/kernel/traps.c:355 handle_exception+0x146/0x152 arch/riscv/kernel/entry.S:197 Also it isn't necessary to require nbd->config_lock, because blk_mq_update_nr_hw_queues() does grab tagset lock for sync everything. Fixes the issue by releasing ->config_lock & retry in case of concurrent updating nr_hw_queues. Fixes: 98e68f6 ("block: prevent adding/deleting disk during updating nr_hw_queues") Reported-by: [email protected] Closes: https://lore.kernel.org/all/[email protected] Reviewed-by: Yu Kuai <[email protected]> Cc: Nilay Shroff <[email protected]> Signed-off-by: Ming Lei <[email protected]> Reviewed-by: Nilay Shroff <[email protected]>
e311dd9 to
b6b569e
Compare
|
branch: linus-master_test |
970dc06 to
f487f9f
Compare
nbd grabs device lock nbd->config_lock for updating nr_hw_queues, this ways cause the following lock dependency: -> #2 (&disk->open_mutex){+.+.}-{4:4}: lock_acquire kernel/locking/lockdep.c:5871 [inline] lock_acquire+0x1ac/0x448 kernel/locking/lockdep.c:5828 __mutex_lock_common kernel/locking/mutex.c:602 [inline] __mutex_lock+0x166/0x1292 kernel/locking/mutex.c:747 mutex_lock_nested+0x14/0x1c kernel/locking/mutex.c:799 __del_gendisk+0x132/0xac6 block/genhd.c:706 del_gendisk+0xf6/0x19a block/genhd.c:819 nbd_dev_remove+0x3c/0xf2 drivers/block/nbd.c:268 nbd_dev_remove_work+0x1c/0x26 drivers/block/nbd.c:284 process_one_work+0x96a/0x1f32 kernel/workqueue.c:3238 process_scheduled_works kernel/workqueue.c:3321 [inline] worker_thread+0x5ce/0xde8 kernel/workqueue.c:3402 kthread+0x39c/0x7d4 kernel/kthread.c:464 ret_from_fork_kernel+0x2a/0xbb2 arch/riscv/kernel/process.c:214 ret_from_fork_kernel_asm+0x16/0x18 arch/riscv/kernel/entry.S:327 -> #1 (&set->update_nr_hwq_lock){++++}-{4:4}: lock_acquire kernel/locking/lockdep.c:5871 [inline] lock_acquire+0x1ac/0x448 kernel/locking/lockdep.c:5828 down_write+0x9c/0x19a kernel/locking/rwsem.c:1577 blk_mq_update_nr_hw_queues+0x3e/0xb86 block/blk-mq.c:5041 nbd_start_device+0x140/0xb2c drivers/block/nbd.c:1476 nbd_genl_connect+0xae0/0x1b24 drivers/block/nbd.c:2201 genl_family_rcv_msg_doit+0x206/0x2e6 net/netlink/genetlink.c:1115 genl_family_rcv_msg net/netlink/genetlink.c:1195 [inline] genl_rcv_msg+0x514/0x78e net/netlink/genetlink.c:1210 netlink_rcv_skb+0x206/0x3be net/netlink/af_netlink.c:2534 genl_rcv+0x36/0x4c net/netlink/genetlink.c:1219 netlink_unicast_kernel net/netlink/af_netlink.c:1313 [inline] netlink_unicast+0x4f0/0x82c net/netlink/af_netlink.c:1339 netlink_sendmsg+0x85e/0xdd6 net/netlink/af_netlink.c:1883 sock_sendmsg_nosec net/socket.c:712 [inline] __sock_sendmsg+0xcc/0x160 net/socket.c:727 ____sys_sendmsg+0x63e/0x79c net/socket.c:2566 ___sys_sendmsg+0x144/0x1e6 net/socket.c:2620 __sys_sendmsg+0x188/0x246 net/socket.c:2652 __do_sys_sendmsg net/socket.c:2657 [inline] __se_sys_sendmsg net/socket.c:2655 [inline] __riscv_sys_sendmsg+0x70/0xa2 net/socket.c:2655 syscall_handler+0x94/0x118 arch/riscv/include/asm/syscall.h:112 do_trap_ecall_u+0x396/0x530 arch/riscv/kernel/traps.c:341 handle_exception+0x146/0x152 arch/riscv/kernel/entry.S:197 -> #0 (&nbd->config_lock){+.+.}-{4:4}: check_noncircular+0x132/0x146 kernel/locking/lockdep.c:2178 check_prev_add kernel/locking/lockdep.c:3168 [inline] check_prevs_add kernel/locking/lockdep.c:3287 [inline] validate_chain kernel/locking/lockdep.c:3911 [inline] __lock_acquire+0x12b2/0x24ea kernel/locking/lockdep.c:5240 lock_acquire kernel/locking/lockdep.c:5871 [inline] lock_acquire+0x1ac/0x448 kernel/locking/lockdep.c:5828 __mutex_lock_common kernel/locking/mutex.c:602 [inline] __mutex_lock+0x166/0x1292 kernel/locking/mutex.c:747 mutex_lock_nested+0x14/0x1c kernel/locking/mutex.c:799 refcount_dec_and_mutex_lock+0x60/0xd8 lib/refcount.c:118 nbd_config_put+0x3a/0x610 drivers/block/nbd.c:1423 nbd_release+0x94/0x15c drivers/block/nbd.c:1735 blkdev_put_whole+0xac/0xee block/bdev.c:721 bdev_release+0x3fe/0x600 block/bdev.c:1144 blkdev_release+0x1a/0x26 block/fops.c:684 __fput+0x382/0xa8c fs/file_table.c:465 ____fput+0x1c/0x26 fs/file_table.c:493 task_work_run+0x16a/0x25e kernel/task_work.c:227 resume_user_mode_work include/linux/resume_user_mode.h:50 [inline] exit_to_user_mode_loop+0x118/0x134 kernel/entry/common.c:114 exit_to_user_mode_prepare include/linux/entry-common.h:330 [inline] syscall_exit_to_user_mode_work include/linux/entry-common.h:414 [inline] syscall_exit_to_user_mode include/linux/entry-common.h:449 [inline] do_trap_ecall_u+0x3f0/0x530 arch/riscv/kernel/traps.c:355 handle_exception+0x146/0x152 arch/riscv/kernel/entry.S:197 Also it isn't necessary to require nbd->config_lock, because blk_mq_update_nr_hw_queues() does grab tagset lock for sync everything. Fixes the issue by releasing ->config_lock & retry in case of concurrent updating nr_hw_queues. Fixes: 98e68f6 ("block: prevent adding/deleting disk during updating nr_hw_queues") Reported-by: [email protected] Closes: https://lore.kernel.org/all/[email protected] Reviewed-by: Yu Kuai <[email protected]> Cc: Nilay Shroff <[email protected]> Signed-off-by: Ming Lei <[email protected]> Reviewed-by: Nilay Shroff <[email protected]>
b6b569e to
ef2c9cd
Compare
|
branch: linus-master_test |
f487f9f to
2771cc9
Compare
|
branch: linus-master_test |
|
branch: linus-master_test |
|
branch: linus-master_test |
1 similar comment
|
branch: linus-master_test |
|
branch: linus-master_test |
|
branch: linus-master_test |
|
branch: linus-master_test |
|
branch: linus-master_test |
|
branch: linus-master_test |
|
branch: linus-master_test |
|
branch: linus-master_test |
|
branch: linus-master_test |
|
branch: linus-master_test |
|
branch: linus-master_test |
|
branch: linus-master_test |
2 similar comments
|
branch: linus-master_test |
|
branch: linus-master_test |
|
branch: linus-master_test |
|
branch: linus-master_test |
|
branch: linus-master_test |
|
branch: linus-master_test |
|
branch: linus-master_test |
|
branch: linus-master_test |
|
branch: linus-master_test |
|
branch: linus-master_test |
|
branch: linus-master_test |
|
branch: linus-master_test |
|
branch: linus-master_test |
branch: linus-master_test
base: linus-master
version: 19272b3