kernel/linux.git/drivers/nvme, branch v5.15.219

nvmet-tcp: Do not WARN on remotely-controlled oversized SGL allocations

2026-08-27T12:28:21+00:00

commit 737a3b535247226f6e1a7988fd9d6e63e7d6fc71 upstream. When fuzzing the nvme target code, I tripped a kernel warning in nvmet_tcp_map_data() because the length passed into the allocator is controlled by the remote initiator. A remote initiator that sends a command with an SGL claiming a huge number, can create a scatterlist and iovec allocation of over 1 million entries, which causes the backing kmalloc call to exceed MAX_PAGE_ORDER and then the page allocator will trip on a WARN_ON_ONCE_GFP() message: WARNING: mm/page_alloc.c:5280 __alloc_frozen_pages_noprof Workqueue: nvmet_tcp_wq nvmet_tcp_io_work ... sgl_alloc_order nvmet_tcp_map_data nvmet_tcp_try_recv_pdu As it's never good to trip a kernel warning remotely due to many systems having panic-on-warn enabled, let's silence it by just add GFP_NOWARN to the allocation flags. Assisted-by: gkh_clanker_2000 Cc: stable Signed-off-by: Greg Kroah-Hartman Signed-off-by: Keith Busch Signed-off-by: Greg Kroah-Hartman

nvmet-fc: fix invalid free in LS IOD error path

2026-08-27T12:28:21+00:00

commit ba98d6796d12258e837ece065d2ecb59d76ce4ff upstream. nvmet_fc_alloc_ls_iodlist() advances iod while initializing the LS IOD array. If an rqstbuf allocation or response buffer DMA mapping fails, the unwind loop decrements iod past the start of the array. The final kfree(iod) therefore frees an address before the allocated object. This can be reproduced with nvme-fcloop and failslab by setting fail-nth to 6 before creating a target port. KASAN reports: BUG: KASAN: invalid-free in nvmet_fc_register_targetport Free of addr ffff88816cf8ff48 by task nvmet_fail_nth/9552 Free the original allocation base stored in tgtport->iod instead. With this fix applied, the same sysfs write with fail-nth=6 returns -ENOMEM without any KASAN report. Fixes: c53432030d86 ("nvme-fabrics: Add target support for FC transport") Cc: stable@vger.kernel.org Reviewed-by: Maurizio Lombardi Assisted-by: Codex:gpt-5 Signed-off-by: Jiang HongHui Signed-off-by: Keith Busch Signed-off-by: Greg Kroah-Hartman

nvmet-tcp: Fix potential UAF when ddgst mismatch

2026-08-19T15:13:51+00:00

commit dbbd07d0a7020b80f6a7028e561908f7b83b3d5a upstream. Shivam Kumar found via vulnerability testing: When data digest is enabled on an NVMe/TCP connection and a digest mismatch occurs on a non-final H2C_DATA PDU during an R2T-based data transfer, the digest error handler in nvmet_tcp_try_recv_ddgst() calls nvmet_req_uninit() - which performs percpu_ref_put() on the submission queue - but does NOT mark the command as completed. It does not set cqe->status, does not modify rbytes_done, and does not clear any flag. When the subsequent fatal error triggers queue teardown, nvmet_tcp_uninit_data_in_cmds() iterates all commands, checks nvmet_tcp_need_data_in() for each one, and finds that the already-uninited command still appears to need data (because rbytes_done < transfer_len and cqe->status == 0). It therefore calls nvmet_req_uninit() a second time on the same command - a double percpu_ref_put against a single percpu_ref_get. Reported-by: Shivam Kumar Reviewed-by: Christoph Hellwig Signed-off-by: Sagi Grimberg Signed-off-by: Keith Busch Signed-off-by: Shivam Kumar Signed-off-by: Sasha Levin

nvmet-tcp: check INIT_FAILED before nvmet_req_uninit in digest error path

2026-07-24T13:51:53+00:00

commit 4606467a75cfc16721937272ed29462a750b60c8 upstream. In nvmet_tcp_try_recv_ddgst(), when a data digest mismatch is detected, nvmet_req_uninit() is called unconditionally. However, if the command arrived via the nvmet_tcp_handle_req_failure() path, nvmet_req_init() had returned false and percpu_ref_tryget_live() was never executed. The unconditional percpu_ref_put() inside nvmet_req_uninit() then causes a refcount underflow, leading to a WARNING in percpu_ref_switch_to_atomic_rcu, a use-after-free diagnostic, and eventually a permanent workqueue deadlock. Check cmd->flags & NVMET_TCP_F_INIT_FAILED before calling nvmet_req_uninit(), matching the existing pattern in nvmet_tcp_execute_request(). Reviewed-by: Christoph Hellwig Signed-off-by: Shivam Kumar Signed-off-by: Keith Busch [shivam: inlined nvmet_tcp_finish_cmd() at the fix site for 5.15.y] Signed-off-by: Shivam Kumar Signed-off-by: Sasha Levin

nvmet-rdma: handle inline data with a nonzero offset

2026-07-24T13:51:51+00:00

commit 48c0162f647bb47e6084ffbc71b8f213f5e2f4f8 upstream. nvmet_rdma_use_inline_sg() maps the host-controlled inline data offset into the per-command inline scatterlist. The bounds check admits any offset with off + len <= inline_data_size, but the mapping still assumes the data begins in the first inline page: sg->offset = off; sg->length = min_t(int, len, PAGE_SIZE - off); When a port is configured with inline_data_size > PAGE_SIZE (settable up to max(SZ_16K, PAGE_SIZE)), an offset in (PAGE_SIZE, inline_data_size] makes "PAGE_SIZE - off" underflow, so sg->length is set to ~4 GiB and the block backend reads far past the first inline page. num_pages(len) also ignores the offset, so an in-bounds offset whose [off, off+len) span crosses a page boundary under-counts the scatterlist. Map the offset properly: split it into a page index and an in-page offset, start the scatterlist at that page, and size the page count from page_off + len. Because the request scatterlist may now start at inline_sg[page_idx] rather than inline_sg[0], generalize the inline-SGL identity test in nvmet_rdma_release_rsp() to a range test; otherwise the persistent inline scatterlist is mistaken for an allocated one and nvmet_req_free_sgls() frees an inline page (and warns in free_large_kmalloc()). Fixes: 0d5ee2b2ab4f ("nvmet-rdma: support max(16KB, PAGE_SIZE) inline data") Cc: stable@vger.kernel.org Suggested-by: Keith Busch Reported-by: Bryam Vargas Signed-off-by: Bryam Vargas Signed-off-by: Keith Busch Signed-off-by: Greg Kroah-Hartman

nvme-multipath: fix flex array size in struct nvme_ns_head

2026-07-24T13:51:30+00:00

[ Upstream commit 001e57554de81aa79c25c18fd53911d8a415c304 ] struct nvme_ns_head contains a flexible array member, current_path[], which is indexed using the NUMA node ID: head->current_path[numa_node_id()] The structure is currently allocated as: size = sizeof(struct nvme_ns_head) + (num_possible_nodes() * sizeof(struct nvme_ns *)); head = kzalloc(size, GFP_KERNEL); This allocation assumes that NUMA node IDs are sequential and densely packed from 0 .. num_possible_nodes() - 1. While this assumption holds on many systems, it is not always true on some architectures such as powerpc. On some powerpc systems, NUMA node IDs can be sparse. For example: NUMA: NUMA node(s): 6 NUMA node0 CPU(s): 80-159 NUMA node8 CPU(s): 0-79 NUMA node252 CPU(s): NUMA node253 CPU(s): NUMA node254 CPU(s): NUMA node255 CPU(s): That is, the possible/online NUMA node IDs are: 0, 8, 252, 253, 254, 255 In this case: num_possible_nodes() = 6 So memory is allocated for only 6 entries in current_path[]. However, the array is later indexed using the actual NUMA node ID. As a result, accesses such as: head->current_path[8] or head->current_path[252] goes out of bounds, leading to the following KASAN splat: ================================================================== BUG: KASAN: slab-out-of-bounds in nvme_mpath_revalidate_paths+0x22c/0x290 [nvme_core] Write of size 8 at addr c00020003bda35b8 by task kworker/u641:2/1997 CPU: 1 UID: 0 PID: 1997 Comm: kworker/u641:2 Not tainted 7.1.0-rc5-dirty #14 PREEMPT(lazy) Hardware name: 8335-GTH POWER9 0x4e1202 opal:skiboot-v6.5.3-35-g1851b2a06 PowerNV Workqueue: async async_run_entry_fn Call Trace: [c000200037fa7510] [c0000000021c23d4] dump_stack_lvl+0x88/0xdc (unreliable) [c000200037fa7540] [c0000000009fda90] print_report+0x22c/0x67c [c000200037fa7630] [c0000000009fd508] kasan_report+0x108/0x220 [c000200037fa7740] [c0000000009fff48] __asan_store8+0xe8/0x120 [c000200037fa7760] [c008000018e76474] nvme_mpath_revalidate_paths+0x22c/0x290 [nvme_core] [c000200037fa7800] [c008000018e6556c] nvme_update_ns_info+0x4a4/0x5e0 [nvme_core] [c000200037fa7a50] [c008000018e66270] nvme_alloc_ns+0x6d8/0x1a70 [nvme_core] [c000200037fa7c20] [c008000018e679fc] nvme_scan_ns+0x3f4/0x630 [nvme_core] [c000200037fa7d10] [c00000000031f22c] async_run_entry_fn+0x9c/0x3a0 [c000200037fa7db0] [c0000000002fa544] process_one_work+0x414/0xa10 [c000200037fa7ec0] [c0000000002fbf00] worker_thread+0x320/0x640 [c000200037fa7f80] [c00000000030d0f8] kthread+0x278/0x290 [c000200037fa7fe0] [c00000000000ded8] start_kernel_thread+0x14/0x18 Allocated by task 1997 on cpu 1 at 35.928317s: The buggy address belongs to the object at c00020003bda3000 which belongs to the cache kmalloc-rnd-15-2k of size 2048 The buggy address is located 16 bytes to the right of allocated 1448-byte region [c00020003bda3000, c00020003bda35a8) The buggy address belongs to the physical page: Memory state around the buggy address: c00020003bda3480: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 c00020003bda3500: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 >c00020003bda3580: 00 00 00 00 00 fc fc fc fc fc fc fc fc fc fc fc ^ c00020003bda3600: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc c00020003bda3680: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc ================================================================== Fix this by allocating the flexible array using nr_node_ids instead of num_possible_nodes(). Since nr_node_ids represents the maximum possible NUMA node IDs, indexing current_path[] using numa_node_id() becomes safe even on systems with sparse node IDs. Fixes: f333444708f8 ("nvme: take node locality into account when selecting a path") Tested-by: Mukesh Kumar Chaurasiya (IBM) Reviewed-by: Mukesh Kumar Chaurasiya (IBM) Reviewed-by: Hannes Reinecke Reviewed-by: John Garry Reviewed-by: Christoph Hellwig Signed-off-by: Nilay Shroff Signed-off-by: Keith Busch Signed-off-by: Sasha Levin

nvme-multipath: set BIO_REMAPPED on bios remapped to per-path namespace disks

2026-07-24T13:51:24+00:00

commit 88bac2c1a72b8f4f71e9845699aa872df04e5850 upstream. When nvme_ns_head_submit_bio() remaps a bio from the multipath head to a per-path namespace, bio_set_dev() clears BIO_REMAPPED. The remapped bio is then resubmitted through submit_bio_noacct() which calls bio_check_eod() because BIO_REMAPPED is not set. This races with nvme_ns_remove() which zeroes the per-path capacity before synchronize_srcu(): CPU 0 (IO submission) --------------------- srcu_read_lock() nvme_find_path() -> ns [NVME_NS_READY is set] CPU 1 (namespace removal) ------------------------- clear_bit(NVME_NS_READY) set_capacity(ns->disk, 0) synchronize_srcu() <- blocks CPU 0 (IO submission) --------------------- bio_set_dev(bio, ns->disk->part0) [clears BIO_REMAPPED] submit_bio_noacct(bio) -> bio_check_eod() sees capacity=0 -> bio fails with IO error The SRCU read lock prevents synchronize_srcu() from completing, but does not prevent set_capacity(0) from executing. The bio fails the EOD check before it reaches the NVMe driver, so nvme_failover_req() never gets a chance to redirect it to another path of multipath. IO errors are reported to the application despite another path being available. On older kernels (before commit 0b64682e78f7 "block: skip unnecessary checks for split bio"), the same race was also reachable through split remainders resubmitted via submit_bio_noacct(). Fix this by setting BIO_REMAPPED after bio_set_dev() in nvme_ns_head_submit_bio(). This skips bio_check_eod() on the per-path device; the EOD check already passed on the multipath head. NVMe per-path namespace devices are always whole disks (bd_partno=0), so the blk_partition_remap() skip also gated by BIO_REMAPPED is a no-op. The flag does not persist across failover and cannot go stale if the namespace geometry changes between attempts: nvme_failover_req() calls bio_set_dev() to redirect the bio back to the multipath head, which clears BIO_REMAPPED. When nvme_requeue_work() resubmits through submit_bio_noacct(), bio_check_eod() runs normally against the current capacity. Same approach as commit 3a905c37c351 ("block: skip bio_check_eod for partition-remapped bios"). Fixes: a7c7f7b2b641 ("nvme: use bio_set_dev to assign ->bi_bdev") Cc: stable@vger.kernel.org Reviewed-by: Christoph Hellwig Signed-off-by: Igor Achkinazi Signed-off-by: Keith Busch Signed-off-by: Greg Kroah-Hartman

nvmet-tcp: fix race between ICReq handling and queue teardown

2026-07-24T13:51:15+00:00

commit 5293a8882c549fab4a878bc76b0b6c951f980a61 upstream. nvmet_tcp_handle_icreq() updates queue->state after sending an Initialization Connection Response (ICResp), but it does so without serializing against target-side queue teardown. If an NVMe/TCP host sends an Initialization Connection Request (ICReq) and immediately closes the connection, target-side teardown may start in softirq context before io_work drains the already buffered ICReq. In that case, nvmet_tcp_schedule_release_queue() sets queue->state to NVMET_TCP_Q_DISCONNECTING and drops the queue reference under state_lock. If io_work later processes that ICReq, nvmet_tcp_handle_icreq() can still overwrite the state back to NVMET_TCP_Q_LIVE. That defeats the DISCONNECTING-state guard in nvmet_tcp_schedule_release_queue() and allows a later socket state change to re-enter teardown and issue a second kref_put() on an already released queue. The ICResp send failure path has the same problem. If teardown has already moved the queue to DISCONNECTING, a send error can still overwrite the state with NVMET_TCP_Q_FAILED, again reopening the window for a second teardown path to drop the queue reference. Fix this by serializing both post-send state transitions with state_lock and bailing out if teardown has already started. Use -ESHUTDOWN as an internal sentinel for that bail-out path rather than propagating it as a transport error like -ECONNRESET. Keep nvmet_tcp_socket_error() setting rcv_state to NVMET_TCP_RECV_ERR before honoring that sentinel so receive-side parsing stays quiesced until the existing release path completes. Fixes: c46a6465bac2 ("nvmet-tcp: add NVMe over TCP target driver") Cc: stable@vger.kernel.org Reported-by: Shivam Kumar Tested-by: Shivam Kumar Signed-off-by: Chaitanya Kulkarni Signed-off-by: Keith Busch [ context diff adaptation: drop `queue->state = NVMET_TCP_Q_FAILED` since the enum introduced in 6.7, 675b453e0241 ("nvmet-tcp: enable TLS handshake upcall" ] Signed-off-by: Philo Lu Signed-off-by: Sasha Levin

use less confusing names for iov_iter direction initializers

2026-06-19T11:33:46+00:00

[ Upstream commit de4eda9de2d957ef2d6a8365a01e26a435e958cb ] READ/WRITE proved to be actively confusing - the meanings are "data destination, as used with read(2)" and "data source, as used with write(2)", but people keep interpreting those as "we read data from it" and "we write data to it", i.e. exactly the wrong way. Call them ITER_DEST and ITER_SOURCE - at least that is harder to misinterpret... Signed-off-by: Al Viro Stable-dep-of: a4f0b001782b ("vsock/virtio: reset connection on receiving queue overflow") Signed-off-by: Sasha Levin Signed-off-by: Greg Kroah-Hartman

nvmet: avoid recursive nvmet-wq flush in nvmet_ctrl_free

2026-06-01T15:35:23+00:00

commit aade8abd8b868b6ffa9697aadaea28ec7f65bee6 upstream. nvmet_tcp_release_queue_work() runs on nvmet-wq and can drop the final controller reference through nvmet_cq_put(). If that triggers nvmet_ctrl_free(), the teardown path flushes ctrl->async_event_work on the same nvmet-wq. Call chain: nvmet_tcp_schedule_release_queue() kref_put(&queue->kref, nvmet_tcp_release_queue) nvmet_tcp_release_queue() queue_work(nvmet_wq, &queue->release_work) <--- nvmet_wq process_one_work() nvmet_tcp_release_queue_work() nvmet_cq_put(&queue->nvme_cq) nvmet_cq_destroy() nvmet_ctrl_put(cq->ctrl) nvmet_ctrl_free() flush_work(&ctrl->async_event_work) <--- nvmet_wq Previously Scheduled by :- nvmet_add_async_event queue_work(nvmet_wq, &ctrl->async_event_work); This trips lockdep with a possible recursive locking warning. [ 5223.015876] run blktests nvme/003 at 2026-04-07 20:53:55 [ 5223.061801] loop0: detected capacity change from 0 to 2097152 [ 5223.072206] nvmet: adding nsid 1 to subsystem blktests-subsystem-1 [ 5223.088368] nvmet_tcp: enabling port 0 (127.0.0.1:4420) [ 5223.126086] nvmet: Created discovery controller 1 for subsystem nqn.2014-08.org.nvmexpress.discovery for NQN nqn.2014-08.org.nvmexpress:uuid:0f01fb42-9f7f-4856-b0b3-51e60b8de349. [ 5223.128453] nvme nvme1: new ctrl: NQN "nqn.2014-08.org.nvmexpress.discovery", addr 127.0.0.1:4420, hostnqn: nqn.2014-08.org.nvmexpress:uuid:0f01fb42-9f7f-4856-b0b3-51e60b8de349 [ 5233.199447] nvme nvme1: Removing ctrl: NQN "nqn.2014-08.org.nvmexpress.discovery" [ 5233.227718] ============================================ [ 5233.231283] WARNING: possible recursive locking detected [ 5233.234696] 7.0.0-rc3nvme+ #20 Tainted: G O N [ 5233.238434] -------------------------------------------- [ 5233.241852] kworker/u192:6/2413 is trying to acquire lock: [ 5233.245429] ffff888111632548 ((wq_completion)nvmet-wq){+.+.}-{0:0}, at: touch_wq_lockdep_map+0x26/0x90 [ 5233.251438] but task is already holding lock: [ 5233.255254] ffff888111632548 ((wq_completion)nvmet-wq){+.+.}-{0:0}, at: process_one_work+0x5cc/0x6e0 [ 5233.261125] other info that might help us debug this: [ 5233.265333] Possible unsafe locking scenario: [ 5233.269217] CPU0 [ 5233.270795] ---- [ 5233.272436] lock((wq_completion)nvmet-wq); [ 5233.275241] lock((wq_completion)nvmet-wq); [ 5233.278020] *** DEADLOCK *** [ 5233.281793] May be due to missing lock nesting notation [ 5233.286195] 3 locks held by kworker/u192:6/2413: [ 5233.289192] #0: ffff888111632548 ((wq_completion)nvmet-wq){+.+.}-{0:0}, at: process_one_work+0x5cc/0x6e0 [ 5233.294569] #1: ffffc9000e2a7e40 ((work_completion)(&queue->release_work)){+.+.}-{0:0}, at: process_one_work+0x1c5/0x6e0 [ 5233.300128] #2: ffffffff82d7dc40 (rcu_read_lock){....}-{1:3}, at: __flush_work+0x62/0x530 [ 5233.304290] stack backtrace: [ 5233.306520] CPU: 4 UID: 0 PID: 2413 Comm: kworker/u192:6 Tainted: G O N 7.0.0-rc3nvme+ #20 PREEMPT(full) [ 5233.306524] Tainted: [O]=OOT_MODULE, [N]=TEST [ 5233.306525] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014 [ 5233.306527] Workqueue: nvmet-wq nvmet_tcp_release_queue_work [nvmet_tcp] [ 5233.306532] Call Trace: [ 5233.306534] [ 5233.306536] dump_stack_lvl+0x73/0xb0 [ 5233.306552] print_deadlock_bug+0x225/0x2f0 [ 5233.306556] __lock_acquire+0x13f0/0x2290 [ 5233.306563] lock_acquire+0xd0/0x300 [ 5233.306565] ? touch_wq_lockdep_map+0x26/0x90 [ 5233.306571] ? __flush_work+0x20b/0x530 [ 5233.306573] ? touch_wq_lockdep_map+0x26/0x90 [ 5233.306577] touch_wq_lockdep_map+0x3b/0x90 [ 5233.306580] ? touch_wq_lockdep_map+0x26/0x90 [ 5233.306583] ? __flush_work+0x20b/0x530 [ 5233.306585] __flush_work+0x268/0x530 [ 5233.306588] ? __pfx_wq_barrier_func+0x10/0x10 [ 5233.306594] ? xen_error_entry+0x30/0x60 [ 5233.306600] nvmet_ctrl_free+0x140/0x310 [nvmet] [ 5233.306617] nvmet_cq_put+0x74/0x90 [nvmet] [ 5233.306629] nvmet_tcp_release_queue_work+0x19f/0x360 [nvmet_tcp] [ 5233.306634] process_one_work+0x206/0x6e0 [ 5233.306640] worker_thread+0x184/0x320 [ 5233.306643] ? __pfx_worker_thread+0x10/0x10 [ 5233.306646] kthread+0xf1/0x130 [ 5233.306648] ? __pfx_kthread+0x10/0x10 [ 5233.306651] ret_from_fork+0x355/0x450 [ 5233.306653] ? __pfx_kthread+0x10/0x10 [ 5233.306656] ret_from_fork_asm+0x1a/0x30 [ 5233.306664] There is also no need to flush async_event_work from controller teardown. The admin queue teardown already fails outstanding AER requests before the final controller put :- nvmet_sq_destroy(admin sq) nvmet_async_events_failall(ctrl) The controller has already been removed from the subsystem list before nvmet_ctrl_free() quiesces outstanding work. Replace flush_work() with cancel_work_sync() so a pending async_event_work item is canceled and a running instance is waited on without recursing into the same workqueue. Fixes: 06406d81a2d7 ("nvmet: cancel fatal error and flush async work before free controller") Cc: stable@vger.kernel.org Reviewed-by: Christoph Hellwig Signed-off-by: Chaitanya Kulkarni Signed-off-by: Keith Busch Signed-off-by: Greg Kroah-Hartman