kernel/linux.git/lib/iov_iter.c, branch v6.18.33

x86-64/arm64/powerpc: clean up and rename __copy_from_user_flushcache

2026-04-22T11:22:31+00:00

commit 809b997a5ce945ab470f70c187048fe4f5df20bf upstream. This finishes the work on these odd functions that were only implemented by a handful of architectures. The 'flushcache' function was only used from the iterator code, and let's make it do the same thing that the nontemporal version does: remove the two underscores and add the user address checking. Yes, yes, the user address checking is also done at iovec import time, but we have long since walked away from the old double-underscore thing where we try to avoid address checking overhead at access time, and these functions shouldn't be so special and old-fashioned. The arm64 version already did the address check, in fact, so there it's just a matter of renaming it. For powerpc and x86-64 we now do the proper user access boilerplate. Signed-off-by: Linus Torvalds Signed-off-by: Greg Kroah-Hartman

x86: rename and clean up __copy_from_user_inatomic_nocache()

2026-04-22T11:22:31+00:00

commit 5de7bcaadf160c1716b20a263cf8f5b06f658959 upstream. Similarly to the previous commit, this renames the somewhat confusingly named function. But in this case, it was at least less confusing: the __copy_from_user_inatomic_nocache is indeed copying from user memory, and it is indeed ok to be used in an atomic context, so it will not warn about it. But the previous commit also removed the NTB mis-use of the __copy_from_user_inatomic_nocache() function, and as a result every call-site is now _actually_ doing a real user copy. That means that we can now do the proper user pointer verification too. End result: add proper address checking, remove the double underscores, and change the "nocache" to "nontemporal" to more accurately describe what this x86-only function actually does. It might be worth noting that only the target is non-temporal: the actual user accesses are normal memory accesses. Also worth noting is that non-x86 targets (and on older 32-bit x86 CPU's before XMM2 in the Pentium III) we end up just falling back on a regular user copy, so nothing can actually depend on the non-temporal semantics, but that has always been true. Signed-off-by: Linus Torvalds Signed-off-by: Greg Kroah-Hartman

iov_iter: remove iov_iter_is_aligned

2025-09-09T16:27:01+00:00

No more callers. Signed-off-by: Keith Busch Reviewed-by: Hannes Reinecke Reviewed-by: Martin K. Petersen Reviewed-by: Christoph Hellwig Reviewed-by: Mike Snitzer Signed-off-by: Christoph Hellwig Signed-off-by: Jens Axboe

iov_iter: use iov_offset for length calculation in iov_iter_aligned_bvec

2025-06-06T05:02:23+00:00

If iov_offset is non-zero, then we need to consider iov_offset in length calculation, otherwise we might pass smaller IOs such as 512 bytes, in below scenario [1]. This issue is reproducible using lib-uring test/fixed-seg.c application with fixed buffer on a 512 LBA formatted device. [1] At present we pass the alignment check, for 512 LBA formatted devices, len_mask = 511 when IO is smaller, i->count = 512 has an offset, i->io_offset = 3584 with bvec values, bvec->bv_offset = 256, bvec->bv_len = 3840. In short, the first 256 bytes are in the current page, next 256 bytes are in the another page. Ideally we expect to fail the IO. I can think of 2 userspace scenarios where we experience this. a: From userspace, we observe a different behaviour when device LBA size is 512 vs 4096 bytes. For 4096 LBA formatted device, I see the same liburing test [2] failing, whereas 512 the test passes without this. This is reproducible everytime. [2] https://github.com/axboe/liburing/ b: Although I was not able to reproduce the below condition, but I suspect below case should be possible from user space for devices with 512 LBA formatted device. Lets say from userspace while allocating a virtually single chunk of memory, if we get 2 physical chunk of memory, and IO happens to be at the boundary of first physical chunk with length crossing first chunk, then we allow IOs to proceed and hence we might map wrong physical address length and proceed with IO rather than failing. : --- a/test/fixed-seg.c : +++ b/test/fixed-seg.c : @@ -64,7 +64,7 @@ static int test(struct io_uring *ring, int fd, int : vec_off) : return T_EXIT_FAIL; : } : : - ret = read_it(ring, fd, 4096, vec_off); : + ret = read_it(ring, fd, 4096, 7*512 + 256); : if (ret) { : fprintf(stderr, "4096 0 failed\n"); : return T_EXIT_FAIL; Effectively this is a write crossing the page boundary. Link: https://lkml.kernel.org/r/20250428095849.11709-1-nj.shetty@samsung.com Fixes: 2263639f96f2 ("iov_iter: streamline iovec/bvec alignment iteration") Reviewed-by: Jens Axboe Reviewed-by: Anuj Gupta Signed-off-by: Nitesh Shetty Cc: Al Viro Cc: Christian Brauner Cc: Keith Busch Signed-off-by: Andrew Morton

iov: remove copy_page_from_iter_atomic()

2025-06-01T05:46:07+00:00

All callers now use copy_folio_from_iter_atomic(), so convert copy_page_from_iter_atomic(). While I'm in there, use kmap_local_folio() and pagefault_disable() instead of kmap_atomic(). That allows preemption and/or task migration to happen during the copy_from_user(). Also use the new folio_test_partial_kmap() predicate instead of open-coding it. Link: https://lkml.kernel.org/r/20250514170607.3000994-4-willy@infradead.org Signed-off-by: Matthew Wilcox (Oracle) Cc: Alexander Viro Cc: Hugh Dickins Cc: Konstantin Komarov Signed-off-by: Andrew Morton

iov_iter: convert iov_iter_extract_xarray_pages() to use folios

2025-05-12T00:48:05+00:00

ITER_XARRAY is exclusively used with xarrays that contain folios, not pages, so extract folio pointers from it, not page pointers. Removes a use of find_subpage(). Link: https://lkml.kernel.org/r/20250402210612.2444135-5-willy@infradead.org Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: David Hildenbrand Signed-off-by: Andrew Morton

iov_iter: convert iter_xarray_populate_pages() to use folios

2025-05-12T00:48:05+00:00

ITER_XARRAY is exclusively used with xarrays that contain folios, not pages, so extract folio pointers from it, not page pointers. Removes a hidden call to compound_head() and a use of find_subpage(). Link: https://lkml.kernel.org/r/20250402210612.2444135-4-willy@infradead.org Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: David Hildenbrand Signed-off-by: Andrew Morton

lib/iov_iter: fix to increase non slab folio refcount

2025-04-12T00:32:36+00:00

When testing EROFS file-backed mount over v9fs on qemu, I encountered a folio UAF issue. The page sanity check reports the following call trace. The root cause is that pages in bvec are coalesced across a folio bounary. The refcount of all non-slab folios should be increased to ensure p9_releas_pages can put them correctly. BUG: Bad page state in process md5sum pfn:18300 page: refcount:0 mapcount:0 mapping:00000000d5ad8e4e index:0x60 pfn:0x18300 head: order:0 mapcount:0 entire_mapcount:0 nr_pages_mapped:0 pincount:0 aops:z_erofs_aops ino:30b0f dentry name(?):"GoogleExtServicesCn.apk" flags: 0x100000000000041(locked|head|node=0|zone=1) raw: 0100000000000041 dead000000000100 dead000000000122 ffff888014b13bd0 raw: 0000000000000060 0000000000000020 00000000ffffffff 0000000000000000 head: 0100000000000041 dead000000000100 dead000000000122 ffff888014b13bd0 head: 0000000000000060 0000000000000020 00000000ffffffff 0000000000000000 head: 0100000000000000 0000000000000000 ffffffffffffffff 0000000000000000 head: 0000000000000010 0000000000000000 00000000ffffffff 0000000000000000 page dumped because: PAGE_FLAGS_CHECK_AT_FREE flag(s) set Call Trace: dump_stack_lvl+0x53/0x70 bad_page+0xd4/0x220 __free_pages_ok+0x76d/0xf30 __folio_put+0x230/0x320 p9_release_pages+0x179/0x1f0 p9_virtio_zc_request+0xa2a/0x1230 p9_client_zc_rpc.constprop.0+0x247/0x700 p9_client_read_once+0x34d/0x810 p9_client_read+0xf3/0x150 v9fs_issue_read+0x111/0x360 netfs_unbuffered_read_iter_locked+0x927/0x1390 netfs_unbuffered_read_iter+0xa2/0xe0 vfs_iocb_iter_read+0x2c7/0x460 erofs_fileio_rq_submit+0x46b/0x5b0 z_erofs_runqueue+0x1203/0x21e0 z_erofs_readahead+0x579/0x8b0 read_pages+0x19f/0xa70 page_cache_ra_order+0x4ad/0xb80 filemap_readahead.isra.0+0xe7/0x150 filemap_get_pages+0x7aa/0x1890 filemap_read+0x320/0xc80 vfs_read+0x6c6/0xa30 ksys_read+0xf9/0x1c0 do_syscall_64+0x9e/0x1a0 entry_SYSCALL_64_after_hwframe+0x71/0x79 Link: https://lkml.kernel.org/r/20250401144712.1377719-1-shengyong1@xiaomi.com Fixes: b9c0e49abfca ("mm: decline to manipulate the refcount on a slab page") Signed-off-by: Sheng Yong Reviewed-by: Matthew Wilcox (Oracle) Acked-by: Vlastimil Babka Cc: Signed-off-by: Andrew Morton

mm: decline to manipulate the refcount on a slab page

2025-03-17T00:40:26+00:00

Slab pages now have a refcount of 0, so nobody should be trying to manipulate the refcount on them. Doing so has little effect; the object could be freed and reallocated to a different purpose, although the slab itself would not be until the refcount was put making it behave rather like TYPESAFE_BY_RCU. Unfortunately, __iov_iter_get_pages_alloc() does take a refcount. Fix that to not change the refcount, and make put_page() silently not change the refcount. get_page() warns so that we can fix any other callers that need to be changed. Long-term, networking needs to stop taking a refcount on the pages that it uses and rely on the caller to hold whatever references are necessary to make the memory stable. In the medium term, more page types are going to hav a zero refcount, so we'll want to move get_page() and put_page() out of line. Link: https://lkml.kernel.org/r/20250310143544.1216127-1-willy@infradead.org Fixes: 9aec2fb0fd5e (slab: allocate frozen pages) Signed-off-by: Matthew Wilcox (Oracle) Reported-by: Hannes Reinecke Closes: https://lore.kernel.org/all/08c29e4b-2f71-4b6d-8046-27e407214d8c@suse.com/ Acked-by: Vlastimil Babka Signed-off-by: Andrew Morton

lib/iov_iter: fix import_iovec_ubuf iovec management

2025-02-18T06:40:00+00:00

import_iovec() says that it should always be fine to kfree the iovec returned in @iovp regardless of the error code. __import_iovec_ubuf() never reallocates it and thus should clear the pointer even in cases when copy_iovec_*() fail. Link: https://lkml.kernel.org/r/378ae26923ffc20fd5e41b4360d673bf47b1775b.1738332461.git.asml.silence@gmail.com Fixes: 3b2deb0e46da ("iov_iter: import single vector iovecs as ITER_UBUF") Signed-off-by: Pavel Begunkov Reviewed-by: Jens Axboe Cc: Al Viro Cc: Christian Brauner Cc: Signed-off-by: Andrew Morton