kernel/linux.git/mm/huge_memory.c, branch v7.2-rc6

mm/huge_memory: unlock i_mmap_rwsem before releasing after-split folios

2026-07-29T00:37:32+00:00

__folio_split() keeps dereferencing the mapping after the split: shmem_uncharge(mapping->host) and remap_page() while the folios are still frozen/locked, and i_mmap_unlock_read(mapping) at the very end, after the after-split folios have been unlocked and freed. Nothing holds an inode reference across that. The split relies on @folio -- which the beyond-EOF drop loop never removes, as it starts at folio_next(folio) -- staying locked and in the page cache to hold off eviction. But the unlock loop unlocks @folio before i_mmap_unlock_read() runs. If the caller's @lock_at is a tail beyond EOF, as memory_failure() passes when splitting a poisoned tail of a shmem THP that reaches past i_size during truncation, it too is gone from the page cache; so once @folio is unlocked no locked, in-cache folio pins the inode, and a concurrent final iput() can evict and RCU-free it before i_mmap_unlock_read() touches i_mmap_rwsem: BUG: KASAN: slab-use-after-free in __up_read+0x634/0x790 i_mmap_unlock_read include/linux/fs.h:537 [inline] __folio_split+0x732/0x1640 mm/huge_memory.c:4100 try_to_split_thp_page+0xab/0x390 mm/memory-failure.c:1675 memory_failure+0x1394/0x26e0 mm/memory-failure.c:2470 Freed by task 4601: shmem_free_in_core_inode+0x54/0xb0 mm/shmem.c:5177 evict+0x57f/0xac0 fs/inode.c:870 Do every mapping dereference while @folio still pins the inode: drop i_mmap_rwsem right after remap_page(), before the loop that unlocks and frees the after-split folios, and clear @mapping so the exit path does not unlock it again. shmem_uncharge() and remap_page() already run before that point, so after this nothing past the unlock loop touches the inode or the mapping. This is now a rule the split depends on, alongside keeping @folio frozen until the page cache is updated: no inode or mapping dereference once the after-split folios start being unlocked. Link: https://lore.kernel.org/20260716095424.471052-1-kirill@shutemov.name Fixes: baa355fd3314 ("thp: file pages support for split_huge_page()") Signed-off-by: Kiryl Shutsemau (Meta) Reported-by: Hao Zhang Closes: https://lore.kernel.org/linux-mm/20260710071344.GA106129@zh-pc Co-developed-by: Hao Zhang Signed-off-by: Hao Zhang Acked-by: David Hildenbrand (Arm) Reviewed-by: Zi Yan Reviewed-by: Baolin Wang Reviewed-by: Miaohe Lin Cc: Baolin Wang Cc: Barry Song Cc: Dev Jain Cc: Lance Yang Cc: Liam R. Howlett Cc: Lorenzo Stoakes Cc: Naoya Horiguchi Cc: Nico Pache Cc: Ryan Roberts Cc: Signed-off-by: Andrew Morton

userfaultfd: wait on source PMD during UFFDIO_MOVE

2026-07-21T00:41:26+00:00

move_pages_huge_pmd() snapshots src_pmdval under src_ptl, drops the lock, and, for migration entries, waits with pmd_migration_entry_wait(). Passing &src_pmdval is wrong. pmd_migration_entry_wait() must lock and re-read the real page-table PMD; on split-PMD-lock kernels, a stack address also resolves to the wrong lock. softleaf_entry_wait_on_locked() then waits without a folio reference, which is safe only while serialized against migration-entry removal by the real PT lock. Pass src_pmd, matching __handle_mm_fault() and hmm_vma_walk_pmd(). Link: https://lore.kernel.org/20260705131231.1499198-1-usama.arif@linux.dev Fixes: adef440691ba ("userfaultfd: UFFDIO_MOVE uABI") Reported-by: sashiko-bot Link: https://sashiko.dev/#/patchset/20260703173903.3789516-1-usama.arif%40linux.dev?part=8 Signed-off-by: Usama Arif Reviewed-by: Rik van Riel Reviewed-by: Baolin Wang Reviewed-by: Lance Yang Acked-by: David Hildenbrand (Arm) Reviewed-by: Lance Yang Reviewed-by: Lorenzo Stoakes Cc: Andrea Arcangeli Cc: Barry Song Cc: Dev Jain Cc: Johannes Weiner Cc: Liam R. Howlett Cc: Nico Pache Cc: Ryan Roberts Cc: Shakeel Butt Cc: Zi Yan Cc: Signed-off-by: Andrew Morton

mm/huge_memory: set PG_has_hwpoisoned only after new folio head is established

2026-07-09T22:48:54+00:00

__split_folio_to_order() copies the hwpoison state onto each new sub-folio while splitting a folio to a non-zero order. It does so via if (handle_hwpoison && page_range_has_hwpoisoned(new_head, new_nr_pages)) folio_set_has_hwpoisoned(new_folio); *before* clear_compound_head(new_head)/prep_compound_page(new_head, ...) turns @new_head from a tail page into a proper folio head. PG_has_hwpoisoned is a FOLIO_SECOND_PAGE flag, so folio_set_has_hwpoisoned() resolves to folio_flags(folio, 1). With the new compound_info-based page-flags layout, folio_flags() asserts the page is not a tail: VM_BUG_ON_PGFLAGS(page->compound_info & 1, page); VM_BUG_ON_PGFLAGS(n > 0 && !test_bit(PG_head, &page->flags.f), page); At the current call site @new_head still has the tail marker (compound_info bit 0 set, PG_head clear), so on CONFIG_DEBUG_VM kernels this hits: kernel BUG at include/linux/page-flags.h:354 folio_flags+0x82 folio_set_has_hwpoisoned __split_folio_to_order __split_unmapped_folio __folio_split truncate_inode_partial_folio (shmem hole-punch / MADV_REMOVE) Reproduced by syzkaller: hwpoison-inject a few subpages of a large shmem folio, then MADV_REMOVE (fallocate punch hole) on the same range, which splits the partial folio to a non-zero order. memory_failure() tries to split the poisoned folio to order 0 first, but that split is best-effort; when it fails the folio is left large with PG_has_hwpoisoned set, the case fa5a06170036 added this hwpoison copying for. Move the folio_set_has_hwpoisoned() call to after clear_compound_head()/prep_compound_page(), where @new_folio is a real order-new_order head folio (handle_hwpoison implies new_order != 0, so a second page always exists). The flag still lands on the same struct page (page[1] of the new folio); only the ordering relative to compound-head setup changes, satisfying the FOLIO_SECOND_PAGE precondition. Link: https://lore.kernel.org/20260701174235.3173401-1-riel@surriel.com Fixes: fa5a06170036 ("mm/huge_memory: preserve PG_has_hwpoisoned if a folio is split to >0 order") Signed-off-by: Rik van Riel Assisted-by: Claude:claude-opus-4-8 Reviewed-by: Zi Yan Acked-by: David Hildenbrand (Arm) Tested-by: Lance Yang Reviewed-by: Lorenzo Stoakes Reviewed-by: Baolin Wang Cc: Barry Song Cc: Dev Jain Cc: Lance Yang Cc: Liam R. Howlett Cc: Nico Pache Cc: Ryan Roberts Cc: Yang Shi Cc: Signed-off-by: Andrew Morton

Merge tag 'mm-stable-2026-06-23-08-55' of git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm

2026-06-23T19:03:44+00:00

Pull more MM updates from Andrew Morton: - "khugepaged: add mTHP collapse support" (Nico Pache) Provide khugepaged with the capability to collapse anonymous memory regions to mTHPs - "Remove CONFIG_READ_ONLY_THP_FOR_FS and enable file THP for writable files" (Zi Yan) Remove the READ_ONLY_THP_FOR_FS check in file_thp_enabled(), so that khugepaged and MADV_COLLAPSE can run on filesystems with PMD THP pagecache support even without READ_ONLY_THP_FOR_FS enabled - "make MM selftests more CI friendly" (Mike Rapoport) General fixes and cleanups to the MM selftests. Also move more MM selftests under the kselftest framework, making them more amenable to ongoing CI testing - "selftests/mm: fix failures and robustness improvements" and "selftests/mm: assorted fixes for hmm-tests" (Sayali Patil) Fix several issues in MM selftests which were revealed by powerpc 64k pagesize * tag 'mm-stable-2026-06-23-08-55' of git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm: (118 commits) Revert "mm: limit filemap_fault readahead to VMA boundaries" mm/vmscan: pass NULL to trace vmscan node reclaim mm: use mapping_mapped to simplify the code selftests/mm: fix exclusive_cow test fork() handling selftests/mm: remove hardcoded THP sizing assumptions in hmm tests selftests/mm: allow PUD-level entries in compound testcase of hmm tests mm/gup_test: reject wrapped user ranges mm/page_frag: reject invalid CPUs in page_frag_test mm/damon/core: always put unsuccessfully committed target pids mm: page_isolation: avoid unsafe folio reads while scanning compound pages mm/shrinker: do not hold RCU lock in shrinker_debugfs_count_show() selftests: mm: fix and speedup "droppable" test mm: merge writeout into pageout MAINTAINERS: add Hao Ge as reviewer for codetag and alloc_tag selftests/mm: clarify alternate unmapping in compaction_test selftests/mm: move hwpoison setup into run_test() and silence modprobe output for memory-failure category selftests/mm: skip uffd-stress test when nr_pages_per_cpu is zero selftests/mm: skip uffd-wp-mremap if UFFD write-protect is unsupported selftests/mm: ensure destination is hugetlb-backed in hugetlb-mremap selftest/mm: register existing mapping with userfaultfd in hugetlb-mremap ...

mm/khugepaged: enable clean pagecache folio collapse for writable files

2026-06-21T18:37:17+00:00

collapse_file() is capable of collapsing pagecache folios from writable files to PMD folios. Now enable clean pagecache folio collapse in addition to read-only pagecache folio collapse by removing the inode_is_open_for_write() from file_thp_enabled() and only performing filemap_flush() if the file is read-only. This means userspace needs to explicitly flush the content of pagecache folios before khugepaged can collapse the folios, or use madvise(MADV_COLLAPSE), which does the flush in the retry. The reason is that blindly enabling dirty pagecache folio from writable files collapse makes khugepaged flush these folios all the time. It is undesirable to cause system level pagecache flushes. To properly support dirty pagecache folio collapse, filemap_flush() needs to be avoided. Potentially, merging associated buffer instead of dropping it with filemap_release_folio() might be needed. NOTE: this breaks khugepaged selftests for writable file pagecache collapse, which is set to fail all the time. The next commit fixes it. Link: https://lore.kernel.org/20260517135416.1434539-14-ziy@nvidia.com Signed-off-by: Zi Yan Reviewed-by: Lance Yang Cc: Al Viro Cc: Baolin Wang Cc: Barry Song Cc: Chris Mason Cc: Christian Brauner Cc: David Hildenbrand (Arm) Cc: David Sterba Cc: Dev Jain Cc: Jan Kara Cc: Liam Howlett Cc: Lorenzo Stoakes Cc: Matthew Wilcox (Oracle) Cc: Michal Hocko Cc: Mike Rapoport Cc: Nico Pache Cc: Ryan Roberts Cc: Shuah Khan Cc: Song Liu Cc: Suren Baghdasaryan Cc: Vlastimil Babka Signed-off-by: Andrew Morton

mm/huge_memory: remove folio split check for READ_ONLY_THP_FOR_FS

2026-06-21T18:37:15+00:00

Without READ_ONLY_THP_FOR_FS, large file-backed folios cannot be created by a FS without large folio support. The check is no longer needed. Link: https://lore.kernel.org/20260517135416.1434539-9-ziy@nvidia.com Signed-off-by: Zi Yan Reviewed-by: Lance Yang Reviewed-by: Lorenzo Stoakes (Oracle) Reviewed-by: Baolin Wang Acked-by: David Hildenbrand (Arm) Cc: Al Viro Cc: Barry Song Cc: Chris Mason Cc: Christian Brauner Cc: David Sterba Cc: Dev Jain Cc: Jan Kara Cc: Liam Howlett Cc: Matthew Wilcox (Oracle) Cc: Michal Hocko Cc: Mike Rapoport Cc: Nico Pache Cc: Ryan Roberts Cc: Shuah Khan Cc: Song Liu Cc: Suren Baghdasaryan Cc: Vlastimil Babka Signed-off-by: Andrew Morton

mm: fs: remove filemap_nr_thps*() functions and their users

2026-06-21T18:37:15+00:00

They are used by READ_ONLY_THP_FOR_FS to handle writes to FSes without large folio support, so that read-only THPs created in these FSes are not seen by the FSes when the underlying fd becomes writable. Now read-only PMD THPs only appear in a FS with large folio support and the supported orders include PMD_ORDER. READ_ONLY_THP_FOR_FS was using mapping->nr_thps, inode->i_writecount, and smp_mb() to prevent writes to a read-only THP and collapsing writable folios into a THP. In collapse_file(), mapping->nr_thps is increased, then smp_mb(), and if inode->i_writecount > 0, collapse is stopped, while do_dentry_open() first increases inode->i_writecount, then a full memory fence, and if mapping->nr_thps > 0, all read-only THPs are truncated. Now this mechanism can be removed along with READ_ONLY_THP_FOR_FS code, since a dirty folio check has been added after try_to_unmap() in collapse_file() to prevent dirty folios from being collapsed as clean. Link: https://lore.kernel.org/20260517135416.1434539-7-ziy@nvidia.com Signed-off-by: Zi Yan Reviewed-by: Matthew Wilcox (Oracle) Acked-by: David Hildenbrand (Arm) Reviewed-by: Baolin Wang Reviewed-by: Lance Yang Cc: Al Viro Cc: Barry Song Cc: Chris Mason Cc: Christian Brauner Cc: David Sterba Cc: Dev Jain Cc: Jan Kara Cc: Liam Howlett Cc: Lorenzo Stoakes Cc: Michal Hocko Cc: Mike Rapoport Cc: Nico Pache Cc: Ryan Roberts Cc: Shuah Khan Cc: Song Liu Cc: Suren Baghdasaryan Cc: Vlastimil Babka Signed-off-by: Andrew Morton

mm/huge_memory: remove READ_ONLY_THP_FOR_FS from file_thp_enabled()

2026-06-21T18:37:14+00:00

Replace it with a check on the max folio order of the file's address space mapping, making sure PMD folio is supported. Keep the inode open-for-write check, since even if collapse_file() now makes sure all to-be-collapsed folios are clean and the created PMD file THP can be handled by FSes properly, the filemap_flush() could perform undesirable write back. Link: https://lore.kernel.org/20260517135416.1434539-4-ziy@nvidia.com Signed-off-by: Zi Yan Acked-by: David Hildenbrand (Arm) Reviewed-by: Lance Yang Reviewed-by: Nico Pache Cc: Al Viro Cc: Baolin Wang Cc: Barry Song Cc: Chris Mason Cc: Christian Brauner Cc: David Sterba Cc: Dev Jain Cc: Jan Kara Cc: Liam Howlett Cc: Lorenzo Stoakes Cc: Matthew Wilcox (Oracle) Cc: Michal Hocko Cc: Mike Rapoport Cc: Ryan Roberts Cc: Shuah Khan Cc: Song Liu Cc: Suren Baghdasaryan Cc: Vlastimil Babka Signed-off-by: Andrew Morton

mm/khugepaged: add per-order mTHP collapse failure statistics

2026-06-21T18:31:31+00:00

Add three new mTHP statistics to track collapse failures for different orders when encountering swap PTEs, excessive none PTEs, and shared PTEs: - collapse_exceed_swap_pte: Increment when mTHP collapse fails due to encountering a swap PTE. - collapse_exceed_none_pte: Counts when mTHP collapse fails due to exceeding the none PTE threshold for the given order - collapse_exceed_shared_pte: Counts when mTHP collapse fails due to encountering a shared PTE. These statistics complement the existing THP_SCAN_EXCEED_* events by providing per-order granularity for mTHP collapse attempts. The stats are exposed via sysfs under `/sys/kernel/mm/transparent_hugepage/hugepages-*/stats/` for each supported hugepage size. As we currently do not support collapsing mTHPs that contain a swap or shared entry, those statistics keep track of how often we are encountering failed mTHP collapses due to these restrictions. We will add support for mTHP collapse for anonymous pages next; lets also track when this happens at the PMD level within the per-mTHP stats. Link: https://lore.kernel.org/20260605161422.213817-9-npache@redhat.com Signed-off-by: Nico Pache Reviewed-by: Lorenzo Stoakes Acked-by: David Hildenbrand (Arm) Reviewed-by: Lance Yang Cc: Alistair Popple Cc: Andrea Arcangeli Cc: Anshuman Khandual Cc: Bagas Sanjaya Cc: Baolin Wang Cc: Barry Song Cc: Brendan Jackman Cc: Byungchul Park Cc: Catalin Marinas Cc: David Rientjes Cc: Dev Jain Cc: Gregory Price Cc: "Huang, Ying" Cc: Hugh Dickins Cc: Jan Kara Cc: Jann Horn Cc: Johannes Weiner Cc: Jonathan Corbet Cc: Joshua Hahn Cc: Kefeng Wang Cc: Liam R. Howlett Cc: "Masami Hiramatsu (Google)" Cc: Mathieu Desnoyers Cc: Matthew Brost Cc: Matthew Wilcox (Oracle) Cc: Michal Hocko Cc: Mike Rapoport Cc: Nanyong Sun Cc: Pedro Falcato Cc: Peter Xu Cc: Rafael Aquini Cc: Rakie Kim Cc: Randy Dunlap Cc: Ryan Roberts Cc: Shivank Garg Cc: Steven Rostedt Cc: Suren Baghdasaryan Cc: Takashi Iwai (SUSE) Cc: Thomas Hellström Cc: Usama Arif Cc: Usama Arif Cc: Vishal Moola (Oracle) Cc: Vlastimil Babka Cc: Wei Yang Cc: Will Deacon Cc: Yang Shi Cc: Zach O'Keefe Cc: Zi Yan Signed-off-by: Andrew Morton

mm/khugepaged: generalize alloc_charge_folio()

2026-06-21T18:31:29+00:00

Pass order to alloc_charge_folio() and update mTHP statistics. Link: https://lore.kernel.org/20260605161422.213817-3-npache@redhat.com Signed-off-by: Dev Jain Co-developed-by: Nico Pache Signed-off-by: Nico Pache Reviewed-by: Wei Yang Reviewed-by: Lance Yang Reviewed-by: Baolin Wang Reviewed-by: Lorenzo Stoakes Reviewed-by: Zi Yan Acked-by: Usama Arif Acked-by: David Hildenbrand (Arm) Cc: Alistair Popple Cc: Andrea Arcangeli Cc: Anshuman Khandual Cc: Bagas Sanjaya Cc: Barry Song Cc: Brendan Jackman Cc: Byungchul Park Cc: Catalin Marinas Cc: David Rientjes Cc: Gregory Price Cc: "Huang, Ying" Cc: Hugh Dickins Cc: Jan Kara Cc: Jann Horn Cc: Johannes Weiner Cc: Jonathan Corbet Cc: Joshua Hahn Cc: Kefeng Wang Cc: Liam R. Howlett Cc: "Masami Hiramatsu (Google)" Cc: Mathieu Desnoyers Cc: Matthew Brost Cc: Matthew Wilcox (Oracle) Cc: Michal Hocko Cc: Mike Rapoport Cc: Nanyong Sun Cc: Pedro Falcato Cc: Peter Xu Cc: Rafael Aquini Cc: Rakie Kim Cc: Randy Dunlap Cc: Ryan Roberts Cc: Shivank Garg Cc: Steven Rostedt Cc: Suren Baghdasaryan Cc: Takashi Iwai (SUSE) Cc: Thomas Hellström Cc: Usama Arif Cc: Vishal Moola (Oracle) Cc: Vlastimil Babka Cc: Will Deacon Cc: Yang Shi Cc: Zach O'Keefe Signed-off-by: Andrew Morton