kernel/linux.git/mm/huge_memory.c, branch v7.1.7

mm/huge_memory: set PG_has_hwpoisoned only after new folio head is established

2026-08-03T09:25:55+00:00

commit e1cd30eceb6908fc13bebce41283b885d71ee8d6 upstream. __split_folio_to_order() copies the hwpoison state onto each new sub-folio while splitting a folio to a non-zero order. It does so via if (handle_hwpoison && page_range_has_hwpoisoned(new_head, new_nr_pages)) folio_set_has_hwpoisoned(new_folio); *before* clear_compound_head(new_head)/prep_compound_page(new_head, ...) turns @new_head from a tail page into a proper folio head. PG_has_hwpoisoned is a FOLIO_SECOND_PAGE flag, so folio_set_has_hwpoisoned() resolves to folio_flags(folio, 1). With the new compound_info-based page-flags layout, folio_flags() asserts the page is not a tail: VM_BUG_ON_PGFLAGS(page->compound_info & 1, page); VM_BUG_ON_PGFLAGS(n > 0 && !test_bit(PG_head, &page->flags.f), page); At the current call site @new_head still has the tail marker (compound_info bit 0 set, PG_head clear), so on CONFIG_DEBUG_VM kernels this hits: kernel BUG at include/linux/page-flags.h:354 folio_flags+0x82 folio_set_has_hwpoisoned __split_folio_to_order __split_unmapped_folio __folio_split truncate_inode_partial_folio (shmem hole-punch / MADV_REMOVE) Reproduced by syzkaller: hwpoison-inject a few subpages of a large shmem folio, then MADV_REMOVE (fallocate punch hole) on the same range, which splits the partial folio to a non-zero order. memory_failure() tries to split the poisoned folio to order 0 first, but that split is best-effort; when it fails the folio is left large with PG_has_hwpoisoned set, the case fa5a06170036 added this hwpoison copying for. Move the folio_set_has_hwpoisoned() call to after clear_compound_head()/prep_compound_page(), where @new_folio is a real order-new_order head folio (handle_hwpoison implies new_order != 0, so a second page always exists). The flag still lands on the same struct page (page[1] of the new folio); only the ordering relative to compound-head setup changes, satisfying the FOLIO_SECOND_PAGE precondition. Link: https://lore.kernel.org/20260701174235.3173401-1-riel@surriel.com Fixes: fa5a06170036 ("mm/huge_memory: preserve PG_has_hwpoisoned if a folio is split to >0 order") Signed-off-by: Rik van Riel Assisted-by: Claude:claude-opus-4-8 Reviewed-by: Zi Yan Acked-by: David Hildenbrand (Arm) Tested-by: Lance Yang Reviewed-by: Lorenzo Stoakes Reviewed-by: Baolin Wang Cc: Barry Song Cc: Dev Jain Cc: Lance Yang Cc: Liam R. Howlett Cc: Nico Pache Cc: Ryan Roberts Cc: Yang Shi Cc: Signed-off-by: Andrew Morton Signed-off-by: Greg Kroah-Hartman

mm/huge_memory: preserve pmd_swp_uffd_wp on device-private PMD downgrade

2026-07-24T14:21:01+00:00

commit f7e2c21bd1f57cd5350eecdfdb5d6025ca6afbab upstream. change_non_present_huge_pmd() rewrites a writable device-private PMD swap entry into a readable one without carrying pmd_swp_uffd_wp() across. The PTE-level change_softleaf_pte() does this correctly; mirror that here, matching what copy_huge_pmd() does for the fork path. Without the carry, a plain mprotect() over a UFFD_WP-marked device-private THP strips the bit and the trap is bypassed on swap-in. Link: https://lore.kernel.org/20260529172331.356655-5-kas@kernel.org Fixes: 368076f52ebe ("mm/huge_memory: add device-private THP support to PMD operations") Signed-off-by: Kiryl Shutsemau Reported-by: Sashiko AI review Reviewed-by: Balbir Singh Cc: David Hildenbrand Cc: Lorenzo Stoakes Cc: Michal Hocko Cc: Mike Rapoport Cc: Peter Xu Cc: Suren Baghdasaryan Cc: Vlastimil Babka Cc: Signed-off-by: Andrew Morton Signed-off-by: Greg Kroah-Hartman

mm: preserve PG_dropbehind flag during folio split

2026-07-24T14:19:31+00:00

[ Upstream commit 0ad186cf167fdbeb6b29a8005c71973781036bd3 ] __split_folio_to_order() copies page flags from the original folio to newly created sub-folios using an explicit allowlist, but PG_dropbehind is not included. When a large folio with PG_dropbehind set is split, only the head sub-folio retains the flag; all tail sub-folios silently lose it and will not be reclaimed eagerly after writeback completes. Add PG_dropbehind to the flag copy mask so that the drop-behind hint is preserved across folio splits. Fixes: a323281cdfec ("mm: add PG_dropbehind folio flag") Assisted-by: Claude:claude-opus-4-6 Signed-off-by: Jeff Layton Link: https://patch.msgid.link/20260511-dontcache-v7-1-2848ddce8090@kernel.org Acked-by: David Hildenbrand (Arm) Reviewed-by: Jan Kara Signed-off-by: Christian Brauner (Amutable) Signed-off-by: Sasha Levin

mm/huge_memory: use correct flags for device private PMD entry

2026-06-03T23:25:51+00:00

Commit 65edfda6f3f2 ("mm/rmap: extend rmap and migration support device-private entries") updated set_pmd_migration_entry() to use pmdp_huge_get_and_clear() in the softleaf case, but made no further adjustments to the function itself. Therefore this function continues to incorrectly use pmd_write(), pmd_soft_dirty() and pmd_uffd_wp() to determine whether the installed migration entry should be marked writable, softdirty or uffd-wp respectively. Whilst all are incorrect, the most problematic of these is pmd_write(), as this can lead to corrupted rmap state. On x86-64 _PAGE_SWP_SOFT_DIRTY is aliased to _PAGE_RW. So calling pmd_write() on a softleaf will return the softdirty state encoded in the entry, assuming CONFIG_MEM_SOFT_DIRTY was enabled. This was observed when running the hmm.hmm_device_private.anon_write_child selftest: 1. The test faults in a range then migrates it such that a device-private THP range is established. 2. The parent then migrates it to a device-private writable PMD entry whose folio is entirely AnonExclusive with entire_mapcount=1, softdirty set (accidentally correct write state). 3. The parent forks and the PMD entries are set to device-private read only entries, entire_mapcount=2, softdirty still set. 4. [BUG] The child writes to the range then migrates to RAM - intending to install non-writable migration entries - but replacing parent and child PMD mappings with WRITABLE entries due to misinterpreting the softdirty bit. 5. In remove_migration_pmd(), if !softleaf_is_migration_read(entry) we set the RMAP_EXCLUSIVE flag when calling folio_add_anon_rmap_pmd() for both parent and child, which are therefore AnonExclusive. 6. [SPLAT] Child sets migrated folio entire_mapcount=1, parent sets entire_mapcount=2 and we end up with an AnonExclusive folio with entire_mapcount=2! Assert fires in __folio_add_anon_rmap(): VM_WARN_ON_FOLIO(folio_test_large(folio) && folio_entire_mapcount(folio) > 1 && PageAnonExclusive(cur_page), folio) This patch fixes the issue by correctly referencing the softleaf entry fields for writable, softdirty and uffd-wp in set_pmd_migration_entry(). It also only updates A/D flags if the entry is present as these are otherwise not meaningful for a softleaf entry. This patch also flips the if (!present) { ... } else { ... } logic in set_pmd_migration_entry() so it is easier to understand, and adds some comments to make things clearer. I was able to bisect this to commit 775465fd26a3 ("lib/test_hmm: add zone device private THP test infrastructure") which first exposes this bug as it was the commit that permitted test_hmm to generate the test. However commit 65edfda6f3f2 ("mm/rmap: extend rmap and migration support device-private entries") is the commit that actually enabled this behaviour. Link: https://lore.kernel.org/20260601083044.57132-1-ljs@kernel.org Fixes: 65edfda6f3f2 ("mm/rmap: extend rmap and migration support device-private entries") Signed-off-by: Lorenzo Stoakes Acked-by: David Hildenbrand (Arm) Reviewed-by: Dev Jain Reviewed-by: Balbir Singh Reviewed-by: Baolin Wang Reviewed-by: Oscar Salvador (SUSE) Reviewed-by: Barry Song Reviewed-by: Lance Yang Reviewed-by: Zi Yan Cc: Baolin Wang Cc: Liam R. Howlett Cc: Nico Pache Cc: Ryan Roberts Cc: SeongJae Park Cc: Wei Yang Cc: Signed-off-by: Andrew Morton

mm/huge_memory: update file PMD counter before folio_put()

2026-06-01T04:50:25+00:00

__split_huge_pmd_locked() updates the file/shmem RSS counter after dropping the PMD mapping's folio reference. If folio_put() drops the last reference, mm_counter_file() can later read freed folio state via folio_test_swapbacked(). Move the counter update before folio_put(). Link: https://lore.kernel.org/20260526101337.1984081-1-yintirui@huawei.com Fixes: fadae2953072 ("thp: use mm_file_counter to determine update which rss counter") Signed-off-by: Yin Tirui Reviewed-by: Lorenzo Stoakes Acked-by: David Hildenbrand (arm) Reviewed-by: Lance Yang Reviewed-by: Dev Jain Cc: Baolin Wang Cc: Barry Song Cc: Chen Jun Cc: Kefeng Wang Cc: Liam R. Howlett Cc: Nico Pache Cc: Ryan Roberts Cc: Vlastimil Babka Cc: Yang Shi Cc: Zi Yan Cc: Signed-off-by: Andrew Morton

mm/huge_memory: update file PUD counter before folio_put()

2026-06-01T04:50:25+00:00

__split_huge_pud_locked() updates the file/shmem RSS counter after dropping the PUD mapping's folio reference. If folio_put() drops the last reference, mm_counter_file() can later read freed folio state via folio_test_swapbacked(). Move the counter update before folio_put(). Link: https://lore.kernel.org/20260526101355.1984244-1-yintirui@huawei.com Fixes: dbe54153296d ("mm/huge_memory: add vmf_insert_folio_pud()") Signed-off-by: Yin Tirui Reviewed-by: Lorenzo Stoakes Acked-by: David Hildenbrand (arm) Reviewed-by: Lance Yang Reviewed-by: Dev Jain Cc: Alistair Popple Cc: Baolin Wang Cc: Barry Song Cc: Chen Jun Cc: Kefeng Wang Cc: Liam R. Howlett Cc: Nico Pache Cc: Ryan Roberts Cc: Zi Yan Cc: Signed-off-by: Andrew Morton

mm: thp: prevent memory cgroup release in folio_split_queue_lock{_irqsave}()

2026-04-18T07:10:46+00:00

In the near future, a folio will no longer pin its corresponding memory cgroup. To ensure safety, it will only be appropriate to hold the rcu read lock or acquire a reference to the memory cgroup returned by folio_memcg(), thereby preventing it from being released. In the current patch, the rcu read lock is employed to safeguard against the release of the memory cgroup in folio_split_queue_lock{_irqsave}(). Link: https://lore.kernel.org/ca2957c0df1126b2c71b40c738018fd5255525a6.1772711148.git.zhengqi.arch@bytedance.com Signed-off-by: Qi Zheng Reviewed-by: Harry Yoo Acked-by: Johannes Weiner Acked-by: Shakeel Butt Acked-by: David Hildenbrand (Red Hat) Acked-by: Muchun Song Cc: Allen Pais Cc: Axel Rasmussen Cc: Baoquan He Cc: Chengming Zhou Cc: Chen Ridong Cc: Hamza Mahfooz Cc: Hugh Dickins Cc: Imran Khan Cc: Kamalesh Babulal Cc: Lance Yang Cc: Liam Howlett Cc: Lorenzo Stoakes (Oracle) Cc: Michal Hocko Cc: Michal Koutný Cc: Mike Rapoport Cc: Muchun Song Cc: Nhat Pham Cc: Roman Gushchin Cc: Suren Baghdasaryan Cc: Usama Arif Cc: Vlastimil Babka Cc: Wei Xu Cc: Yosry Ahmed Cc: Yuanchu Xie Cc: Zi Yan Signed-off-by: Andrew Morton

mm: rename unlock_page_lruvec_irq and its variants

2026-04-18T07:10:44+00:00

It is inappropriate to use folio_lruvec_lock() variants in conjunction with unlock_page_lruvec() variants, as this involves the inconsistent operation of locking a folio while unlocking a page. To rectify this, the functions unlock_page_lruvec{_irq, _irqrestore} are renamed to lruvec_unlock{_irq,_irqrestore}. Link: https://lore.kernel.org/4e5e05271a250df4d1812e1832be65636a78c957.1772711148.git.zhengqi.arch@bytedance.com Signed-off-by: Muchun Song Signed-off-by: Qi Zheng Acked-by: Roman Gushchin Acked-by: Johannes Weiner Reviewed-by: Harry Yoo Reviewed-by: Chen Ridong Acked-by: David Hildenbrand (Red Hat) Acked-by: Shakeel Butt Cc: Allen Pais Cc: Axel Rasmussen Cc: Baoquan He Cc: Chengming Zhou Cc: Hamza Mahfooz Cc: Hugh Dickins Cc: Imran Khan Cc: Kamalesh Babulal Cc: Lance Yang Cc: Liam Howlett Cc: Lorenzo Stoakes (Oracle) Cc: Michal Hocko Cc: Michal Koutný Cc: Mike Rapoport Cc: Muchun Song Cc: Nhat Pham Cc: Suren Baghdasaryan Cc: Usama Arif Cc: Vlastimil Babka Cc: Wei Xu Cc: Yosry Ahmed Cc: Yuanchu Xie Cc: Zi Yan Signed-off-by: Andrew Morton

mm: fix deferred split queue races during migration

2026-04-05T20:53:47+00:00

migrate_folio_move() records the deferred split queue state from src and replays it on dst. Replaying it after remove_migration_ptes(src, dst, 0) makes dst visible before it is requeued, so a concurrent rmap-removal path can mark dst partially mapped and trip the WARN in deferred_split_folio(). Move the requeue before remove_migration_ptes() so dst is back on the deferred split queue before it becomes visible again. Because migration still holds dst locked at that point, teach deferred_split_scan() to requeue a folio when folio_trylock() fails. Otherwise a fully mapped underused folio can be dequeued by the shrinker and silently lost from split_queue. [ziy@nvidia.com: move the comment] Link: https://lkml.kernel.org/r/FB71A764-0F10-4E5A-B4A0-BA4C7F138408@nvidia.com Link: https://syzkaller.appspot.com/bug?extid=a7067a757858ac8eb085 Link: https://lkml.kernel.org/r/20260401131032.13011-1-lance.yang@linux.dev Fixes: 8a8ca142a488 ("mm: migrate: requeue destination folio on deferred split queue") Signed-off-by: Lance Yang Signed-off-by: Zi Yan Reported-by: syzbot+a7067a757858ac8eb085@syzkaller.appspotmail.com Closes: https://lore.kernel.org/linux-mm/69ccb65b.050a0220.183828.003a.GAE@google.com/ Suggested-by: David Hildenbrand (Arm) Acked-by: David Hildenbrand (Arm) Acked-by: Zi Yan Cc: Alistair Popple Cc: Baolin Wang Cc: Barry Song Cc: Byungchul Park Cc: David Hildenbrand Cc: Deepanshu Kartikey Cc: Dev Jain Cc: Gregory Price Cc: "Huang, Ying" Cc: Joshua Hahn Cc: Lance Yang Cc: Liam Howlett Cc: Lorenzo Stoakes (Oracle) Cc: Matthew Brost Cc: Nico Pache Cc: Rakie Kim Cc: Ryan Roberts Cc: Wei Yang Cc: Ying Huang Cc: Usama Arif Cc: Signed-off-by: Andrew Morton

mm/huge_memory: add and use has_deposited_pgtable()

2026-04-05T20:53:47+00:00

Rather than thread has_deposited through zap_huge_pmd(), make things clearer by adding has_deposited_pgtable() with comments describing why in each case. [ljs@kernel.org: fix folio_put()-before-recheck issue, per Sashiko] Link: https://lkml.kernel.org/r/0a917f80-902f-49b0-a75f-1bbaf23d7f94@lucifer.local Link: https://lkml.kernel.org/r/f9db59ca90937e39913d50ecb4f662e2bad17bbb.1774029655.git.ljs@kernel.org Signed-off-by: Lorenzo Stoakes (Oracle) Reviewed-by: Suren Baghdasaryan Cc: Baolin Wang Cc: Barry Song Cc: David Hildenbrand Cc: Dev Jain Cc: Lance Yang Cc: Liam Howlett Cc: Michal Hocko Cc: Mike Rapoport Cc: Nico Pache Cc: Qi Zheng Cc: Ryan Roberts Cc: Zi Yan Signed-off-by: Andrew Morton