kernel/linux.git/include/linux/memcontrol.h, branch v7.3-rc1

Merge tag 'mm-stable-2026-08-26-15-22' of git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm

2026-08-27T16:17:06+00:00

Pull more MM updates from Andrew Morton: - "mm/rmap: index MAP_PRIVATE file-backed folios by anonymous pgoff" (Lorenzo Stoakes) Index MAP_PRIVATE file-backed folios by their anonymous page offset to resolve confusion around reverse mapping for zeroed and CoW'd file-backed memory. Use this new VMA anonymous page offset tracking to eliminate index conflicts and lay the foundation for scalable CoW performance improvements. - "promote mapped executable folios after first usage for MGLRU" (Baolin Wang) Make MGLRU's protection of mapped executable file folios more reliable. Follow the classical LRU's logic, promoting mapped executable file folios after their first usage to give executable code a better chance to stay in memory and improve workload performance. - "mm: vmscan: fix node reclaim ignoring swappiness parameter" (Ridong Chen) Fix per-node proactive reclaim interface's ignoring the swappiness parameter when CONFIG_MEMCG is disabled by consolidating sc_swappiness() into a single function that checks proactive_swappiness regardless of kernel configuration. - "mm/vmscan: reduce lru_lock contention via vmstat-derived scan-balance cost" (Usama Arif) Reduce lru_lock contention in the reclaim path by deriving scan-balance costs from vmstat counters rather than lock-acquired producer updates. Read and decay these cost signals on the reclaim side under a dedicated per-lruvec lock, reducing total LRU lock wait time by over 60% without impacting scan throughput. - "zram: fix zram issues reported by sashiko" (Sergey Senozhatsky) Fix two low-risk zram bugs which Sashiko spotted in drive-by review. - "Honor XA_FLAGS_ACCOUNT in xas_split_alloc() and charge to folio's memcg" (Zi Yan) Fix xas_split_alloc() by enabling target folio memcg charging during splits and adding the missing __GFP_ACCOUNT flag for proper XArray node memory accounting. - "selftests/mm: use pattern matching in .gitignore" (Pratyush Mallick) Replace hardcoded binary names in selftests/mm/.gitignore with a generic pattern-matching rule to automatically ignore generated test files and avoid manual updates when adding new tests. - "mm/page_ext: remove pgdat_page_ext_init()" (Sang-Heon Jeon) Make the incompatibility between FLATMEM and NUMA explicit in mm/Kconfig and remove the unused pgdat_page_ext_init() function. - "zram: fix zstd error paths and add parameter validation" (Haoqin Huang) Clean up zram compression backends by removing redundant error cleanup, adding parameter and dictionary validation, auto-prefixing algorithm error logs, and resetting parameters prior to reinitialization. - "zram: fix stale scan bounds after reinitialization" (Longlong Xia) Prevent out-of-bounds slot accesses during concurrent zram resets by moving table scan bound calculations under dev_lock in writeback_store() and read_block_state(). - "add anon mTHP collapse test cases" (Baolin Wang) Extend selftests helper functions to support arbitrary page orders and add new test cases and options for mTHP collapse in khugepaged. - "selftests/mm: Handle unsupported and transient test conditions" (Muhammad Usama Anjum) Update MM selftests to report a SKIP status instead of a failure when required kernel or filesystem features are unsupported, while adding retry logic for transient page migration errors. - "mm/zswap: Fixes and improves the zswap shrink" (Hao Jia) Fix the missing zswap global shrinker when CONFIG_MEMCG is disabled and extend shrink_memcg() to support batch writeback for improved writeback efficiency. - "alloc_tag: introduce IOCTL-based filtering for MAP" (Suren Baghdasaryan) Introduce an IOCTL-based binary interface for memory allocation profiling that enables kernel-side filtering before per-CPU counter aggregation. This eliminates the text-parsing overhead of /proc/allocinfo and provides up to a 20x speedup by transferring only filtered allocation data to userspace. - "better block swap batching and a different take on swap_ops v5" (Christoph Hellwig) Refactor block swap I/O to use swap_iocb for batching instead of single-bio requests and rebase the swap_ops interface, achieving faster swap throughput during kernel builds. - "mm: kmemleak: reduce transient false positives by confirming leaks" (Catalin Marinas) Reduce false-positive kmemleak reports by combining two kmemleak enhancements that add a second confirmation scan and a configurable minimum unreferenced scan count module parameter. - "mm: kmemleak: default min_unref_scans to 2 for verbose kernels" (Breno Leitao) Auto-scanning kernels can generate false-positive memory leak reports on single scans, so this patch defaults min_unref_scans to 2 when CONFIG_DEBUG_KMEMLEAK_VERBOSE is enabled to require a second confirming scan. - "swap_ops updates" (Christoph Hellwig) Batching I/O for synchronous swap devices causes performance regressions and filesystem-based swap suffers from double-indirection overhead. This series resolves both issues by reintroducing per-folio writes for synchronous swap and allowing filesystems to directly export their own swap_ops. - "mm/khugepaged: several cleanups" (Nico Pache) khugepaged accumulated redundant state-checking patterns and outdated comments following mTHP integration. Introduce dedicated helpers for PTE validation and event counting while refreshing the internal documentation. - "maple_tree: lock checking and clean ups" (Liam Howlett) Syzbot reports incorrectly blame memory management exit paths for locking bugs, maple tree erase operations risk allocation failures without gfp flags and internal documentation lacks clarity. Improve lock error detection, update docs, fix race and allocation edge cases and optimize erase allocations using a fallback to GFP_KERNEL | GFP_NOFAIL. * tag 'mm-stable-2026-08-26-15-22' of git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm: (172 commits) selftests/proc: make proc-maps-race work with READ_IMPLIES_EXEC memcg: move LRU size accounting on reparenting instead of copying it mm/vmscan: fix comment logic in balance_pgdat maple_tree: add helper mas_make_walkable() maple_tree: avoid extra gap calculation maple_tree: fix argument name in header maple_tree: change two GFP flags in tests maple_tree: document erase and allocations better maple_tree: avoid mas_erase() and mtree_erase() failures maple_tree: document that erase may use GFP_KERNEL for allocations maple_tree: catch race in mas_alloc_cyclic() maple_tree: add bulk parent set helper maple_tree: micro optimisation of mas_wr_store_type() maple_tree: optimise mas_wr_node_store() when not in rcu mode maple_tree: use prefetched value in mas_wr_store_type() maple_tree: clarify comments on mas_nomem() maple_tree: drop MAPLE_ALLOC_SLOTS maple_tree: drop dead code from mas_extend_spanning_null() maple_tree: documentation fix maple_tree: add write lock checking with lockdep sequence numbers ...

mm/memcontrol: avoid false sharing between vmstats and events

2026-08-25T01:43:25+00:00

Moving v1 userspace eventfd handling into memcontrol-v1.c shrank struct vmpressure from 112 to 24 bytes when CONFIG_MEMCG_V1 is disabled. This moved memory_events_local[MEMCG_SWAP_FAIL] and the hot vmstats_percpu pointer onto the same cacheline. The stress-ng mremap stressor exercises MADV_PAGEOUT with swap disabled, generating about 20 million MEMCG_SWAP_FAIL updates per 60-second run on a 176-CPU test system. Those writes bounce the line while memcg statistics paths load vmstats_percpu. Move cgwb_list into the existing alignment gap and cacheline-align vmstats_percpu. This separates the pointer from the event counters without increasing the size of struct mem_cgroup in the tested configuration. The blamed commit reduced median mremap throughput by 4.38% on the test system with one socket. The patched kernel brings the performance to within 0.5% of the parent which is within the observed boot-to-boot spread (up to 1.2%). Link: https://lore.kernel.org/20260817103835.2937733-1-usama.arif@linux.dev Fixes: ea928e9e18da ("mm/vmpressure: move v1 userspace eventfd code into memcontrol-v1.c") Signed-off-by: Usama Arif Reported-by: kernel test robot Closes: https://lore.kernel.org/oe-lkp/202608131743.c6a7dda4-lkp@intel.com Tested-by: kernel test robot Link: http://lore.kernel.org/aoAABX59IzUXz/Rv@ly-workstation Acked-by: Shakeel Butt Acked-by: Michal Hocko Cc: David Hildenbrand Cc: Johannes Weiner Cc: Muchun Song Cc: Roman Gushchin Cc: Yi Lai Signed-off-by: Andrew Morton

mm/vmstat, mm/memcontrol: add _monotonic vmstat readers

2026-08-25T01:42:55+00:00

Patch series "mm/vmscan: reduce lru_lock contention via vmstat-derived scan-balance cost", v5. The anon/file scan balance heuristic in get_scan_count() is fed by two scalars in struct lruvec (anon_cost, file_cost) that every reclaim producer updates under lruvec->lru_lock. The cost-recording work itself is trivial, but it both contends for and contributes to contention on lru_lock - which is often a contention point on memory-pressured workloads. Specifically: - shrink_inactive_list() re-acquires lru_lock at function exit just to call lru_note_cost_unlock_irq(). - shrink_active_list() does the same after rotation accounting. - workingset_refault() takes folio_lruvec_lock_irq() purely to record the refault cost. - prepare_scan_control() snapshots anon_cost/file_cost under lru_lock. - lru_note_cost_unlock_irq() itself walks parent_lruvec() and re-acquires lru_lock on every ancestor, multiplying the cost of every update by memcg-hierarchy depth. This series removes those producer-side acquisitions entirely. The rotation inputs become per-LRU PGROTATE_{ANON,FILE} vmstat counters. NR_VMSCAN_WRITE already captures reclaim-driven pageout at writeout(); charge it through lruvec_stat_mod_folio() so it is available per lruvec and aggregated through the memcg hierarchy. Reclaim does not submit filesystem folios for writeback from this path, so pageout contributes only to anon cost. WORKINGSET_RESTORE_* already captures the refault input. PGROTATE_* are also useful independently of scan balancing. They are cumulative base-page events, not unique-page counts. Classic inactive reclaim records scan work that does not produce immediate reclaim or demotion, while active reclaim records referenced executable file folios retained on the active list. MGLRU records initially isolated pages that remain unreclaimed after its retry passes. Read alongside pgscan_* and pgsteal_*, their deltas identify which LRU type is consuming reclaim CPU without producing immediate yield. Unlike the existing pgrotated event, they do not imply a move to the inactive-list tail. prepare_scan_control() reads the raw cost signals without lru_lock: anon = PGROTATE_ANON + (NR_VMSCAN_WRITE + WORKINGSET_RESTORE_ANON) * SWAP_CLUSTER_MAX file = PGROTATE_FILE + WORKINGSET_RESTORE_FILE * SWAP_CLUSTER_MAX It folds the deltas into a per-lruvec accumulator. A dedicated per-lruvec cost_lock, not touched by isolate_lru_folios(), move_folios_to_lru(), or folio_add_lru(), serialises the accumulator RMW and the lrusize/4 halving check. Hierarchy aggregation is implicit in rstat propagation, so the parent_lruvec() walk and the lru_reparent_memcg() cost-splice both disappear. Moving accumulation and decay to the reclaim side also improves the cost model across reclaim gaps. With producer-side decay, events that happen while reclaim is idle still age each other before reclaim ever samples the costs. If a workload refaults a large anon set and then a smaller file set before reclaim runs again, the later file activity can age the earlier anon activity out of the cost model. The new scheme observes the whole between-reclaim delta and decays anon and file proportionally, so the scan-balance history better represents what happened since the last reclaim pass. Trade-offs: - Cost reads see rstat-aggregated values that can lag until periodic / reader-triggered flushing. - Per-lruvec footprint grows by 4 unsigned longs + a spinlock (a struct lru_cost { count, last_rotated, last_io } per side), which is a small cost. - NR_VMSCAN_WRITE now also updates the folio's lruvec/memcg stat, adding memcg stat accounting to the reclaim writeout path while preserving the existing node-level total. == Numbers == Tested on a 176-core, 256 GB host. The benchmark drives sustained swap-out/refault inside a tight memcg using vm-scalability/usemem: usemem -n 16 --prealloc --prefault --random $((256*1024*1024)) run inside a two-level memcg with memory.max=512M on the leaf (4 GB anon working set has to fit in 512 MB -> continuous shrink_inactive_list + workingset_refault). A 16 GB swap file is used. Measurement is a 30 s `perf lock record -a` window over otherwise-idle hardware. Workload rates are identical on both kernels (the bench drives the same memory pressure): baseline patched delta pgscan_direct / s 172,662 171,817 ~0% pgsteal_direct / s 67,162 66,306 ~0% workingset_refault_anon / s 40,696 39,830 ~0% perf lock contention (total wait per 30 s window): Lock Name Before After % change shrink_lruvec+0x770 722.84 ms 0 -100% (eliminated) (= lru_note_cost_unlock_irq) workingset_refault+0x167 385.26 ms 0 -100% (eliminated) (= lru_note_cost_refault) shrink_node+0x4ad 689.43 ms 26.95 ms -96% shrink_active_list 208.34 ms 15.97 ms -92% lru_add_drain_cpu+0x34 1.96 s 917.71 ms -53% Total LRU lock wait ~4.23 s ~1.66 s -61% The two specific contention sites the patch removes (shrink_lruvec+0x770 = lru_note_cost_unlock_irq; workingset_refault+0x167 = lru_note_cost_refault) are completely absent from the patched perf-lock-contention output. Secondary reductions in shrink_node, shrink_active_list, lru_add_drain_cpu and pgrefill/pgactivate look like knock-on effects from removing the cost-recording overhead and the parent_lruvec walk. The remaining ~1.66 s of LRU lock wait on the patched kernel is dominated by the per-CPU pagevec drain (lru_add_drain_cpu) and the main reclaim path in shrink_lruvec. The numbers above can be reproduced using the script in [1]. This patch (of 3): lruvec_page_state(), node_page_state(), and global_node_page_state() all clamp negative reads to zero on CONFIG_SMP so that a transient per-CPU delta skew presents as zero pages rather than as a garbage unsigned value. This is the right behaviour for non-monotonic page-count readers. It is however incorrect for callers that snapshot a monotonically- incremented event counter and compute a delta from two samples. Once the underlying signed long wraps past LONG_MAX, the clamped read drops to zero while the previously-recorded snapshot still holds the pre-wrap value; the unsigned subtraction then underflows into a ~2^31 spurious delta for 32-bit architecture and corrupts the caller's accumulator. Add non-clamping siblings that return the underlying state value cast to unsigned long: global_node_page_state_monotonic() node_page_state_monotonic() lruvec_page_state_monotonic() With both samples read via the _monotonic variant, unsigned modular subtraction stays correct across a signed-long wraparound as long as the true growth between two samples fits in unsigned long (< 2^32 on 32-bit, < 2^64 on 64-bit); the 32-bit bound is the practically-reachable one that motivates this helper. The variants are only safe for monotonically-incremented counters. Non-monotonic page-count readers must keep using the existing clamped helpers so transient negative reads still present as zero. This is a prerequisite for a later patch which replaces the producer-side anon_cost/file_cost accumulators with a read-side accumulator in prepare_scan_control() that samples monotonic per-LRU vmstat counters (PGROTATE_*, NR_VMSCAN_WRITE, WORKINGSET_RESTORE_*) via lruvec_page_state_monotonic() and folds their unsigned modular deltas into lruvec->cost[].count. Link: https://lore.kernel.org/20260727162550.2032-1-usama.arif@linux.dev Link: https://lore.kernel.org/20260727162550.2032-2-usama.arif@linux.dev Link: https://gist.github.com/uarif1/a4eb33a86c5b2d7bbc55b42f0956e884 [1] Signed-off-by: Usama Arif Acked-by: Johannes Weiner Acked-by: Shakeel Butt Acked-by: Vlastimil Babka (SUSE) Cc: Axel Rasmussen Cc: Baoquan He Cc: Chris Li Cc: David Hildenbrand Cc: David Rientjes Cc: Kairui Song Cc: Liam R. Howlett Cc: Lorenzo Stoakes Cc: Michal Hocko Cc: Mike Rapoport Cc: Muchun Song Cc: Nhat Pham Cc: Roman Gushchin Cc: Suren Baghdasaryan Cc: Wei Xu Cc: Yuanchu Xie Signed-off-by: Andrew Morton

memcg: move mem_cgroup_swappiness and vm_swappiness to mm/swap.h

2026-08-25T01:42:55+00:00

Patch series "mm: vmscan: fix node reclaim ignoring swappiness parameter", v4. The per-node proactive reclaim interface (/sys/devices/system/node/nodeX/reclaim) accepts a swappiness parameter, but it is silently ignored when CONFIG_MEMCG is disabled. The root cause is that sc_swappiness() has separate implementations for CONFIG_MEMCG and !CONFIG_MEMCG, and the latter never checks proactive_swappiness. Patch 1 moves mem_cgroup_swappiness() and vm_swappiness out of the public include/linux/swap.h into the mm-private mm/swap.h, and makes the helper handle both CONFIG_MEMCG and !CONFIG_MEMCG in a single inline function. This is a prerequisite for unifying sc_swappiness(). Patch 2 consolidates sc_swappiness() into a single definition that works regardless of CONFIG_MEMCG, fixing the node reclaim swappiness bug. This patch (of 2): The per-memcg swappiness knob is v1-only; v2 always uses global vm_swappiness and ignores the per-cgroup field. Both mem_cgroup_swappiness() and vm_swappiness are only used within mm/ (memcontrol.c, memcontrol-v1.c, vmscan.c), so move them out of the public include/linux/swap.h into the mm-private mm/swap.h. This keeps unrelated declarations out of include/linux/swap.h. Guard memcg->swappiness with CONFIG_MEMCG_V1 as well, so v2-only kernels drop the unused field. No functional change for v1; v2-only kernels drop the unused field. Link: https://lore.kernel.org/20260723032434.2016749-1-ridong.chen@linux.dev Link: https://lore.kernel.org/20260723032434.2016749-2-ridong.chen@linux.dev Signed-off-by: Ridong Chen Acked-by: Johannes Weiner Reviewed-by: Barry Song Reviewed-by: Song Hu Acked-by: Shakeel Butt Cc: Axel Rasmussen Cc: Baoquan He Cc: Chris Li Cc: David Hildenbrand Cc: Davidlohr Bueso Cc: Kairui Song Cc: Kemeng Shi Cc: Lorenzo Stoakes Cc: Michal Hocko Cc: Muchun Song Cc: Nhat Pham Cc: Roman Gushchin Cc: Wei Xu Cc: Yuanchu Xie Cc: Qi Zheng Signed-off-by: Andrew Morton

Merge tag 'slab-for-7.3' of git://git.kernel.org/pub/scm/linux/kernel/git/vbabka/slab

2026-08-24T17:58:57+00:00

Pull slab updates from Vlastimil Babka: - Add kfree_rcu_nolock() that can be used from contexts where spinning on a lock might be unsafe, such as a BPF program attached to an arbitrary function, or in NMI context. This complements the existing kfree_nolock() support (Harry Yoo) - Runtime instead of compile-time slabobj_ext sizing. Avoid wasting memory when memory allocation profiling is compiled but not enabled, with initial partial support to also avoid wasting memory for objcg pointers when those are not needed, while profiling is enabled (Vlastimil Babka) - Various non-urgent fixes, cleanups and optimizations (Hao Li, Hongling Zeng, Li RongQing, Li Xiasong, Seongjun Hong, Shengming Hu) * tag 'slab-for-7.3' of git://git.kernel.org/pub/scm/linux/kernel/git/vbabka/slab: (31 commits) mm/slab, kfence, memcg: completely remove obj_ext for kfence objects mm/slab: stop allocating objcg pointers when unnecessary mm/slab: add cache_ and slab_needs_objcg() helpers mm/slab: stop exporting kvfree_rcu_barrier[_on_cache]() slub_kunit: extend the test for kfree_rcu_nolock() mm/slab: introduce kfree_rcu_nolock() mm/slab: introduce struct kvfree_rcu_head for kvfree_rcu batching mm/slab: reduce slabobj_ext memory with allocation profiling disabled mm/slab: introduce slab_obj_ext_has_codetag() mm/slab: allow kfree_rcu_sheaf() on PREEMPT_RT mm/slab: extend deferred free mechanism to handle rcu sheaves mm/slab: use call_rcu() in unknown context if irqs are enabled mm/slab: handle the !allow_spin case in kfree_rcu_sheaf() mm/slab: change struct slabobj_ext to a union mm/slab: replace slab.stride with obj_exts_in_object mm/slab: abstract slabobj_ext.ref access mm/slab: abstract slabobj_ext.objcg access mm/slab: make slab_obj_ext() determine object index mm: move struct slabobj_ext to mm/slab.h mm/slab: remove objs_per_slab() ...

mm: move struct slabobj_ext to mm/slab.h

2026-07-31T13:50:47+00:00

Users of include/linux/memcontrol.h don't need to see this internal structure. Further changes to the struct will reduce recompiling. Reviewed-by: Suren Baghdasaryan Reviewed-by: Harry Yoo (Oracle) Reviewed-by: Hao Li Link: https://patch.msgid.link/20260727-b4-objext_split-v3-3-c29ef0f1f257@kernel.org Signed-off-by: Vlastimil Babka (SUSE)

Merge tag 'mm-hotfixes-stable-2026-07-27-14-18' of git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm

2026-07-27T21:36:26+00:00

Pull misc fixes from Andrew Morton: "13 hotfixes. All are cc:stable. 11 are for MM. All are singletons - please see the changelogs for details" * tag 'mm-hotfixes-stable-2026-07-27-14-18' of git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm: fs/proc/task_mmu: fix PAGEMAP_SCAN written state for PMD holes mm/hugetlb: fix list corruption in allocate_file_region_entries() mm: mglru: fix stale batch updates after memcg reparenting selftest: fix headers in fclog.c ocfs2: fix boundary check in ocfs2_check_dir_entry() to use buffer offset mm/percpu-km: fix bitmap overflow and accounting in pcpu_create_chunk() mm/util: don't read __page_2 for order-1 folios in snapshot_page() mm/hugetlb: fix swap entry corruption when clearing uffd-wp at fork() mm: migrate_device: fix pte_pfn/pte_dirty called on non-present PTE fs/proc/task_mmu: fix PAGEMAP_SCAN written state for unpopulated ptes userfaultfd: wait on source PMD during UFFDIO_MOVE lib: test_hmm: use device devt for coherent device range selection mm/vmstat: fold stranded per-cpu node stats when a node comes online

fs: push nr_cached_objects memcg gating into individual filesystems

2026-07-23T09:35:02+00:00

Commit 0baad6f9b997 ("fs/super: skip non-memcg-aware nr_cached_objects in memcg slab shrink") added a check in fs/super.c that skipped every ->nr_cached_objects() hook whenever the shrinker was invoked for a non-root memcg, on the assumption that none of them honour sc->memcg. That assumption is wrong for XFS, whose inode-reclaim hook is intentionally driven from per-memcg contexts to free memcg-charged slab. Encoding a blanket "never memcg-aware" policy in fs/super.c short-circuits that path. Push the check down into the callbacks whose counters really are irrelevant to per-memcg reclaim - btrfs_nr_cached_objects() and shmem_unused_huge_count() - and drop the fs/super.c gate. Each filesystem can now lift the restriction independently if its counter later grows memcg awareness, without touching fs/super.c. Introduce mem_cgroup_shrink_is_root() in so the callbacks don't open-code "sc->memcg is NULL or root". Fixes: 0baad6f9b997 ("fs/super: skip non-memcg-aware nr_cached_objects in memcg slab shrink") Acked-by: Qi Zheng Reviewed-by: Jan Kara Reviewed-by: Shakeel Butt Signed-off-by: Usama Arif Link: https://patch.msgid.link/20260715103516.2410175-1-usama.arif@linux.dev Acked-by: David Sterba Reviewed-by: Baolin Wang Signed-off-by: Christian Brauner (Amutable)

mm: mglru: fix stale batch updates after memcg reparenting

2026-07-21T00:41:28+00:00

The mglru page table walker batches per-generation size deltas in walk->nr_pages while walking page tables without holding the lruvec lock. The reset_batch_size() later folds those deltas into walk->lruvec under the lruvec lock. The page table walker can run concurrently with the memcg reparenting path as follows: CPU0 CPU1 ==== ==== walk_mm --> walk_page_range --> update_batch_size --> walk->nr_pages += delta mem_cgroup_css_offline --> memcg_reparent_objcgs --> lock lruvec lru_gen_reparent_memcg --> reparent child folios to parent unlock lruvec lock lruvec reset_batch_size --> child lrugen->nr_pages += delta This will trigger the following warning in lru_gen_exit_memcg(): VM_WARN_ON_ONCE(memchr_inv(lruvec->lrugen.nr_pages, 0, sizeof(lruvec->lrugen.nr_pages))); And the user-visible impact of underestimated nr_pages in MGLRU was premature OOMs because MGLRU does not try to reclaim memory when nr_pages reaches zero, but there are still more pages. To fix it, make reset_batch_size() check CSS_DYING under RCU before flushing the pending batch. A non-dying memcg keeps the original lruvec stable against RCU-delayed offlining; a dying memcg redirects the deltas to the first non-dying ancestor. Link: https://lore.kernel.org/20260710154318.75388-1-qi.zheng@linux.dev Fixes: f304652609ea ("mm: vmscan: prepare for reparenting MGLRU folios") Signed-off-by: Qi Zheng Reported-by: Peiyang He Closes: https://lore.kernel.org/all/5A9E929D82717101+12fcf643-efb8-4b9a-a53a-1e28cc894f0b@smail.nju.edu.cn Reviewed-by: Harry Yoo (Oracle) Acked-by: Johannes Weiner Acked-by: Shakeel Butt Cc: Axel Rasmussen Cc: Barry Song Cc: David Hildenbrand Cc: Kairui Song Cc: Lorenzo Stoakes Cc: Michal Hocko Cc: Muchun Song Cc: Roman Gushchin Cc: Wei Xu Cc: Yuanchu Xie Cc: Signed-off-by: Andrew Morton

mm: switch deferred split shrinker to list_lru

2026-06-09T01:21:25+00:00

The deferred split queue handles cgroups in a suboptimal fashion. The queue is per-NUMA node or per-cgroup, not the intersection. That means on a cgrouped system, a node-restricted allocation entering reclaim can end up splitting large pages on other nodes: alloc/unmap deferred_split_folio() list_add_tail(memcg->split_queue) set_shrinker_bit(memcg, node, deferred_shrinker_id) for_each_zone_zonelist_nodemask(restricted_nodes) mem_cgroup_iter() shrink_slab(node, memcg) shrink_slab_memcg(node, memcg) if test_shrinker_bit(memcg, node, deferred_shrinker_id) deferred_split_scan() walks memcg->split_queue The shrinker bit adds an imperfect guard rail. As soon as the cgroup has a single large page on the node of interest, all large pages owned by that memcg, including those on other nodes, will be split. list_lru properly sets up per-node, per-cgroup lists. As a bonus, it streamlines a lot of the list operations and reclaim walks. It's used widely by other major shrinkers already. Convert the deferred split queue as well. The list_lru per-memcg heads are instantiated on demand when the first object of interest is allocated for a cgroup, by calling folio_memcg_alloc_deferred(). Add calls to where splittable pages are created: anon faults, swapin faults, khugepaged collapse. These calls create all possible node heads for the cgroup at once, so the migration code (between nodes) doesn't need any special care. [akpm@linux-foundation.org: fix build with CONFIG_TRANSPARENT_HUGEPAGE=n] Link: https://lore.kernel.org/202605281620.lc3rtkBm-lkp@intel.com [hannes@cmpxchg.org: fix cgroup.memory=nokmem handling] Link: https://lore.kernel.org/ah9PGv12mqai84ES@cmpxchg.org Link: https://lore.kernel.org/20260527204757.2544958-10-hannes@cmpxchg.org Signed-off-by: Johannes Weiner Reported-by: Mikhail Zaslonko Tested-by: Mikhail Zaslonko Acked-by: Shakeel Butt Reviewed-by: Lorenzo Stoakes (Oracle) Acked-by: Usama Arif Reviewed-by: Kairui Song Cc: Baolin Wang Cc: Barry Song Cc: Dave Chinner Cc: David Hildenbrand (Arm) Cc: Dev Jain Cc: Lance Yang Cc: Liam R. Howlett Cc: Michal Hocko Cc: Muchun Song Cc: Nico Pache Cc: Roman Gushchin Cc: Ryan Roberts Cc: Vasily Gorbik Cc: Vlastimil Babka Cc: Zi Yan Cc: kernel test robot Signed-off-by: Andrew Morton