kernel/linux.git/include/linux/mm.h, branch v7.3-rc1

mm: introduce linear_anon_page_index()

2026-08-25T01:42:50+00:00

This function provides the anonymous equivalent of linear_page_index(), instead offsetting based on the anonymous page offset of the VMA. It is valid only for anonymous or MAP_PRIVATE file-backed mappings, in other words CoW mappings. For pure anon VMAs, this will be equal to linear_page_index(). Assert that both of these invariants are true in linear_anon_page_index() and implement the algorithm in __linear_anon_page_index(). Note that MAP_PRIVATE-/dev/zero mappings will satisfy vma_is_anonymous() but not fulfill this invariant, so when asserting this we check vma->vm_file to account for this. We do not update callsites yet, so no functional change intended. Also const-ify vma_is_anonymous() to make it compatible with the const-ified linear_anon_page_index(). While we're here, update linear_page_index() to be more succinct. VMA userland tests are also updated accordingly. Link: https://lore.kernel.org/20260813-b4-scalable-cow-virt-pgoff-v5-3-c21581c0c3c8@kernel.org Signed-off-by: Lorenzo Stoakes (ARM) Reviewed-by: Gregory Price (Meta) Acked-by: David Hildenbrand (Arm) Cc: Adrian Hunter Cc: Alexander Deucher Cc: Alexander Gordeev Cc: Alexander Shishkin Cc: Alistair Popple Cc: Arnaldo Carvalho de Melo Cc: Arnd Bergmann Cc: Baolin Wang Cc: Baoquan He Cc: Barry Song Cc: Boris Brezillon Cc: Byungchul Park Cc: Chengming Zhou Cc: Chris Li Cc: Christan König Cc: Christian Borntraeger Cc: Claudio Imbrenda Cc: Dave Airlie Cc: Dev Jain Cc: Gerald Schaefer Cc: Greg Kroah-Hartman Cc: Harry Yoo Cc: Heiko Carstens Cc: Huang Ray Cc: "Huang, Ying" Cc: Ian Rogers Cc: Ingo Molnar Cc: James Clark Cc: Jan Kara Cc: Jann Horn Cc: Janosch Frank Cc: Jason Gunthorpe Cc: Jiri Olsa Cc: John Hubbard Cc: Joshua Hahn Cc: Kairui Song Cc: Kees Cook Cc: Kemeng Shi Cc: Lance Yang Cc: Liam R. Howlett Cc: Liviu Dudau Cc: Maarten Lankhorst Cc: Marc Rutland Cc: "Masami Hiramatsu (Google)" Cc: Matthew Auld Cc: Matthew Brost Cc: Matthew Wilcox (Oracle) Cc: Maxime Ripard Cc: Miaohe Lin Cc: Michal Hocko Cc: Mike Rapoport Cc: Muchun Song Cc: Namhyung kim Cc: Naoya Horiguchi Cc: Nhat Pham Cc: Nico Pache Cc: Oleg Nesterov Cc: Oscar Salvador Cc: Pedro Falcato Cc: Peter Xu Cc: Peter Zijlstra Cc: Rakie Kim Cc: Rik van Riel Cc: Rodrigo Vivi Cc: Ryan Roberts Cc: Steven Price Cc: Suren Baghdasaryan Cc: Sven Schnelle Cc: Thomas Hellström Cc: Thomas Zimemrmann Cc: Vasily Gorbik Cc: Vlastimil Babka Cc: xu xin Cc: Zi Yan Signed-off-by: Andrew Morton

mm: provide vma_[flags_]is_cow_mapping() and remove is_cow_mapping()

2026-08-25T01:42:50+00:00

All remaining callers of is_cow_mapping() are invoking it in the form of is_cow_mapping(vma->vm_flags) or an indirected version of this. Therefore, provide a helper - vma_is_cow_mapping() to directly test the VMA. Additionally provide a new helper vma_flags_is_cow_mapping() which performs the check using the new vma_flags_t type, and share this logic between vma_is_cow_mapping() and vma_desc_is_cow_mapping(). With these changes, no callers of is_cow_mapping() remain, so remove it. Also update the userland VMA tests to reflect the change. No functional change intended. [akpm@linux-foundation.org: fix kerneldoc comment typo, per Lorenzo] Link: https://lore.kernel.org/aob1goSSPH6sTN9y@gremlin Link: https://lore.kernel.org/20260813-b4-scalable-cow-virt-pgoff-v5-2-c21581c0c3c8@kernel.org Signed-off-by: Lorenzo Stoakes (ARM) Acked-by: David Hildenbrand (Arm) Cc: Adrian Hunter Cc: Alexander Deucher Cc: Alexander Gordeev Cc: Alexander Shishkin Cc: Alistair Popple Cc: Arnaldo Carvalho de Melo Cc: Arnd Bergmann Cc: Baolin Wang Cc: Baoquan He Cc: Barry Song Cc: Boris Brezillon Cc: Byungchul Park Cc: Chengming Zhou Cc: Chris Li Cc: Christan König Cc: Christian Borntraeger Cc: Claudio Imbrenda Cc: Dave Airlie Cc: Dev Jain Cc: Gerald Schaefer Cc: Greg Kroah-Hartman Cc: Gregory Price (Meta) Cc: Harry Yoo Cc: Heiko Carstens Cc: Huang Ray Cc: "Huang, Ying" Cc: Ian Rogers Cc: Ingo Molnar Cc: James Clark Cc: Jan Kara Cc: Jann Horn Cc: Janosch Frank Cc: Jason Gunthorpe Cc: Jiri Olsa Cc: John Hubbard Cc: Joshua Hahn Cc: Kairui Song Cc: Kees Cook Cc: Kemeng Shi Cc: Lance Yang Cc: Liam R. Howlett Cc: Liviu Dudau Cc: Maarten Lankhorst Cc: Marc Rutland Cc: "Masami Hiramatsu (Google)" Cc: Matthew Auld Cc: Matthew Brost Cc: Matthew Wilcox (Oracle) Cc: Maxime Ripard Cc: Miaohe Lin Cc: Michal Hocko Cc: Mike Rapoport Cc: Muchun Song Cc: Namhyung kim Cc: Naoya Horiguchi Cc: Nhat Pham Cc: Nico Pache Cc: Oleg Nesterov Cc: Oscar Salvador Cc: Pedro Falcato Cc: Peter Xu Cc: Peter Zijlstra Cc: Rakie Kim Cc: Rik van Riel Cc: Rodrigo Vivi Cc: Ryan Roberts Cc: Steven Price Cc: Suren Baghdasaryan Cc: Sven Schnelle Cc: Thomas Hellström Cc: Thomas Zimemrmann Cc: Vasily Gorbik Cc: Vlastimil Babka Cc: xu xin Cc: Zi Yan Signed-off-by: Andrew Morton

mm/vma: introduce VMA anon page offset field and add helpers

2026-08-25T01:42:49+00:00

Patch series "mm/rmap: index MAP_PRIVATE file-backed folios by anonymous pgoff", v5. In memory management we've managed to manufacture a great deal of confusion around the concept of anonymous memory. We have: 1. 'Pure anon' memory - anonymous VMAs whose folios are anonymous and swap-backed (thus for reclaim purposes, treated as anonymous). These are simple enough. 2. shmem - file-backed VMAs, file-backed folios (from rmap perspective) so present in the page cache and mapped by an address_space object, but whose folios are also swap-backed (thus treated as anonymous for reclaim purposes). 3. MAP_PRIVATE-mapped /dev/zero - a strange beast whose VMAs have vma->vm_file set, but which clears vma->vm_ops to satisfy vma_is_anonymous(), resulting in VMAs that were mmap()'d referencing a file, but are in every other sense anonymous, including the folios. 4. Other MAP_PRIVATE-file backed mappings - These possess file-backed VMAs and have file-backed folios until CoW'd, at which point those CoW'd folios are anonymous. This series fixes issue 3. In order for us to traverse VMAs using the reverse mapping, we require two fields - folio->mapping and folio->index. The first tells the rmap code where to look for VMAs, and the second tells it at which offset the folio starts within the referenced object. For anonymous folios, folio->mapping points at an anon_vma object. For file-backed folios, it points at an address_space. And: * For file-backed folios folio->index is simply the page offset of the start of the folio within the file. * For anonymous folios belonging to pure anon mappings, folio->index is equal to the anonymous page offset of the folio. * For anonymous folios belonging to file-backed mappings (i.e. CoW'd folios of a MAP_PRIVATE file-backed mapping), folio->index is equal to the file page offset. This series establishes a new anonymous page offset property of VMAs to allow us to map anonymous folios at their anonymous page offset, consistent with pure anon. The purpose of doing so is to lay the foundations for the scalable CoW work. This is necessary because scalable CoW looks in the maple tree for the VMA located at folio->index << PAGE_SHIFT, before falling back to looking up tracked remaps if necessary. The MAP_PRIVATE file-backed case means that folio indices will very often conflict with one another and this remap tracking becomes substantially more contended, and of course the fast path can never be used. This also makes it possible, in future, to unshare anonymously mapped folios with deep fork hierarchies on remap, eliminating the need for remap tracking in the vast majority of cases. Similar to page offset of pure anonymous VMAs, we update the anonymous page offset of unfaulted file-backed VMAs on remap, but do not once CoW'd (i.e. vma->anon_vma is non-NULL). Overall, there is little impact on mergeability, which remains exactly the same for pure anonymous and shared file-backed mappings, with the only impact being on MAP_PRIVATE-mapped file-backed mappings, which must now match on anonymous page offset as well as file page offset to be merged. To fail to merge like this would require CoW'ing the mapping, then finding another VMA with identical file and compatible page offset to remap next to. This is therefore very much an edge case that should have very little impact (and which scalable CoW may very well address in any case). This patch (of 16): Establish fields in vm_area_struct to store the anonymous page offset of VMAs. Initially, the anonymous page offset of a VMA is vma->vm_start >> PAGE_SHIFT. When a VMA is remapped to new_address its anonymous page offset is either updated to new_address >> PAGE_SHIFT if unfaulted or, if faulted, remains equal to the anonymous page offset it had when first faulted. Currently, anonymous folios belonging to CoW'd MAP_PRIVATE-mapped file-backed VMAs are tracked by their file offsets. By adding anonymous offset as a property of VMAs, we can now track them by their anonymous page offset instead. By tracking this, we provide the means by which to eliminate this inconsistency, and more importantly lay the foundations for future work for the scalable CoW anonymous rmap rework. This patch simply adds the fields and some simple helpers. Subsequent patches will update mm code to make use of these fields correctly. The fields chosen are packed in the VMA such that, for 64-bit kernel builds, no additional space is taken up. The first field is present on cacheline 0 containing key VMA fields, and the second on cacheline 3, which contains file-backed reverse mapping fields. Given the relative time spent accessing reverse mapping fields as well as updating them, there shouldn't be any performance impact here from false sharing. Update the VMA userland tests to account for this change. No callsites are updated yet, so no functional change intended. Link: https://lore.kernel.org/20260813-b4-scalable-cow-virt-pgoff-v5-0-c21581c0c3c8@kernel.org Link: https://lore.kernel.org/20260813-b4-scalable-cow-virt-pgoff-v5-1-c21581c0c3c8@kernel.org Signed-off-by: Lorenzo Stoakes (ARM) Acked-by: David Hildenbrand (Arm) Reviewed-by: Gregory Price (Meta) Reviewed-by: Xu Xin Cc: Adrian Hunter Cc: Alexander Deucher Cc: Alexander Gordeev Cc: Alexander Shishkin Cc: Alistair Popple Cc: Arnaldo Carvalho de Melo Cc: Arnd Bergmann Cc: Baolin Wang Cc: Baoquan He Cc: Barry Song Cc: Boris Brezillon Cc: Byungchul Park Cc: Chengming Zhou Cc: Chris Li Cc: Christan König Cc: Christian Borntraeger Cc: Claudio Imbrenda Cc: Dave Airlie Cc: Dev Jain Cc: Gerald Schaefer Cc: Greg Kroah-Hartman Cc: Harry Yoo Cc: Heiko Carstens Cc: Huang Ray Cc: "Huang, Ying" Cc: Ian Rogers Cc: Ingo Molnar Cc: James Clark Cc: Jan Kara Cc: Jann Horn Cc: Janosch Frank Cc: Jason Gunthorpe Cc: Jiri Olsa Cc: John Hubbard Cc: Joshua Hahn Cc: Kairui Song Cc: Kees Cook Cc: Kemeng Shi Cc: Lance Yang Cc: Liam R. Howlett Cc: Liviu Dudau Cc: Maarten Lankhorst Cc: Marc Rutland Cc: "Masami Hiramatsu (Google)" Cc: Matthew Auld Cc: Matthew Brost Cc: Matthew Wilcox (Oracle) Cc: Maxime Ripard Cc: Miaohe Lin Cc: Michal Hocko Cc: Mike Rapoport Cc: Muchun Song Cc: Namhyung kim Cc: Naoya Horiguchi Cc: Nhat Pham Cc: Nico Pache Cc: Oleg Nesterov Cc: Oscar Salvador Cc: Pedro Falcato Cc: Peter Xu Cc: Peter Zijlstra Cc: Rakie Kim Cc: Rik van Riel Cc: Rodrigo Vivi Cc: Ryan Roberts Cc: Steven Price Cc: Suren Baghdasaryan Cc: Sven Schnelle Cc: Thomas Hellström Cc: Thomas Zimemrmann Cc: Vasily Gorbik Cc: Vlastimil Babka Cc: xu xin Cc: Zi Yan Signed-off-by: Andrew Morton

mm/mseal: limit scope of mseal address zero to address zero

2026-08-07T01:57:12+00:00

Commit 44f65d900698 ("binfmt_elf: mseal address zero") unconditionally provided do_mseal() to any internal kernel caller in order to address a corner case slated for possible removal. It also incorrectly attempts to mseal without checking to see whether the mapping even succeeded. Restrict the scope to the corner case by providing mseal_mmap_page_zero() which asserts the MMAP_PAGE_ZERO personality. Avoid unnecessary checks in the start, end range by abstracting the actual mseal()'ing to mseal_range() and have mseal_mmap_page_zero() call that instead. Also only try to seal the VMA if we mapped the VMA. This isn't strictly necessary as the operation would error out anyway, but it's useless work and could be problematic if me make future changes to mseal semantics. Link: https://lore.kernel.org/20260717-mseal-fixups-v2-2-0daa0014b813@kernel.org Signed-off-by: Lorenzo Stoakes (ARM) Acked-by: David Hildenbrand (Arm) Cc: Al Viro Cc: Christian Brauner Cc: Jan Kara Cc: Jann Horn Cc: Kees Cook Cc: Liam R. Howlett Cc: Michal Hocko Cc: Mike Rapoport Cc: Pedro Falcato Cc: Suren Baghdasaryan Cc: Vlastimil Babka Signed-off-by: Andrew Morton

mm/vma: update create_init_stack_vma() to use vma_flags_t

2026-08-07T01:57:03+00:00

Replace use of the legacy vm_flags_t flags with vma_flags_t values in create_init_stack_vma(). As part of this change we add VMA_STACK_EARLY and VMA_STACK_INCOMPLETE vma_flags_t defines, and slightly rework create_init_stack_vma() for clarity. No functional change intended. Link: https://lore.kernel.org/20260711-b4-vma-flags-mm-v2-9-0fa2357d5431@kernel.org Signed-off-by: Lorenzo Stoakes Reviewed-by: Zi Yan Reviewed-by: Lance Yang Reviewed-by: Vlastimil Babka (SUSE) Cc: Baolin Wang Cc: Barry Song Cc: Christian Brauner Cc: Dave Airlie Cc: David Hildenbrand Cc: Dev Jain Cc: Jani Nikula Cc: Jan Kara Cc: Jann Horn Cc: Mike Rapoport Cc: Muchun Song Cc: Nico Pache Cc: Oscar Salvador Cc: Pedro Falcato Cc: Suren Baghdasaryan Cc: Thomas Zimmermann Signed-off-by: Andrew Morton

mm: introduce vma_get_page_prot() and use it

2026-08-07T01:57:03+00:00

There's a large number of vm_get_page_prot(vma->vm_flags) invocations. Make life easier by introducing vma_get_page_prot() parameterised by the VMA. This also makes converting vm_get_page_prot() to vma_flags_t easier. Also update the userland VMA tests to reflect the change. No functional change intended. Link: https://lore.kernel.org/20260711-b4-vma-flags-mm-v2-8-0fa2357d5431@kernel.org Signed-off-by: Lorenzo Stoakes Acked-by: Zi Yan Acked-by: Jani Nikula # for i915 Reviewed-by: Thomas Zimmermann [DRM] Reviewed-by: Vlastimil Babka (SUSE) Cc: Baolin Wang Cc: Barry Song Cc: Christian Brauner Cc: Dave Airlie Cc: David Hildenbrand Cc: Dev Jain Cc: Jan Kara Cc: Jann Horn Cc: Lance Yang Cc: Mike Rapoport Cc: Muchun Song Cc: Nico Pache Cc: Oscar Salvador Cc: Pedro Falcato Cc: Suren Baghdasaryan Signed-off-by: Andrew Morton

mm/vma: rename vma_get_page_prot to vma_flags_to_page_prot

2026-08-07T01:57:02+00:00

Having vma_get_page_prot() refer to VMA flags and vma_set_page_prot() refer to a VMA is confusing. Rename vma_get_page_prot() to vma_flags_to_page_prot() to resolve this confusion. No functional change intended. Link: https://lore.kernel.org/20260711-b4-vma-flags-mm-v2-7-0fa2357d5431@kernel.org Signed-off-by: Lorenzo Stoakes Reviewed-by: Lance Yang Reviewed-by: Zi Yan Reviewed-by: Vlastimil Babka (SUSE) Cc: Baolin Wang Cc: Barry Song Cc: Christian Brauner Cc: Dave Airlie Cc: David Hildenbrand Cc: Dev Jain Cc: Jani Nikula Cc: Jan Kara Cc: Jann Horn Cc: Mike Rapoport Cc: Muchun Song Cc: Nico Pache Cc: Oscar Salvador Cc: Pedro Falcato Cc: Suren Baghdasaryan Cc: Thomas Zimmermann Signed-off-by: Andrew Morton

mm: convert __get_unmapped_area() to use vma_flags_t

2026-08-07T01:57:01+00:00

Update __get_unmapped_area() to be parameterised by vma_flags_t rather than vm_flags_t as part of the effort to move VMA flags from a system word to a bitmap. We cascade the changes up to arch_get_unmapped_area_topdown() and arch_get_unmapped_area(), where, for now, we use vma_flags_to_legacy() in order to propagate the VMA flags. No functional change intended. Link: https://lore.kernel.org/20260711-b4-vma-flags-mm-v2-3-0fa2357d5431@kernel.org Signed-off-by: Lorenzo Stoakes Reviewed-by: Lance Yang Reviewed-by: Zi Yan Reviewed-by: Vlastimil Babka (SUSE) Cc: Baolin Wang Cc: Barry Song Cc: Christian Brauner Cc: Dave Airlie Cc: David Hildenbrand Cc: Dev Jain Cc: Jani Nikula Cc: Jan Kara Cc: Jann Horn Cc: Mike Rapoport Cc: Muchun Song Cc: Nico Pache Cc: Oscar Salvador Cc: Pedro Falcato Cc: Suren Baghdasaryan Cc: Thomas Zimmermann Signed-off-by: Andrew Morton

mm/vma: update do_mmap() to use vma_flags_t

2026-08-07T01:57:01+00:00

The core do_mmap() function accepts a vm_flags_t parameter which it then manipulates before passing to mmap_region() to do the heavy lifting of the memory mapping. Update do_mmap() to instead accept a vma_flags_t parameter, and adjust all the logic within do_mmap() to manipulate this instead. This is as part of the ongoing effort to convert VMA flags from a system word size to a bitmap type which allows us to unrestrict the number of VMA flags, as well as gain control over how VMA flag manipulation occurs. We do not cascade these changes to all functions which accept vm_flags_t, but rather use vma_flags_to_legacy() where necessary, specifically deferring converting calc_vm_prot_bits(), calc_vm_flag_bits() and __get_unmapped_area() to vma_flags_t. Also utilise the new vma_flags_can_grow() predicate which correctly handles the case of architectures without upward growing stacks. As part of this change, introduce VMA_SHADOW_STACK so we can correctly handle the case of the shadow stack not being defined. No functional change intended. Link: https://lore.kernel.org/20260711-b4-vma-flags-mm-v2-2-0fa2357d5431@kernel.org Signed-off-by: Lorenzo Stoakes Reviewed-by: Lance Yang Reviewed-by: Zi Yan Reviewed-by: Vlastimil Babka (SUSE) Cc: Baolin Wang Cc: Barry Song Cc: Christian Brauner Cc: Dave Airlie Cc: David Hildenbrand Cc: Dev Jain Cc: Jani Nikula Cc: Jan Kara Cc: Jann Horn Cc: Mike Rapoport Cc: Muchun Song Cc: Nico Pache Cc: Oscar Salvador Cc: Pedro Falcato Cc: Suren Baghdasaryan Cc: Thomas Zimmermann Signed-off-by: Andrew Morton

mm: introduce vma_flags_can_grow() and vma_can_grow()

2026-08-07T01:57:01+00:00

Patch series "mm: convert more vm_flags_t users to vma_flags_t", v2. This series makes further progress in converting usage of the deprecated vm_flags_t type to its replacement, vma_flags_t. It focuses on mm, though updates some users of mm APIs also. It updates: * The core do_mmap() code path for VMA mapping. * Unmapped area logic. * The usage of mm->def_vma_flags. * VMA page protection bit logic. * General usage of VMA flags in core mm code, mlock, mprotect, mremap. This patch (of 13): These test whether the VMA has stack semantics, i.e. is able to grow upwards or downwards depending on the architecture. In order to account for arches which do not support upward-growing stacks, introduce VMA_GROWSUP whose definition depends on the architecture supporting it, and use vma_flags_test_single_mask() in vma_flags_can_grow() to account for this. No functional change intended. Link: https://lore.kernel.org/20260711-b4-vma-flags-mm-v2-0-0fa2357d5431@kernel.org Link: https://lore.kernel.org/20260711-b4-vma-flags-mm-v2-1-0fa2357d5431@kernel.org Signed-off-by: Lorenzo Stoakes Reviewed-by: Zi Yan Reviewed-by: Lance Yang Reviewed-by: Vlastimil Babka (SUSE) Cc: Baolin Wang Cc: Barry Song Cc: Christian Brauner Cc: Dave Airlie Cc: David Hildenbrand Cc: Dev Jain Cc: Jan Kara Cc: Jann Horn Cc: Lance Yang Cc: Mike Rapoport Cc: Muchun Song Cc: Nico Pache Cc: Oscar Salvador Cc: Pedro Falcato Cc: Suren Baghdasaryan Cc: Zi Yan Cc: Jani Nikula Cc: Thomas Zimmermann Signed-off-by: Andrew Morton