summaryrefslogtreecommitdiff
path: root/include/linux
diff options
context:
space:
mode:
authorHrushikesh Salunke <hsalunke@amd.com>2026-04-22 13:26:58 +0300
committerAndrew Morton <akpm@linux-foundation.org>2026-05-29 07:04:39 +0300
commitb001cf7d16dd18f14bd372a8018ecbf48197289d (patch)
treee52f2fe516fa6b477beecf3fbc3b38958ef575df /include/linux
parent04cf82a741e096bf74e77bb8cf11e66481b4dcdf (diff)
downloadlinux-b001cf7d16dd18f14bd372a8018ecbf48197289d.tar.xz
mm/page_alloc: replace kernel_init_pages() with batch page clearing
When init_on_alloc is enabled, kernel_init_pages() clears every page one at a time via clear_highpage_kasan_tagged(), which incurs per-page kmap_local_page()/kunmap_local() overhead and prevents the architecture clearing primitive from operating on contiguous ranges. Introduce clear_highpages_kasan_tagged() as a static batch clearing helper in page_alloc.c that calls clear_pages() for the full contiguous range on !HIGHMEM systems, bypassing the per-page kmap overhead and allowing a single invocation of the arch clearing primitive across the entire allocation. The HIGHMEM path falls back to per-page clearing since those pages require kmap. Replace kernel_init_pages() with direct calls to the new helper, as it becomes a trivial wrapper. Allocating 8192 x 2MB HugeTLB pages (16GB) with init_on_alloc=1: Before: 0.445s After: 0.166s (-62.7%, 2.68x faster) Kernel time (sys) reduction per workload with init_on_alloc=1: Workload Before After Change Graph500 64C128T 30m 41.8s 15m 14.8s -50.3% Graph500 16C32T 15m 56.7s 9m 43.7s -39.0% Pagerank 32T 1m 58.5s 1m 12.8s -38.5% Pagerank 128T 2m 36.3s 1m 40.4s -35.7% [hsalunke@amd.com: move clear_highpages_kasan_tagged() to page_alloc.c] Link: https://lore.kernel.org/20260504063942.553438-1-hsalunke@amd.com Link: https://lore.kernel.org/20260422102729.166599-1-hsalunke@amd.com Signed-off-by: Hrushikesh Salunke <hsalunke@amd.com> Acked-by: Vlastimil Babka (SUSE) <vbabka@kernel.org> Acked-by: Zi Yan <ziy@nvidia.com> Acked-by: Pankaj Gupta <pankaj.gupta@amd.com> Acked-by: David Hildenbrand (Arm) <david@kernel.org> Acked-by: Lorenzo Stoakes <ljs@kernel.org> Cc: Ankur Arora <ankur.a.arora@oracle.com> Cc: Bharata B Rao <bharata@amd.com> Cc: Brendan Jackman <jackmanb@google.com> Cc: Johannes Weiner <hannes@cmpxchg.org> Cc: Liam Howlett <liam.howlett@oracle.com> Cc: Michal Hocko <mhocko@suse.com> Cc: Mike Rapoport <rppt@kernel.org> Cc: Shivank Garg <shivankg@amd.com> Cc: Suren Baghdasaryan <surenb@google.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Diffstat (limited to 'include/linux')
0 files changed, 0 insertions, 0 deletions