kernel/linux.git/lib/codetag.c, branch v7.3-rc1

alloc_tag: add ioctl to /proc/allocinfo

2026-08-25T01:43:12+00:00

Patch series "alloc_tag: introduce IOCTL-based filtering for MAP", v8. Currently, memory allocation profiling data is primarily exposed through /proc/allocinfo. While useful for manual inspection, this text-based interface poses challenges for production monitoring and large-scale analysis: 1. Userspace must parse large amounts of text to extract specific fields. 2. To find specific tags, userspace must read the entire dataset, requiring many context switches and high data copying. 3. The kernel currently aggregates per-CPU counters for every allocation size, even those the user intends to filter out immediately. This series introduces a new IOCTL-based binary interface for allocinfo that supports kernel-side filtering. By allowing the user to specify a filter mask, we significantly reduce the work performed in-kernel and the amount of data transferred to userspace. The IOCTL mechanism was chosen for allocinfo to address the per-CPU counter aggregation bottleneck. A traditional read() operation must report the total allocation count and sizes for every code tag in the system. Doing so requires iterating across all CPUs to sum their per-CPU counters for thousands of tags, which introduces substantial runtime overhead. The IOCTL interface allows userspace to push selective filtering criteria directly into the kernel before the per-CPU counter aggregation. The kernel aggregates per-CPU counters only for a small subset of tags that match the filter. This results in significant performance improvement. Beyond fast filtered retrieval, the IOCTL foundation allows introducing a context capture mechanism in the future to capture the context for specific allocations. Performance measurements were conducted on an Intel Xeon Platinum 8481C (224 CPUs) with caches dropped before each run. The IOCTL mechanism shows a ~20x performance improvement for filtered queries. The kernel avoids the expensive per-CPU counter aggregation (alloc_tag_read) for any tags that fail the initial string or location filters. Scenario 1: Specific File Filtering (arch/x86/events/rapl.c) 1. Traditional (cat /proc/allocinfo | grep): 22ms (sys) 2. IOCTL Interface: 1ms (sys) Scenario 2: Compound Filtering (Filename + Size) 1. Traditional: (cat ... | grep | awk): 21ms (sys) 2. IOCTL Interface: 1ms (sys) Scenario 3: Size-Based Filtering (min_size = 1MB) 1. Traditional: (cat ... | awk): 21ms (sys) 2. IOCTL Interface: 14ms (sys) This patch (of 6): Add the following ioctl commands for /proc/allocinfo file: ALLOCINFO_IOC_CONTENT_ID - gets content identifier which can be used to check whether the file content has changed specifically due to module load/unload. Every time a module is loaded / unloaded, the returned value will be different. By comparing the identifier value at the beginning and at the end of the content retrieval operation, users can validate retrieved information for consistency. ALLOCINFO_IOC_GET_AT - gets the record at the specified position. This is the position of a record in /proc/allocinfo. ALLOCINFO_IOC_GET_NEXT - gets the record next to the last retrieved one. If no records were previously retrieved, returns the first record. Note, function file and module names often have the same prefixes, therefore when filtering for them, we compare the last 64 characters to minimize the chances of name collisions. [akpm@linux-foundation.org: include compat.h, per Suren] Closes: https://lore.kernel.org/oe-kbuild-all/202607091820.qbjlGhKK-lkp@intel.com/ Link: https://lore.kernel.org/cover.1783532853.git.abhishekbapat@google.com Link: https://lore.kernel.org/15596de2607ef13e7c77c6d74763f4ae992ec475.1783532853.git.abhishekbapat@google.com Signed-off-by: Suren Baghdasaryan Signed-off-by: Abhishek Bapat Acked-by: Hao Ge Cc: Jonathan Corbet Cc: Kent Overstreet Cc: Sourav Panda Signed-off-by: Andrew Morton

lib: split codetag_lock_module_list()

2026-06-21T18:31:28+00:00

Letting a function argument indicate whether a lock or unlock operation should be performed is incompatible with compile-time analysis of locking operations by sparse and Clang. Hence, split codetag_lock_module_list() into two functions: a function that locks cttype->mod_lock and another function that unlocks cttype->mod_lock. No functionality has been changed. See also commit 916cc5167cc6 ("lib: code tagging framework"). Link: https://lore.kernel.org/20260324214226.3684605-1-bvanassche@acm.org Signed-off-by: Bart Van Assche Acked-by: Suren Baghdasaryan Cc: Kent Overstreet Cc: Nathan Chancellor Signed-off-by: Andrew Morton

Convert 'alloc_obj' family to use the new default GFP_KERNEL argument

2026-02-22T01:09:51+00:00

This was done entirely with mindless brute force, using git grep -l '\

treewide: Replace kmalloc with kmalloc_obj for non-scalar types

2026-02-21T09:02:28+00:00

This is the result of running the Coccinelle script from scripts/coccinelle/api/kmalloc_objs.cocci. The script is designed to avoid scalar types (which need careful case-by-case checking), and instead replace kmalloc-family calls that allocate struct or union object instances: Single allocations: kmalloc(sizeof(TYPE), ...) are replaced with: kmalloc_obj(TYPE, ...) Array allocations: kmalloc_array(COUNT, sizeof(TYPE), ...) are replaced with: kmalloc_objs(TYPE, COUNT, ...) Flex array allocations: kmalloc(struct_size(PTR, FAM, COUNT), ...) are replaced with: kmalloc_flex(*PTR, FAM, COUNT, ...) (where TYPE may also be *VAR) The resulting allocations no longer return "void *", instead returning "TYPE *". Signed-off-by: Kees Cook

alloc_tag: add sequence number for module and iterator

2025-07-10T05:42:06+00:00

Codetag iterator use pair to guarantee the validness. But both id and address can be reused, there is theoretical possibility when module inserted right after another module removed, kmalloc returns an address same as the address kfree by previous module and IDR key reuses the key recently removed. Add a sequence number to codetag_module and code_iterator, the sequence number is strickly incremented whenever a module is loaded. An iterator is valid if and only if its sequence number match codetag_module's. Link: https://lkml.kernel.org/r/20250609064200.112639-1-00107082@163.com Signed-off-by: David Wang <00107082@163.com> Acked-by: Suren Baghdasaryan Cc: Kent Overstreet Cc: Tim Chen Signed-off-by: Andrew Morton

alloc_tag: handle module codetag load errors as module load failures

2025-06-06T05:02:23+00:00

Failures inside codetag_load_module() are currently ignored. As a result an error there would not cause a module load failure and freeing of the associated resources. Correct this behavior by propagating the error code to the caller and handling possible errors. With this change, error to allocate percpu counters, which happens at this stage, will not be ignored and will cause a module load failure and freeing of resources. With this change we also do not need to disable memory allocation profiling when this error happens, instead we fail to load the module. Link: https://lkml.kernel.org/r/20250521160602.1940771-1-surenb@google.com Fixes: 10075262888b ("alloc_tag: allocate percpu counters for module tags dynamically") Signed-off-by: Suren Baghdasaryan Reported-by: Casey Chen Closes: https://lore.kernel.org/all/20250520231620.15259-1-cachen@purestorage.com/ Cc: Daniel Gomez Cc: David Wang <00107082@163.com> Cc: Kent Overstreet Cc: Luis Chamberalin Cc: Petr Pavlu Cc: Sami Tolvanen Cc: Signed-off-by: Andrew Morton

alloc_tag: allocate percpu counters for module tags dynamically

2025-05-25T07:53:48+00:00

When a module gets unloaded it checks whether any of its tags are still in use and if so, we keep the memory containing module's allocation tags alive until all tags are unused. However percpu counters referenced by the tags are freed by free_module(). This will lead to UAF if the memory allocated by a module is accessed after module was unloaded. To fix this we allocate percpu counters for module allocation tags dynamically and we keep it alive for tags which are still in use after module unloading. This also removes the requirement of a larger PERCPU_MODULE_RESERVE when memory allocation profiling is enabled because percpu memory for counters does not need to be reserved anymore. Link: https://lkml.kernel.org/r/20250517000739.5930-1-surenb@google.com Fixes: 0db6f8d7820a ("alloc_tag: load module tags into separate contiguous memory") Signed-off-by: Suren Baghdasaryan Reported-by: David Wang <00107082@163.com> Closes: https://lore.kernel.org/all/20250516131246.6244-1-00107082@163.com/ Tested-by: David Wang <00107082@163.com> Cc: Christoph Lameter (Ampere) Cc: Dennis Zhou Cc: Kent Overstreet Cc: Pasha Tatashin Cc: Tejun Heo Cc: Signed-off-by: Andrew Morton

alloc_tag: support for page allocation tag compression

2024-11-07T22:25:16+00:00

Implement support for storing page allocation tag references directly in the page flags instead of page extensions. sysctl.vm.mem_profiling boot parameter it extended to provide a way for a user to request this mode. Enabling compression eliminates memory overhead caused by page_ext and results in better performance for page allocations. However this mode will not work if the number of available page flag bits is insufficient to address all kernel allocations. Such condition can happen during boot or when loading a module. If this condition is detected, memory allocation profiling gets disabled with an appropriate warning. By default compression mode is disabled. Link: https://lkml.kernel.org/r/20241023170759.999909-7-surenb@google.com Signed-off-by: Suren Baghdasaryan Reviewed-by: Pasha Tatashin Cc: Ard Biesheuvel Cc: Arnd Bergmann Cc: Borislav Petkov (AMD) Cc: Christoph Hellwig Cc: Daniel Gomez Cc: David Hildenbrand Cc: Davidlohr Bueso Cc: David Rientjes Cc: Dennis Zhou Cc: Johannes Weiner Cc: John Hubbard Cc: Jonathan Corbet Cc: Joonsoo Kim Cc: Kalesh Singh Cc: Kees Cook Cc: Kent Overstreet Cc: Liam R. Howlett Cc: Luis Chamberlain Cc: Matthew Wilcox Cc: Michal Hocko Cc: Mike Rapoport (Microsoft) Cc: Minchan Kim Cc: Paul E. McKenney Cc: Petr Pavlu Cc: Roman Gushchin Cc: Sami Tolvanen Cc: Sourav Panda Cc: Steven Rostedt (Google) Cc: Thomas Gleixner Cc: Thomas Huth Cc: Uladzislau Rezki (Sony) Cc: Vlastimil Babka Cc: Xiongwei Song Cc: Yu Zhao Signed-off-by: Andrew Morton

alloc_tag: load module tags into separate contiguous memory

2024-11-07T22:25:16+00:00

When a module gets unloaded there is a possibility that some of the allocations it made are still used and therefore the allocation tags corresponding to these allocations are still referenced. As such, the memory for these tags can't be freed. This is currently handled as an abnormal situation and module's data section is not being unloaded. To handle this situation without keeping module's data in memory, allow codetags with longer lifespan than the module to be loaded into their own separate memory. The in-use memory areas and gaps after module unloading in this separate memory are tracked using maple trees. Allocation tags arrange their separate memory so that it is virtually contiguous and that will allow simple allocation tag indexing later on in this patchset. The size of this virtually contiguous memory is set to store up to 100000 allocation tags. [surenb@google.com: fix empty codetag module section handling] Link: https://lkml.kernel.org/r/20241101000017.3856204-1-surenb@google.com [akpm@linux-foundation.org: update comment, per Dan] Link: https://lkml.kernel.org/r/20241023170759.999909-4-surenb@google.com Signed-off-by: Suren Baghdasaryan Reviewed-by: Pasha Tatashin Cc: Ard Biesheuvel Cc: Arnd Bergmann Cc: Borislav Petkov (AMD) Cc: Christoph Hellwig Cc: Daniel Gomez Cc: David Hildenbrand Cc: Davidlohr Bueso Cc: David Rientjes Cc: Dennis Zhou Cc: Johannes Weiner Cc: John Hubbard Cc: Jonathan Corbet Cc: Joonsoo Kim Cc: Kalesh Singh Cc: Kees Cook Cc: Kent Overstreet Cc: Liam R. Howlett Cc: Luis Chamberlain Cc: Matthew Wilcox Cc: Michal Hocko Cc: Mike Rapoport (Microsoft) Cc: Minchan Kim Cc: Paul E. McKenney Cc: Petr Pavlu Cc: Roman Gushchin Cc: Sami Tolvanen Cc: Sourav Panda Cc: Steven Rostedt (Google) Cc: Thomas Gleixner Cc: Thomas Huth Cc: Uladzislau Rezki (Sony) Cc: Vlastimil Babka Cc: Xiongwei Song Cc: Yu Zhao Cc: Dan Carpenter Signed-off-by: Andrew Morton

lib: alloc_tag_module_unload must wait for pending kfree_rcu calls

2024-10-17T07:28:07+00:00

Ben Greear reports following splat: ------------[ cut here ]------------ net/netfilter/nf_nat_core.c:1114 module nf_nat func:nf_nat_register_fn has 256 allocated at module unload WARNING: CPU: 1 PID: 10421 at lib/alloc_tag.c:168 alloc_tag_module_unload+0x22b/0x3f0 Modules linked in: nf_nat(-) btrfs ufs qnx4 hfsplus hfs minix vfat msdos fat ... Hardware name: Default string Default string/SKYBAY, BIOS 5.12 08/04/2020 RIP: 0010:alloc_tag_module_unload+0x22b/0x3f0 codetag_unload_module+0x19b/0x2a0 ? codetag_load_module+0x80/0x80 nf_nat module exit calls kfree_rcu on those addresses, but the free operation is likely still pending by the time alloc_tag checks for leaks. Wait for outstanding kfree_rcu operations to complete before checking resolves this warning. Reproducer: unshare -n iptables-nft -t nat -A PREROUTING -p tcp grep nf_nat /proc/allocinfo # will list 4 allocations rmmod nft_chain_nat rmmod nf_nat # will WARN. [akpm@linux-foundation.org: add comment] Link: https://lkml.kernel.org/r/20241007205236.11847-1-fw@strlen.de Fixes: a473573964e5 ("lib: code tagging module support") Signed-off-by: Florian Westphal Reported-by: Ben Greear Closes: https://lore.kernel.org/netdev/bdaaef9d-4364-4171-b82b-bcfc12e207eb@candelatech.com/ Cc: Uladzislau Rezki Cc: Vlastimil Babka Cc: Suren Baghdasaryan Cc: Kent Overstreet Cc: Signed-off-by: Andrew Morton