kernel/linux.git/include/linux/nodemask.h, branch v6.18.21

bitmap: generalize node_random()

2025-07-08T23:11:57+00:00

Generalize node_random() and make it available to general bitmaps and cpumasks users. Notice, find_first_bit() is generally faster than find_nth_bit(), and we employ it when there's a single set bit in the bitmap. See commit 3e061d924fe9c7b4 ("lib/nodemask: optimize node_random for nodemask with single NUMA node"). CC: Andrew Morton Signed-off-by: "Yury Norov [NVIDIA]"

topology: make for_each_node_with_cpus() O(N)

2025-05-13T15:40:04+00:00

for_each_node_with_cpus() calls nr_cpus_node() at every iteration, which makes it O(N^2). Kernel tracks such nodes with N_CPU record in node_states array. Switching to it makes for_each_node_with_cpus() O(N). Andrea: Now we can include also offline nodes with CPUs assigned (assuming it's possible). If checking the online state is required, the user must use node_online() within the loop. CC: Andrea Righi CC:Tejun Heo Signed-off-by: Yury Norov [NVIDIA]

nodemask: drop nodes_shift

2025-04-29T19:58:37+00:00

nodes_shift_{left,right} are not used. Drop them. Signed-off-by: Yury Norov [NVIDIA]

nodemask: numa: reorganize inclusion path

2025-02-16T16:52:19+00:00

Nodemasks now pull linux/numa.h for MAX_NUMNODES and NUMA_NO_NODE macros. This series makes numa.h depending on nodemasks, so we hit a circular dependency. Nodemasks library is highly employed by NUMA code, and it would be logical to resolve the circular dependency by making NUMA headers dependent nodemask.h. Signed-off-by: Yury Norov [NVIDIA] Signed-off-by: Andrea Righi Signed-off-by: Tejun Heo

nodemask: add nodes_copy()

2025-02-16T16:52:19+00:00

Nodemasks API misses the plain nodes_copy() which is required in this series. Signed-off-by: Yury Norov [NVIDIA] Signed-off-by: Andrea Righi Signed-off-by: Tejun Heo

nodemask: Switch from inline to __always_inline

2024-08-18T17:08:00+00:00

'inline' keyword is only a recommendation for compiler. If it decides to not inline nodemask functions, the whole small_const_nbits() machinery doesn't work. This is how a standard GCC 11.3.0 does for my x86_64 build now. This patch replaces 'inline' directive with unconditional '__always_inline' to make sure that there's always a chance for compile-time optimization. It doesn't change size of kernel image, according to bloat-o-meter. [[ Brian: split out from: Subject: [PATCH 1/3] bitmap: switch from inline to __always_inline https://lore.kernel.org/all/20221027043810.350460-2-yury.norov@gmail.com/ But rewritten, as there were too many conflicts. ]] Co-developed-by: Brian Norris Signed-off-by: Brian Norris Reviewed-by: Kees Cook Reviewed-by: Nathan Chancellor Signed-off-by: Yury Norov

nodemask: Split out include/linux/nodemask_types.h

2023-12-21T00:26:30+00:00

sched.h, which defines task_struct, needs nodemask_t - but sched.h is a frequently used header and ideally shouldn't be pulling in any more code that it needs to. This splits out nodemask_types.h which has the definition sched.h needs, which will avoid a circular header dependency in the alloc tagging patch series, and as a bonus should speed up kernel build times. Signed-off-by: Kent Overstreet Signed-off-by: Suren Baghdasaryan Cc: Ingo Molnar Cc: Peter Zijlstra

nodemask: Drop duplicate check in for_each_node_mask()

2023-06-22T20:57:41+00:00

The return value type is changed from 'int' to 'unsigned int' since commit 0dfe54071d7c8 ("nodemask: Fix return values to be unsigned"). Besides, the conversion between 'int' and 'unsigned int' on the parameter @node is guaranteed to be safe due to the limited range of MAX_NUMNODES and CONFIG_NODES_SHIFT. By the way, '(node >= 0)' should have been '(node) >= 0' actually. It's unnecessary to check if their return values are greater or equal to 0 in for_each_node_mask(). Remove it. No functional change intended. Signed-off-by: Gavin Shan Reviewed-by: Andy Shevchenko Signed-off-by: Yury Norov

treewide: use get_random_u32_below() instead of deprecated function

2022-11-18T01:15:15+00:00

This is a simple mechanical transformation done by: @@ expression E; @@ - prandom_u32_max + get_random_u32_below (E) Reviewed-by: Kees Cook Reviewed-by: Greg Kroah-Hartman Acked-by: Darrick J. Wong # for xfs Reviewed-by: SeongJae Park # for damon Reviewed-by: Jason Gunthorpe # for infiniband Reviewed-by: Russell King (Oracle) # for arm Acked-by: Ulf Hansson # for mmc Signed-off-by: Jason A. Donenfeld

treewide: use prandom_u32_max() when possible, part 1

2022-10-11T23:42:55+00:00

Rather than incurring a division or requesting too many random bytes for the given range, use the prandom_u32_max() function, which only takes the minimum required bytes from the RNG and avoids divisions. This was done mechanically with this coccinelle script: @basic@ expression E; type T; identifier get_random_u32 =~ "get_random_int|prandom_u32|get_random_u32"; typedef u64; @@ ( - ((T)get_random_u32() % (E)) + prandom_u32_max(E) | - ((T)get_random_u32() & ((E) - 1)) + prandom_u32_max(E * XXX_MAKE_SURE_E_IS_POW2) | - ((u64)(E) * get_random_u32() >> 32) + prandom_u32_max(E) | - ((T)get_random_u32() & ~PAGE_MASK) + prandom_u32_max(PAGE_SIZE) ) @multi_line@ identifier get_random_u32 =~ "get_random_int|prandom_u32|get_random_u32"; identifier RAND; expression E; @@ - RAND = get_random_u32(); ... when != RAND - RAND %= (E); + RAND = prandom_u32_max(E); // Find a potential literal @literal_mask@ expression LITERAL; type T; identifier get_random_u32 =~ "get_random_int|prandom_u32|get_random_u32"; position p; @@ ((T)get_random_u32()@p & (LITERAL)) // Add one to the literal. @script:python add_one@ literal << literal_mask.LITERAL; RESULT; @@ value = None if literal.startswith('0x'): value = int(literal, 16) elif literal[0] in '123456789': value = int(literal, 10) if value is None: print("I don't know how to handle %s" % (literal)) cocci.include_match(False) elif value == 2**32 - 1 or value == 2**31 - 1 or value == 2**24 - 1 or value == 2**16 - 1 or value == 2**8 - 1: print("Skipping 0x%x for cleanup elsewhere" % (value)) cocci.include_match(False) elif value & (value + 1) != 0: print("Skipping 0x%x because it's not a power of two minus one" % (value)) cocci.include_match(False) elif literal.startswith('0x'): coccinelle.RESULT = cocci.make_expr("0x%x" % (value + 1)) else: coccinelle.RESULT = cocci.make_expr("%d" % (value + 1)) // Replace the literal mask with the calculated result. @plus_one@ expression literal_mask.LITERAL; position literal_mask.p; expression add_one.RESULT; identifier FUNC; @@ - (FUNC()@p & (LITERAL)) + prandom_u32_max(RESULT) @collapse_ret@ type T; identifier VAR; expression E; @@ { - T VAR; - VAR = (E); - return VAR; + return E; } @drop_var@ type T; identifier VAR; @@ { - T VAR; ... when != VAR } Reviewed-by: Greg Kroah-Hartman Reviewed-by: Kees Cook Reviewed-by: Yury Norov Reviewed-by: KP Singh Reviewed-by: Jan Kara # for ext4 and sbitmap Reviewed-by: Christoph Böhmwalder # for drbd Acked-by: Jakub Kicinski Acked-by: Heiko Carstens # for s390 Acked-by: Ulf Hansson # for mmc Acked-by: Darrick J. Wong # for xfs Signed-off-by: Jason A. Donenfeld