kernel/linux.git/lib/cpumask.c, branch v6.6.131

sched/topology: Handle NUMA_NO_NODE in sched_numa_find_nth_cpu()

2024-08-29T15:33:24+00:00

[ Upstream commit 9ecea9ae4d3127a09fb5dfcea87f248937a39ff5 ] sched_numa_find_nth_cpu() doesn't handle NUMA_NO_NODE properly, and may crash kernel if passed with it. On the other hand, the only user of sched_numa_find_nth_cpu() has to check NUMA_NO_NODE case explicitly. It would be easier for users if this logic will get moved into sched_numa_find_nth_cpu(). Signed-off-by: Yury Norov Signed-off-by: Ingo Molnar Cc: Mel Gorman Link: https://lore.kernel.org/r/20230819141239.287290-6-yury.norov@gmail.com Signed-off-by: Sasha Levin

cpumask: eliminate kernel-doc warnings

2023-07-17T22:47:21+00:00

Update lib/cpumask.c and to fix all kernel-doc warnings: include/linux/cpumask.h:185: warning: Function parameter or member 'srcp1' not described in 'cpumask_first_and' include/linux/cpumask.h:185: warning: Function parameter or member 'srcp2' not described in 'cpumask_first_and' include/linux/cpumask.h:185: warning: Excess function parameter 'src1p' description in 'cpumask_first_and' include/linux/cpumask.h:185: warning: Excess function parameter 'src2p' description in 'cpumask_first_and' lib/cpumask.c:59: warning: Function parameter or member 'node' not described in 'alloc_cpumask_var_node' lib/cpumask.c:169: warning: Function parameter or member 'src1p' not described in 'cpumask_any_and_distribute' lib/cpumask.c:169: warning: Function parameter or member 'src2p' not described in 'cpumask_any_and_distribute' Fixes: 7b4967c53204 ("cpumask: Add alloc_cpumask_var_node()") Fixes: 839cad5fa54b ("cpumask: fix function description kernel-doc notation") Fixes: 93ba139ba819 ("cpumask: use find_first_and_bit()") Signed-off-by: Randy Dunlap Reviewed-by: Andy Shevchenko Signed-off-by: Yury Norov

cpumask: fix function description kernel-doc notation

2023-06-22T20:57:41+00:00

Use kernel-doc notation for the function description to prevent a warning: lib/cpumask.c:160: warning: This comment starts with '/**', but isn't a kernel-doc comment. Refer Documentation/doc-guide/kernel-doc.rst * Returns an arbitrary cpu within srcp1 & srcp2. Signed-off-by: Randy Dunlap Reviewed-by: Andy Shevchenko Signed-off-by: Yury Norov

lib/cpumask: update comment for cpumask_local_spread()

2023-02-08T02:20:00+00:00

Now that we have an iterator-based alternative for a very common case of using cpumask_local_spread for all cpus in a row, it's worth to mention that in comment to cpumask_local_spread(). Signed-off-by: Yury Norov Reviewed-by: Valentin Schneider Reviewed-by: Tariq Toukan Signed-off-by: Jakub Kicinski

lib/cpumask: reorganize cpumask_local_spread() logic

2023-02-08T02:20:00+00:00

Now after moving all NUMA logic into sched_numa_find_nth_cpu(), else-branch of cpumask_local_spread() is just a function call, and we can simplify logic by using ternary operator. While here, replace BUG() with WARN_ON(). Signed-off-by: Yury Norov Acked-by: Tariq Toukan Reviewed-by: Jacob Keller Reviewed-by: Peter Lafreniere Signed-off-by: Jakub Kicinski

cpumask: improve on cpumask_local_spread() locality

2023-02-08T02:20:00+00:00

Switch cpumask_local_spread() to use newly added sched_numa_find_nth_cpu(), which takes into account distances to each node in the system. For the following NUMA configuration: root@debian:~# numactl -H available: 4 nodes (0-3) node 0 cpus: 0 1 2 3 node 0 size: 3869 MB node 0 free: 3740 MB node 1 cpus: 4 5 node 1 size: 1969 MB node 1 free: 1937 MB node 2 cpus: 6 7 node 2 size: 1967 MB node 2 free: 1873 MB node 3 cpus: 8 9 10 11 12 13 14 15 node 3 size: 7842 MB node 3 free: 7723 MB node distances: node 0 1 2 3 0: 10 50 30 70 1: 50 10 70 30 2: 30 70 10 50 3: 70 30 50 10 The new cpumask_local_spread() traverses cpus for each node like this: node 0: 0 1 2 3 6 7 4 5 8 9 10 11 12 13 14 15 node 1: 4 5 8 9 10 11 12 13 14 15 0 1 2 3 6 7 node 2: 6 7 0 1 2 3 8 9 10 11 12 13 14 15 4 5 node 3: 8 9 10 11 12 13 14 15 4 5 6 7 0 1 2 3 Signed-off-by: Yury Norov Acked-by: Tariq Toukan Reviewed-by: Jacob Keller Reviewed-by: Peter Lafreniere Signed-off-by: Jakub Kicinski

lib/find_bit: add find_next{,_and}_bit_wrap

2022-10-01T17:22:57+00:00

The helper is better optimized for the worst case: in case of empty cpumask, current code traverses 2 * size: next = cpumask_next_and(prev, src1p, src2p); if (next >= nr_cpu_ids) next = cpumask_first_and(src1p, src2p); At bitmap level we can stop earlier after checking 'size + offset' bits. Signed-off-by: Yury Norov

cpumask: add cpumask_nth_{,and,andnot}

2022-09-26T19:19:12+00:00

Add cpumask_nth_{,and,andnot} as wrappers around corresponding find functions, and use it in cpumask_local_spread(). Signed-off-by: Yury Norov

lib/cpumask: drop always-true preprocessor guard

2022-08-15T18:00:44+00:00

Since lib/cpumask.o is only built for CONFIG_SMP=y, NR_CPUS will always be greater than 1 at compile time. This makes checking for that condition unnecesarry, so it can be dropped. Signed-off-by: Sander Vanheule Signed-off-by: Yury Norov

Merge tag 'bitmap-6.0-rc1' of https://github.com/norov/linux

2022-08-08T00:52:35+00:00

Pull bitmap updates from Yury Norov: - fix the duplicated comments on bitmap_to_arr64() (Qu Wenruo) - optimize out non-atomic bitops on compile-time constants (Alexander Lobakin) - cleanup bitmap-related headers (Yury Norov) - x86/olpc: fix 'logical not is only applied to the left hand side' (Alexander Lobakin) - lib/nodemask: inline wrappers around bitmap (Yury Norov) * tag 'bitmap-6.0-rc1' of https://github.com/norov/linux: (26 commits) lib/nodemask: inline next_node_in() and node_random() powerpc: drop dependency on in archrandom.h x86/olpc: fix 'logical not is only applied to the left hand side' lib/cpumask: move some one-line wrappers to header file headers/deps: mm: align MANITAINERS and Docs with new gfp.h structure headers/deps: mm: Split out of headers/deps: mm: Optimize header dependencies lib/cpumask: move trivial wrappers around find_bit to the header lib/cpumask: change return types to unsigned where appropriate cpumask: change return types to bool where appropriate lib/bitmap: change type of bitmap_weight to unsigned long lib/bitmap: change return types to bool where appropriate arm: align find_bit declarations with generic kernel iommu/vt-d: avoid invalid memory access via node_online(NUMA_NO_NODE) lib/test_bitmap: test the tail after bitmap_to_arr64() lib/bitmap: fix off-by-one in bitmap_to_arr64() lib: test_bitmap: add compile-time optimization/evaluations assertions bitmap: don't assume compiler evaluates small mem*() builtins calls net/ice: fix initializing the bitmap in the switch code bitops: let optimize out non-atomic bitops on compile-time constants ...