kernel/linux.git/lib/idr.c, branch v4.19.148

idr: Fix idr_get_next_ul race with idr_remove

2019-12-17T19:36:02+00:00

[ Upstream commit 5a74ac4c4a97bd8b7dba054304d598e2a882fea6 ] Commit 5c089fd0c734 ("idr: Fix idr_get_next race with idr_remove") neglected to fix idr_get_next_ul(). As far as I can tell, nobody's actually using this interface under the RCU read lock, but fix it now before anybody decides to use it. Fixes: 5c089fd0c734 ("idr: Fix idr_get_next race with idr_remove") Signed-off-by: Matthew Wilcox (Oracle) Signed-off-by: Sasha Levin

idr: Fix idr_get_next race with idr_remove

2019-11-24T07:19:11+00:00

commit 5c089fd0c73411f2170ab795c9ffc16718c7d007 upstream. If the entry is deleted from the IDR between the call to radix_tree_iter_find() and rcu_dereference_raw(), idr_get_next() will return NULL, which will end the iteration prematurely. We should instead continue to the next entry in the IDR. This only happens if the iteration is protected by the RCU lock. Most IDR users use a spinlock or semaphore to exclude simultaneous modifications. It was noticed once the PID allocator was converted to use the IDR, as it uses the RCU lock, but there may be other users elsewhere in the kernel. We can't use the normal pattern of calling radix_tree_deref_retry() (which catches both a retry entry in a leaf node and a node entry in the root) as the IDR supports storing entries which are unaligned, which will trigger an infinite loop if they are encountered. Instead, we have to explicitly check whether the entry is a retry entry. Fixes: 0a835c4f090a ("Reimplement IDR and IDA using the radix tree") Reported-by: Brendan Gregg Tested-by: Brendan Gregg Signed-off-by: Matthew Wilcox (Oracle) Signed-off-by: Greg Kroah-Hartman

ida: Change ida_get_new_above to return the id

2018-08-22T03:54:21+00:00

This calling convention makes more sense for the implementation as well as the callers. It even shaves 32 bytes off the compiled code size. Signed-off-by: Matthew Wilcox

ida: Remove old API

2018-08-22T03:54:21+00:00

Delete ida_pre_get(), ida_get_new(), ida_get_new_above() and ida_remove() from the public API. Some of these functions still exist as internal helpers, but they should not be called by consumers. Signed-off-by: Matthew Wilcox

ida: Add new API

2018-08-22T03:54:13+00:00

Add ida_alloc(), ida_alloc_min(), ida_alloc_max(), ida_alloc_range() and ida_free(). The ida_alloc_max() and ida_alloc_range() functions differ from ida_simple_get() in that they take an inclusive 'max' parameter instead of an exclusive 'end' parameter. Callers are about evenly split whether they'd like inclusive or exclusive parameters and 'max' is easier to document than 'end'. Change the IDA allocation to first attempt to allocate a bit using existing memory, and only allocate memory afterwards. Also change the behaviour of 'min' > INT_MAX from being a BUG() to returning -ENOSPC. Leave compatibility wrappers in place for ida_simple_get() and ida_simple_remove() to avoid changing all callers. Signed-off-by: Matthew Wilcox

ida: Lock the IDA in ida_destroy

2018-08-22T03:49:31+00:00

The user has no need to handle locking between ida_simple_get() and ida_simple_remove(). They shouldn't be forced to think about whether ida_destroy() might be called at the same time as any of their other IDA manipulation calls. Improve the documnetation while I'm in here. Signed-off-by: Matthew Wilcox

lib/idr.c: remove simple_ida_lock

2018-06-08T00:34:39+00:00

Improve the scalability of the IDA by using the per-IDA xa_lock rather than the global simple_ida_lock. IDAs are not typically used in performance-sensitive locations, but since we have this lock anyway, we can use it. It is also a step towards converting the IDA from the radix tree to the XArray. [akpm@linux-foundation.org: idr.c needs xarray.h] Link: http://lkml.kernel.org/r/20180331125332.GF13332@bombadil.infradead.org Signed-off-by: Matthew Wilcox Reviewed-by: Andrew Morton Cc: Rasmus Villemoes Cc: Daniel Vetter Cc: Tejun Heo Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds

idr: Fix handling of IDs above INT_MAX

2018-02-26T19:39:30+00:00

Khalid reported that the kernel selftests are currently failing: selftests: test_bpf.sh ======================================== test_bpf: [FAIL] not ok 1..8 selftests: test_bpf.sh [FAIL] He bisected it to 6ce711f2750031d12cec91384ac5cfa0a485b60a ("idr: Make 1-based IDRs more efficient"). The root cause is doing a signed comparison in idr_alloc_u32() instead of an unsigned comparison. I went looking for any similar problems and found a couple (which would each result in the failure to warn in two situations that aren't supposed to happen). I knocked up a few test-cases to prove that I was right and added them to the test-suite. Reported-by: Khalid Aziz Tested-by: Khalid Aziz Signed-off-by: Matthew Wilcox

ida: do zeroing in ida_pre_get()

2018-02-21T23:35:43+00:00

As far as I can tell, the only place the per-cpu ida_bitmap is populated is in ida_pre_get. The pre-allocated element is stolen in two places in ida_get_new_above, in both cases immediately followed by a memset(0). Since ida_get_new_above is called with locks held, do the zeroing in ida_pre_get, or rather let kmalloc() do it. Also, apparently gcc generates ~44 bytes of code to do a memset(, 0, 128): $ scripts/bloat-o-meter vmlinux.{0,1} add/remove: 0/0 grow/shrink: 2/1 up/down: 5/-88 (-83) Function old new delta ida_pre_get 115 119 +4 vermagic 27 28 +1 ida_get_new_above 715 627 -88 Link: http://lkml.kernel.org/r/20180108225634.15340-1-linux@rasmusvillemoes.dk Signed-off-by: Rasmus Villemoes Acked-by: Matthew Wilcox Cc: Eric Biggers Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds

idr: Make 1-based IDRs more efficient

2018-02-06T21:41:29+00:00

About 20% of the IDR users in the kernel want the allocated IDs to start at 1. The implementation currently searches all the way down the left hand side of the tree, finds no free ID other than ID 0, walks all the way back up, and then all the way down again. This patch 'rebases' the ID so we fill the entire radix tree, rather than leave a gap at 0. Chris Wilson says: "I did the quick hack of allocating index 0 of the idr and that eradicated idr_get_free() from being at the top of the profiles for the many-object stress tests. This improvement will be much appreciated." Signed-off-by: Matthew Wilcox