kernel/linux.git/drivers/base/cacheinfo.c, branch v6.3.3

cacheinfo: Check cache properties are present in DT

2023-05-11T14:17:26+00:00

[ Upstream commit cde0fbff07eff7e4e0e85fa053fe19a24c86b1e0 ] If a Device Tree (DT) is used, the presence of cache properties is assumed. Not finding any is not considered. For arm64 platforms, cache information can be fetched from the clidr_el1 register. Checking whether cache information is available in the DT allows to switch to using clidr_el1. init_of_cache_level() \-of_count_cache_leaves() will assume there a 2 cache leaves (L1 data/instruction caches), which can be different from clidr_el1 information. cache_setup_of_node() tries to read cache properties in the DT. If there are none, this is considered a success. Knowing no information was available would allow to switch to using clidr_el1. Fixes: de0df442ee49 ("cacheinfo: Check 'cache-unified' property to count cache leaves") Reported-by: Alexandre Ghiti Link: https://lore.kernel.org/all/20230404-hatred-swimmer-6fecdf33b57a@spud/ Signed-off-by: Pierre Gondois Reviewed-by: Conor Dooley Link: https://lore.kernel.org/r/20230414081453.244787-3-pierre.gondois@arm.com Signed-off-by: Sudeep Holla Signed-off-by: Sasha Levin

cacheinfo: Check sib_leaf in cache_leaves_are_shared()

2023-05-11T14:17:26+00:00

[ Upstream commit 7a306e3eabf2b2fd8cffa69b87b32dbf814d79ce ] If there is no ACPI/DT information, it is assumed that L1 caches are private and L2 (and higher) caches are shared. A cache is 'shared' between two CPUs if it is accessible from these two CPUs. Each CPU owns a representation (i.e. has a dedicated cacheinfo struct) of the caches it has access to. cache_leaves_are_shared() tries to identify whether two representations are designating the same actual cache. In cache_leaves_are_shared(), if 'this_leaf' is a L2 cache (or higher) and 'sib_leaf' is a L1 cache, the caches are detected as shared as only this_leaf's cache level is checked. This is leads to setting sib_leaf as being shared with another CPU, which is incorrect as this is a L1 cache. Check 'sib_leaf->level'. Also update the comment as the function is called when populating 'shared_cpu_map'. Fixes: f16d1becf96f ("cacheinfo: Use cache identifiers to check if the caches are shared if available") Signed-off-by: Pierre Gondois Reviewed-by: Conor Dooley Link: https://lore.kernel.org/r/20230414081453.244787-2-pierre.gondois@arm.com Signed-off-by: Sudeep Holla Signed-off-by: Sasha Levin

cacheinfo: Fix LLC is not exported through sysfs

2023-03-29T10:04:10+00:00

After entering 6.3-rc1 the LLC cacheinfo is not exported on our ACPI based arm64 server. This is because the LLC cacheinfo is partly reset when secondary CPUs boot up. On arm64 the primary cpu will allocate and setup cacheinfo: init_cpu_topology() for_each_possible_cpu() fetch_cache_info() // Allocate cacheinfo and init levels detect_cache_attributes() cache_shared_cpu_map_setup() if (!last_level_cache_is_valid()) // not valid, setup LLC cache_setup_properties() // setup LLC On secondary CPU boot up: detect_cache_attributes() populate_cache_leaves() get_cache_type() // Get cache type from clidr_el1, // for LLC type=CACHE_TYPE_NOCACHE cache_shared_cpu_map_setup() if (!last_level_cache_is_valid()) // Valid and won't go to this branch, // leave LLC's type=CACHE_TYPE_NOCACHE The last_level_cache_is_valid() use cacheinfo->{attributes, fw_token} to test it's valid or not, but populate_cache_leaves() will only reset LLC's type, so we won't try to re-setup LLC's type and leave it CACHE_TYPE_NOCACHE and won't export it through sysfs. This patch tries to fix this by not re-populating the cache leaves if the LLC is valid. Fixes: 5944ce092b97 ("arch_topology: Build cacheinfo from primary CPU") Signed-off-by: Yicong Yang Reviewed-by: Pierre Gondois Reviewed-by: Sudeep Holla Link: https://lore.kernel.org/r/20230328114915.33340-1-yangyicong@huawei.com Signed-off-by: Greg Kroah-Hartman

cacheinfo: Initialize variables in fetch_cache_info()

2023-01-31T15:02:02+00:00

Set potentially uninitialized variables to 0. This is particularly relevant when CONFIG_ACPI_PPTT is not set. Reported-by: kernel test robot Link: https://lore.kernel.org/all/202301052307.JYt1GWaJ-lkp@intel.com/ Reported-by: Dan Carpenter Link: https://lore.kernel.org/all/Y86iruJPuwNN7rZw@kili/ Fixes: 5944ce092b97 ("arch_topology: Build cacheinfo from primary CPU") Signed-off-by: Pierre Gondois Reviewed-by: Conor Dooley Link: https://lore.kernel.org/r/20230124154053.355376-2-pierre.gondois@arm.com Signed-off-by: Greg Kroah-Hartman

cacheinfo: Fix shared_cpu_map to handle shared caches at different levels

2023-01-18T09:58:40+00:00

The cacheinfo sets up the shared_cpu_map by checking whether the caches with the same index are shared between CPUs. However, this will trigger slab-out-of-bounds access if the CPUs do not have the same cache hierarchy. Another problem is the mismatched shared_cpu_map when the shared cache does not have the same index between CPUs. CPU0 I D L3 index 0 1 2 x ^ ^ ^ ^ index 0 1 2 3 CPU1 I D L2 L3 This patch checks each cache is shared with all caches on other CPUs. Reviewed-by: Pierre Gondois Signed-off-by: Yong-Xuan Wang Link: https://lore.kernel.org/r/20230117105133.4445-2-yongxuan.wang@sifive.com Signed-off-by: Sudeep Holla

arch_topology: Build cacheinfo from primary CPU

2023-01-18T09:58:40+00:00

commit 3fcbf1c77d08 ("arch_topology: Fix cache attributes detection in the CPU hotplug path") adds a call to detect_cache_attributes() to populate the cacheinfo before updating the siblings mask. detect_cache_attributes() allocates memory and can take the PPTT mutex (on ACPI platforms). On PREEMPT_RT kernels, on secondary CPUs, this triggers a: 'BUG: sleeping function called from invalid context' [1] as the code is executed with preemption and interrupts disabled. The primary CPU was previously storing the cache information using the now removed (struct cpu_topology).llc_id: commit 5b8dc787ce4a ("arch_topology: Drop LLC identifier stash from the CPU topology") allocate_cache_info() tries to build the cacheinfo from the primary CPU prior secondary CPUs boot, if the DT/ACPI description contains cache information. If allocate_cache_info() fails, then fallback to the current state for the cacheinfo allocation. [1] will be triggered in such case. When unplugging a CPU, the cacheinfo memory cannot be freed. If it was, then the memory would be allocated early by the re-plugged CPU and would trigger [1]. Note that populate_cache_leaves() might be called multiple times due to populate_leaves being moved up. This is required since detect_cache_attributes() might be called with per_cpu_cacheinfo(cpu) being allocated but not populated. [1]: | BUG: sleeping function called from invalid context at kernel/locking/spinlock_rt.c:46 | in_atomic(): 1, irqs_disabled(): 128, non_block: 0, pid: 0, name: swapper/111 | preempt_count: 1, expected: 0 | RCU nest depth: 1, expected: 1 | 3 locks held by swapper/111/0: | #0: (&pcp->lock){+.+.}-{3:3}, at: get_page_from_freelist+0x218/0x12c8 | #1: (rcu_read_lock){....}-{1:3}, at: rt_spin_trylock+0x48/0xf0 | #2: (&zone->lock){+.+.}-{3:3}, at: rmqueue_bulk+0x64/0xa80 | irq event stamp: 0 | hardirqs last enabled at (0): 0x0 | hardirqs last disabled at (0): copy_process+0x5dc/0x1ab8 | softirqs last enabled at (0): copy_process+0x5dc/0x1ab8 | softirqs last disabled at (0): 0x0 | Preemption disabled at: | migrate_enable+0x30/0x130 | CPU: 111 PID: 0 Comm: swapper/111 Tainted: G W 6.0.0-rc4-rt6-[...] | Call trace: | __kmalloc+0xbc/0x1e8 | detect_cache_attributes+0x2d4/0x5f0 | update_siblings_masks+0x30/0x368 | store_cpu_topology+0x78/0xb8 | secondary_start_kernel+0xd0/0x198 | __secondary_switched+0xb0/0xb4 Signed-off-by: Pierre Gondois Reviewed-by: Sudeep Holla Acked-by: Palmer Dabbelt Link: https://lore.kernel.org/r/20230104183033.755668-7-pierre.gondois@arm.com Signed-off-by: Sudeep Holla

cacheinfo: Check 'cache-unified' property to count cache leaves

2023-01-18T09:58:31+00:00

The DeviceTree Specification v0.3 specifies that the cache node '[d-|i-|]cache-size' property is required. The 'cache-unified' property is specifies whether the cache level is separate or unified. If the cache-size property is missing, no cache leaves is accounted. This can lead to a 'BUG: KASAN: slab-out-of-bounds' [1] bug. Check 'cache-unified' property and always account for at least one cache leaf when parsing the device tree. [1] https://lore.kernel.org/all/0f19cb3f-d6cf-4032-66d2-dedc9d09a0e3@linaro.org/ Reported-by: Krzysztof Kozlowski Signed-off-by: Pierre Gondois Tested-by: Krzysztof Kozlowski Link: https://lore.kernel.org/r/20230104183033.755668-4-pierre.gondois@arm.com Signed-off-by: Sudeep Holla

cacheinfo: Return error code in init_of_cache_level()

2023-01-17T22:00:06+00:00

Make init_of_cache_level() return an error code when the cache information parsing fails to help detecting missing information. init_of_cache_level() is only called for riscv. Returning an error code instead of 0 will prevent detect_cache_attributes() to allocate memory if an incomplete DT is parsed. Signed-off-by: Pierre Gondois Acked-by: Palmer Dabbelt Link: https://lore.kernel.org/r/20230104183033.755668-3-pierre.gondois@arm.com Signed-off-by: Sudeep Holla

cacheinfo: Use RISC-V's init_cache_level() as generic OF implementation

2023-01-17T21:59:52+00:00

RISC-V's implementation of init_of_cache_level() is following the Devicetree Specification v0.3 regarding caches, cf.: - s3.7.3 'Internal (L1) Cache Properties' - s3.8 'Multi-level and Shared Cache Nodes' Allow reusing the implementation by moving it. Also make 'levels', 'leaves' and 'level' unsigned int. Signed-off-by: Pierre Gondois Reviewed-by: Conor Dooley Acked-by: Palmer Dabbelt Link: https://lore.kernel.org/r/20230104183033.755668-2-pierre.gondois@arm.com Signed-off-by: Sudeep Holla

cacheinfo: Remove of_node_put() for fw_token

2022-11-23T19:03:51+00:00

fw_token is used for DT/ACPI systems to identify CPUs sharing caches. For DT based systems, fw_token is set to a pointer to a DT node. commit 3da72e18371c ("cacheinfo: Decrement refcount in cache_setup_of_node()") doesn't increment the refcount of fw_token anymore in cache_setup_of_node(). fw_token is indeed used as a token and not as a (struct device_node*), so no reference to fw_token should be kept. However, [1] is triggered when hotplugging a CPU multiple times since cache_shared_cpu_map_remove() decrements the refcount to fw_token at each CPU unplugging, eventually reaching 0. Remove of_node_put() for fw_token in cache_shared_cpu_map_remove(). [1] ------------[ cut here ]------------ refcount_t: saturated; leaking memory. WARNING: CPU: 4 PID: 32 at lib/refcount.c:22 refcount_warn_saturate (lib/refcount.c:22 (discriminator 3)) Modules linked in: CPU: 4 PID: 32 Comm: cpuhp/4 Tainted: G W 6.1.0-rc1-14091-g9fdf2ca7b9c8 #76 Hardware name: ARM LTD ARM Juno Development Platform/ARM Juno Development Platform, BIOS EDK II Oct 31 2022 pstate: 600000c5 (nZCv daIF -PAN -UAO -TCO -DIT -SSBS BTYPE=--) pc : refcount_warn_saturate (lib/refcount.c:22 (discriminator 3)) lr : refcount_warn_saturate (lib/refcount.c:22 (discriminator 3)) [...] Call trace: [...] of_node_release (drivers/of/dynamic.c:335) kobject_put (lib/kobject.c:677 lib/kobject.c:704 ./include/linux/kref.h:65 lib/kobject.c:721) of_node_put (drivers/of/dynamic.c:49) free_cache_attributes.part.0 (drivers/base/cacheinfo.c:712) cacheinfo_cpu_pre_down (drivers/base/cacheinfo.c:718) cpuhp_invoke_callback (kernel/cpu.c:247 (discriminator 4)) cpuhp_thread_fun (kernel/cpu.c:785) smpboot_thread_fn (kernel/smpboot.c:164 (discriminator 3)) kthread (kernel/kthread.c:376) ret_from_fork (arch/arm64/kernel/entry.S:861) ---[ end trace 0000000000000000 ]--- Fixes: 3da72e18371c ("cacheinfo: Decrement refcount in cache_setup_of_node()") Reported-by: Geert Uytterhoeven Reported-by: Marek Szyprowski Tested-by: Geert Uytterhoeven Tested-by: Sudeep Holla Reviewed-by: Sudeep Holla Signed-off-by: Pierre Gondois Link: https://lore.kernel.org/r/20221116094958.2141072-1-pierre.gondois@arm.com Signed-off-by: Greg Kroah-Hartman