kernel/linux.git/drivers/base/cacheinfo.c, branch v6.6.132

cacheinfo: Allocate memory during CPU hotplug if not done from the primary CPU

2024-12-14T18:59:55+00:00

commit b3fce429a1e030b50c1c91351d69b8667eef627b upstream. Commit 5944ce092b97 ("arch_topology: Build cacheinfo from primary CPU") adds functionality that architectures can use to optionally allocate and build cacheinfo early during boot. Commit 6539cffa9495 ("cacheinfo: Add arch specific early level initializer") lets secondary CPUs correct (and reallocate memory) cacheinfo data if needed. If the early build functionality is not used and cacheinfo does not need correction, memory for cacheinfo is never allocated. x86 does not use the early build functionality. Consequently, during the cacheinfo CPU hotplug callback, last_level_cache_is_valid() attempts to dereference a NULL pointer: BUG: kernel NULL pointer dereference, address: 0000000000000100 #PF: supervisor read access in kernel mode #PF: error_code(0x0000) - not present page PGD 0 P4D 0 Oops: 0000 [#1] PREEPMT SMP NOPTI CPU: 0 PID 19 Comm: cpuhp/0 Not tainted 6.4.0-rc2 #1 RIP: 0010: last_level_cache_is_valid+0x95/0xe0a Allocate memory for cacheinfo during the cacheinfo CPU hotplug callback if not done earlier. Moreover, before determining the validity of the last-level cache info, ensure that it has been allocated. Simply checking for non-zero cache_leaves() is not sufficient, as some architectures (e.g., Intel processors) have non-zero cache_leaves() before allocation. Dereferencing NULL cacheinfo can occur in update_per_cpu_data_slice_size(). This function iterates over all online CPUs. However, a CPU may have come online recently, but its cacheinfo may not have been allocated yet. While here, remove an unnecessary indentation in allocate_cache_info(). [ bp: Massage. ] Fixes: 6539cffa9495 ("cacheinfo: Add arch specific early level initializer") Signed-off-by: Ricardo Neri Signed-off-by: Borislav Petkov (AMD) Reviewed-by: Radu Rendec Reviewed-by: Nikolay Borisov Reviewed-by: Andreas Herrmann Reviewed-by: Sudeep Holla Cc: stable@vger.kernel.org # 6.3+ Link: https://lore.kernel.org/r/20241128002247.26726-2-ricardo.neri-calderon@linux.intel.com Signed-off-by: Greg Kroah-Hartman

drivers: base: cacheinfo: Update cpu_map_populated during CPU Hotplug

2023-05-31T19:36:47+00:00

Until commit 5c2712387d48 ("cacheinfo: Fix LLC is not exported through sysfs"), cacheinfo called populate_cache_leaves() for CPU coming online which let the arch specific functions handle (at least on x86) populating the shared_cpu_map. However, with the changes in the aforementioned commit, populate_cache_leaves() is not called when a CPU comes online as a result of hotplug since last_level_cache_is_valid() returns true as the cacheinfo data is not discarded. The CPU coming online is not present in shared_cpu_map, however, it will not be added since the cpu_cacheinfo->cpu_map_populated flag is set (it is set in populate_cache_leaves() when cacheinfo is first populated for x86) This can lead to inconsistencies in the shared_cpu_map when an offlined CPU comes online again. Example below depicts the inconsistency in the shared_cpu_list in cacheinfo when CPU8 is offlined and onlined again on a 3rd Generation EPYC processor: # for i in /sys/devices/system/cpu/cpu8/cache/index*/shared_cpu_list; do echo -n "$i: "; cat $i; done /sys/devices/system/cpu/cpu8/cache/index0/shared_cpu_list: 8,136 /sys/devices/system/cpu/cpu8/cache/index1/shared_cpu_list: 8,136 /sys/devices/system/cpu/cpu8/cache/index2/shared_cpu_list: 8,136 /sys/devices/system/cpu/cpu8/cache/index3/shared_cpu_list: 8-15,136-143 # echo 0 > /sys/devices/system/cpu/cpu8/online # echo 1 > /sys/devices/system/cpu/cpu8/online # for i in /sys/devices/system/cpu/cpu8/cache/index*/shared_cpu_list; do echo -n "$i: "; cat $i; done /sys/devices/system/cpu/cpu8/cache/index0/shared_cpu_list: 8 /sys/devices/system/cpu/cpu8/cache/index1/shared_cpu_list: 8 /sys/devices/system/cpu/cpu8/cache/index2/shared_cpu_list: 8 /sys/devices/system/cpu/cpu8/cache/index3/shared_cpu_list: 8 # cat /sys/devices/system/cpu/cpu136/cache/index0/shared_cpu_list 136 # cat /sys/devices/system/cpu/cpu136/cache/index3/shared_cpu_list 9-15,136-143 Clear the flag when the CPU is removed from shared_cpu_map when cache_shared_cpu_map_remove() is called during CPU hotplug. This will allow cache_shared_cpu_map_setup() to add the CPU coming back online in the shared_cpu_map. Set the flag again when the shared_cpu_map is setup. Following are results of performing the same test as described above with the changes: # for i in /sys/devices/system/cpu/cpu8/cache/index*/shared_cpu_list; do echo -n "$i: "; cat $i; done /sys/devices/system/cpu/cpu8/cache/index0/shared_cpu_list: 8,136 /sys/devices/system/cpu/cpu8/cache/index1/shared_cpu_list: 8,136 /sys/devices/system/cpu/cpu8/cache/index2/shared_cpu_list: 8,136 /sys/devices/system/cpu/cpu8/cache/index3/shared_cpu_list: 8-15,136-143 # echo 0 > /sys/devices/system/cpu/cpu8/online # echo 1 > /sys/devices/system/cpu/cpu8/online # for i in /sys/devices/system/cpu/cpu8/cache/index*/shared_cpu_list; do echo -n "$i: "; cat $i; done /sys/devices/system/cpu/cpu8/cache/index0/shared_cpu_list: 8,136 /sys/devices/system/cpu/cpu8/cache/index1/shared_cpu_list: 8,136 /sys/devices/system/cpu/cpu8/cache/index2/shared_cpu_list: 8,136 /sys/devices/system/cpu/cpu8/cache/index3/shared_cpu_list: 8-15,136-143 # cat /sys/devices/system/cpu/cpu136/cache/index0/shared_cpu_list 8,136 # cat /sys/devices/system/cpu/cpu136/cache/index3/shared_cpu_list 8-15,136-143 Fixes: 5c2712387d48 ("cacheinfo: Fix LLC is not exported through sysfs") Signed-off-by: K Prateek Nayak Reviewed-by: Yicong Yang Reviewed-by: Sudeep Holla Link: https://lore.kernel.org/r/20230508084115.1157-3-kprateek.nayak@amd.com Signed-off-by: Greg Kroah-Hartman

drivers: base: cacheinfo: Fix shared_cpu_map changes in event of CPU hotplug

2023-05-31T19:36:46+00:00

While building the shared_cpu_map, check if the cache level and cache type matches. On certain systems that build the cache topology based on the instance ID, there are cases where the same ID may repeat across multiple cache levels, leading inaccurate topology. In event of CPU offlining, the cache_shared_cpu_map_remove() does not consider if IDs at same level are being compared. As a result, when same IDs repeat across different cache levels, the CPU going offline is not removed from all the shared_cpu_map. Below is the output of cache topology of CPU8 and it's SMT sibling after CPU8 is offlined on a dual socket 3rd Generation AMD EPYC processor (2 x 64C/128T) running kernel release v6.3: # for i in /sys/devices/system/cpu/cpu8/cache/index*/shared_cpu_list; do echo -n "$i: "; cat $i; done /sys/devices/system/cpu/cpu8/cache/index0/shared_cpu_list: 8,136 /sys/devices/system/cpu/cpu8/cache/index1/shared_cpu_list: 8,136 /sys/devices/system/cpu/cpu8/cache/index2/shared_cpu_list: 8,136 /sys/devices/system/cpu/cpu8/cache/index3/shared_cpu_list: 8-15,136-143 # echo 0 > /sys/devices/system/cpu/cpu8/online # for i in /sys/devices/system/cpu/cpu136/cache/index*/shared_cpu_list; do echo -n "$i: "; cat $i; done /sys/devices/system/cpu/cpu136/cache/index0/shared_cpu_list: 136 /sys/devices/system/cpu/cpu136/cache/index1/shared_cpu_list: 8,136 /sys/devices/system/cpu/cpu136/cache/index2/shared_cpu_list: 8,136 /sys/devices/system/cpu/cpu136/cache/index3/shared_cpu_list: 9-15,136-143 CPU8 is removed from index0 (L1i) but remains in the shared_cpu_list of index1 (L1d) and index2 (L2). Since L1i, L1d, and L2 are shared by the SMT siblings, and they have the same cache instance ID, CPU 2 is only removed from the first index with matching ID which is index1 (L1i) in this case. With this fix, the results are as expected when performing the same experiment on the same system: # for i in /sys/devices/system/cpu/cpu8/cache/index*/shared_cpu_list; do echo -n "$i: "; cat $i; done /sys/devices/system/cpu/cpu8/cache/index0/shared_cpu_list: 8,136 /sys/devices/system/cpu/cpu8/cache/index1/shared_cpu_list: 8,136 /sys/devices/system/cpu/cpu8/cache/index2/shared_cpu_list: 8,136 /sys/devices/system/cpu/cpu8/cache/index3/shared_cpu_list: 8-15,136-143 # echo 0 > /sys/devices/system/cpu/cpu8/online # for i in /sys/devices/system/cpu/cpu136/cache/index*/shared_cpu_list; do echo -n "$i: "; cat $i; done /sys/devices/system/cpu/cpu136/cache/index0/shared_cpu_list: 136 /sys/devices/system/cpu/cpu136/cache/index1/shared_cpu_list: 136 /sys/devices/system/cpu/cpu136/cache/index2/shared_cpu_list: 136 /sys/devices/system/cpu/cpu136/cache/index3/shared_cpu_list: 9-15,136-143 When rebuilding topology, the same problem appears as cache_shared_cpu_map_setup() implements a similar logic. Consider the same 3rd Generation EPYC processor: CPUs in Core 1, that share the L1 and L2 caches, have L1 and L2 instance ID as 1. For all the CPUs on the second chiplet, the L3 ID is also 1 leading to grouping on CPUs from Core 1 (1, 17) and the entire second chiplet (8-15, 24-31) as CPUs sharing one cache domain. This went undetected since x86 processors depended on arch specific populate_cache_leaves() method to repopulate the shared_cpus_map when CPU came back online until kernel release v6.3-rc5. Fixes: 198102c9103f ("cacheinfo: Fix shared_cpu_map to handle shared caches at different levels") Signed-off-by: K Prateek Nayak Reviewed-by: Sudeep Holla Link: https://lore.kernel.org/r/20230508084115.1157-2-kprateek.nayak@amd.com Signed-off-by: Greg Kroah-Hartman

Merge tag 'driver-core-6.4-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/driver-core

2023-04-27T18:53:57+00:00

Pull driver core updates from Greg KH: "Here is the large set of driver core changes for 6.4-rc1. Once again, a busy development cycle, with lots of changes happening in the driver core in the quest to be able to move "struct bus" and "struct class" into read-only memory, a task now complete with these changes. This will make the future rust interactions with the driver core more "provably correct" as well as providing more obvious lifetime rules for all busses and classes in the kernel. The changes required for this did touch many individual classes and busses as many callbacks were changed to take const * parameters instead. All of these changes have been submitted to the various subsystem maintainers, giving them plenty of time to review, and most of them actually did so. Other than those changes, included in here are a small set of other things: - kobject logging improvements - cacheinfo improvements and updates - obligatory fw_devlink updates and fixes - documentation updates - device property cleanups and const * changes - firwmare loader dependency fixes. All of these have been in linux-next for a while with no reported problems" * tag 'driver-core-6.4-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/driver-core: (120 commits) device property: make device_property functions take const device * driver core: update comments in device_rename() driver core: Don't require dynamic_debug for initcall_debug probe timing firmware_loader: rework crypto dependencies firmware_loader: Strip off \n from customized path zram: fix up permission for the hot_add sysfs file cacheinfo: Add use_arch[|_cache]_info field/function arch_topology: Remove early cacheinfo error message if -ENOENT cacheinfo: Check cache properties are present in DT cacheinfo: Check sib_leaf in cache_leaves_are_shared() cacheinfo: Allow early level detection when DT/ACPI info is missing/broken cacheinfo: Add arm64 early level initializer implementation cacheinfo: Add arch specific early level initializer tty: make tty_class a static const structure driver core: class: remove struct class_interface * from callbacks driver core: class: mark the struct class in struct class_interface constant driver core: class: make class_register() take a const * driver core: class: mark class_release() as taking a const * driver core: remove incorrect comment for device_create* MIPS: vpe-cmp: remove module owner pointer from struct class usage. ...

Merge tag 'devicetree-for-6.4-2' of git://git.kernel.org/pub/scm/linux/kernel/git/robh/linux

2023-04-27T17:09:05+00:00

Pull more devicetree updates from Rob Herring: - First part of DT header detangling dropping cpu.h from of_device.h and replacing some includes with forward declarations. A handful of drivers needed some adjustment to their includes as a result. - Refactor of_device.h to be used by bus drivers rather than various device drivers. This moves non-bus related functions out of of_device.h. The end goal is for of_platform.h and of_device.h to stop including each other. - Refactor open coded parsing of "ranges" in some bus drivers to use DT address parsing functions - Add some new address parsing functions of_property_read_reg(), of_range_count(), and of_range_to_resource() in preparation to convert more open coded parsing of DT addresses to use them. - Treewide clean-ups to use of_property_read_bool() and of_property_present() as appropriate. The ones here are the ones that didn't get picked up elsewhere. * tag 'devicetree-for-6.4-2' of git://git.kernel.org/pub/scm/linux/kernel/git/robh/linux: (34 commits) bus: tegra-gmi: Replace of_platform.h with explicit includes hte: Use of_property_present() for testing DT property presence w1: w1-gpio: Use of_property_read_bool() for boolean properties virt: fsl: Use of_property_present() for testing DT property presence soc: fsl: Use of_property_present() for testing DT property presence sbus: display7seg: Use of_property_read_bool() for boolean properties sparc: Use of_property_read_bool() for boolean properties sparc: Use of_property_present() for testing DT property presence bus: mvebu-mbus: Remove open coded "ranges" parsing of/address: Add of_property_read_reg() helper of/address: Add of_range_count() helper of/address: Add support for 3 address cell bus of/address: Add of_range_to_resource() helper of: unittest: Add bus address range parsing tests of: Drop cpu.h include from of_device.h OPP: Adjust includes to remove of_device.h irqchip: loongson-eiointc: Add explicit include for cpuhotplug.h cpuidle: Adjust includes to remove of_device.h cpufreq: sun50i: Add explicit include for cpu.h cpufreq: Adjust includes to remove of_device.h ...

Merge tag 'cacheinfo-updates-6.4' of git://git.kernel.org/pub/scm/linux/kernel/git/sudeep.holla/linux into driver-core-next

2023-04-19T13:09:40+00:00

Sudeep writes: cacheinfo and arch_topology updates for v6.4 The cache information can be extracted from either a Device Tree(DT), the PPTT ACPI table, or arch registers (clidr_el1 for arm64). When the DT is used but no cache properties are advertised, the current code doesn't correctly fallback to using arch information. The changes fixes the same and also assuse the that L1 data/instruction caches are private and L2/higher caches are shared when the cache information is missing in DT/ACPI and is derived form clidr_el1/arch registers. Currently the cacheinfo is built from the primary CPU prior to secondary CPUs boot, if the DT/ACPI description contains cache information. However, if not present, it still reverts to the old behavior, which allocates the cacheinfo memory on each secondary CPUs which causes RT kernels to triggers a "BUG: sleeping function called from invalid context". The changes here attempts to enable automatic detection for RT kernels when no DT/ACPI cache information is available, by pre-allocating cacheinfo memory on the primary CPU. * tag 'cacheinfo-updates-6.4' of git://git.kernel.org/pub/scm/linux/kernel/git/sudeep.holla/linux: cacheinfo: Add use_arch[|_cache]_info field/function arch_topology: Remove early cacheinfo error message if -ENOENT cacheinfo: Check cache properties are present in DT cacheinfo: Check sib_leaf in cache_leaves_are_shared() cacheinfo: Allow early level detection when DT/ACPI info is missing/broken cacheinfo: Add arm64 early level initializer implementation cacheinfo: Add arch specific early level initializer

cacheinfo: Add use_arch[|_cache]_info field/function

2023-04-14T09:13:39+00:00

The cache information can be extracted from either a Device Tree (DT), the PPTT ACPI table, or arch registers (clidr_el1 for arm64). The clidr_el1 register is used only if DT/ACPI information is not available. It does not states how caches are shared among CPUs. Add a use_arch_cache_info field/function to identify when the DT/ACPI doesn't provide cache information. Use this information to assume L1 caches are privates and L2 and higher are shared among all CPUs. Signed-off-by: Pierre Gondois Link: https://lore.kernel.org/r/20230414081453.244787-5-pierre.gondois@arm.com Signed-off-by: Sudeep Holla

cacheinfo: Check cache properties are present in DT

2023-04-14T09:13:38+00:00

If a Device Tree (DT) is used, the presence of cache properties is assumed. Not finding any is not considered. For arm64 platforms, cache information can be fetched from the clidr_el1 register. Checking whether cache information is available in the DT allows to switch to using clidr_el1. init_of_cache_level() \-of_count_cache_leaves() will assume there a 2 cache leaves (L1 data/instruction caches), which can be different from clidr_el1 information. cache_setup_of_node() tries to read cache properties in the DT. If there are none, this is considered a success. Knowing no information was available would allow to switch to using clidr_el1. Fixes: de0df442ee49 ("cacheinfo: Check 'cache-unified' property to count cache leaves") Reported-by: Alexandre Ghiti Link: https://lore.kernel.org/all/20230404-hatred-swimmer-6fecdf33b57a@spud/ Signed-off-by: Pierre Gondois Reviewed-by: Conor Dooley Link: https://lore.kernel.org/r/20230414081453.244787-3-pierre.gondois@arm.com Signed-off-by: Sudeep Holla

cacheinfo: Check sib_leaf in cache_leaves_are_shared()

2023-04-14T09:13:38+00:00

If there is no ACPI/DT information, it is assumed that L1 caches are private and L2 (and higher) caches are shared. A cache is 'shared' between two CPUs if it is accessible from these two CPUs. Each CPU owns a representation (i.e. has a dedicated cacheinfo struct) of the caches it has access to. cache_leaves_are_shared() tries to identify whether two representations are designating the same actual cache. In cache_leaves_are_shared(), if 'this_leaf' is a L2 cache (or higher) and 'sib_leaf' is a L1 cache, the caches are detected as shared as only this_leaf's cache level is checked. This is leads to setting sib_leaf as being shared with another CPU, which is incorrect as this is a L1 cache. Check 'sib_leaf->level'. Also update the comment as the function is called when populating 'shared_cpu_map'. Fixes: f16d1becf96f ("cacheinfo: Use cache identifiers to check if the caches are shared if available") Signed-off-by: Pierre Gondois Reviewed-by: Conor Dooley Link: https://lore.kernel.org/r/20230414081453.244787-2-pierre.gondois@arm.com Signed-off-by: Sudeep Holla

cacheinfo: Adjust includes to remove of_device.h

2023-04-13T22:46:34+00:00

Now that of_cpu_device_node_get() is defined in of.h, of_device.h is just implicitly including other includes, and is no longer needed. Update the includes to use of.h instead of of_device.h. Acked-by: Sudeep Holla Acked-by: Greg Kroah-Hartman Link: https://lore.kernel.org/r/20230329-dt-cpu-header-cleanups-v1-10-581e2605fe47@kernel.org Signed-off-by: Rob Herring