kernel/linux.git/tools/testing/selftests/resctrl/cmt_test.c, branch v7.2-rc2

selftests/resctrl: Reduce interference from L2 occupancy during cache occupancy test

2026-05-05T00:40:02+00:00

The CMT test creates a new control group that is also capable of monitoring and assigns the workload to it. The workload allocates a buffer that by default fills a portion of the L3 and keeps reading from the buffer, measuring the L3 occupancy at intervals. The test passes if the workload's L3 occupancy is within 15% of the buffer size. The CMT test does not take into account that some of the workload's data may land in L2/L1. Matching L3 occupancy to the size of the buffer while a portion of the buffer can be allocated into L2 is not accurate. Take the L2 cache into account to improve test accuracy: - Reduce the workload's L2 cache allocation to the minimum on systems that support L2 cache allocation. Do so with a new utility in preparation for all L3 cache allocation tests needing the same capability. - Increase the buffer size to accommodate data that may be allocated into the L2 cache. Use a buffer size double the L3 portion to keep using the L3 portion size as goal for L3 occupancy while taking into account that some of the data may be in L2. Running the CMT test on a sample system while introducing significant cache misses using "stress-ng --matrix-3d 0 --matrix-3d-zyx" shows significant improvement in L3 cache occupancy: Before: # Starting CMT test ... # Mounting resctrl to "/sys/fs/resctrl" # Cache size :335544320 # Writing benchmark parameters to resctrl FS # Write schema "L3:0=fffe0" to resctrl FS # Write schema "L3:0=1f" to resctrl FS # Benchmark PID: 7089 # Checking for pass/fail # Pass: Check cache miss rate within 15% # Percent diff=12 # Number of bits: 5 # Average LLC val: 73269248 # Cache span (bytes): 83886080 ok 1 CMT: test After: # Starting CMT test ... # Mounting resctrl to "/sys/fs/resctrl" # Cache size :335544320 # Writing benchmark parameters to resctrl FS # Write schema "L3:0=fffe0" to resctrl FS # Write schema "L3:0=1f" to resctrl FS # Write schema "L2:1=0x1" to resctrl FS # Benchmark PID: 7171 # Checking for pass/fail # Pass: Check cache miss rate within 15% # Percent diff=0 # Number of bits: 5 # Average LLC val: 83755008 # Cache span (bytes): 83886080 ok 1 CMT: test Link: https://lore.kernel.org/r/00445fa64c251b86b86023f87220ee1ad8561460.1775266384.git.reinette.chatre@intel.com Reported-by: Dave Martin Signed-off-by: Reinette Chatre Tested-by: Chen Yu Reviewed-by: Ilpo Järvinen Link: https://lore.kernel.org/lkml/aO+7MeSMV29VdbQs@e133380.arm.com/ Signed-off-by: Shuah Khan

selftests/resctrl: Improve accuracy of cache occupancy test

2026-05-05T00:40:02+00:00

Dave Martin reported inconsistent CMT test failures. In one experiment the first run of the CMT test failed because of too large (24%) difference between measured and achievable cache occupancy while the second run passed with an acceptable 4% difference. The CMT test is susceptible to interference from the rest of the system. This can be demonstrated with a utility like stress-ng by running the CMT test while introducing cache misses using: stress-ng --matrix-3d 0 --matrix-3d-zyx Below shows an example of the CMT test failing because of a significant difference between measured and achievable cache occupancy when run with interference: # Starting CMT test ... # Mounting resctrl to "/sys/fs/resctrl" # Cache size :335544320 # Writing benchmark parameters to resctrl FS # Benchmark PID: 7011 # Checking for pass/fail # Fail: Check cache miss rate within 15% # Percent diff=99 # Number of bits: 5 # Average LLC val: 235929 # Cache span (bytes): 83886080 not ok 1 CMT: test The CMT test creates a new control group that is also capable of monitoring and assigns the workload to it. The workload allocates a buffer that by default fills a portion of the L3 and keeps reading from the buffer, measuring the L3 occupancy at intervals. The test passes if the workload's L3 occupancy is within 15% of the buffer size. By not adjusting any capacity bitmasks the workload shares the cache with the rest of the system. Any other task that may be running could evict the workload's data from the cache causing it to have low cache occupancy. Reduce interference from the rest of the system by ensuring that the workload's control group uses the capacity bitmask found in the user parameters for L3 and that the rest of the system can only allocate into the inverse of the workload's L3 cache portion. Other tasks can thus no longer evict the workload's data from L3. With the above adjustments the CMT test is more consistent. Repeating the CMT test while generating interference with stress-ng on a sample system after applying the fixes show significant improvement in test accuracy: # Starting CMT test ... # Mounting resctrl to "/sys/fs/resctrl" # Cache size :335544320 # Writing benchmark parameters to resctrl FS # Write schema "L3:0=fffe0" to resctrl FS # Write schema "L3:0=1f" to resctrl FS # Benchmark PID: 7089 # Checking for pass/fail # Pass: Check cache miss rate within 15% # Percent diff=12 # Number of bits: 5 # Average LLC val: 73269248 # Cache span (bytes): 83886080 ok 1 CMT: test Link: https://lore.kernel.org/r/b160592179f88069cdc679563e152007998a0d76.1775266384.git.reinette.chatre@intel.com Reported-by: Dave Martin Signed-off-by: Reinette Chatre Tested-by: Chen Yu Reviewed-by: Ilpo Järvinen Link: https://lore.kernel.org/lkml/aO+7MeSMV29VdbQs@e133380.arm.com/ Signed-off-by: Shuah Khan

selftests/resctrl: Discover SNC kernel support and adjust messages

2025-01-15T00:06:32+00:00

Resctrl selftest prints a message on test failure that Sub-Numa Clustering (SNC) could be enabled and points the user to check their BIOS settings. No actual check is performed before printing that message so it is not very accurate in pinpointing a problem. When there is SNC support for kernel's resctrl subsystem and SNC is enabled then sub node files are created for each node in the resctrlfs. The sub node files exist in each regular node's L3 monitoring directory. The reliable path to check for existence of sub node files is /sys/fs/resctrl/mon_data/mon_L3_00/mon_sub_L3_00. Add helper that checks for mon_sub_L3_00 existence. Correct old messages to account for kernel support of SNC in resctrl. Signed-off-by: Maciej Wieczor-Retman Reviewed-by: Reinette Chatre Signed-off-by: Shuah Khan

selftests/resctrl: Keep results from first test run

2024-11-05T00:02:03+00:00

The resctrl selftests drop the results from every first test run to avoid (per comment) "inaccurate due to monitoring setup transition phase" data. Previously inaccurate data resulted from workloads needing some time to "settle" and also the measurements themselves to account for earlier measurements to measure across needed timeframe. commit da50de0a92f3 ("selftests/resctrl: Calculate resctrl FS derived mem bw over sleep(1) only") ensured that measurements accurately measure just the time frame of interest. The default "fill_buf" benchmark since separated the buffer prepare phase from the benchmark run phase reducing the need for the tests themselves to accommodate the benchmark's "settle" time. With these enhancements there are no remaining portions needing to "settle" and the first test run can contribute to measurements. Signed-off-by: Reinette Chatre Reviewed-by: Ilpo Järvinen Signed-off-by: Shuah Khan

selftests/resctrl: Make benchmark parameter passing robust

2024-11-05T00:02:03+00:00

The benchmark used during the CMT, MBM, and MBA tests can be provided by the user via (-b) parameter, if not provided the default "fill_buf" benchmark is used. The user is additionally able to override any of the "fill_buf" default parameters when running the tests with "-b fill_buf ". The "fill_buf" parameters are managed as an array of strings. Using an array of strings is complex because it requires transformations to/from strings at every producer and consumer. This is made worse for the individual tests where the default benchmark parameters values may not be appropriate and additional data wrangling is required. For example, the CMT test duplicates the entire array of strings in order to replace one of the parameters. More issues appear when combining the usage of an array of strings with the use case of user overriding default parameters by specifying "-b fill_buf ". This use case is fragile with opportunities to trigger a SIGSEGV because of opportunities for NULL pointers to exist in the array of strings. For example, by running below (thus by specifying "fill_buf" should be used but all parameters are NULL): $ sudo resctrl_tests -t mbm -b fill_buf Replace the "array of strings" parameters used for "fill_buf" with new struct fill_buf_param that contains the "fill_buf" parameters that can be used directly without transformations to/from strings. Two instances of struct fill_buf_param may exist at any point in time: * If the user provides new parameters to "fill_buf", the user parameter structure (struct user_params) will point to a fully initialized and immutable struct fill_buf_param containing the user provided parameters. * If "fill_buf" is the benchmark that should be used by a test, then the test parameter structure (struct resctrl_val_param) will point to a fully initialized struct fill_buf_param. The latter may contain (a) the user provided parameters verbatim, (b) user provided parameters adjusted to be appropriate for the test, or (c) the default parameters for "fill_buf" that is appropriate for the test if the user did not provide "fill_buf" parameters nor an alternate benchmark. The existing behavior of CMT test is to use test defined value for the buffer size even if the user provides another value via command line. This behavior is maintained since the test requires that the buffer size matches the size of the cache allocated, and the amount of cache allocated can instead be changed by the user with the "-n" command line parameter. Signed-off-by: Reinette Chatre Reviewed-by: Ilpo Järvinen Signed-off-by: Shuah Khan

selftests/resctrl: Remove test name comparing from write_bm_pid_to_resctrl()

2024-07-11T17:23:54+00:00

write_bm_pid_to_resctrl() uses resctrl_val to check test name which is not a good interface generic resctrl FS functions should provide. Tests define mongrp when needed. Remove the test name check in write_bm_pid_to_resctrl() to only rely on the mongrp parameter being non-NULL. Remove write_bm_pid_to_resctrl() resctrl_val parameter and resctrl_val member from the struct resctrl_val_param that are not used anymore. Similarly, remove the test name constants that are no longer used. Signed-off-by: Ilpo Järvinen Tested-by: Babu Moger Reviewed-by: Reinette Chatre Signed-off-by: Shuah Khan

selftests/resctrl: Remove mongrp from CMT test

2024-07-11T17:23:54+00:00

The CMT selftest instantiates a monitor group to read LLC occupancy. Since the test also creates a control group, it is unnecessary to create another one for monitoring because control groups already provide monitoring too. Remove the unnecessary monitor group from the CMT selftest. Suggested-by: Reinette Chatre Signed-off-by: Ilpo Järvinen Tested-by: Babu Moger Reviewed-by: Reinette Chatre Signed-off-by: Shuah Khan

selftests/resctrl: Add ->init() callback into resctrl_val_param

2024-07-11T17:23:54+00:00

The struct resctrl_val_param is there to customize behavior inside resctrl_val() which is currently not used to full extent and there are number of strcmp()s for test name in resctrl_val done by resctrl_val(). Create ->init() hook into the struct resctrl_val_param to cleanly do per test initialization. Remove also unused branches to setup paths and the related #defines for CMT test. While touching kerneldoc, make the adjacent line consistent with the newly added form (callback vs call back). Signed-off-by: Ilpo Järvinen Tested-by: Babu Moger Reviewed-by: Reinette Chatre Signed-off-by: Shuah Khan

selftests/resctrl: Add ->measure() callback to resctrl_val_param

2024-07-11T17:23:54+00:00

The measurement done in resctrl_val() varies depending on test type. The decision for how to measure is decided based on the string compare to test name which is quite inflexible. Add ->measure() callback into the struct resctrl_val_param to allow each test to provide necessary code as a function which simplifies what resctrl_val() has to do. Signed-off-by: Ilpo Järvinen Tested-by: Babu Moger Reviewed-by: Reinette Chatre Signed-off-by: Shuah Khan

selftests/resctrl: fix clang build warnings related to abs(), labs() calls

2024-05-08T22:53:19+00:00

When building with clang, via: make LLVM=1 -C tools/testing/selftests ...two types of warnings occur: warning: absolute value function 'abs' given an argument of type 'long' but has parameter of type 'int' which may cause truncation of value warning: taking the absolute value of unsigned type 'unsigned long' has no effect Fix these by: a) using labs() in place of abs(), when long integers are involved, and b) Change to use signed integer data types, in places where subtraction is used (and could end up with negative values). c) Remove a duplicate abs() call in cmt_test.c. Cc: Ilpo Järvinen Reviewed-by: Reinette Chatre Signed-off-by: John Hubbard Signed-off-by: Shuah Khan