kernel/linux.git/lib/kunit/test.c, branch v6.6.132

kunit: run test suites only after module initialization completes

2024-02-05T20:14:34+00:00

[ Upstream commit a1af6a2bfa0cb46d70b7df5352993e750da6c79b ] Commit 2810c1e99867 ("kunit: Fix wild-memory-access bug in kunit_free_suite_set()") fixed a wild-memory-access bug that could have happened during the loading phase of test suites built and executed as loadable modules. However, it also introduced a problematic side effect that causes test suites modules to crash when they attempt to register fake devices. When a module is loaded, it traverses the MODULE_STATE_UNFORMED and MODULE_STATE_COMING states before reaching the normal operating state MODULE_STATE_LIVE. Finally, when the module is removed, it moves to MODULE_STATE_GOING before being released. However, if the loading function load_module() fails between complete_formation() and do_init_module(), the module goes directly from MODULE_STATE_COMING to MODULE_STATE_GOING without passing through MODULE_STATE_LIVE. This behavior was causing kunit_module_exit() to be called without having first executed kunit_module_init(). Since kunit_module_exit() is responsible for freeing the memory allocated by kunit_module_init() through kunit_filter_suites(), this behavior was resulting in a wild-memory-access bug. Commit 2810c1e99867 ("kunit: Fix wild-memory-access bug in kunit_free_suite_set()") fixed this issue by running the tests when the module is still in MODULE_STATE_COMING. However, modules in that state are not fully initialized, lacking sysfs kobjects. Therefore, if a test module attempts to register a fake device, it will inevitably crash. This patch proposes a different approach to fix the original wild-memory-access bug while restoring the normal module execution flow by making kunit_module_exit() able to detect if kunit_module_init() has previously initialized the tests suite set. In this way, test modules can once again register fake devices without crashing. This behavior is achieved by checking whether mod->kunit_suites is a virtual or direct mapping address. If it is a virtual address, then kunit_module_init() has allocated the suite_set in kunit_filter_suites() using kmalloc_array(). On the contrary, if mod->kunit_suites is still pointing to the original address that was set when looking up the .kunit_test_suites section of the module, then the loading phase has failed and there's no memory to be freed. v4: - rebased on 6.8 - noted that kunit_filter_suites() must return a virtual address v3: - add a comment to clarify why the start address is checked v2: - add include Fixes: 2810c1e99867 ("kunit: Fix wild-memory-access bug in kunit_free_suite_set()") Reviewed-by: David Gow Tested-by: Rae Moar Tested-by: Richard Fitzgerald Reviewed-by: Javier Martinez Canillas Signed-off-by: Marco Pagani Signed-off-by: Shuah Khan Signed-off-by: Sasha Levin

kunit: Reset suite counter right before running tests

2024-01-20T10:51:37+00:00

[ Upstream commit 2e3c94aed51eabbe9c1c0ee515371ea5441c2fa7 ] Today we reset the suite counter as part of the suite cleanup, called from the module exit callback, but it might not work that well as one can try to collect results without unloading a previous test (either unintentionally or due to dependencies). For easy reproduction try to load the kunit-test.ko and then collect and parse results from the kunit-example-test.ko load. Parser will complain about mismatch of expected test number: [ ] KTAP version 1 [ ] 1..1 [ ] # example: initializing suite [ ] KTAP version 1 [ ] # Subtest: example .. [ ] # example: pass:5 fail:0 skip:4 total:9 [ ] # Totals: pass:6 fail:0 skip:6 total:12 [ ] ok 7 example [ ] [ERROR] Test: example: Expected test number 1 but found 7 [ ] ===================== [PASSED] example ===================== [ ] ============================================================ [ ] Testing complete. Ran 12 tests: passed: 6, skipped: 6, errors: 1 Since we are now printing suite test plan on every module load, right before running suite tests, we should make sure that suite counter will also start from 1. Easiest solution seems to be move counter reset to the __kunit_test_suites_init() function. Signed-off-by: Michal Wajdeczko Cc: David Gow Cc: Rae Moar Reviewed-by: David Gow Signed-off-by: Shuah Khan Signed-off-by: Sasha Levin

kunit: Warn if tests are slow

2024-01-20T10:51:37+00:00

[ Upstream commit f8f2847f739dc899d0e563eac01299dadefa64ff ] Kunit recently gained support to setup attributes, the first one being the speed of a given test, then allowing to filter out slow tests. A slow test is defined in the documentation as taking more than one second. There's an another speed attribute called "super slow" but whose definition is less clear. Add support to the test runner to check the test execution time, and report tests that should be marked as slow but aren't. Signed-off-by: Maxime Ripard Reviewed-by: David Gow Signed-off-by: Shuah Khan Signed-off-by: Sasha Levin

kunit: Fix wild-memory-access bug in kunit_free_suite_set()

2023-09-05T18:29:50+00:00

Inject fault while probing kunit-example-test.ko, if kstrdup() fails in mod_sysfs_setup() in load_module(), the mod->state will switch from MODULE_STATE_COMING to MODULE_STATE_GOING instead of from MODULE_STATE_LIVE to MODULE_STATE_GOING, so only kunit_module_exit() will be called without kunit_module_init(), and the mod->kunit_suites is no set correctly and the free in kunit_free_suite_set() will cause below wild-memory-access bug. The mod->state state machine when load_module() succeeds: MODULE_STATE_UNFORMED ---> MODULE_STATE_COMING ---> MODULE_STATE_LIVE ^ | | | delete_module +---------------- MODULE_STATE_GOING <---------+ The mod->state state machine when load_module() fails at mod_sysfs_setup(): MODULE_STATE_UNFORMED ---> MODULE_STATE_COMING ---> MODULE_STATE_GOING ^ | | | +-----------------------------------------------+ Call kunit_module_init() at MODULE_STATE_COMING state to fix the issue because MODULE_STATE_LIVE is transformed from it. Unable to handle kernel paging request at virtual address ffffff341e942a88 KASAN: maybe wild-memory-access in range [0x0003f9a0f4a15440-0x0003f9a0f4a15447] Mem abort info: ESR = 0x0000000096000004 EC = 0x25: DABT (current EL), IL = 32 bits SET = 0, FnV = 0 EA = 0, S1PTW = 0 FSC = 0x04: level 0 translation fault Data abort info: ISV = 0, ISS = 0x00000004, ISS2 = 0x00000000 CM = 0, WnR = 0, TnD = 0, TagAccess = 0 GCS = 0, Overlay = 0, DirtyBit = 0, Xs = 0 swapper pgtable: 4k pages, 48-bit VAs, pgdp=00000000441ea000 [ffffff341e942a88] pgd=0000000000000000, p4d=0000000000000000 Internal error: Oops: 0000000096000004 [#1] PREEMPT SMP Modules linked in: kunit_example_test(-) cfg80211 rfkill 8021q garp mrp stp llc ipv6 [last unloaded: kunit_example_test] CPU: 3 PID: 2035 Comm: modprobe Tainted: G W N 6.5.0-next-20230828+ #136 Hardware name: linux,dummy-virt (DT) pstate: a0000005 (NzCv daif -PAN -UAO -TCO -DIT -SSBS BTYPE=--) pc : kfree+0x2c/0x70 lr : kunit_free_suite_set+0xcc/0x13c sp : ffff8000829b75b0 x29: ffff8000829b75b0 x28: ffff8000829b7b90 x27: 0000000000000000 x26: dfff800000000000 x25: ffffcd07c82a7280 x24: ffffcd07a50ab300 x23: ffffcd07a50ab2e8 x22: 1ffff00010536ec0 x21: dfff800000000000 x20: ffffcd07a50ab2f0 x19: ffffcd07a50ab2f0 x18: 0000000000000000 x17: 0000000000000000 x16: 0000000000000000 x15: ffffcd07c24b6764 x14: ffffcd07c24b63c0 x13: ffffcd07c4cebb94 x12: ffff700010536ec7 x11: 1ffff00010536ec6 x10: ffff700010536ec6 x9 : dfff800000000000 x8 : 00008fffefac913a x7 : 0000000041b58ab3 x6 : 0000000000000000 x5 : 1ffff00010536ec5 x4 : ffff8000829b7628 x3 : dfff800000000000 x2 : ffffff341e942a80 x1 : ffffcd07a50aa000 x0 : fffffc0000000000 Call trace: kfree+0x2c/0x70 kunit_free_suite_set+0xcc/0x13c kunit_module_notify+0xd8/0x360 blocking_notifier_call_chain+0xc4/0x128 load_module+0x382c/0x44a4 init_module_from_file+0xd4/0x128 idempotent_init_module+0x2c8/0x524 __arm64_sys_finit_module+0xac/0x100 invoke_syscall+0x6c/0x258 el0_svc_common.constprop.0+0x160/0x22c do_el0_svc+0x44/0x5c el0_svc+0x38/0x78 el0t_64_sync_handler+0x13c/0x158 el0t_64_sync+0x190/0x194 Code: aa0003e1 b25657e0 d34cfc42 8b021802 (f9400440) ---[ end trace 0000000000000000 ]--- Kernel panic - not syncing: Oops: Fatal exception SMP: stopping secondary CPUs Kernel Offset: 0x4d0742200000 from 0xffff800080000000 PHYS_OFFSET: 0xffffee43c0000000 CPU features: 0x88000203,3c020000,1000421b Memory Limit: none Rebooting in 1 seconds.. Fixes: 3d6e44623841 ("kunit: unify module and builtin suite definitions") Signed-off-by: Jinjie Ruan Reviewed-by: Rae Moar Reviewed-by: David Gow Signed-off-by: Shuah Khan

kunit: Allow kunit test modules to use test filtering

2023-08-08T19:46:18+00:00

External tools, e.g., Intel GPU tools (IGT), support execution of individual selftests provided by kernel modules. That could be also applicable to kunit test modules if they provided test filtering. But test filtering is now possible only when kunit code is built into the kernel. Moreover, a filter can be specified only at boot time, then reboot is required each time a different filter is needed. Build the test filtering code also when kunit is configured as a module, expose test filtering functions to other kunit source files, and use them in kunit module notifier callback functions. Userspace can then reload the kunit module with a value of the filter_glob parameter tuned to a specific kunit test module every time it wants to limit the scope of tests executed on that module load. Make the kunit.filter* parameters visible in sysfs for user convenience. v5: Refresh on tpp of attributes filtering fix v4: Refresh on top of newly applied attributes patches and changes introdced by new versions of other patches submitted in series with this one. v3: Fix CONFIG_GLOB, required by filtering functions, not selected when building as a module (lkp@intel.com). v2: Fix new name of a structure moved to kunit namespace not updated across all uses (lkp@intel.com). Signed-off-by: Janusz Krzysztofik Reviewed-by: David Gow Signed-off-by: Shuah Khan

kunit: Make 'list' action available to kunit test modules

2023-08-08T19:46:13+00:00

Results from kunit tests reported via dmesg may be interleaved with other kernel messages. When parsing dmesg for modular kunit results in real time, external tools, e.g., Intel GPU tools (IGT), may want to insert their own test name markers into dmesg at the start of each test, before any kernel message related to that test appears there, so existing upper level test result parsers have no doubt which test to blame for a specific kernel message. Unfortunately, kunit reports names of tests only at their completion (with the exeption of a not standarized "# Subtest: " header above a test plan of each test suite or parametrized test). External tools could be able to insert their own "start of the test" markers with test names included if they new those names in advance. Test names could be learned from a list if provided by a kunit test module. There exists a feature of listing kunit tests without actually executing them, but it is now limited to configurations with the kunit module built in and covers only built-in tests, already available at boot time. Moreover, switching from list to normal mode requires reboot. If that feature was also available when kunit is built as a module, userspace could load the module with action=list parameter, load some kunit test modules they are interested in and learn about the list of tests provided by those modules, then unload them, reload the kunit module in normal mode and execute the tests with their lists already known. Extend kunit module notifier initialization callback with a processing path for only listing the tests provided by a module if the kunit action parameter is set to "list" or "list_attr". For user convenience, make the kunit.action parameter visible in sysfs. v2: Don't use a different format, use kunit_exec_list_tests() (Rae), - refresh on top of new attributes patches, handle newly introduced kunit.action=list_attr case (Rae). Signed-off-by: Janusz Krzysztofik Cc: Rae Moar Reviewed-by: David Gow Signed-off-by: Shuah Khan

kunit: Report the count of test suites in a module

2023-08-08T19:46:05+00:00

According to KTAP specification[1], results should always start from a header that provides a TAP protocol version, followed by a test plan with a count of items to be executed. That pattern should be followed at each nesting level. In the current implementation of the top-most, i.e., test suite level, those rules apply only for test suites built into the kernel, executed and reported on boot. Results submitted to dmesg from kunit test modules loaded later are missing those top-level headers. As a consequence, if a kunit test module provides more than one test suite then, without the top level test plan, external tools that are parsing dmesg for kunit test output are not able to tell how many test suites should be expected and whether to continue parsing after complete output from the first test suite is collected. Submit the top-level headers also from the kunit test module notifier initialization callback. v3: Fix new name of a structure moved to kunit namespace not updated in executor_test functions (lkp@intel.com). v2: Use kunit_exec_run_tests() (Mauro, Rae), but prevent it from emitting the headers when called on load of non-test modules. [1] https://docs.kernel.org/dev-tools/ktap.html# Signed-off-by: Janusz Krzysztofik Cc: Mauro Carvalho Chehab Cc: Rae Moar Reviewed-by: Rae Moar Reviewed-by: David Gow Signed-off-by: Shuah Khan

kunit: Add ability to filter attributes

2023-07-26T19:29:15+00:00

Add filtering of test attributes. Users can filter tests using the module_param called "filter". Filters are imputed in the format: Example: kunit.filter="speed>slow" Operations include: >, <, >=, <=, !=, and =. These operations will act the same for attributes of the same type but may not between types. Note multiple filters can be inputted by separating them with a comma. Example: kunit.filter="speed=slow, module!=example" Since both suites and test cases can have attributes, there may be conflicts. The process of filtering follows these rules: - Filtering always operates at a per-test level. - If a test has an attribute set, then the test's value is filtered on. - Otherwise, the value falls back to the suite's value. - If neither are set, the attribute has a global "default" value, which is used. Filtered tests will not be run or show in output. The tests can instead be skipped using the configurable option "kunit.filter_action=skip". Note the default settings for running tests remains unfiltered. Finally, add "filter" methods for the speed and module attributes to parse and compare attribute values. Note this filtering functionality will be added to kunit.py in the next patch. Reviewed-by: David Gow Signed-off-by: Rae Moar Signed-off-by: Shuah Khan

kunit: Add test attributes API structure

2023-07-26T19:28:57+00:00

Add the basic structure of the test attribute API to KUnit, which can be used to save and access test associated data. Add attributes.c and attributes.h to hold associated structs and functions for the API. Create a struct that holds a variety of associated helper functions for each test attribute. These helper functions will be used to get the attribute value, convert the value to a string, and filter based on the value. This struct is flexible by design to allow for attributes of numerous types and contexts. Add a method to print test attributes in the format of "# [.]: ". Example for a suite: "# speed: slow" Example for a test case: "# test_case.speed: very_slow" Use this method to report attributes in the KTAP output (KTAP spec: https://docs.kernel.org/dev-tools/ktap.html) and _list_tests output when kernel's new kunit.action=list_attr option is used. Note this is derivative of the kunit.action=list option. In test.h, add fields and associated helper functions to test cases and suites to hold user-inputted test attributes. Reviewed-by: David Gow Signed-off-by: Rae Moar Signed-off-by: Shuah Khan

kunit: Move kunit_abort() call out of kunit_do_failed_assertion()

2023-06-01T19:04:46+00:00

KUnit aborts the current thread when an assertion fails. Currently, this is done conditionally as part of the kunit_do_failed_assertion() function, but this hides the kunit_abort() call from the compiler (particularly if it's in another module). This, in turn, can lead to both suboptimal code generation (the compiler can't know if kunit_do_failed_assertion() will return), and to static analysis tools like smatch giving false positives. Moving the kunit_abort() call into the macro should give the compiler and tools a better chance at understanding what's going on. Doing so requires exporting kunit_abort(), though it's recommended to continue to use assertions in lieu of aborting directly. In addition, kunit_abort() and kunit_do_failed_assertion() are renamed to make it clear they they're intended for internal KUnit use, to: __kunit_do_failed_assertion() and __kunit_abort() Suggested-by: Dan Carpenter Signed-off-by: David Gow Reviewed-by: Miguel Ojeda Reviewed-by: Daniel Latypov Signed-off-by: Shuah Khan