summaryrefslogtreecommitdiff
path: root/lib/kunit
AgeCommit message (Collapse)AuthorFilesLines
2023-08-21kunit: fix struct kunit_attr headerRae Moar1-0/+2
Add parameter descriptions to struct kunit_attr header for the parameters attr_default and print. Fixes: 39e92cb1e4a1 ("kunit: Add test attributes API structure") Reported-by: kernel test robot <lkp@intel.com> Closes: https://lore.kernel.org/oe-kbuild-all/202308180127.VD7YRPGa-lkp@intel.com/ Signed-off-by: Rae Moar <rmoar@google.com> Reviewed-by: David Gow <davidgow@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2023-08-08kunit: Allow kunit test modules to use test filteringJanusz Krzysztofik3-27/+55
External tools, e.g., Intel GPU tools (IGT), support execution of individual selftests provided by kernel modules. That could be also applicable to kunit test modules if they provided test filtering. But test filtering is now possible only when kunit code is built into the kernel. Moreover, a filter can be specified only at boot time, then reboot is required each time a different filter is needed. Build the test filtering code also when kunit is configured as a module, expose test filtering functions to other kunit source files, and use them in kunit module notifier callback functions. Userspace can then reload the kunit module with a value of the filter_glob parameter tuned to a specific kunit test module every time it wants to limit the scope of tests executed on that module load. Make the kunit.filter* parameters visible in sysfs for user convenience. v5: Refresh on tpp of attributes filtering fix v4: Refresh on top of newly applied attributes patches and changes introdced by new versions of other patches submitted in series with this one. v3: Fix CONFIG_GLOB, required by filtering functions, not selected when building as a module (lkp@intel.com). v2: Fix new name of a structure moved to kunit namespace not updated across all uses (lkp@intel.com). Signed-off-by: Janusz Krzysztofik <janusz.krzysztofik@linux.intel.com> Reviewed-by: David Gow <davidgow@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2023-08-08kunit: Make 'list' action available to kunit test modulesJanusz Krzysztofik2-14/+32
Results from kunit tests reported via dmesg may be interleaved with other kernel messages. When parsing dmesg for modular kunit results in real time, external tools, e.g., Intel GPU tools (IGT), may want to insert their own test name markers into dmesg at the start of each test, before any kernel message related to that test appears there, so existing upper level test result parsers have no doubt which test to blame for a specific kernel message. Unfortunately, kunit reports names of tests only at their completion (with the exeption of a not standarized "# Subtest: <name>" header above a test plan of each test suite or parametrized test). External tools could be able to insert their own "start of the test" markers with test names included if they new those names in advance. Test names could be learned from a list if provided by a kunit test module. There exists a feature of listing kunit tests without actually executing them, but it is now limited to configurations with the kunit module built in and covers only built-in tests, already available at boot time. Moreover, switching from list to normal mode requires reboot. If that feature was also available when kunit is built as a module, userspace could load the module with action=list parameter, load some kunit test modules they are interested in and learn about the list of tests provided by those modules, then unload them, reload the kunit module in normal mode and execute the tests with their lists already known. Extend kunit module notifier initialization callback with a processing path for only listing the tests provided by a module if the kunit action parameter is set to "list" or "list_attr". For user convenience, make the kunit.action parameter visible in sysfs. v2: Don't use a different format, use kunit_exec_list_tests() (Rae), - refresh on top of new attributes patches, handle newly introduced kunit.action=list_attr case (Rae). Signed-off-by: Janusz Krzysztofik <janusz.krzysztofik@linux.intel.com> Cc: Rae Moar <rmoar@google.com> Reviewed-by: David Gow <davidgow@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2023-08-08kunit: Report the count of test suites in a moduleJanusz Krzysztofik3-32/+52
According to KTAP specification[1], results should always start from a header that provides a TAP protocol version, followed by a test plan with a count of items to be executed. That pattern should be followed at each nesting level. In the current implementation of the top-most, i.e., test suite level, those rules apply only for test suites built into the kernel, executed and reported on boot. Results submitted to dmesg from kunit test modules loaded later are missing those top-level headers. As a consequence, if a kunit test module provides more than one test suite then, without the top level test plan, external tools that are parsing dmesg for kunit test output are not able to tell how many test suites should be expected and whether to continue parsing after complete output from the first test suite is collected. Submit the top-level headers also from the kunit test module notifier initialization callback. v3: Fix new name of a structure moved to kunit namespace not updated in executor_test functions (lkp@intel.com). v2: Use kunit_exec_run_tests() (Mauro, Rae), but prevent it from emitting the headers when called on load of non-test modules. [1] https://docs.kernel.org/dev-tools/ktap.html# Signed-off-by: Janusz Krzysztofik <janusz.krzysztofik@linux.intel.com> Cc: Mauro Carvalho Chehab <mchehab@kernel.org> Cc: Rae Moar <rmoar@google.com> Reviewed-by: Rae Moar <rmoar@google.com> Reviewed-by: David Gow <davidgow@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2023-08-04kunit: fix uninitialized variables bug in attributes filteringRae Moar3-31/+29
Fix smatch warnings regarding uninitialized variables in the filtering patch of the new KUnit Attributes feature. Fixes: 529534e8cba3 ("kunit: Add ability to filter attributes") Reported-by: kernel test robot <lkp@intel.com> Reported-by: Dan Carpenter <dan.carpenter@linaro.org> Closes: https://lore.kernel.org/r/202307270610.s0w4NKEn-lkp@intel.com/ Signed-off-by: Rae Moar <rmoar@google.com> Reviewed-by: David Gow <davidgow@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2023-08-04kunit: fix possible memory leak in kunit_filter_suites()Ruan Jinjie1-5/+12
Inject fault while probing drm_kunit_helpers.ko, if one of kunit_next_attr_filter(), kunit_filter_glob_tests() and kunit_filter_attr_tests() fails, parsed_filters, parsed_glob.suite_glob/test_glob alloced in kunit_parse_glob_filter() is leaked. And the filtered_suite->test_cases alloced in kunit_filter_glob_tests() or kunit_filter_attr_tests() may also be leaked. unreferenced object 0xff110001067e4800 (size 1024): comm "kunit_try_catch", pid 96, jiffies 4294671796 (age 763.547s) hex dump (first 32 bytes): 73 75 69 74 65 32 00 00 00 00 00 00 00 00 00 00 suite2.......... 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ backtrace: [<00000000116e8eba>] __kmalloc_node_track_caller+0x4e/0x140 [<00000000e2f9cce9>] kmemdup+0x2c/0x60 [<000000002a36710b>] kunit_filter_suites+0x3e4/0xa50 [<0000000045779fb9>] filter_suites_test+0x1b7/0x440 [<00000000cd1104a7>] kunit_try_run_case+0x119/0x270 [<00000000c654c917>] kunit_generic_run_threadfn_adapter+0x4e/0xa0 [<00000000d195ac13>] kthread+0x2c7/0x3c0 [<00000000b79c1ee9>] ret_from_fork+0x2c/0x70 [<000000001167f7e6>] ret_from_fork_asm+0x1b/0x30 unreferenced object 0xff11000105d79b00 (size 192): comm "kunit_try_catch", pid 96, jiffies 4294671796 (age 763.547s) hex dump (first 32 bytes): f0 e1 5a 88 ff ff ff ff 60 59 bb 8a ff ff ff ff ..Z.....`Y...... 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ backtrace: [<000000000d6e4891>] __kmalloc+0x4d/0x140 [<000000006afe50bd>] kunit_filter_suites+0x424/0xa50 [<0000000045779fb9>] filter_suites_test+0x1b7/0x440 [<00000000cd1104a7>] kunit_try_run_case+0x119/0x270 [<00000000c654c917>] kunit_generic_run_threadfn_adapter+0x4e/0xa0 [<00000000d195ac13>] kthread+0x2c7/0x3c0 [<00000000b79c1ee9>] ret_from_fork+0x2c/0x70 [<000000001167f7e6>] ret_from_fork_asm+0x1b/0x30 unreferenced object 0xff110001067e6000 (size 1024): comm "kunit_try_catch", pid 98, jiffies 4294671798 (age 763.545s) hex dump (first 32 bytes): 73 75 69 74 65 32 00 00 00 00 00 00 00 00 00 00 suite2.......... 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ backtrace: [<00000000116e8eba>] __kmalloc_node_track_caller+0x4e/0x140 [<00000000e2f9cce9>] kmemdup+0x2c/0x60 [<000000002a36710b>] kunit_filter_suites+0x3e4/0xa50 [<00000000f452f130>] filter_suites_test_glob_test+0x1b7/0x660 [<00000000cd1104a7>] kunit_try_run_case+0x119/0x270 [<00000000c654c917>] kunit_generic_run_threadfn_adapter+0x4e/0xa0 [<00000000d195ac13>] kthread+0x2c7/0x3c0 [<00000000b79c1ee9>] ret_from_fork+0x2c/0x70 [<000000001167f7e6>] ret_from_fork_asm+0x1b/0x30 unreferenced object 0xff11000103f3a800 (size 96): comm "kunit_try_catch", pid 98, jiffies 4294671798 (age 763.545s) hex dump (first 32 bytes): f0 e1 5a 88 ff ff ff ff 40 39 bb 8a ff ff ff ff ..Z.....@9...... 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ backtrace: [<000000000d6e4891>] __kmalloc+0x4d/0x140 [<000000006afe50bd>] kunit_filter_suites+0x424/0xa50 [<00000000f452f130>] filter_suites_test_glob_test+0x1b7/0x660 [<00000000cd1104a7>] kunit_try_run_case+0x119/0x270 [<00000000c654c917>] kunit_generic_run_threadfn_adapter+0x4e/0xa0 [<00000000d195ac13>] kthread+0x2c7/0x3c0 [<00000000b79c1ee9>] ret_from_fork+0x2c/0x70 [<000000001167f7e6>] ret_from_fork_asm+0x1b/0x30 unreferenced object 0xff11000101a72ac0 (size 16): comm "kunit_try_catch", pid 104, jiffies 4294671814 (age 763.529s) hex dump (first 16 bytes): 00 00 00 00 00 00 00 00 e0 2a a7 01 01 00 11 ff .........*...... backtrace: [<000000000d6e4891>] __kmalloc+0x4d/0x140 [<00000000c7b724e7>] kunit_filter_suites+0x108/0xa50 [<00000000bad5427d>] filter_attr_test+0x1e9/0x6a0 [<00000000cd1104a7>] kunit_try_run_case+0x119/0x270 [<00000000c654c917>] kunit_generic_run_threadfn_adapter+0x4e/0xa0 [<00000000d195ac13>] kthread+0x2c7/0x3c0 [<00000000b79c1ee9>] ret_from_fork+0x2c/0x70 [<000000001167f7e6>] ret_from_fork_asm+0x1b/0x30 unreferenced object 0xff11000103caf880 (size 32): comm "kunit_try_catch", pid 104, jiffies 4294671814 (age 763.547s) hex dump (first 32 bytes): 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ backtrace: [<000000000d6e4891>] __kmalloc+0x4d/0x140 [<00000000c47b0f75>] kunit_filter_suites+0x189/0xa50 [<00000000bad5427d>] filter_attr_test+0x1e9/0x6a0 [<00000000cd1104a7>] kunit_try_run_case+0x119/0x270 [<00000000c654c917>] kunit_generic_run_threadfn_adapter+0x4e/0xa0 [<00000000d195ac13>] kthread+0x2c7/0x3c0 [<00000000b79c1ee9>] ret_from_fork+0x2c/0x70 [<000000001167f7e6>] ret_from_fork_asm+0x1b/0x30 unreferenced object 0xff11000101a72ae0 (size 16): comm "kunit_try_catch", pid 106, jiffies 4294671823 (age 763.538s) hex dump (first 16 bytes): 00 00 00 00 00 00 00 00 00 2b a7 01 01 00 11 ff .........+...... backtrace: [<000000000d6e4891>] __kmalloc+0x4d/0x140 [<00000000c7b724e7>] kunit_filter_suites+0x108/0xa50 [<0000000096255c51>] filter_attr_empty_test+0x1b0/0x310 [<00000000cd1104a7>] kunit_try_run_case+0x119/0x270 [<00000000c654c917>] kunit_generic_run_threadfn_adapter+0x4e/0xa0 [<00000000d195ac13>] kthread+0x2c7/0x3c0 [<00000000b79c1ee9>] ret_from_fork+0x2c/0x70 [<000000001167f7e6>] ret_from_fork_asm+0x1b/0x30 unreferenced object 0xff11000103caf9c0 (size 32): comm "kunit_try_catch", pid 106, jiffies 4294671823 (age 763.538s) hex dump (first 32 bytes): 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ backtrace: [<000000000d6e4891>] __kmalloc+0x4d/0x140 [<00000000c47b0f75>] kunit_filter_suites+0x189/0xa50 [<0000000096255c51>] filter_attr_empty_test+0x1b0/0x310 [<00000000cd1104a7>] kunit_try_run_case+0x119/0x270 [<00000000c654c917>] kunit_generic_run_threadfn_adapter+0x4e/0xa0 [<00000000d195ac13>] kthread+0x2c7/0x3c0 [<00000000b79c1ee9>] ret_from_fork+0x2c/0x70 [<000000001167f7e6>] ret_from_fork_asm+0x1b/0x30 unreferenced object 0xff11000101a72b00 (size 16): comm "kunit_try_catch", pid 108, jiffies 4294671832 (age 763.529s) hex dump (first 16 bytes): 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ backtrace: [<000000000d6e4891>] __kmalloc+0x4d/0x140 [<00000000c47b0f75>] kunit_filter_suites+0x189/0xa50 [<00000000881258cc>] filter_attr_skip_test+0x148/0x770 [<00000000cd1104a7>] kunit_try_run_case+0x119/0x270 [<00000000c654c917>] kunit_generic_run_threadfn_adapter+0x4e/0xa0 [<00000000d195ac13>] kthread+0x2c7/0x3c0 [<00000000b79c1ee9>] ret_from_fork+0x2c/0x70 [<000000001167f7e6>] ret_from_fork_asm+0x1b/0x30 Fixes: 5d31f71efcb6 ("kunit: add kunit.filter_glob cmdline option to filter suites") Fixes: 529534e8cba3 ("kunit: Add ability to filter attributes") Signed-off-by: Ruan Jinjie <ruanjinjie@huawei.com> Reviewed-by: Rae Moar <rmoar@google.com> Reviewed-by: David Gow <davidgow@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2023-07-31kunit: fix wild-memory-access bug in kunit_filter_suites()Ruan Jinjie1-1/+2
As for kunit_filter_suites(), When the filters arg = NULL, such as the call of kunit_filter_suites(&suite_set, "suite2", NULL, NULL, &err) in filter_suites_test() tese case in kunit, both filter_count and parsed_filters will not be initialized. So it's possible to enter kunit_filter_attr_tests(), and the use of uninitialized parsed_filters will cause below wild-memory-access. RIP: 0010:kunit_filter_suites+0x780/0xa40 Code: fe ff ff e8 42 87 4d ff 41 83 c6 01 49 83 c5 10 49 89 dc 44 39 74 24 50 0f 8e 81 fe ff ff e8 27 87 4d ff 4c 89 e8 48 c1 e8 03 <66> 42 83 3c 38 00 0f 85 af 01 00 00 49 8b 75 00 49 8b 55 08 4c 89 RSP: 0000:ff1100010743fc38 EFLAGS: 00010203 RAX: 03fc4400041d0ff1 RBX: ff1100010389a900 RCX: ffffffff9f940ad9 RDX: ff11000107429740 RSI: 0000000000000000 RDI: ff110001037ec920 RBP: ff1100010743fd50 R08: 0000000000000000 R09: ffe21c0020e87f1e R10: 0000000000000003 R11: 0000000000032001 R12: ff110001037ec800 R13: 1fe2200020e87f8c R14: 0000000000000000 R15: dffffc0000000000 FS: 0000000000000000(0000) GS:ff1100011b000000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: ff11000115201000 CR3: 0000000113066001 CR4: 0000000000771ef0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 PKRU: 55555554 Call Trace: <TASK> ? die_addr+0x3c/0xa0 ? exc_general_protection+0x148/0x220 ? asm_exc_general_protection+0x26/0x30 ? kunit_filter_suites+0x779/0xa40 ? kunit_filter_suites+0x780/0xa40 ? kunit_filter_suites+0x779/0xa40 ? __pfx_kunit_filter_suites+0x10/0x10 ? __pfx_kfree+0x10/0x10 ? kunit_add_action_or_reset+0x3d/0x50 filter_suites_test+0x1b7/0x440 ? __pfx_filter_suites_test+0x10/0x10 ? __pfx___schedule+0x10/0x10 ? try_to_wake_up+0xa8e/0x1210 ? _raw_spin_lock_irqsave+0x86/0xe0 ? __pfx__raw_spin_lock_irqsave+0x10/0x10 ? set_cpus_allowed_ptr+0x7c/0xb0 kunit_try_run_case+0x119/0x270 ? __kthread_parkme+0xdc/0x160 ? __pfx_kunit_try_run_case+0x10/0x10 kunit_generic_run_threadfn_adapter+0x4e/0xa0 ? __pfx_kunit_generic_run_threadfn_adapter+0x10/0x10 kthread+0x2c7/0x3c0 ? __pfx_kthread+0x10/0x10 ret_from_fork+0x2c/0x70 ? __pfx_kthread+0x10/0x10 ret_from_fork_asm+0x1b/0x30 </TASK> Modules linked in: Dumping ftrace buffer: (ftrace buffer empty) ---[ end trace 0000000000000000 ]--- RIP: 0010:kunit_filter_suites+0x780/0xa40 Code: fe ff ff e8 42 87 4d ff 41 83 c6 01 49 83 c5 10 49 89 dc 44 39 74 24 50 0f 8e 81 fe ff ff e8 27 87 4d ff 4c 89 e8 48 c1 e8 03 <66> 42 83 3c 38 00 0f 85 af 01 00 00 49 8b 75 00 49 8b 55 08 4c 89 RSP: 0000:ff1100010743fc38 EFLAGS: 00010203 RAX: 03fc4400041d0ff1 RBX: ff1100010389a900 RCX: ffffffff9f940ad9 RDX: ff11000107429740 RSI: 0000000000000000 RDI: ff110001037ec920 RBP: ff1100010743fd50 R08: 0000000000000000 R09: ffe21c0020e87f1e R10: 0000000000000003 R11: 0000000000032001 R12: ff110001037ec800 R13: 1fe2200020e87f8c R14: 0000000000000000 R15: dffffc0000000000 FS: 0000000000000000(0000) GS:ff1100011b000000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: ff11000115201000 CR3: 0000000113066001 CR4: 0000000000771ef0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 PKRU: 55555554 Kernel panic - not syncing: Fatal exception Dumping ftrace buffer: (ftrace buffer empty) Kernel Offset: 0x1da00000 from 0xffffffff81000000 (relocation range: 0xffffffff80000000-0xffffffffbfffffff) Rebooting in 1 seconds.. Fixes: 529534e8cba3 ("kunit: Add ability to filter attributes") Signed-off-by: Ruan Jinjie <ruanjinjie@huawei.com> Reviewed-by: David Gow <davidgow@google.com> Tested-by: Guenter Roeck <linux@roeck-us.net> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2023-07-26kunit: add tests for filtering attributesRae Moar1-0/+116
Add four tests to executor_test.c to test behavior of filtering attributes. - parse_filter_attr_test - to test the parsing of inputted filters - filter_attr_test - to test the filtering procedure on attributes - filter_attr_empty_test - to test the behavior when all tests are filtered out - filter_attr_skip_test - to test the configurable filter_action=skip option Reviewed-by: David Gow <davidgow@google.com> Signed-off-by: Rae Moar <rmoar@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2023-07-26kunit: Add ability to filter attributesRae Moar4-28/+358
Add filtering of test attributes. Users can filter tests using the module_param called "filter". Filters are imputed in the format: <attribute_name><operation><value> Example: kunit.filter="speed>slow" Operations include: >, <, >=, <=, !=, and =. These operations will act the same for attributes of the same type but may not between types. Note multiple filters can be inputted by separating them with a comma. Example: kunit.filter="speed=slow, module!=example" Since both suites and test cases can have attributes, there may be conflicts. The process of filtering follows these rules: - Filtering always operates at a per-test level. - If a test has an attribute set, then the test's value is filtered on. - Otherwise, the value falls back to the suite's value. - If neither are set, the attribute has a global "default" value, which is used. Filtered tests will not be run or show in output. The tests can instead be skipped using the configurable option "kunit.filter_action=skip". Note the default settings for running tests remains unfiltered. Finally, add "filter" methods for the speed and module attributes to parse and compare attribute values. Note this filtering functionality will be added to kunit.py in the next patch. Reviewed-by: David Gow <davidgow@google.com> Signed-off-by: Rae Moar <rmoar@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2023-07-26kunit: Add module attributeRae Moar1-0/+25
Add module attribute to the test attribute API. This attribute stores the module name associated with the test using KBUILD_MODNAME. The name of a test suite and the module name often do not match. A reference to the module name associated with the suite could be extremely helpful in running tests as modules without needing to check the codebase. This attribute will be printed for each suite. Reviewed-by: David Gow <davidgow@google.com> Signed-off-by: Rae Moar <rmoar@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2023-07-26kunit: Add speed attributeRae Moar2-1/+52
Add speed attribute to the test attribute API. This attribute will allow users to mark tests with a category of speed. Currently the categories of speed proposed are: normal, slow, and very_slow (outlined in enum kunit_speed). These are outlined in the enum kunit_speed. The assumed default speed for tests is "normal". This indicates that the test takes a relatively trivial amount of time (less than 1 second), regardless of the machine it is running on. Any test slower than this could be marked as "slow" or "very_slow". Add the macro KUNIT_CASE_SLOW to set a test as slow, as this is likely a common use of the attributes API. Add an example of marking a slow test to kunit-example-test.c. Reviewed-by: David Gow <davidgow@google.com> Signed-off-by: Rae Moar <rmoar@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2023-07-26kunit: Add test attributes API structureRae Moar4-12/+109
Add the basic structure of the test attribute API to KUnit, which can be used to save and access test associated data. Add attributes.c and attributes.h to hold associated structs and functions for the API. Create a struct that holds a variety of associated helper functions for each test attribute. These helper functions will be used to get the attribute value, convert the value to a string, and filter based on the value. This struct is flexible by design to allow for attributes of numerous types and contexts. Add a method to print test attributes in the format of "# [<test_name if not suite>.]<attribute_name>: <attribute_value>". Example for a suite: "# speed: slow" Example for a test case: "# test_case.speed: very_slow" Use this method to report attributes in the KTAP output (KTAP spec: https://docs.kernel.org/dev-tools/ktap.html) and _list_tests output when kernel's new kunit.action=list_attr option is used. Note this is derivative of the kunit.action=list option. In test.h, add fields and associated helper functions to test cases and suites to hold user-inputted test attributes. Reviewed-by: David Gow <davidgow@google.com> Signed-off-by: Rae Moar <rmoar@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2023-06-28Merge tag 'mm-nonmm-stable-2023-06-24-19-23' of ↵Linus Torvalds1-0/+1
git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm Pull non-mm updates from Andrew Morton: - Arnd Bergmann has fixed a bunch of -Wmissing-prototypes in top-level directories - Douglas Anderson has added a new "buddy" mode to the hardlockup detector. It permits the detector to work on architectures which cannot provide the required interrupts, by having CPUs periodically perform checks on other CPUs - Zhen Lei has enhanced kexec's ability to support two crash regions - Petr Mladek has done a lot of cleanup on the hard lockup detector's Kconfig entries - And the usual bunch of singleton patches in various places * tag 'mm-nonmm-stable-2023-06-24-19-23' of git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm: (72 commits) kernel/time/posix-stubs.c: remove duplicated include ocfs2: remove redundant assignment to variable bit_off watchdog/hardlockup: fix typo in config HARDLOCKUP_DETECTOR_PREFER_BUDDY powerpc: move arch_trigger_cpumask_backtrace from nmi.h to irq.h devres: show which resource was invalid in __devm_ioremap_resource() watchdog/hardlockup: define HARDLOCKUP_DETECTOR_ARCH watchdog/sparc64: define HARDLOCKUP_DETECTOR_SPARC64 watchdog/hardlockup: make HAVE_NMI_WATCHDOG sparc64-specific watchdog/hardlockup: declare arch_touch_nmi_watchdog() only in linux/nmi.h watchdog/hardlockup: make the config checks more straightforward watchdog/hardlockup: sort hardlockup detector related config values a logical way watchdog/hardlockup: move SMP barriers from common code to buddy code watchdog/buddy: simplify the dependency for HARDLOCKUP_DETECTOR_PREFER_BUDDY watchdog/buddy: don't copy the cpumask in watchdog_next_cpu() watchdog/buddy: cleanup how watchdog_buddy_check_hardlockup() is called watchdog/hardlockup: remove softlockup comment in touch_nmi_watchdog() watchdog/hardlockup: in watchdog_hardlockup_check() use cpumask_copy() watchdog/hardlockup: don't use raw_cpu_ptr() in watchdog_hardlockup_kick() watchdog/hardlockup: HAVE_NMI_WATCHDOG must implement watchdog_hardlockup_probe() watchdog/hardlockup: keep kernel.nmi_watchdog sysctl as 0444 if probe fails ...
2023-06-10kunit: include debugfs header fileArnd Bergmann1-0/+1
An extra #include statement is needed to ensure the prototypes for debugfs interfaces are visible, avoiding this warning: lib/kunit/debugfs.c:28:6: error: no previous prototype for 'kunit_debugfs_cleanup' [-Werror=missing-prototypes] lib/kunit/debugfs.c:33:6: error: no previous prototype for 'kunit_debugfs_init' [-Werror=missing-prototypes] lib/kunit/debugfs.c:102:6: error: no previous prototype for 'kunit_debugfs_create_suite' [-Werror=missing-prototypes] lib/kunit/debugfs.c:118:6: error: no previous prototype for 'kunit_debugfs_destroy_suite' [-Werror=missing-prototypes] Link: https://lkml.kernel.org/r/20230517131102.934196-10-arnd@kernel.org Signed-off-by: Arnd Bergmann <arnd@arndb.de> Reviewed-by: David Gow <davidgow@google.com> Cc: Boqun Feng <boqun.feng@gmail.com> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Christoph Lameter <cl@linux.com> Cc: Dennis Zhou <dennis@kernel.org> Cc: Eric Paris <eparis@redhat.com> Cc: Heiko Carstens <hca@linux.ibm.com> Cc: Helge Deller <deller@gmx.de> Cc: Ingo Molnar <mingo@redhat.com> Cc: Michael Ellerman <mpe@ellerman.id.au> Cc: Michal Simek <monstr@monstr.eu> Cc: Palmer Dabbelt <palmer@dabbelt.com> Cc: Paul Moore <paul@paul-moore.com> Cc: Pavel Machek <pavel@ucw.cz> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Rafael J. Wysocki <rafael@kernel.org> Cc: Russell King <linux@armlinux.org.uk> Cc: Tejun Heo <tj@kernel.org> Cc: Thomas Bogendoerfer <tsbogend@alpha.franken.de> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: Waiman Long <longman@redhat.com> Cc: Will Deacon <will@kernel.org> Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
2023-06-01kunit: Move kunit_abort() call out of kunit_do_failed_assertion()David Gow1-6/+4
KUnit aborts the current thread when an assertion fails. Currently, this is done conditionally as part of the kunit_do_failed_assertion() function, but this hides the kunit_abort() call from the compiler (particularly if it's in another module). This, in turn, can lead to both suboptimal code generation (the compiler can't know if kunit_do_failed_assertion() will return), and to static analysis tools like smatch giving false positives. Moving the kunit_abort() call into the macro should give the compiler and tools a better chance at understanding what's going on. Doing so requires exporting kunit_abort(), though it's recommended to continue to use assertions in lieu of aborting directly. In addition, kunit_abort() and kunit_do_failed_assertion() are renamed to make it clear they they're intended for internal KUnit use, to: __kunit_do_failed_assertion() and __kunit_abort() Suggested-by: Dan Carpenter <dan.carpenter@linaro.org> Signed-off-by: David Gow <davidgow@google.com> Reviewed-by: Miguel Ojeda <ojeda@kernel.org> Reviewed-by: Daniel Latypov <dlatypov@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2023-05-26kunit: Update kunit_print_ok_not_ok functionMichal Wajdeczko1-17/+28
There is no need use opaque test_or_suite pointer and is_test flag as we don't use anything from the suite struct. Always expect test pointer and use NULL as indication that provided results are from the suite so we can treat them differently. Since results could be from nested tests, like parameterized tests, add explicit level parameter to properly indent output messages and thus allow to reuse this function from other places. While around, remove small code duplication near skip directive. Signed-off-by: Michal Wajdeczko <michal.wajdeczko@intel.com> Cc: David Gow <davidgow@google.com> Cc: Rae Moar <rmoar@google.com> Reviewed-by: David Gow <davidgow@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2023-05-26kunit: Fix reporting of the skipped parameterized testsMichal Wajdeczko1-2/+4
Logs from the parameterized tests that were skipped don't include SKIP directive thus they are displayed as PASSED. Fix that. Signed-off-by: Michal Wajdeczko <michal.wajdeczko@intel.com> Cc: David Gow <davidgow@google.com> Reviewed-by: Rae Moar <rmoar@google.com> Reviewed-by: David Gow <davidgow@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2023-05-26kunit/test: Add example test showing parameterized testingMichal Wajdeczko1-0/+34
Use of parameterized testing is documented [1] but such use case is not present in demo kunit test. Add small subtest for that. [1] https://kernel.org/doc/html/latest/dev-tools/kunit/usage.html#parameterized-testing Signed-off-by: Michal Wajdeczko <michal.wajdeczko@intel.com> Cc: David Gow <davidgow@google.com> Reviewed-by: David Gow <davidgow@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2023-05-25kunit: kmalloc_array: Use kunit_add_action()David Gow1-39/+9
The kunit_add_action() function is much simpler and cleaner to use that the full KUnit resource API for simple things like the kunit_kmalloc_array() functionality. Replacing it allows us to get rid of a number of helper functions, and leaves us with no uses of kunit_alloc_resource(), which has some usability problems and is going to have its behaviour modified in an upcoming patch. Note that we need to use kunit_defer_trigger_all() to implement kunit_kfree(). Reviewed-by: Benjamin Berg <benjamin.berg@intel.com> Reviewed-by: Maxime Ripard <maxime@cerno.tech> Tested-by: Maxime Ripard <maxime@cerno.tech> Signed-off-by: David Gow <davidgow@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2023-05-25kunit: executor_test: Use kunit_add_action()David Gow1-7/+4
Now we have the kunit_add_action() function, we can use it to implement kfree_at_end() and free_subsuite_at_end() without the need for extra helper functions. Reviewed-by: Benjamin Berg <benjamin.berg@intel.com> Reviewed-by: Maxime Ripard <maxime@cerno.tech> Tested-by: Maxime Ripard <maxime@cerno.tech> Signed-off-by: David Gow <davidgow@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2023-05-25kunit: Add kunit_add_action() to defer a call until test exitDavid Gow2-1/+186
Many uses of the KUnit resource system are intended to simply defer calling a function until the test exits (be it due to success or failure). The existing kunit_alloc_resource() function is often used for this, but was awkward to use (requiring passing NULL init functions, etc), and returned a resource without incrementing its reference count, which -- while okay for this use-case -- could cause problems in others. Instead, introduce a simple kunit_add_action() API: a simple function (returning nothing, accepting a single void* argument) can be scheduled to be called when the test exits. Deferred actions are called in the opposite order to that which they were registered. This mimics the devres API, devm_add_action(), and also provides kunit_remove_action(), to cancel a deferred action, and kunit_release_action() to trigger one early. This is implemented as a resource under the hood, so the ordering between resource cleanup and deferred functions is maintained. Reviewed-by: Benjamin Berg <benjamin.berg@intel.com> Reviewed-by: Maxime Ripard <maxime@cerno.tech> Tested-by: Maxime Ripard <maxime@cerno.tech> Signed-off-by: David Gow <davidgow@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2023-05-12kunit: example: Provide example exit functionsDavid Gow1-0/+22
Add an example .exit and .suite_exit function to the KUnit example suite. Given exit functions are a bit more subtle than init functions (due to running in a different kthread, and running even after tests or test init functions fail), providing an easy place to experiment with them is useful. Reviewed-by: Rae Moar <rmoar@google.com> Signed-off-by: David Gow <davidgow@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2023-05-12kunit: Always run cleanup from a test kthreadDavid Gow1-8/+48
KUnit tests run in a kthread, with the current->kunit_test pointer set to the test's context. This allows the kunit_get_current_test() and kunit_fail_current_test() macros to work. Normally, this pointer is still valid during test shutdown (i.e., the suite->exit function, and any resource cleanup). However, if the test has exited early (e.g., due to a failed assertion), the cleanup is done in the parent KUnit thread, which does not have an active context. Instead, in the event test terminates early, run the test exit and cleanup from a new 'cleanup' kthread, which sets current->kunit_test, and better isolates the rest of KUnit from issues which arise in test cleanup. If a test cleanup function itself aborts (e.g., due to an assertion failing), there will be no further attempts to clean up: an error will be logged and the test failed. For example: # example_simple_test: test aborted during cleanup. continuing without cleaning up This should also make it easier to get access to the KUnit context, particularly from within resource cleanup functions, which may, for example, need access to data in test->priv. Reviewed-by: Benjamin Berg <benjamin.berg@intel.com> Reviewed-by: Maxime Ripard <maxime@cerno.tech> Tested-by: Maxime Ripard <maxime@cerno.tech> Signed-off-by: David Gow <davidgow@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2023-04-05kunit: add tests for using current KUnit test fieldRae Moar1-1/+41
Create test suite called "kunit_current" to add test coverage for the use of current->kunit_test, which returns the current KUnit test. Add two test cases: - kunit_current_test to test current->kunit_test and the method kunit_get_current_test(), which utilizes current->kunit_test. - kunit_current_fail_test to test the method kunit_fail_current_test(), which utilizes current->kunit_test. Signed-off-by: Rae Moar <rmoar@google.com> Reviewed-by: Daniel Latypov <dlatypov@google.com> Reviewed-by: David Gow <davidgow@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2023-03-10kunit: fix bug of extra newline characters in debugfs logsRae Moar2-12/+41
Fix bug of the extra newline characters in debugfs logs. When a line is added to debugfs with a newline character at the end, an extra line appears in the debugfs log. This is due to a discrepancy between how the lines are printed and how they are added to the logs. Remove this discrepancy by checking if a newline character is present before adding a newline character. This should closely match the printk behavior. Add kunit_log_newline_test to provide test coverage for this issue. (Also, move kunit_log_test above suite definition to remove the unnecessary declaration prior to the suite definition) As an example, say we add these two lines to the log: kunit_log(..., "KTAP version 1\n"); kunit_log(..., "1..1"); The debugfs log before this fix: KTAP version 1 1..1 The debugfs log after this fix: KTAP version 1 1..1 Signed-off-by: Rae Moar <rmoar@google.com> Reviewed-by: David Gow <davidgow@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2023-03-10kunit: fix bug in the order of lines in debugfs logsRae Moar2-9/+26
Fix bug in debugfs logs that causes an incorrect order of lines in the debugfs log. Currently, the test counts lines that show the number of tests passed, failed, and skipped, as well as any suite diagnostic lines, appear prior to the individual results, which is a bug. Ensure the order of printing for the debugfs log is correct. Additionally, add a KTAP header to so the debugfs logs can be valid KTAP. This is an example of a log prior to these fixes: KTAP version 1 # Subtest: kunit_status 1..2 # kunit_status: pass:2 fail:0 skip:0 total:2 # Totals: pass:2 fail:0 skip:0 total:2 ok 1 kunit_status_set_failure_test ok 2 kunit_status_mark_skipped_test ok 1 kunit_status Note the two lines with stats are out of order. This is the same debugfs log after the fixes (in combination with the third patch to remove the extra line): KTAP version 1 1..1 KTAP version 1 # Subtest: kunit_status 1..2 ok 1 kunit_status_set_failure_test ok 2 kunit_status_mark_skipped_test # kunit_status: pass:2 fail:0 skip:0 total:2 # Totals: pass:2 fail:0 skip:0 total:2 ok 1 kunit_status Signed-off-by: Rae Moar <rmoar@google.com> Reviewed-by: David Gow <davidgow@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2023-03-10kunit: fix bug in debugfs logs of parameterized testsRae Moar1-6/+12
Fix bug in debugfs logs that causes individual parameterized results to not appear because the log is reinitialized (cleared) when each parameter is run. Ensure these results appear in the debugfs logs, increase log size to allow for the size of parameterized results. As a result, append lines to the log directly rather than using an intermediate variable that can cause stack size warnings due to the increased log size. Here is the debugfs log of ext4_inode_test which uses parameterized tests before the fix: KTAP version 1 # Subtest: ext4_inode_test 1..1 # Totals: pass:16 fail:0 skip:0 total:16 ok 1 ext4_inode_test As you can see, this log does not include any of the individual parametrized results. After (in combination with the next two fixes to remove extra empty line and ensure KTAP valid format): KTAP version 1 1..1 KTAP version 1 # Subtest: ext4_inode_test 1..1 KTAP version 1 # Subtest: inode_test_xtimestamp_decoding ok 1 1901-12-13 Lower bound of 32bit < 0 timestamp, no extra bits ... (the rest of the individual parameterized tests) ok 16 2446-05-10 Upper bound of 32bit >=0 timestamp. All extra # inode_test_xtimestamp_decoding: pass:16 fail:0 skip:0 total:16 ok 1 inode_test_xtimestamp_decoding # Totals: pass:16 fail:0 skip:0 total:16 ok 1 ext4_inode_test Signed-off-by: Rae Moar <rmoar@google.com> Reviewed-by: David Gow <davidgow@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2023-02-28kunit: Fix 'hooks.o' build by recursing into kunitDavid Gow1-1/+1
KUnit's 'hooks.o' file need to be built-in whenever KUnit is enabled (even if CONFIG_KUNIT=m). We'd previously attemtped to do this by adding 'kunit/hooks.o' to obj-y in lib/Makefile, but this caused hooks.c to be rebuilt even when it was unchanged. Instead, always recurse into lib/kunit using obj-y when KUnit is enabled, and add the hooks there. Fixes: 7170b7ed6acb ("kunit: Add "hooks" to call into KUnit when it's built as a module"). Reported-by: Linus Torvalds <torvalds@linux-foundation.org> Link: https://lore.kernel.org/linux-kselftest/CAHk-=wiEf7irTKwPJ0jTMOF3CS-13UXmF6Fns3wuWpOZ_wGyZQ@mail.gmail.com/ Signed-off-by: David Gow <davidgow@google.com> Reviewed-by: Brendan Higgins <brendanhiggins@google.com> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2023-02-09kunit: Add printf attribute to fail_current_test_implDavid Gow2-2/+4
Add the gnu_printf (__printf()) attribute to the kunit_fail_current_test() implementation in __kunit_fail_current_test_impl(). While it's not actually useful here, as this function is never called directly, it nevertheless was triggering -Wsuggest-attribute=format warnings, so we should add it to reduce the noise. Fixes: cc3ed2fe5c93 ("kunit: Add "hooks" to call into KUnit when it's built as a module") Reported-by: kernel test robot <lkp@intel.com> Signed-off-by: David Gow <davidgow@google.com> Reviewed-by: Rae Moar <rmoar@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2023-02-09kunit: Expose 'static stub' API to redirect functionsDavid Gow4-0/+164
Add a simple way of redirecting calls to functions by including a special prologue in the "real" function which checks to see if the replacement function should be called (and, if so, calls it). To redirect calls to a function, make the first (non-declaration) line of the function: KUNIT_STATIC_STUB_REDIRECT(function_name, [function arguments]); (This will compile away to nothing if KUnit is not enabled, otherwise it will check if a redirection is active, call the replacement function, and return. This check is protected by a static branch, so has very little overhead when there are no KUnit tests running.) Calls to the real function can be redirected to a replacement using: kunit_activate_static_stub(test, real_fn, replacement_fn); The redirection will only affect calls made from within the kthread of the current test, and will be automatically disabled when the test completes. It can also be manually disabled with kunit_deactivate_static_stub(). The 'example' KUnit test suite has a more complete example. Co-developed-by: Daniel Latypov <dlatypov@google.com> Signed-off-by: Daniel Latypov <dlatypov@google.com> Signed-off-by: David Gow <davidgow@google.com> Reviewed-by: Brendan Higgins <brendanhiggins@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2023-02-09kunit: Add "hooks" to call into KUnit when it's built as a moduleDavid Gow4-8/+58
KUnit has several macros and functions intended for use from non-test code. These hooks, currently the kunit_get_current_test() and kunit_fail_current_test() macros, didn't work when CONFIG_KUNIT=m. In order to support this case, the required functions and static data need to be available unconditionally, even when KUnit itself is not built-in. The new 'hooks.c' file is therefore always included, and has both the static key required for kunit_get_current_test(), and a table of function pointers in struct kunit_hooks_table. This is filled in with the real implementations by kunit_install_hooks(), which is kept in hooks-impl.h and called when the kunit module is loaded. This can be extended for future features which require similar "hook" behaviour, such as static stubs, by simply adding new entries to the struct, and the appropriate code to set them. Fixed white-space errors during commit: Shuah Khan <skhan@linuxfoundation.org> Resolved merge conflicts with: db105c37a4d6 ("kunit: Export kunit_running()") This patch supersedes the above. Shuah Khan <skhan@linuxfoundation.org> Signed-off-by: David Gow <davidgow@google.com> Reviewed-by: Rae Moar <rmoar@google.com> Reviewed-by: Brendan Higgins <brendanhiggins@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2023-01-31kunit: fix bug in KUNIT_EXPECT_MEMEQRae Moar1-15/+25
In KUNIT_EXPECT_MEMEQ and KUNIT_EXPECT_MEMNEQ, add check if one of the inputs is NULL and fail if this is the case. Currently, the kernel crashes if one of the inputs is NULL. Instead, fail the test and add an appropriate error message. Fixes: b8a926bea8b1 ("kunit: Introduce KUNIT_EXPECT_MEMEQ and KUNIT_EXPECT_MEMNEQ macros") This was found by the kernel test robot: https://lore.kernel.org/all/202212191448.D6EDPdOh-lkp@intel.com/ Reported-by: kernel test robot <lkp@intel.com> Signed-off-by: Rae Moar <rmoar@google.com> Reviewed-by: David Gow <davidgow@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2023-01-20kunit: Export kunit_running()Arnd Bergmann1-0/+1
Using kunit_fail_current_test() in a loadable module causes a link error like: ERROR: modpost: "kunit_running" [drivers/gpu/drm/vc4/vc4.ko] undefined! Export the symbol to allow using it from modules. Fixes: da43ff045c3f ("drm/vc4: tests: Fail the current test if we access a register") Signed-off-by: Arnd Bergmann <arnd@arndb.de> Reviewed-by: David Gow <davidgow@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2022-12-27kunit: alloc_string_stream_fragment error handling bug fixYoungJun.park1-1/+3
When it fails to allocate fragment, it does not free and return error. And check the pointer inappropriately. Fixed merge conflicts with commit 618887768bb7 ("kunit: update NULL vs IS_ERR() tests") Shuah Khan <skhan@linuxfoundation.org> Signed-off-by: YoungJun.park <her0gyugyu@gmail.com> Reviewed-by: David Gow <davidgow@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2022-12-13Merge tag 'linux-kselftest-kunit-next-6.2-rc1' of ↵Linus Torvalds6-14/+83
git://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest Pull KUnit updates from Shuah Khan: "Several enhancements, fixes, clean-ups, documentation updates, improvements to logging and KTAP compliance of KUnit test output: - log numbers in decimal and hex - parse KTAP compliant test output - allow conditionally exposing static symbols to tests when KUNIT is enabled - make static symbols visible during kunit testing - clean-ups to remove unused structure definition" * tag 'linux-kselftest-kunit-next-6.2-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest: (29 commits) Documentation: dev-tools: Clarify requirements for result description apparmor: test: make static symbols visible during kunit testing kunit: add macro to allow conditionally exposing static symbols to tests kunit: tool: make parser preserve whitespace when printing test log Documentation: kunit: Fix "How Do I Use This" / "Next Steps" sections kunit: tool: don't include KTAP headers and the like in the test log kunit: improve KTAP compliance of KUnit test output kunit: tool: parse KTAP compliant test output mm: slub: test: Use the kunit_get_current_test() function kunit: Use the static key when retrieving the current test kunit: Provide a static key to check if KUnit is actively running tests kunit: tool: make --json do nothing if --raw_ouput is set kunit: tool: tweak error message when no KTAP found kunit: remove KUNIT_INIT_MEM_ASSERTION macro Documentation: kunit: Remove redundant 'tips.rst' page Documentation: KUnit: reword description of assertions Documentation: KUnit: make usage.rst a superset of tips.rst, remove duplication kunit: eliminate KUNIT_INIT_*_ASSERT_STRUCT macros kunit: tool: remove redundant file.close() call in unit test kunit: tool: unit tests all check parser errors, standardize formatting a bit ...
2022-12-13kunit: improve KTAP compliance of KUnit test outputRae Moar3-7/+10
Change KUnit test output to better comply with KTAP v1 specifications found here: https://kernel.org/doc/html/latest/dev-tools/ktap.html. 1) Use "KTAP version 1" instead of "TAP version 14" as test output header 2) Remove '-' between test number and test name on test result lines 2) Add KTAP version lines to each subtest header as well Note that the new KUnit output still includes the “# Subtest” line now located after the KTAP version line. This does not completely match the KTAP v1 spec but since it is classified as a diagnostic line, it is not expected to be disruptive or break any existing parsers. This “# Subtest” line comes from the TAP 14 spec (https://testanything.org/tap-version-14-specification.html) and it is used to define the test name before the results. Original output: TAP version 14 1..1 # Subtest: kunit-test-suite 1..3 ok 1 - kunit_test_1 ok 2 - kunit_test_2 ok 3 - kunit_test_3 # kunit-test-suite: pass:3 fail:0 skip:0 total:3 # Totals: pass:3 fail:0 skip:0 total:3 ok 1 - kunit-test-suite New output: KTAP version 1 1..1 KTAP version 1 # Subtest: kunit-test-suite 1..3 ok 1 kunit_test_1 ok 2 kunit_test_2 ok 3 kunit_test_3 # kunit-test-suite: pass:3 fail:0 skip:0 total:3 # Totals: pass:3 fail:0 skip:0 total:3 ok 1 kunit-test-suite Signed-off-by: Rae Moar <rmoar@google.com> Reviewed-by: Daniel Latypov <dlatypov@google.com> Reviewed-by: David Gow <davidgow@google.com> Tested-by: Anders Roxell <anders.roxell@linaro.org> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2022-12-13kunit: Provide a static key to check if KUnit is actively running testsDavid Gow1-0/+6
KUnit does a few expensive things when enabled. This hasn't been a problem because KUnit was only enabled on test kernels, but with a few people enabling (but not _using_) KUnit on production systems, we need a runtime way of handling this. Provide a 'kunit_running' static key (defaulting to false), which allows us to hide any KUnit code behind a static branch. This should reduce the performance impact (on other code) of having KUnit enabled to a single NOP when no tests are running. Note that, while it looks unintuitive, tests always run entirely within __kunit_test_suites_init(), so it's safe to decrement the static key at the end of this function, rather than in __kunit_test_suites_exit(), which is only there to clean up results in debugfs. Signed-off-by: David Gow <davidgow@google.com> Reviewed-by: Daniel Latypov <dlatypov@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2022-10-27kunit: remove unused structure definitionYoungJun.park1-5/+0
remove unused string_stream_alloc_context structure definition. Signed-off-by: YoungJun.park <her0gyugyu@gmail.com> Reviewed-by: David Gow <davidgow@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2022-10-27kunit: Add KUnit memory block assertions to the example_all_expect_macros_testMaíra Canal1-0/+7
Augment the example_all_expect_macros_test with the KUNIT_EXPECT_MEMEQ and KUNIT_EXPECT_MEMNEQ macros by creating a test with memory block assertions. Signed-off-by: Maíra Canal <mairacanal@riseup.net> Reviewed-by: Daniel Latypov <dlatypov@google.com> Reviewed-by: David Gow <davidgow@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2022-10-27kunit: Introduce KUNIT_EXPECT_MEMEQ and KUNIT_EXPECT_MEMNEQ macrosMaíra Canal1-0/+56
Currently, in order to compare memory blocks in KUnit, the KUNIT_EXPECT_EQ or KUNIT_EXPECT_FALSE macros are used in conjunction with the memcmp function, such as: KUNIT_EXPECT_EQ(test, memcmp(foo, bar, size), 0); Although this usage produces correct results for the test cases, when the expectation fails, the error message is not very helpful, indicating only the return of the memcmp function. Therefore, create a new set of macros KUNIT_EXPECT_MEMEQ and KUNIT_EXPECT_MEMNEQ that compare memory blocks until a specified size. In case of expectation failure, those macros print the hex dump of the memory blocks, making it easier to debug test failures for memory blocks. That said, the expectation KUNIT_EXPECT_EQ(test, memcmp(foo, bar, size), 0); would translate to the expectation KUNIT_EXPECT_MEMEQ(test, foo, bar, size); Signed-off-by: Maíra Canal <mairacanal@riseup.net> Reviewed-by: Daniel Latypov <dlatypov@google.com> Reviewed-by: Muhammad Usama Anjum <usama.anjum@collabora.com> Reviewed-by: David Gow <davidgow@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2022-10-27kunit: log numbers in decimal and hexMark Rutland1-2/+4
When KUNIT_EXPECT_EQ() or KUNIT_ASSERT_EQ() log a failure, they log the two values being compared, with numerical values logged in decimal. In some cases, decimal output is painful to consume, and hexadecimal output would be more helpful. For example, this is the case for tests I'm currently developing for the arm64 insn encoding/decoding code, where comparing two 32-bit instruction opcodes results in output such as: | # test_insn_add_shifted_reg: EXPECTATION FAILED at arch/arm64/lib/test_insn.c:2791 | Expected obj_insn == gen_insn, but | obj_insn == 2332164128 | gen_insn == 1258422304 To make this easier to consume, this patch logs the values in both decimal and hexadecimal: | # test_insn_add_shifted_reg: EXPECTATION FAILED at arch/arm64/lib/test_insn.c:2791 | Expected obj_insn == gen_insn, but | obj_insn == 2332164128 (0x8b020020) | gen_insn == 1258422304 (0x4b020020) As can be seen from the example, having hexadecimal makes it significantly easier for a human to spot which specific bits are incorrect. Signed-off-by: Mark Rutland <mark.rutland@arm.com> Cc: Brendan Higgins <brendan.higgins@linux.dev> Cc: David Gow <davidgow@google.com> Cc: linux-kselftest@vger.kernel.org Cc: kunit-dev@googlegroups.com Acked-by: Daniel Latypov <dlatypov@google.com> Reviewed-by: David Gow <davidgow@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2022-10-19kunit: update NULL vs IS_ERR() testsDan Carpenter2-3/+3
The alloc_string_stream() functions were changed from returning NULL on error to returning error pointers so these caller needs to be updated as well. Fixes: 78b1c6584fce ("kunit: string-stream: Simplify resource use") Signed-off-by: Dan Carpenter <dan.carpenter@oracle.com> Reviewed-by: Daniel Latypov <dlatypov@google.com> Reviewed-by: David Gow <davidgow@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2022-10-07kunit: remove format func from struct kunit_assert, get it to 0 bytesDaniel Latypov1-3/+4
Each calll to a KUNIT_EXPECT_*() macro creates a local variable which contains a struct kunit_assert. Normally, we'd hope the compiler would be able to optimize this away, but we've seen cases where it hasn't, see https://groups.google.com/g/kunit-dev/c/i3fZXgvBrfA/m/GbrMNej2BAAJ. In changes like commit 21957f90b28f ("kunit: split out part of kunit_assert into a static const"), we've moved more and more parts out of struct kunit_assert and its children types (kunit_binary_assert). This patch removes the final field and gets us to: sizeof(struct kunit_assert) == 0 sizeof(struct kunit_binary_assert) == 24 (on UML x86_64). This also reduces the amount of macro plumbing going on at the cost of passing in one more arg to the base KUNIT_ASSERTION macro and kunit_do_failed_assertion(). Signed-off-by: Daniel Latypov <dlatypov@google.com> Reviewed-by: David Gow <davidgow@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2022-10-07kunit: make kunit_kfree(NULL) a no-op to match kfree()Daniel Latypov1-0/+3
The real kfree() function will silently return when given a NULL. So a user might reasonably think they can write the following code: char *buffer = NULL; if (param->use_buffer) buffer = kunit_kzalloc(test, 10, GFP_KERNEL); ... kunit_kfree(test, buffer); As-is, kunit_kfree() will mark the test as FAILED when buffer is NULL. (And in earlier times, it would segfault). Let's match the semantics of kfree(). Suggested-by: David Gow <davidgow@google.com> Signed-off-by: Daniel Latypov <dlatypov@google.com> Reviewed-by: David Gow <davidgow@google.com> Reviewed-by: Brendan Higgins <brendanhiggins@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2022-10-07kunit: make kunit_kfree() not segfault on invalid inputsDaniel Latypov1-12/+2
kunit_kfree() can only work on data ("resources") allocated by KUnit. Currently for code like this, > void *ptr = kmalloc(4, GFP_KERNEL); > kunit_kfree(test, ptr); kunit_kfree() will segfault. It'll try and look up the kunit_resource associated with `ptr` and get a NULL back, but it won't check for this. This means we also segfault if you double-free. Change kunit_kfree() so it'll notice these invalid pointers and respond by failing the test. Implementation: kunit_destroy_resource() does what kunit_kfree() does, but is more generic and returns -ENOENT when it can't find the resource. Sadly, unlike just letting it crash, this means we don't get a stack trace. But kunit_kfree() is so infrequently used it shouldn't be hard to track down the bad callsite anyways. After this change, the above code gives: > # example_simple_test: EXPECTATION FAILED at lib/kunit/test.c:702 > kunit_kfree: 00000000626ec200 already freed or not allocated by kunit Signed-off-by: Daniel Latypov <dlatypov@google.com> Reviewed-by: David Gow <davidgow@google.com> Reviewed-by: Brendan Higgins <brendanhiggins@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2022-10-07kunit: make kunit_kfree() only work on pointers from kunit_malloc() and friendsDaniel Latypov2-2/+15
kunit_kfree() exists to clean up allocations from kunit_kmalloc() and friends early instead of waiting for this to happen automatically at the end of the test. But it can be used on *anything* registered with the kunit resource API. E.g. the last 2 statements are equivalent: struct kunit_resource *res = something(); kfree(res->data); kunit_put_resource(res); The problem is that there could be multiple resources that point to the same `data`. E.g. you can have a named resource acting as a pseudo-global variable in a test. If you point it to data allocated with kunit_kmalloc(), then calling `kunit_kfree(ptr)` has the chance to delete either the named resource or to kfree `ptr`. Which one it does depends on the order the resources are registered as kunit_kfree() will delete resources in LIFO order. So this patch restricts kunit_kfree() to only working on resources created by kunit_kmalloc(). Calling it is therefore guaranteed to free the memory, not do anything else. Note: kunit_resource_instance_match() wasn't used outside of KUnit, so it should be safe to remove from the public interface. It's also generally dangerous, as shown above, and shouldn't be used. Signed-off-by: Daniel Latypov <dlatypov@google.com> Reviewed-by: David Gow <davidgow@google.com> Reviewed-by: Brendan Higgins <brendanhiggins@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2022-10-07kunit: drop test pointer in string_stream_fragmentDaniel Latypov2-6/+5
We already store the `struct kunit *test` in the string_stream object itself, so we need don't need to store a copy of this pointer in every fragment in the stream. Drop it, getting string_stream_fragment down the bare minimum: a list_head and the `char *` with the actual fragment. Signed-off-by: Daniel Latypov <dlatypov@google.com> Reviewed-by: David Gow <davidgow@google.com> Reviewed-by: Brendan Higgins <brendanhiggins@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2022-10-07kunit: string-stream: Simplify resource useDavid Gow3-76/+18
Currently, KUnit's string streams are themselves "KUnit resources". This is redundant since the stream itself is already allocated with kunit_kzalloc() and will thus be freed automatically at the end of the test. string-stream is only used internally within KUnit, and isn't using the extra features that resources provide like reference counting, being able to locate them dynamically as "test-local variables", etc. Indeed, the resource's refcount is never incremented when the pointer is returned. The fact that it's always manually destroyed is more evidence that the reference counting is unused. Signed-off-by: David Gow <davidgow@google.com> Signed-off-by: Daniel Latypov <dlatypov@google.com> Reviewed-by: Brendan Higgins <brendanhiggins@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2022-09-30kunit: add kunit.enable to enable/disable KUnit testJoe Fradley3-0/+39
This patch adds the kunit.enable module parameter that will need to be set to true in addition to KUNIT being enabled for KUnit tests to run. The default value is true giving backwards compatibility. However, for the production+testing use case the new config option KUNIT_DEFAULT_ENABLED can be set to N requiring the tester to opt-in by passing kunit.enable=1 to the kernel. Signed-off-by: Joe Fradley <joefradley@google.com> Reviewed-by: David Gow <davidgow@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2022-07-14kunit: executor: Fix a memory leak on failure in kunit_filter_testsDavid Gow1-1/+3
It's possible that memory allocation for 'filtered' will fail, but for the copy of the suite to succeed. In this case, the copy could be leaked. Properly free 'copy' in the error case for the allocation of 'filtered' failing. Note that there may also have been a similar issue in kunit_filter_subsuites, before it was removed in "kunit: flatten kunit_suite*** to kunit_suite** in .kunit_test_suites". This was reported by clang-analyzer via the kernel test robot, here: https://lore.kernel.org/all/c8073b8e-7b9e-0830-4177-87c12f16349c@intel.com/ And by smatch via Dan Carpenter and the kernel test robot: https://lore.kernel.org/all/202207101328.ASjx88yj-lkp@intel.com/ Fixes: a02353f49162 ("kunit: bail out of test filtering logic quicker if OOM") Reported-by: kernel test robot <yujie.liu@intel.com> Reported-by: kernel test robot <lkp@intel.com> Reported-by: Dan Carpenter <dan.carpenter@oracle.com> Reviewed-by: Daniel Latypov <dlatypov@google.com> Reviewed-by: Brendan Higgins <brendanhiggins@google.com> Signed-off-by: David Gow <davidgow@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
generated by cgit v1.2.3 (git 2.39.0) at 2024-11-24 00:00:13 +0300