summaryrefslogtreecommitdiff
path: root/tools/testing/kunit/kunit.py
AgeCommit message (Collapse)AuthorFilesLines
2021-08-13kunit: tool: make --raw_output support only showing kunit outputDaniel Latypov1-5/+15
--raw_output is nice, but it would be nicer if could show only output after KUnit tests have started. So change the flag to allow specifying a string ('kunit'). Make it so `--raw_output` alone will default to `--raw_output=all` and have the same original behavior. Drop the small kunit_parser.raw_output() function since it feels wrong to put it in "kunit_parser.py" when the point of it is to not parse anything. E.g. $ ./tools/testing/kunit/kunit.py run --raw_output=kunit ... [15:24:07] Starting KUnit Kernel ... TAP version 14 1..1 # Subtest: example 1..3 # example_simple_test: initializing ok 1 - example_simple_test # example_skip_test: initializing # example_skip_test: You should not see a line below. ok 2 - example_skip_test # SKIP this test should be skipped # example_mark_skipped_test: initializing # example_mark_skipped_test: You should see a line below. # example_mark_skipped_test: You should see this line. ok 3 - example_mark_skipped_test # SKIP this test should be skipped ok 1 - example [15:24:10] Elapsed time: 6.487s total, 0.001s configuring, 3.510s building, 0.000s running Signed-off-by: Daniel Latypov <dlatypov@google.com> Reviewed-by: David Gow <davidgow@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2021-08-13kunit: tool: add --kernel_args to allow setting module paramsDaniel Latypov1-4/+12
kunit.py currently does not make it possible for users to specify module parameters (/kernel arguments more generally) unless one directly tweaks the kunit.py code itself. This hasn't mattered much so far, but this would make it easier to port existing tests that expose module parameters over to KUnit and/or let current KUnit tests take advantage of them. Tested using an kunit internal parameter: $ ./tools/testing/kunit/kunit.py run --kunitconfig=lib/kunit \ --kernel_args=kunit.filter_glob=kunit_status ... Testing complete. 2 tests run. 0 failed. 0 crashed. 0 skipped. Signed-off-by: Daniel Latypov <dlatypov@google.com> Reviewed-by: David Gow <davidgow@google.com> Reviewed-by: Brendan Higgins <brendanhiggins@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2021-07-12kunit: tool: Assert the version requirementSeongJae Park1-0/+2
Commit 87c9c1631788 ("kunit: tool: add support for QEMU") on the 'next' tree adds 'from __future__ import annotations' in 'kunit_kernel.py'. Because it is supported on only >=3.7 Python, people using older Python will get below error: Traceback (most recent call last): File "./tools/testing/kunit/kunit.py", line 20, in <module> import kunit_kernel File "/home/sjpark/linux/tools/testing/kunit/kunit_kernel.py", line 9 from __future__ import annotations ^ SyntaxError: future feature annotations is not defined This commit adds a version assertion in 'kunit.py', so that people get more explicit error message like below: Traceback (most recent call last): File "./tools/testing/kunit/kunit.py", line 15, in <module> assert sys.version_info >= (3, 7), "Python version is too old" AssertionError: Python version is too old Signed-off-by: SeongJae Park <sjpark@amazon.de> Acked-by: Daniel Latypov <dlatypov@google.com> Reviewed-by: Brendan Higgins <brendanhiggins@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2021-06-12kunit: tool: add support for QEMUBrendan Higgins1-8/+49
Add basic support to run QEMU via kunit_tool. Add support for i386, x86_64, arm, arm64, and a bunch more. Signed-off-by: Brendan Higgins <brendanhiggins@google.com> Tested-by: David Gow <davidgow@google.com> Reviewed-by: David Gow <davidgow@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2021-04-02kunit: tool: make --kunitconfig accept dirs, add lib/kunit fragmentDaniel Latypov1-1/+3
TL;DR $ ./tools/testing/kunit/kunit.py run --kunitconfig=lib/kunit Per suggestion from Ted [1], we can reduce the amount of typing by assuming a convention that these files are named '.kunitconfig'. In the case of [1], we now have $ ./tools/testing/kunit/kunit.py run --kunitconfig=fs/ext4 Also add in such a fragment for kunit itself so we can give that as an example more close to home (and thus less likely to be accidentally broken). [1] https://lore.kernel.org/linux-ext4/YCNF4yP1dB97zzwD@mit.edu/ Signed-off-by: Daniel Latypov <dlatypov@google.com> Reviewed-by: David Gow <davidgow@google.com> Reviewed-by: Brendan Higgins <brendanhiggins@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2021-02-09kunit: tool: add support for filtering suites by globDaniel Latypov1-5/+16
This allows running different subsets of tests, e.g. $ ./tools/testing/kunit/kunit.py build $ ./tools/testing/kunit/kunit.py exec 'list*' $ ./tools/testing/kunit/kunit.py exec 'kunit*' This passes the "kunit_filter.glob" commandline option to the UML kernel, which currently only supports filtering by suite name. Signed-off-by: Daniel Latypov <dlatypov@google.com> Reviewed-by: Brendan Higgins <brendanhiggins@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2021-02-09kunit: make kunit_tool accept optional path to .kunitconfig fragmentDaniel Latypov1-3/+6
Currently running tests via KUnit tool means tweaking a .kunitconfig file, which you'd keep around locally and never commit. This changes makes it so users can pass in a path to a kunitconfig. One of the imagined use cases is having kunitconfig fragments in-tree to formalize interesting sets of tests for features/subsystems, e.g. $ ./tools/testing/kunit/kunit.py run --kunticonfig=fs/ext4/kunitconfig For now, this hypothetical fs/ext4/kunitconfig would contain CONFIG_KUNIT=y CONFIG_EXT4_FS=y CONFIG_EXT4_KUNIT_TESTS=y At the moment, it's not hard to manually whip up this file, but as more and more tests get added, this will get tedious. It also opens the door to documenting how to run all the tests relevant to a specific subsystem or feature as a simple one-liner. This can be seen as an analogue to tools/testing/selftests/*/config But in the case of KUnit, the tests live in the same directory as the code-under-test, so it feels more natural to allow the kunitconfig fragments to live anywhere. (Though, people could create a separate directory if wanted; this patch imposes no restrictions on the path). Signed-off-by: Daniel Latypov <dlatypov@google.com> Reviewed-by: Brendan Higgins <brendanhiggins@google.com> Tested-by: Brendan Higgins <brendanhiggins@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2021-01-16kunit: tool: move kunitconfig parsing into __init__, make it optionalDaniel Latypov1-16/+4
LinuxSourceTree will unceremoniously crash if the user doesn't call read_kunitconfig() first in a number of functions. And currently every place we create an instance, the caller also calls create_kunitconfig() and read_kunitconfig(). Move these instead into __init__() so they can't be forgotten and to reduce copy-paste. The https://github.com/google/pytype type-checker complained that _config wasn't initialized. With this, kunit_tool now type checks under both pytype and mypy. Add an optional boolean that can be used to disable this for use cases in the future where we might not need/want to load the config. Signed-off-by: Daniel Latypov <dlatypov@google.com> Reviewed-by: Brendan Higgins <brendanhiggins@google.com> Tested-by: Brendan Higgins <brendanhiggins@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2021-01-16kunit: tool: surface and address more typing issuesDaniel Latypov1-7/+7
The authors of this tool were more familiar with a different type-checker, https://github.com/google/pytype. That's open source, but mypy seems more prevalent (and runs faster). And unlike pytype, mypy doesn't try to infer types so it doesn't check unanotated functions. So annotate ~all functions in kunit tool to increase type-checking coverage. Note: per https://www.python.org/dev/peps/pep-0484/, `__init__()` should be annotated as `-> None`. Doing so makes mypy discover a number of new violations. Exclude main() since we reuse `request` for the different types of requests, which mypy isn't happy about. This commit fixes all but one error, where `TestSuite.status` might be None. Signed-off-by: Daniel Latypov <dlatypov@google.com> Reviewed-by: David Gow <davidgow@google.com> Tested-by: Brendan Higgins <brendanhiggins@google.com> Acked-by: Brendan Higgins <brendanhiggins@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2020-12-08tweewide: Fix most Shebang linesFinn Behrens1-1/+1
Change every shebang which does not need an argument to use /usr/bin/env. This is needed as not every distro has everything under /usr/bin, sometimes not even bash. Signed-off-by: Finn Behrens <me@kloenk.de> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
2020-11-10kunit: Do not pollute source directory with generated files (.kunitconfig)Andy Shevchenko1-13/+12
When --build_dir is provided use it and do not pollute source directory which even can be mounted over network or read-only. Signed-off-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com> Reviewed-by: Brendan Higgins <brendanhiggins@google.com> Tested-by: Brendan Higgins <brendanhiggins@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2020-11-10kunit: Fix kunit.py parse subcommand (use null build_dir)David Gow1-1/+1
When JSON support was added in [1], the KunitParseRequest tuple was updated to contain a 'build_dir' field, but kunit.py parse doesn't accept --build_dir as an option. The code nevertheless tried to access it, resulting in this error: AttributeError: 'Namespace' object has no attribute 'build_dir' Given that the parser only uses the build_dir variable to set the 'build_environment' json field, we set it to None (which gives the JSON 'null') for now. Ultimately, we probably do want to be able to set this, but since it's new functionality which (for the parse subcommand) never worked, this is the quickest way of getting it back up and running. [1]: https://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest.git/commit/?h=kunit-fixes&id=21a6d1780d5bbfca0ce9b8104ca6233502fcbf86 Fixes: 21a6d1780d5b ("kunit: tool: allow generating test results in JSON") Signed-off-by: David Gow <davidgow@google.com> Reviewed-by: Brendan Higgins <brendanhiggins@google.com> Tested-by: Brendan Higgins <brendanhiggins@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2020-10-02kunit: tool: handle when .kunit exists but .kunitconfig does notBrendan Higgins1-4/+8
Right now .kunitconfig and the build dir are automatically created if the build dir does not exists; however, if the build dir is present and .kunitconfig is not, kunit_tool will crash. Fix this by checking for both the build dir as well as the .kunitconfig. NOTE: This depends on commit 5578d008d9e0 ("kunit: tool: fix running kunit_tool from outside kernel tree") Link: https://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest.git/commit/?id=5578d008d9e06bb531fb3e62dd17096d9fd9c853 Signed-off-by: Brendan Higgins <brendanhiggins@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2020-09-01kunit: tool: allow generating test results in JSONHeidi Fahim1-6/+29
Add a --json flag, which when specified generates JSON formatted test results conforming to the KernelCI API test_group spec[1]. The user can use the new flag to specify a filename to print the json formatted results to. Link[1]: https://api.kernelci.org/schema-test-group.html#post Signed-off-by: Heidi Fahim <heidifahim@google.com> Signed-off-by: Brendan Higgins <brendanhiggins@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2020-09-01kunit: tool: fix running kunit_tool from outside kernel treeBrendan Higgins1-8/+5
Currently kunit_tool does not work correctly when executed from a path outside of the kernel tree, so make sure that the current working directory is correct and the kunit_dir is properly initialized before running. Signed-off-by: Brendan Higgins <brendanhiggins@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2020-08-05Merge tag 'linux-kselftest-kunit-5.9-rc1' of ↵Linus Torvalds1-24/+0
git://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest Pull kunit updates from Shuah Khan: - Add a generic kunit_resource API extending it to support resources that are passed in to kunit in addition kunit allocated resources. In addition, KUnit resources are now refcounted to avoid passed in resources being released while in use by kunit. - Add support for named resources. - Important bug fixes from Brendan Higgins and Will Chen * tag 'linux-kselftest-kunit-5.9-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest: kunit: tool: fix improper treatment of file location kunit: tool: fix broken default args in unit tests kunit: capture stderr on all make subprocess calls Documentation: kunit: Remove references to --defconfig kunit: add support for named resources kunit: generalize kunit_resource API beyond allocated resources
2020-07-17kunit: tool: fix improper treatment of file locationBrendan Higgins1-24/+0
Commit 01397e822af4 ("kunit: Fix TabError, remove defconfig code and handle when there is no kunitconfig") and commit 45ba7a893ad8 ("kunit: kunit_tool: Separate out config/build/exec/parse") introduced two closely related issues which built off of each other: they excessively created the build directory when not present and modified a constant (constants in Python only exist by convention). Together these issues broken a number of unit tests for KUnit tool, so fix them. Fixed up commit log to fic checkpatch commit description style error. Shuah Khan <skhan@linuxfoundation.org> Fixes: 01397e822af4 ("kunit: Fix TabError, remove defconfig code and handle when there is no kunitconfig") Fixes: 45ba7a893ad8 ("kunit: kunit_tool: Separate out config/build/exec/parse") Signed-off-by: Brendan Higgins <brendanhiggins@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2020-06-26kunit: kunit_tool: Fix invalid result when build failsDavid Gow1-1/+3
When separating out different phases of running tests[1] (build/exec/parse/etc), the format of the KunitResult tuple changed (adding an elapsed_time variable). This is not populated during a build failure, causing kunit.py to crash. This fixes [1] to probably populate the result variable, causing a failing build to be reported properly. [1]: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=45ba7a893ad89114e773b3dc32f6431354c465d6 Signed-off-by: David Gow <davidgow@google.com> Reviewed-by: Brendan Higgins <brendanhiggins@google.com> Tested-by: Brendan Higgins <brendanhiggins@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2020-06-01kunit: Fix TabError, remove defconfig code and handle when there is no ↵Vitor Massaru Iha1-9/+18
kunitconfig The identation before this code (`if not os.path.exists(cli_args.build_dir):``) was with spaces instead of tabs after fixed up merge conflits, this commit revert spaces to tabs: [iha@bbking linux]$ tools/testing/kunit/kunit.py run File "tools/testing/kunit/kunit.py", line 247 if not linux: ^ TabError: inconsistent use of tabs and spaces in indentation [iha@bbking linux]$ tools/testing/kunit/kunit.py run Traceback (most recent call last): File "tools/testing/kunit/kunit.py", line 338, in <module> main(sys.argv[1:]) File "tools/testing/kunit/kunit.py", line 215, in main add_config_opts(config_parser) [iha@bbking linux]$ tools/testing/kunit/kunit.py run Traceback (most recent call last): File "tools/testing/kunit/kunit.py", line 337, in <module> main(sys.argv[1:]) File "tools/testing/kunit/kunit.py", line 255, in main result = run_tests(linux, request) File "tools/testing/kunit/kunit.py", line 133, in run_tests request.defconfig, AttributeError: 'KunitRequest' object has no attribute 'defconfig' Handles when there is no .kunitconfig, the error due to merge conflicts between the following: commit 9bdf64b35117 ("kunit: use KUnit defconfig by default") commit 45ba7a893ad8 ("kunit: kunit_tool: Separate out config/build/exec/parse") [iha@bbking linux]$ tools/testing/kunit/kunit.py run Traceback (most recent call last): File "tools/testing/kunit/kunit.py", line 335, in <module> main(sys.argv[1:]) File "tools/testing/kunit/kunit.py", line 246, in main linux = kunit_kernel.LinuxSourceTree() File "../tools/testing/kunit/kunit_kernel.py", line 109, in __init__ self._kconfig.read_from_file(kunitconfig_path) File "t../ools/testing/kunit/kunit_config.py", line 88, in read_from_file with open(path, 'r') as f: FileNotFoundError: [Errno 2] No such file or directory: '.kunit/.kunitconfig' Signed-off-by: Vitor Massaru Iha <vitor@massaru.org> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2020-05-23kunit: use KUnit defconfig by defaultVitor Massaru Iha1-11/+3
To improve the usability of KUnit, defconfig is used by default if no kunitconfig is present. * https://bugzilla.kernel.org/show_bug.cgi?id=205259 Fixed up minor merge conflicts - Shuah Khan <skhan@linuxfoundation.org> Signed-off-by: Vitor Massaru Iha <vitor@massaru.org> Reviewed-by: Brendan Higgins <brendanhiggins@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2020-05-22kunit: use --build_dir=.kunit as defaultVitor Massaru Iha1-7/+6
To make KUnit easier to use, and to avoid overwriting object and .config files, the default KUnit build directory is set to .kunit * Related bug: https://bugzilla.kernel.org/show_bug.cgi?id=205221 Fixed up minor merge conflicts - Shuah Khan <skhan@linuxfoundation.org> Signed-off-by: Vitor Massaru Iha <vitor@massaru.org> Reviewed-by: Brendan Higgins <brendanhiggins@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2020-05-12kunit: kunit_tool: Separate out config/build/exec/parseDavid Gow1-56/+237
Add new subcommands to kunit.py to allow stages of the existing 'run' subcommand to be run independently: - 'config': Verifies that .config is a subset of .kunitconfig - 'build': Compiles a UML kernel for KUnit - 'exec': Runs the kernel, and outputs the test results. - 'parse': Parses test results from a file or stdin The 'run' command continues to behave as before. Signed-off-by: David Gow <davidgow@google.com> Reviewed-by: Brendan Higgins <brendanhiggins@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2020-03-24kunit: add --make_optionsGreg Thelen1-4/+10
The kunit.py utility builds an ARCH=um kernel and then runs it. Add optional --make_options flag to kunit.py allowing for the operator to specify extra build options. This allows use of the clang compiler for kunit: tools/testing/kunit/kunit.py run --defconfig \ --make_options CC=clang --make_options HOSTCC=clang Signed-off-by: Greg Thelen <gthelen@google.com> Reviewed-by: Brendan Higgins <brendanhiggins@google.com> Reviewed-by: Nathan Chancellor <natechancellor@gmail.com> Tested-by: David Gow <davidgow@google.com> Signed-off-by: Brendan Higgins <brendanhiggins@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2020-03-21kunit: Run all KUnit tests through allyesconfigHeidi Fahim1-12/+18
Implemented the functionality to run all KUnit tests through kunit_tool by specifying an --alltests flag, which builds UML with allyesconfig enabled, and consequently runs every KUnit test. A new function was added to kunit_kernel: make_allyesconfig. Firstly, if --alltests is specified, kunit.py triggers build_um_kernel which call make_allyesconfig. This function calls the make command, disables the broken configs that would otherwise prevent UML from building, then starts the kernel with all possible configurations enabled. All stdout and stderr is sent to test.log and read from there then fed through kunit_parser to parse the tests to the user. Also added a signal_handler in case kunit is interrupted while running. Tested: Run under different conditions such as testing with --raw_output, testing program interrupt then immediately running kunit again without --alltests and making sure to clean the console. Signed-off-by: Heidi Fahim <heidifahim@google.com> Reviewed-by: Brendan Higgins <brendanhiggins@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2020-02-20kunit: run kunit_tool from any directoryHeidi Fahim1-0/+12
Implemented small fix so that the script changes work directories to the root of the linux kernel source tree from which kunit.py is run. This enables the user to run kunit from any working directory. Originally considered using os.path.join but this is more error prone as we would have to find all file path usages and modify them accordingly. Using os.chdir ensures that the entire script is run within /linux. Signed-off-by: Heidi Fahim <heidifahim@google.com> Reviewed-by: Brendan Higgins <brendanhiggins@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2019-12-23kunit: Rename 'kunitconfig' to '.kunitconfig'SeongJae Park1-1/+1
This commit renames 'kunitconfig' to '.kunitconfig' so that it can be automatically ignored by git and do not disturb people who want to type 'kernel/' by pressing only the 'k' and then 'tab' key. Signed-off-by: SeongJae Park <sjpark@amazon.de> Reviewed-by: Brendan Higgins <brendanhiggins@google.com> Tested-by: Brendan Higgins <brendanhiggins@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2019-12-23kunit: Place 'test.log' under the 'build_dir'SeongJae Park1-1/+1
'kunit' writes the 'test.log' under the kernel source directory even though a 'build_dir' option is given. As users who use the option might expect the outputs to be placed under the specified directory, this commit modifies the logic to write the log file under the 'build_dir'. Signed-off-by: SeongJae Park <sjpark@amazon.de> Reviewed-by: Brendan Higgins <brendanhiggins@google.com> Tested-by: Brendan Higgins <brendanhiggins@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2019-12-23kunit: Create default config in '--build_dir'SeongJae Park1-2/+9
If both '--build_dir' and '--defconfig' are given, the handling of '--defconfig' ignores '--build_dir' option. This commit modifies the behavior to respect '--build_dir' option. Reported-by: Brendan Higgins <brendanhiggins@google.com> Suggested-by: Brendan Higgins <brendanhiggins@google.com> Signed-off-by: SeongJae Park <sjpark@amazon.de> Reviewed-by: Brendan Higgins <brendanhiggins@google.com> Tested-by: Brendan Higgins <brendanhiggins@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2019-12-23kunit: Remove duplicated defconfig creationSeongJae Park1-3/+0
'--defconfig' option is handled by the 'main() of the 'kunit.py' but again handled in following 'run_tests()'. This commit removes this duplicated handling of the option in the 'run_tests()'. Signed-off-by: SeongJae Park <sjpark@amazon.de> Reviewed-by: Brendan Higgins <brendanhiggins@google.com> Tested-by: Brendan Higgins <brendanhiggins@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2019-10-08kunit: Fix '--build_dir' optionSeongJae Park1-2/+4
Running kunit with '--build_dir' option gives following error message: ``` $ ./tools/testing/kunit/kunit.py run --build_dir ../linux.out.kunit/ [00:57:24] Building KUnit Kernel ... [00:57:29] Starting KUnit Kernel ... Traceback (most recent call last): File "./tools/testing/kunit/kunit.py", line 136, in <module> main(sys.argv[1:]) File "./tools/testing/kunit/kunit.py", line 129, in main result = run_tests(linux, request) File "./tools/testing/kunit/kunit.py", line 68, in run_tests test_result = kunit_parser.parse_run_tests(kunit_output) File "/home/sjpark/linux/tools/testing/kunit/kunit_parser.py", line 283, in parse_run_tests test_result = parse_test_result(list(isolate_kunit_output(kernel_output))) File "/home/sjpark/linux/tools/testing/kunit/kunit_parser.py", line 54, in isolate_kunit_output for line in kernel_output: File "/home/sjpark/linux/tools/testing/kunit/kunit_kernel.py", line 145, in run_kernel process = self._ops.linux_bin(args, timeout, build_dir) File "/home/sjpark/linux/tools/testing/kunit/kunit_kernel.py", line 69, in linux_bin stderr=subprocess.PIPE) File "/usr/lib/python3.5/subprocess.py", line 947, in __init__ restore_signals, start_new_session) File "/usr/lib/python3.5/subprocess.py", line 1551, in _execute_child raise child_exception_type(errno_num, err_msg) FileNotFoundError: [Errno 2] No such file or directory: './linux' ``` This error occurs because the '--build_dir' option value is not passed to the 'run_kernel()' function. Consequently, the function assumes the kernel image that built for the tests, which is under the '--build_dir' directory, is in kernel source directory and finally raises the 'FileNotFoundError'. This commit fixes the problem by properly passing the '--build_dir' option value to the 'run_kernel()'. Signed-off-by: SeongJae Park <sj38.park@gmail.com> Reviewed-by: Brendan Higgins <brendanhiggins@google.com> Tested-by: Brendan Higgins <brendanhiggins@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2019-10-01kunit: defconfig: add defconfigs for building KUnit testsBrendan Higgins1-4/+24
Add defconfig for UML and a fragment that can be used to configure other architectures for building KUnit tests. Add option to kunit_tool to use a defconfig to create the kunitconfig. Signed-off-by: Brendan Higgins <brendanhiggins@google.com> Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> Reviewed-by: Logan Gunthorpe <logang@deltatee.com> Reviewed-by: Stephen Boyd <sboyd@kernel.org> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
2019-10-01kunit: tool: add Python wrappers for running KUnit testsFelix Guo1-0/+116
The ultimate goal is to create minimal isolated test binaries; in the meantime we are using UML to provide the infrastructure to run tests, so define an abstract way to configure and run tests that allow us to change the context in which tests are built without affecting the user. This also makes pretty and dynamic error reporting, and a lot of other nice features easier. kunit_config.py: - parse .config and Kconfig files. kunit_kernel.py: provides helper functions to: - configure the kernel using kunitconfig. - build the kernel with the appropriate configuration. - provide function to invoke the kernel and stream the output back. kunit_parser.py: parses raw logs returned out by kunit_kernel and displays them in a user friendly way. test_data/*: samples of test data for testing kunit.py, kunit_config.py, etc. Signed-off-by: Felix Guo <felixguoxiuping@gmail.com> Signed-off-by: Brendan Higgins <brendanhiggins@google.com> Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> Reviewed-by: Logan Gunthorpe <logang@deltatee.com> Reviewed-by: Stephen Boyd <sboyd@kernel.org> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>