kernel/linux.git/tools/testing/selftests/lib.mk, branch v4.17.5

selftests: Fix lib.mk run_tests target shell script

2018-04-27T22:06:36+00:00

Within run_tests target, the whole script needs to be executed within the same shell and not as separate subshells, so the initial test_num variable set to 0 is still present when executing "test_num=`echo $$test_num+1 | bc`;". Demonstration of the issue (make run_tests): TAP version 13 (standard_in) 1: syntax error selftests: basic_test ======================================== ok 1.. selftests: basic_test [PASS] (standard_in) 1: syntax error selftests: basic_percpu_ops_test ======================================== ok 1.. selftests: basic_percpu_ops_test [PASS] (standard_in) 1: syntax error selftests: param_test ======================================== ok 1.. selftests: param_test [PASS] With fix applied: TAP version 13 selftests: basic_test ======================================== ok 1..1 selftests: basic_test [PASS] selftests: basic_percpu_ops_test ======================================== ok 1..2 selftests: basic_percpu_ops_test [PASS] selftests: param_test ======================================== ok 1..3 selftests: param_test [PASS] Signed-off-by: Mathieu Desnoyers Fixes: 1f87c7c15d7 ("selftests: lib.mk: change RUN_TESTS to print messages in TAP13 format") CC: Shuah Khan CC: linux-kselftest@vger.kernel.org Signed-off-by: Shuah Khan (Samsung OSG)

selftests: lib.mk set KSFT_TAP_LEVEL to prevent nested TAP headers

2018-03-06T02:10:35+00:00

Set KSFT_TAP_LEVEL before running tests to prevent nested TAP header printing from tests. Signed-off-by: Shuah Khan

selftests: Fix loss of test output in run_kselftests.sh

2018-01-16T16:31:31+00:00

Commit fbcab13d2e25 ("selftests: silence test output by default") changed the run_tests logic as well as the logic to generate run_kselftests.sh to redirect test output away from the console. As discussed on the list and at kernel summit, this is not a desirable default as it means in order to debug a failure the console output is not sufficient, you also need access to the test machine to get the full test logs. Additionally it's impolite to write directly to /tmp/$TEST_NAME on shared systems. The change to the run_tests logic was reverted in commit a323335e62cc ("selftests: lib.mk: print individual test results to console by default"), and instead a summary option was added so that quiet output could be requested. However the change to run_kselftests.sh was left as-is. This commit applies the same logic to the run_kselftests.sh code, ie. the script now takes a "--summary" option which suppresses the output, but shows all output by default. Additionally instead of writing to /tmp/$TEST_NAME the output is redirected to the directory where the generated test script is located. Fixes: fbcab13d2e25 ("selftests: silence test output by default") Signed-off-by: Michael Ellerman Signed-off-by: Shuah Khan

selftests: lib.mk: print individual test results to console by default

2017-10-31T19:24:00+00:00

Change run_tests to print individual test results to console by default. Introduce "summary" option to print individual test results to a file /tmp/test_name and just print the summary to the console. This change is necessary to support use-cases where test machines get rebooted once tests are run and the console log should contain the full results. In the following example, individual test results with "summary=1" option are written to /tmp/kcmp_test make --silent TARGETS=kcmp kselftest TAP version 13 selftests: kcmp_test ======================================== pid1: 30126 pid2: 30127 FD: 2 FILES: 2 VM: 1 FS: 2 SIGHAND: 2 IO: 0 SYSVSEM: 0 INV: -1 PASS: 0 returned as expected PASS: 0 returned as expected FAIL: 0 expected but -1 returned (Invalid argument) Pass 2 Fail 1 Xfail 0 Xpass 0 Skip 0 Error 0 1..3 Bail out! Pass 2 Fail 1 Xfail 0 Xpass 0 Skip 0 Error 0 1..3 Pass 0 Fail 0 Xfail 0 Xpass 0 Skip 0 Error 0 1..0 ok 1..1 selftests: kcmp_test [PASS] make --silent TARGETS=kcmp summary=1 kselftest TAP version 13 selftests: kcmp_test ======================================== ok 1..1 selftests: kcmp_test [PASS] Signed-off-by: Shuah Khan

selftests: lib.mk: copy test scripts and test files for make O=dir run

2017-09-21T13:55:40+00:00

For make O=dir run_tests to work, test scripts, test files, and other dependencies need to be copied over to the object directory. Running tests from the object directory is necessary to avoid making the source tree dirty. Signed-off-by: Shuah Khan

selftests: lib.mk: add TEST_CUSTOM_PROGS to allow custom test run/install

2017-09-21T13:55:39+00:00

Some tests such as sync can't use generic build rules in lib.mk and require custom rules. Currently there is no provision to allow custom builds and test such as sync use TEST_PROGS which is reserved for test shell scripts. Add TEST_CUSTOM_PROGS variable to lib.mk to run and install custom tests built by individual test make files. Signed-off-by: Shuah Khan

selftests: lib.mk: fix test executable status check to use full path

2017-09-21T13:55:37+00:00

Fix test executable status check to use full path for make O=dir case,m when tests are relocated to user specified object directory. Without the full path, this check fails to find the file and fails the test. Signed-off-by: Shuah Khan

selftests: lib.mk: kselftest and kselftest-clean fail for make O=dir case

2017-09-21T13:55:36+00:00

kselftest and kselftest-clean targets fail when object directory is specified to relocate objects. Main Makefile make O= path clears the built-in defines LINK.c, COMPILE.S, LINK.S, and RM that are used in lib.mk to build and clean targets. Define them. Signed-off-by: Shuah Khan

selftests: silence test output by default

2017-09-19T20:08:50+00:00

Some of the networking tests are very noisy and make it impossible to see if we actually passed the tests as they run. Default to suppressing the output from any tests run in order to make it easier to track what failed. Signed-off-by: Josef Bacik Signed-off-by: Shuah Khan

selftests: lib.mk: change RUN_TESTS to print messages in TAP13 format

2017-08-25T16:17:12+00:00

Change common RUN_TESTS to print messages in user friendly TAP13 format. This change add TAP13 header at the start of RUN_TESTS target run, and prints the resulting pass/fail messages with test number information in the TAP 13 format for each test in the run tests list. This change covers test scripts as well as test programs. Test programs have an option to use ksft_ API, however test scripts won't be able to. With this change, test scripts can print TAP13 format output without any changes to individual scripts. Test programs can provide TAP13 format output as needed as some tests already do. Tests that haven't been converted will benefit from this change. Tests that are converted benefit from the test counts for all the tests in each test directory. Running firmware tests: make --silent -C tools/testing/selftests/firmware/ run_tests Before the change: modprobe: ERROR: could not insert 'test_firmware': Operation not permitted ./fw_filesystem.sh: /sys/devices/virtual/misc/test_firmware not present You must have the following enabled in your kernel: CONFIG_TEST_FIRMWARE=y selftests: fw_filesystem.sh [FAIL] modprobe: ERROR: could not insert 'test_firmware': Operation not permitted selftests: fw_fallback.sh [FAIL] After the change: TAP version 13 selftests: fw_filesystem.sh ======================================== modprobe: ERROR: could not insert 'test_firmware': Operation not permitted ./fw_filesystem.sh: /sys/devices/virtual/misc/test_firmware not present You must have the following enabled in your kernel: CONFIG_TEST_FIRMWARE=y not ok 1..1 selftests: fw_filesystem.sh [FAIL] selftests: fw_fallback.sh ======================================== modprobe: ERROR: could not insert 'test_firmware': Operation not permitted not ok 1..2 selftests: fw_fallback.sh [FAIL] Signed-off-by: Shuah Khan