kernel/linux.git/scripts/clang-tools, branch v6.12.80

gen_compile_commands: fix invalid escape sequence warning

2024-02-14T21:57:19+00:00

With python 3.12, '\#' results in this warning SyntaxWarning: invalid escape sequence '\#' Signed-off-by: Andrew Ballance Reviewed-by: Justin Stitt Signed-off-by: Masahiro Yamada

gen_compile_commands.py: fix path resolve with symlinks in it

2023-12-23T14:24:03+00:00

When a path contains relative symbolic links, os.path.abspath() might not follow the symlinks and instead return the absolute path with just the relative paths resolved, resulting in an incorrect path. 1. Say "drivers/hdf/" has some symlinks: # ls -l drivers/hdf/ total 364 drwxrwxr-x 2 ... 4096 ... evdev lrwxrwxrwx 1 ... 44 ... framework -> ../../../../../../drivers/hdf_core/framework -rw-rw-r-- 1 ... 359010 ... hdf_macro_test.h lrwxrwxrwx 1 ... 55 ... inner_api -> ../../../../../../drivers/hdf_core/interfaces/inner_api lrwxrwxrwx 1 ... 53 ... khdf -> ../../../../../../drivers/hdf_core/adapter/khdf/linux -rw-r--r-- 1 ... 74 ... Makefile drwxrwxr-x 3 ... 4096 ... wifi 2. One .cmd file records that: # head -1 ./framework/core/manager/src/.devmgr_service.o.cmd cmd_drivers/hdf/khdf/manager/../../../../framework/core/manager/src/devmgr_service.o := ... \ /path/to/src/drivers/hdf/khdf/manager/../../../../framework/core/manager/src/devmgr_service.c 3. os.path.abspath returns "/path/to/src/framework/core/manager/src/devmgr_service.c", not correct: # ./scripts/clang-tools/gen_compile_commands.py INFO: Could not add line from ./framework/core/manager/src/.devmgr_service.o.cmd: File \ /path/to/src/framework/core/manager/src/devmgr_service.c not found Use os.path.realpath(), which resolves the symlinks and normalizes the paths correctly. # cat compile_commands.json ... { "command": ... "directory": ... "file": "/path/to/bla/drivers/hdf_core/framework/core/manager/src/devmgr_service.c" }, ... Also fix it in parse_arguments(). Signed-off-by: Jialu Xu Signed-off-by: Masahiro Yamada

run-clang-tools: Add pass through checks and and header-filter arguments

2023-10-12T17:01:55+00:00

Add a -checks argument to allow the checks passed to the clang-tool to be set on the command line. Add a pass through -header-filter option. Don't run analysis on non-C or CPP files. Signed-off-by: Ian Rogers Reviewed-by: Nick Desaulniers Cc: Ravi Bangoria Cc: Mark Rutland Cc: Yang Jihong Cc: Peter Zijlstra Cc: Adrian Hunter Cc: Arnaldo Carvalho de Melo Cc: Huacai Chen Cc: Jiri Olsa Cc: Nathan Chancellor Cc: Alexander Shishkin Cc: Kan Liang Cc: llvm@lists.linux.dev Cc: Ming Wang Cc: Ingo Molnar Cc: Tom Rix Cc: bpf@vger.kernel.org Cc: linux-kernel@vger.kernel.org Cc: linux-perf-users@vger.kernel.org Link: https://lore.kernel.org/r/20231009183920.200859-4-irogers@google.com Signed-off-by: Namhyung Kim

gen_compile_commands: Sort output compile commands by file name

2023-10-12T17:01:55+00:00

Make the output more stable and deterministic. Signed-off-by: Ian Rogers Reviewed-by: Nick Desaulniers Cc: Ravi Bangoria Cc: Mark Rutland Cc: Yang Jihong Cc: Peter Zijlstra Cc: Adrian Hunter Cc: Arnaldo Carvalho de Melo Cc: Huacai Chen Cc: Jiri Olsa Cc: Nathan Chancellor Cc: Alexander Shishkin Cc: Kan Liang Cc: llvm@lists.linux.dev Cc: Ming Wang Cc: Ingo Molnar Cc: Tom Rix Cc: bpf@vger.kernel.org Cc: linux-kernel@vger.kernel.org Cc: linux-perf-users@vger.kernel.org Link: https://lore.kernel.org/r/20231009183920.200859-3-irogers@google.com Signed-off-by: Namhyung Kim

gen_compile_commands: Allow the line prefix to still be cmd_

2023-10-12T17:01:55+00:00

Builds in tools still use the cmd_ prefix in .cmd files, so don't require the saved part. Name the groups in the line pattern match so that changing the regular expression is more robust and works with the addition of a new match group. Signed-off-by: Ian Rogers Reviewed-by: Nick Desaulniers Cc: Ravi Bangoria Cc: Mark Rutland Cc: Yang Jihong Cc: Peter Zijlstra Cc: Adrian Hunter Cc: Arnaldo Carvalho de Melo Cc: Huacai Chen Cc: Jiri Olsa Cc: Nathan Chancellor Cc: Alexander Shishkin Cc: Kan Liang Cc: llvm@lists.linux.dev Cc: Ming Wang Cc: Ingo Molnar Cc: Tom Rix Cc: bpf@vger.kernel.org Cc: linux-kernel@vger.kernel.org Cc: linux-perf-users@vger.kernel.org Link: https://lore.kernel.org/r/20231009183920.200859-2-irogers@google.com Signed-off-by: Namhyung Kim

gen_compile_commands: add assembly files to compilation database

2023-07-23T13:36:07+00:00

Like C source files, tooling can find it useful to have the assembly source file compilation recorded. The .S extension appears to used across all architectures. Signed-off-by: Benjamin Gray Reviewed-by: Fangrui Song Reviewed-by: Nathan Chancellor Reviewed-by: Nick Desaulniers Signed-off-by: Masahiro Yamada

scripts: handle BrokenPipeError for python scripts

2023-01-26T03:43:33+00:00

In the follow-up of commit fb3041d61f68 ("kbuild: fix SIGPIPE error message for AR=gcc-ar and AR=llvm-ar"), Kees Cook pointed out that tools should _not_ catch their own SIGPIPEs [1] [2]. Based on his feedback, LLVM was fixed [3]. However, Python's default behavior is to show noisy bracktrace when SIGPIPE is sent. So, scripts written in Python are basically in the same situation as the buggy llvm tools. Example: $ make -s allnoconfig $ make -s allmodconfig $ scripts/diffconfig .config.old .config | head -n1 -ALIX n Traceback (most recent call last): File "/home/masahiro/linux/scripts/diffconfig", line 132, in main() File "/home/masahiro/linux/scripts/diffconfig", line 130, in main print_config("+", config, None, b[config]) File "/home/masahiro/linux/scripts/diffconfig", line 64, in print_config print("+%s %s" % (config, new_value)) BrokenPipeError: [Errno 32] Broken pipe Python documentation [4] notes how to make scripts die immediately and silently: """ Piping output of your program to tools like head(1) will cause a SIGPIPE signal to be sent to your process when the receiver of its standard output closes early. This results in an exception like BrokenPipeError: [Errno 32] Broken pipe. To handle this case, wrap your entry point to catch this exception as follows: import os import sys def main(): try: # simulate large output (your code replaces this loop) for x in range(10000): print("y") # flush output here to force SIGPIPE to be triggered # while inside this try block. sys.stdout.flush() except BrokenPipeError: # Python flushes standard streams on exit; redirect remaining output # to devnull to avoid another BrokenPipeError at shutdown devnull = os.open(os.devnull, os.O_WRONLY) os.dup2(devnull, sys.stdout.fileno()) sys.exit(1) # Python exits with error code 1 on EPIPE if __name__ == '__main__': main() Do not set SIGPIPE’s disposition to SIG_DFL in order to avoid BrokenPipeError. Doing that would cause your program to exit unexpectedly whenever any socket connection is interrupted while your program is still writing to it. """ Currently, tools/perf/scripts/python/intel-pt-events.py seems to be the only script that fixes the issue that way. tools/perf/scripts/python/compaction-times.py uses another approach signal.signal(signal.SIGPIPE, signal.SIG_DFL) but the Python documentation clearly says "Don't do it". I cannot fix all Python scripts since there are so many. I fixed some in the scripts/ directory. [1]: https://lore.kernel.org/all/202211161056.1B9611A@keescook/ [2]: https://github.com/llvm/llvm-project/issues/59037 [3]: https://github.com/llvm/llvm-project/commit/4787efa38066adb51e2c049499d25b3610c0877b [4]: https://docs.python.org/3/library/signal.html#note-on-sigpipe Signed-off-by: Masahiro Yamada Reviewed-by: Nick Desaulniers Reviewed-by: Nicolas Schier

kbuild: rename cmd_$@ to savedcmd_$@ in *.cmd files

2023-01-22T14:43:33+00:00

The cmd-check macro compares $(cmd_$@) and $(cmd_$1), but a pitfall is that you cannot use cmd_ as the variable name for the command. For example, the following code will not work in the top Makefile or ./Kbuild. quiet_cmd_foo = GEN $@ cmd_foo = touch $@ targets += foo foo: FORCE $(call if_changed,foo) In this case, both $@ and $1 are expanded to 'foo', so $(cmd_check) is always empty. We do not need to use the same prefix for cmd_$@ and cmd_$1. Rename the former to savedcmd_$@. Signed-off-by: Masahiro Yamada Reviewed-by: Nicolas Schier

kbuild: change module.order to list .o instead of .ko

2022-12-14T06:42:40+00:00

scripts/Makefile.build replaces the suffix .o with .ko, then scripts/Makefile.modpost calls the sed command to change .ko back to the original .o suffix. Instead of converting the suffixes back-and-forth, store the .o paths in modules.order, and replace it with .ko in 'make modules_install'. This avoids the unneeded sed command. Signed-off-by: Masahiro Yamada Reviewed-by: Luis Chamberlain

scripts/clang-tools: Convert clang-tidy args to list

2022-10-14T13:32:52+00:00

Convert list of clang-tidy arguments to a list for ease of adding to them and extending them as required. Signed-off-by: Guru Das Srinagesh Suggested-by: Nick Desaulniers Reviewed-by: Nick Desaulniers Signed-off-by: Masahiro Yamada

kernel/linux.git/scripts/clang-tools, branch v6.12.80

gen_compile_commands: fix invalid escape sequence warning

gen_compile_commands.py: fix path resolve with symlinks in it

run-clang-tools: Add pass through checks and and header-filter arguments

gen_compile_commands: Sort output compile commands by file name

gen_compile_commands: Allow the line prefix to still be cmd_

gen_compile_commands: add assembly files to compilation database

scripts: handle BrokenPipeError for python scripts

kbuild: rename cmd_$@ to savedcmd_$@ in *.cmd files

kbuild: change module.order to list *.o instead of *.ko

scripts/clang-tools: Convert clang-tidy args to list

kbuild: change module.order to list .o instead of .ko