kernel/linux.git/Documentation/sphinx/kernel_abi.py, branch v6.18.21

sphinx: kernel_abi: fix performance regression with O=

2025-07-24T14:36:17+00:00

The logic there which adds a dependency note to Sphinx cache is not taking into account that the build dir may not be the source dir. This causes a performance regression: $ time make O=/tmp/foo SPHINXDIRS=admin-guide htmldocs [OUTDATED] Added: set() Changed: {'abi-obsolete', 'abi-removed', 'abi-stable-files', 'abi-obsolete-files', 'abi-stable', 'abi', 'abi-removed-files', 'abi-testing-files', 'abi-testing', 'gpio/index', 'gpio/obsolete'} Removed: set() All docs count: 385 Found docs count: 385 real 0m11,324s user 0m15,783s sys 0m1,164s To get the root cause of the problem (ABI files reported as changed), I used this changeset: diff --git a/Documentation/conf.py b/Documentation/conf.py index e8766e689c1b..ab486623bd8b 100644 --- a/Documentation/conf.py +++ b/Documentation/conf.py @@ -571,3 +571,16 @@ def setup(app): """Patterns need to be updated at init time on older Sphinx versions""" app.connect('config-inited', update_patterns) + app.connect('env-get-outdated', on_outdated) + +def on_outdated(app, env, added, changed, removed): + """Track cache outdated due to added/changed/removed files""" + print("\n[OUTDATED]") + print(f"Added: {added}") + print(f"Changed: {changed}") + print(f"Removed: {removed}") + print(f"All docs count: {len(env.all_docs)}") + print(f"Found docs count: {len(env.found_docs)}") + + # Just return what we have + return added | changed | removed Reported-by: Akira Yokosawa Closes: https://lore.kernel.org/linux-doc/c174f7c5-ec21-4eae-b1c3-f643cca90d9d@gmail.com/ Signed-off-by: Mauro Carvalho Chehab Tested-by: Akira Yokosawa Signed-off-by: Jonathan Corbet Link: https://lore.kernel.org/r/e25673d87357457bc54ee863e97ff8f75956580d.1752752211.git.mchehab+huawei@kernel.org

docs: extensions: don't use utf-8 syntax for descriptions

2025-02-13T18:33:32+00:00

None of the descriptions at the Sphinx extensions are using non-ascii characters, so no need to place them under u""" notation. Also, according with: https://docs.python.org/3/deprecations/pending-removal-in-3.16.html the 'u' format code is scheduled to be removed in Python 3.16. So, let's just use """. Signed-off-by: Mauro Carvalho Chehab Reviewed-by: Kees Cook Signed-off-by: Jonathan Corbet Link: https://lore.kernel.org/r/8a42f6be53464af4b866492a7e9ddf29c0429997.1739254187.git.mchehab+huawei@kernel.org

docs: sphinx/kernel_abi: avoid warnings during Sphinx module init

2025-02-10T18:19:57+00:00

Sphinx logging system doesn't like warnings during module load, as it understands that such logs are produced at the wrong time: WARNING: while setting up extension automarkup: /sys/devices/system/cpu/cpuX/topology/physical_package_id is defined 2 times: /new_devel/v4l/docs/Documentation/ABI/stable/sysfs-devices-system-cpu:27; /new_devel/v4l/docs/Documentation/ABI/testing/sysfs-devices-system-cpu:70 WARNING: while setting up extension automarkup: /sys/devices/system/cpu/cpuX/topology/ppin is defined 2 times: /new_devel/v4l/docs/Documentation/ABI/stable/sysfs-devices-system-cpu:89; /new_devel/v4l/docs/Documentation/ABI/testing/sysfs-devices-system-cpu:70 So, use a function to allocate/process ABI files and use it to be called at kernel_abi.py, as automarkup also needs it to produce the right cross-references. Signed-off-by: Mauro Carvalho Chehab Signed-off-by: Jonathan Corbet Link: https://lore.kernel.org/r/b0e79dc60d556e3b39fa6774d3b7bf734b73f352.1739182025.git.mchehab+huawei@kernel.org

docs: sphinx/kernel_abi: parse ABI files only once

2025-02-10T18:19:57+00:00

Right now, the logic parses ABI files on 4 steps, one for each directory. While this is fine in principle, by doing that, not all symbol cross-references will be created. Change the logic to do the parsing only once in order to get a global dictionary to be used when creating ABI cross-references. Signed-off-by: Mauro Carvalho Chehab Signed-off-by: Jonathan Corbet Link: https://lore.kernel.org/r/5205c53838b6ea25f4cdd4cc1e3d17c0141e75a6.1739182025.git.mchehab+huawei@kernel.org

docs: sphinx/kernel_abi: properly split lines

2025-02-10T18:19:57+00:00

Sphinx doesn't like to have lines split with str.split("\n"). Instead, it uses its own splitter, with handles line breaks the way Spinx expects. Not using it cause issues at the output files. Signed-off-by: Mauro Carvalho Chehab Signed-off-by: Jonathan Corbet Link: https://lore.kernel.org/r/d4ad5b977799616544376210364d5cec686119ef.1739182025.git.mchehab+huawei@kernel.org

docs: sphinx/kernel_abi: reduce buffer usage for ABI messages

2025-02-10T18:19:57+00:00

Instead of producing a big message with all ABI contents and then parse as a whole, simplify the code by handling each ABI symbol in separate. As an additional benefit, there's no need to place file/line nubers inlined at the data and use a regex to convert them. Signed-off-by: Mauro Carvalho Chehab Signed-off-by: Jonathan Corbet Link: https://lore.kernel.org/r/15be22955e3c6df49d7256c8fd24f62b397ad0ff.1739182025.git.mchehab+huawei@kernel.org

docs: sphinx/kernel_abi: use AbiParser directly

2025-02-10T18:19:57+00:00

Instead of running get_abi.py script, import AbiParser class and handle messages directly there using an interactor. This shold save some memory, as there's no need to exec python inside the Sphinx python extension. Signed-off-by: Mauro Carvalho Chehab Signed-off-by: Jonathan Corbet Link: https://lore.kernel.org/r/8dbc244dcda97112c1b694e2512a5d600e62873b.1739182025.git.mchehab+huawei@kernel.org

docs: use get_abi.py for ABI generation

2025-02-10T18:19:56+00:00

Use the new script instead of the old one when generating ABI docs. For now, execute it via exec. Future changes will better integrate it by using the class defined there directly. Signed-off-by: Mauro Carvalho Chehab Signed-off-by: Jonathan Corbet Link: https://lore.kernel.org/r/e7fcb121c0612c94f6f54f0d742cd3a26a46cd7d.1739182025.git.mchehab+huawei@kernel.org

docs: sphinx/kernel_abi: adjust coding style

2025-02-10T18:19:56+00:00

Make pylint and flake8 happier with this module's coding style Signed-off-by: Mauro Carvalho Chehab Signed-off-by: Jonathan Corbet Link: https://lore.kernel.org/r/02a9ec0fab61e4c94b9c2ff555bc0e9d93f59100.1739182025.git.mchehab+huawei@kernel.org

docs: sphinx: remove kernellog.py file

2025-02-10T18:19:56+00:00

In the past, there was a need for a wrapper due to different Sphinx versions support (before Sphinx 1.6). This is long gone, and now it is just a wrapper. Get rig of it to simplify the code. Signed-off-by: Mauro Carvalho Chehab Signed-off-by: Jonathan Corbet Link: https://lore.kernel.org/r/48bf16f199250d7048ca164d1b90773861915157.1739182025.git.mchehab+huawei@kernel.org