From 212fa2f84dbf36682ee6491f203471a27997043e Mon Sep 17 00:00:00 2001 From: Mauro Carvalho Chehab Date: Fri, 14 Jul 2017 13:41:16 -0300 Subject: changes.rst: Update Sphinx minimal requirements The kfigure module doesn't work with Sphinx version 1.2. So, update the minimal requirements accordingly. Signed-off-by: Mauro Carvalho Chehab Signed-off-by: Jonathan Corbet --- Documentation/process/changes.rst | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/Documentation/process/changes.rst b/Documentation/process/changes.rst index adbb50ae5246..ceddf1d1e646 100644 --- a/Documentation/process/changes.rst +++ b/Documentation/process/changes.rst @@ -53,7 +53,7 @@ mcelog 0.6 mcelog --version iptables 1.4.2 iptables -V openssl & libcrypto 1.0.0 openssl version bc 1.06.95 bc --version -Sphinx\ [#f1]_ 1.2 sphinx-build --version +Sphinx\ [#f1]_ 1.3 sphinx-build --version ====================== =============== ======================================== .. [#f1] Sphinx is needed only to build the Kernel documentation @@ -310,8 +310,8 @@ Sphinx ------ The ReST markups currently used by the Documentation/ files are meant to be -built with ``Sphinx`` version 1.2 or upper. If you're desiring to build -PDF outputs, it is recommended to use version 1.4.6. +built with ``Sphinx`` version 1.3 or upper. If you're desiring to build +PDF outputs, it is recommended to use version 1.4.6 or upper. .. note:: -- cgit v1.2.3 From b8b07b5c8dfc1fdf2f93d7419e0cac1034abc937 Mon Sep 17 00:00:00 2001 From: Mauro Carvalho Chehab Date: Fri, 14 Jul 2017 13:41:17 -0300 Subject: docs-rst: move Sphinx install instructions to sphinx.rst The toolchain used by Sphinx is somewhat complex, and installing it should be part of the doc-guide. Move it out of changes.rst. Signed-off-by: Mauro Carvalho Chehab Signed-off-by: Jonathan Corbet --- Documentation/doc-guide/sphinx.rst | 17 +++++++++++++++++ Documentation/process/changes.rst | 14 ++------------ 2 files changed, 19 insertions(+), 12 deletions(-) diff --git a/Documentation/doc-guide/sphinx.rst b/Documentation/doc-guide/sphinx.rst index 84e8e8a9cbdb..1ee109b19322 100644 --- a/Documentation/doc-guide/sphinx.rst +++ b/Documentation/doc-guide/sphinx.rst @@ -19,6 +19,23 @@ Finally, there are thousands of plain text documentation files scattered around ``Documentation``. Some of these will likely be converted to reStructuredText over time, but the bulk of them will remain in plain text. +.. _sphinx_install: + +Sphinx Install +============== + +The ReST markups currently used by the Documentation/ files are meant to be +built with ``Sphinx`` version 1.3 or upper. If you're desiring to build +PDF outputs, it is recommended to use version 1.4.6 or upper. + +.. note:: + + Please notice that, for PDF and LaTeX output, you'll also need ``XeLaTeX`` + version 3.14159265. Depending on the distribution, you may also need to + install a series of ``texlive`` packages that provide the minimal set of + functionalities required for ``XeLaTex`` to work. For PDF output you'll also + need ``convert(1)`` from ImageMagick (https://www.imagemagick.org). + Sphinx Build ============ diff --git a/Documentation/process/changes.rst b/Documentation/process/changes.rst index ceddf1d1e646..560beaef5a7c 100644 --- a/Documentation/process/changes.rst +++ b/Documentation/process/changes.rst @@ -309,18 +309,8 @@ Kernel documentation Sphinx ------ -The ReST markups currently used by the Documentation/ files are meant to be -built with ``Sphinx`` version 1.3 or upper. If you're desiring to build -PDF outputs, it is recommended to use version 1.4.6 or upper. - -.. note:: - - Please notice that, for PDF and LaTeX output, you'll also need ``XeLaTeX`` - version 3.14159265. Depending on the distribution, you may also need to - install a series of ``texlive`` packages that provide the minimal set of - functionalities required for ``XeLaTex`` to work. For PDF output you'll also - need ``convert(1)`` from ImageMagick (https://www.imagemagick.org). - +Please see :ref:`sphinx_install` in ``Documentation/doc-guide/sphinx.rst`` +for details about Sphinx requirements. Getting updated software ======================== -- cgit v1.2.3 From 58ef4a42dd42580945985c533d4d5c3d6628ef92 Mon Sep 17 00:00:00 2001 From: Mauro Carvalho Chehab Date: Fri, 14 Jul 2017 13:41:18 -0300 Subject: sphinx.rst: explain the usage of virtual environment As the Sphinx build seems very fragile, specially for PDF output, add a notice about how to use it on a virtual environment. Signed-off-by: Mauro Carvalho Chehab Signed-off-by: Jonathan Corbet --- Documentation/doc-guide/sphinx.rst | 32 ++++++++++++++++++++++++++++++++ 1 file changed, 32 insertions(+) diff --git a/Documentation/doc-guide/sphinx.rst b/Documentation/doc-guide/sphinx.rst index 1ee109b19322..3278262164da 100644 --- a/Documentation/doc-guide/sphinx.rst +++ b/Documentation/doc-guide/sphinx.rst @@ -28,6 +28,38 @@ The ReST markups currently used by the Documentation/ files are meant to be built with ``Sphinx`` version 1.3 or upper. If you're desiring to build PDF outputs, it is recommended to use version 1.4.6 or upper. +Most distributions are shipped with Sphinx, but its toolchain is fragile, +and it is not uncommon that upgrading it or some other Python packages +on your machine would cause the documentation build to break. + +A way to get rid of that is to use a different version than the one shipped +on your distributions. In order to do that, it is recommended to install +Sphinx inside a virtual environment, using ``virtualenv-3`` +or ``virtualenv``, depending on how your distribution packaged Python 3. + +.. note:: + + #) Sphinx versions below 1.5 don't work properly with Python's + docutils version 0.13.1 or upper. So, if you're willing to use + those versions, you should run ``pip install 'docutils==0.12'``. + + #) It is recommended to use the RTD theme for html output. Depending + on the Sphinx version, it should be installed in separate, + with ``pip install sphinx_rtd_theme``. + +In summary, if you want to install Sphinx version 1.4.9, you should do:: + + $ virtualenv sphinx_1.4 + $ . sphinx_1.4/bin/activate + (sphinx_1.4) $ pip install 'docutils==0.12' + (sphinx_1.4) $ pip install 'Sphinx==1.4.9' + (sphinx_1.4) $ pip install sphinx_rtd_theme + +After running ``. sphinx_1.4/bin/activate``, the prompt will change, +in order to indicate that you're using the new environment. If you +open a new shell, you need to rerun this command to enter again at +the virtual environment before building the documentation. + .. note:: Please notice that, for PDF and LaTeX output, you'll also need ``XeLaTeX`` -- cgit v1.2.3 From 29fd35bd02568f27b463657218889276215b1662 Mon Sep 17 00:00:00 2001 From: Mauro Carvalho Chehab Date: Fri, 14 Jul 2017 13:41:19 -0300 Subject: sphinx.rst: fix unknown reference There's no "Sphinx C Domain" reference at the Kernel documentation. So, don't use references for it. Signed-off-by: Mauro Carvalho Chehab Signed-off-by: Jonathan Corbet --- Documentation/doc-guide/sphinx.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Documentation/doc-guide/sphinx.rst b/Documentation/doc-guide/sphinx.rst index 3278262164da..4c6cb454eaa0 100644 --- a/Documentation/doc-guide/sphinx.rst +++ b/Documentation/doc-guide/sphinx.rst @@ -167,7 +167,7 @@ Here are some specific guidelines for the kernel documentation: the C domain ------------ -The `Sphinx C Domain`_ (name c) is suited for documentation of C API. E.g. a +The **Sphinx C Domain** (name c) is suited for documentation of C API. E.g. a function prototype: .. code-block:: rst -- cgit v1.2.3 From d43e5ae9748f327c60014973e3e290a6f02577ea Mon Sep 17 00:00:00 2001 From: Mauro Carvalho Chehab Date: Fri, 14 Jul 2017 13:41:20 -0300 Subject: sphinx.rst: describe the install requirements for kfigure As we now have a document describing the install requirements for Sphinx, add there the need for GraphViz and ImageMagick. Signed-off-by: Mauro Carvalho Chehab Signed-off-by: Jonathan Corbet --- Documentation/doc-guide/sphinx.rst | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/Documentation/doc-guide/sphinx.rst b/Documentation/doc-guide/sphinx.rst index 4c6cb454eaa0..3a41483ed499 100644 --- a/Documentation/doc-guide/sphinx.rst +++ b/Documentation/doc-guide/sphinx.rst @@ -68,6 +68,18 @@ the virtual environment before building the documentation. functionalities required for ``XeLaTex`` to work. For PDF output you'll also need ``convert(1)`` from ImageMagick (https://www.imagemagick.org). +Image output +------------ + +The kernel documentation build system contains an extension that +handles images on both GraphViz and SVG formats (see +:ref:`sphinx_kfigure`). + +For it to work, you need to install both GraphViz and ImageMagick +packages. If those packages are not installed, the build system will +still build the documentation, but won't include any images at the +output. + Sphinx Build ============ @@ -278,6 +290,7 @@ Rendered as: - column 3 +.. _sphinx_kfigure: Figures & Images ================ -- cgit v1.2.3 From 6e322a17fbe574a1558abab10bb89205031ec921 Mon Sep 17 00:00:00 2001 From: Mauro Carvalho Chehab Date: Fri, 14 Jul 2017 13:41:21 -0300 Subject: sphinx.rst: better organize the documentation about PDF build Instead of having it on just one note, add a separate section. This way, we could later improve it, providing a better guide about the needed steps for PDF builds. Signed-off-by: Mauro Carvalho Chehab Signed-off-by: Jonathan Corbet --- Documentation/doc-guide/sphinx.rst | 21 +++++++++++++-------- 1 file changed, 13 insertions(+), 8 deletions(-) diff --git a/Documentation/doc-guide/sphinx.rst b/Documentation/doc-guide/sphinx.rst index 3a41483ed499..5521213efa68 100644 --- a/Documentation/doc-guide/sphinx.rst +++ b/Documentation/doc-guide/sphinx.rst @@ -60,14 +60,6 @@ in order to indicate that you're using the new environment. If you open a new shell, you need to rerun this command to enter again at the virtual environment before building the documentation. -.. note:: - - Please notice that, for PDF and LaTeX output, you'll also need ``XeLaTeX`` - version 3.14159265. Depending on the distribution, you may also need to - install a series of ``texlive`` packages that provide the minimal set of - functionalities required for ``XeLaTex`` to work. For PDF output you'll also - need ``convert(1)`` from ImageMagick (https://www.imagemagick.org). - Image output ------------ @@ -80,6 +72,19 @@ packages. If those packages are not installed, the build system will still build the documentation, but won't include any images at the output. +PDF and LaTeX builds +-------------------- + +Such builds are currently supported only with Sphinx versions 1.4 and 1.5. + +Currently, it is not possible to do pdf builds with Sphinx version 1.6. + +For PDF and LaTeX output, you'll also need ``XeLaTeX`` version 3.14159265. + +Depending on the distribution, you may also need to install a series of +``texlive`` packages that provide the minimal set of functionalities +required for ``XeLaTex`` to work. + Sphinx Build ============ -- cgit v1.2.3 From 7673f5b14f6b28416ff3f4ed28bf97db7dc21546 Mon Sep 17 00:00:00 2001 From: Geert Uytterhoeven Date: Mon, 17 Jul 2017 15:39:28 +0200 Subject: Documentation: arm: Replace use of virt_to_phys with __pa_symbol All low-level PM/SMP code using virt_to_phys() should actually use __pa_symbol() against kernel symbols. Update the documentation to move away from virt_to_phys(). Cfr. commit 6996cbb2372189f7 ("ARM: 8641/1: treewide: Replace uses of virt_to_phys with __pa_symbol") Signed-off-by: Geert Uytterhoeven Signed-off-by: Jonathan Corbet --- Documentation/arm/firmware.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Documentation/arm/firmware.txt b/Documentation/arm/firmware.txt index da6713adac8a..7f175dbb427e 100644 --- a/Documentation/arm/firmware.txt +++ b/Documentation/arm/firmware.txt @@ -60,7 +60,7 @@ Example of using a firmware operation: /* some platform code, e.g. SMP initialization */ - __raw_writel(virt_to_phys(exynos4_secondary_startup), + __raw_writel(__pa_symbol(exynos4_secondary_startup), CPU1_BOOT_REG); /* Call Exynos specific smc call */ -- cgit v1.2.3 From 916f677adab9b0d4b0a5028ba5113a16ddf11192 Mon Sep 17 00:00:00 2001 From: Markus Heiser Date: Sat, 17 Jun 2017 10:17:20 +0200 Subject: docs RDT theme: fix bottom margin of lists items List items with two ore more blocks are not well rendered. E.g. the gap between last block (l1-b2) of the first list item and the following list item (L2) is to small:: * L1 xxxxxxxxxx xxxxxxxxxxxxx l1-b2 xxxxxxx xxxxxxxxxxxxx * L2 xxxxxxxxxx xxxxxxxxxxxxx So that it can be read more liquidly, a distance was added to the last block (l1-b2):: * L1 xxxxxxxxxx xxxxxxxxxxxxx l1-b2 xxxxxxx xxxxxxxxxxxxx * L2 xxxxxxxxxx xxxxxxxxxxxxx Signed-off-by: Markus Heiser Signed-off-by: Jonathan Corbet --- Documentation/sphinx-static/theme_overrides.css | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/Documentation/sphinx-static/theme_overrides.css b/Documentation/sphinx-static/theme_overrides.css index d5764a4de5a2..1c9a9ab0f48c 100644 --- a/Documentation/sphinx-static/theme_overrides.css +++ b/Documentation/sphinx-static/theme_overrides.css @@ -56,6 +56,12 @@ font-family: "Courier New", Courier, monospace } + /* fix bottom margin of lists items */ + + .rst-content .section ul li:last-child, .rst-content .section ul li p:last-child { + margin-bottom: 12px; + } + /* inline literal: drop the borderbox, padding and red color */ code, .rst-content tt, .rst-content code { -- cgit v1.2.3 From 4c0477899f2e4a865db070b0526987ca31e525b1 Mon Sep 17 00:00:00 2001 From: Markus Heiser Date: Sat, 17 Jun 2017 10:17:21 +0200 Subject: docs RTD theme: code-block with line nos - lines and line numbers don't line up. In a code-block with line numbers (option :lineno:) there is a misalignment of the rendered source code lines on the right side and the line numbers on the left side. https://github.com/rtfd/sphinx_rtd_theme/issues/419 Since this issue is reported to the RTD theme project, it might be fixed in the future (take this as a interim solution). Signed-off-by: Markus Heiser Signed-off-by: Jonathan Corbet --- Documentation/sphinx-static/theme_overrides.css | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/Documentation/sphinx-static/theme_overrides.css b/Documentation/sphinx-static/theme_overrides.css index 1c9a9ab0f48c..522b6d4c49d4 100644 --- a/Documentation/sphinx-static/theme_overrides.css +++ b/Documentation/sphinx-static/theme_overrides.css @@ -4,6 +4,17 @@ * */ +/* Interim: Code-blocks with line nos - lines and line numbers don't line up. + * see: https://github.com/rtfd/sphinx_rtd_theme/issues/419 + */ + +div[class^="highlight"] pre { + line-height: normal; +} +.rst-content .highlight > pre { + line-height: normal; +} + @media screen { /* content column -- cgit v1.2.3 From 127d4b376431cd8091ed98e9c284744838975ae9 Mon Sep 17 00:00:00 2001 From: Jonathan Corbet Date: Mon, 17 Jul 2017 13:55:58 -0600 Subject: docs: Get module_init() docs from module.h The docs build complains: ./include/linux/init.h:1: warning: no structured comments found The problem is that the comments in question were moved to module.h in commit 0fd972a7d91d (module: relocate module_init from init.h to module.h). Fix basics.rst to match. Signed-off-by: Jonathan Corbet --- Documentation/driver-api/basics.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Documentation/driver-api/basics.rst b/Documentation/driver-api/basics.rst index ab82250c7727..cbfb5a825077 100644 --- a/Documentation/driver-api/basics.rst +++ b/Documentation/driver-api/basics.rst @@ -4,7 +4,7 @@ Driver Basics Driver Entry and Exit points ---------------------------- -.. kernel-doc:: include/linux/init.h +.. kernel-doc:: include/linux/module.h :internal: Driver device table -- cgit v1.2.3 From b18b12ae608ae2a108e5872bef9ab8122e01904d Mon Sep 17 00:00:00 2001 From: Jonathan Corbet Date: Mon, 17 Jul 2017 14:04:50 -0600 Subject: docs: Do not include kerneldoc comments from kernel/sys.c ...because there are none there, and I cannot figure out what would ever have been of interest there. This eliminates this warning: ./kernel/sys.c:1: warning: no structured comments found from the build. Signed-off-by: Jonathan Corbet --- Documentation/driver-api/basics.rst | 3 --- 1 file changed, 3 deletions(-) diff --git a/Documentation/driver-api/basics.rst b/Documentation/driver-api/basics.rst index cbfb5a825077..73fa7d42bbba 100644 --- a/Documentation/driver-api/basics.rst +++ b/Documentation/driver-api/basics.rst @@ -103,9 +103,6 @@ Kernel utility functions .. kernel-doc:: kernel/panic.c :export: -.. kernel-doc:: kernel/sys.c - :export: - .. kernel-doc:: kernel/rcu/tree.c :export: -- cgit v1.2.3 From 9b158d860f7a87d4ff341aae684b32fc9f6e371f Mon Sep 17 00:00:00 2001 From: Jonathan Corbet Date: Mon, 17 Jul 2017 14:07:49 -0600 Subject: docs: Do not include from .../seqno-fence.c There are no kerneldoc comments in drivers/dma-buf/seqno-fence.c, and it appears there never have been. Stop looking for comments there to eliminate this warning: ./drivers/dma-buf/seqno-fence.c:1: warning: no structured comments found Signed-off-by: Jonathan Corbet --- Documentation/driver-api/dma-buf.rst | 3 --- 1 file changed, 3 deletions(-) diff --git a/Documentation/driver-api/dma-buf.rst b/Documentation/driver-api/dma-buf.rst index 31671b469627..dc384f2f7f34 100644 --- a/Documentation/driver-api/dma-buf.rst +++ b/Documentation/driver-api/dma-buf.rst @@ -139,9 +139,6 @@ DMA Fences Seqno Hardware Fences ~~~~~~~~~~~~~~~~~~~~~ -.. kernel-doc:: drivers/dma-buf/seqno-fence.c - :export: - .. kernel-doc:: include/linux/seqno-fence.h :internal: -- cgit v1.2.3 From 405d4c5a14ba305444e0c290edb20105499a02e6 Mon Sep 17 00:00:00 2001 From: Jonathan Corbet Date: Mon, 17 Jul 2017 14:19:25 -0600 Subject: docs: Get the struct cmbdata kernel doc from the right file Back in 2012, commit 9807f75955ea (UAPI: (Scripted) Disintegrate arch/s390/include/asm) moved struct cmbdata (and its kerneldoc comments) to another file, but did not update the docs to match. The result is this warning: ./arch/s390/include/asm/cmb.h:1: warning: no structured comments found ...and no documentation for that structure. Update the docs to get the information from the right place. Signed-off-by: Jonathan Corbet --- Documentation/driver-api/s390-drivers.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Documentation/driver-api/s390-drivers.rst b/Documentation/driver-api/s390-drivers.rst index 7060da136095..ecf8851d3565 100644 --- a/Documentation/driver-api/s390-drivers.rst +++ b/Documentation/driver-api/s390-drivers.rst @@ -75,7 +75,7 @@ The channel-measurement facility provides a means to collect measurement data which is made available by the channel subsystem for each channel attached device. -.. kernel-doc:: arch/s390/include/asm/cmb.h +.. kernel-doc:: arch/s390/include/uapi/asm/cmb.h :internal: .. kernel-doc:: drivers/s390/cio/cmf.c -- cgit v1.2.3 From 5ee5432b045a7e0fb8b72c196ae3d7077c6111eb Mon Sep 17 00:00:00 2001 From: Jonathan Corbet Date: Mon, 17 Jul 2017 14:27:19 -0600 Subject: docs: Do not include from drivers/scsi/constants.c The only function of interest in that file was scsi_print_status(). That function was removed in commit 7ac7076344d9 (scsi: remove scsi_print_status()) but the docs were not changed to match, yielding this warning: ./drivers/scsi/constants.c:1: warning: no structured comments found There's nothing there anymore, so just remove that section from the docs. Signed-off-by: Jonathan Corbet --- Documentation/driver-api/scsi.rst | 8 -------- 1 file changed, 8 deletions(-) diff --git a/Documentation/driver-api/scsi.rst b/Documentation/driver-api/scsi.rst index 859fb672319f..5a2aa7a377d9 100644 --- a/Documentation/driver-api/scsi.rst +++ b/Documentation/driver-api/scsi.rst @@ -224,14 +224,6 @@ mid to lowlevel SCSI driver interface .. kernel-doc:: drivers/scsi/hosts.c :export: -drivers/scsi/constants.c -~~~~~~~~~~~~~~~~~~~~~~~~ - -mid to lowlevel SCSI driver interface - -.. kernel-doc:: drivers/scsi/constants.c - :export: - Transport classes ----------------- -- cgit v1.2.3 From 14994a9bb4f4f042da9f1195ec76677552801eee Mon Sep 17 00:00:00 2001 From: Zhouyi Zhou Date: Fri, 7 Jul 2017 15:11:46 +0800 Subject: docs: disable KASLR when debugging kernel commit 6807c84652b0 ("x86: Enable KASLR by default") enables KASLR by default on x86. While KASLR will confuse gdb which resolve kernel symbol address from symbol table of vmlinux. We should turn off KASLR for kernel debugging. Signed-off-by: Zhouyi Zhou Signed-off-by: Jonathan Corbet --- Documentation/dev-tools/kgdb.rst | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/Documentation/dev-tools/kgdb.rst b/Documentation/dev-tools/kgdb.rst index 75273203a35a..d38be58f872a 100644 --- a/Documentation/dev-tools/kgdb.rst +++ b/Documentation/dev-tools/kgdb.rst @@ -348,6 +348,15 @@ default behavior is always set to 0. - ``echo 1 > /sys/module/debug_core/parameters/kgdbreboot`` - Enter the debugger on reboot notify. +Kernel parameter: ``nokaslr`` +----------------------------- + +If the architecture that you are using enable KASLR by default, +you should consider turning it off. KASLR randomizes the +virtual address where the kernel image is mapped and confuse +gdb which resolve kernel symbol address from symbol table +of vmlinux. + Using kdb ========= @@ -358,7 +367,7 @@ This is a quick example of how to use kdb. 1. Configure kgdboc at boot using kernel parameters:: - console=ttyS0,115200 kgdboc=ttyS0,115200 + console=ttyS0,115200 kgdboc=ttyS0,115200 nokaslr OR -- cgit v1.2.3 From e604f1cb85367d2e5fd4cf253296d190996da81a Mon Sep 17 00:00:00 2001 From: Zhouyi Zhou Date: Fri, 7 Jul 2017 16:51:45 +0800 Subject: docs: disable KASLR when debugging kernel commit 6807c84652b0 ("x86: Enable KASLR by default") enables KASLR by default on x86. While KASLR will confuse gdb which resolve kernel symbol address from symbol table of vmlinux. We should turn off KASLR for kernel debugging. Signed-off-by: Zhouyi Zhou Reviewed-by: Kieran Bingham Acked-by: Jan Kiszka Signed-off-by: Jonathan Corbet --- Documentation/dev-tools/gdb-kernel-debugging.rst | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/Documentation/dev-tools/gdb-kernel-debugging.rst b/Documentation/dev-tools/gdb-kernel-debugging.rst index 5e93c9bc6619..19df79286f00 100644 --- a/Documentation/dev-tools/gdb-kernel-debugging.rst +++ b/Documentation/dev-tools/gdb-kernel-debugging.rst @@ -31,11 +31,13 @@ Setup CONFIG_DEBUG_INFO_REDUCED off. If your architecture supports CONFIG_FRAME_POINTER, keep it enabled. -- Install that kernel on the guest. +- Install that kernel on the guest, turn off KASLR if necessary by adding + "nokaslr" to the kernel command line. Alternatively, QEMU allows to boot the kernel directly using -kernel, -append, -initrd command line switches. This is generally only useful if you do not depend on modules. See QEMU documentation for more details on - this mode. + this mode. In this case, you should build the kernel with + CONFIG_RANDOMIZE_BASE disabled if the architecture supports KASLR. - Enable the gdb stub of QEMU/KVM, either -- cgit v1.2.3 From 18afab8c1d3c2a463eece561e9f15a1704b5eff9 Mon Sep 17 00:00:00 2001 From: Mauro Carvalho Chehab Date: Mon, 17 Jul 2017 18:46:35 -0300 Subject: docs: Makefile: remove no-ops targets After removal of DocBook, those targets are bogus. Reported-by: Jim Davis Signed-off-by: Mauro Carvalho Chehab Signed-off-by: Jonathan Corbet --- Documentation/Makefile | 10 ---------- Documentation/translations/zh_CN/HOWTO | 2 -- Makefile | 2 +- 3 files changed, 1 insertion(+), 13 deletions(-) diff --git a/Documentation/Makefile b/Documentation/Makefile index a42320385df3..d75c00e3aadb 100644 --- a/Documentation/Makefile +++ b/Documentation/Makefile @@ -95,16 +95,6 @@ endif # HAVE_SPHINX # The following targets are independent of HAVE_SPHINX, and the rules should # work or silently pass without Sphinx. -# no-ops for the Sphinx toolchain -sgmldocs: - @: -psdocs: - @: -mandocs: - @: -installmandocs: - @: - cleandocs: $(Q)rm -rf $(BUILDDIR) $(Q)$(MAKE) BUILDDIR=$(abspath $(BUILDDIR)) $(build)=Documentation/media clean diff --git a/Documentation/translations/zh_CN/HOWTO b/Documentation/translations/zh_CN/HOWTO index 11be075ba5fa..5f6d09edc9ac 100644 --- a/Documentation/translations/zh_CN/HOWTO +++ b/Documentation/translations/zh_CN/HOWTO @@ -149,9 +149,7 @@ Linux内核代码中包含有大量的文档。这些文档对于学习如何与 核源码的主目录中使用以下不同命令将会分别生成PDF、Postscript、HTML和手册 页等不同格式的文档： make pdfdocs - make psdocs make htmldocs - make mandocs 如何成为内核开发者 diff --git a/Makefile b/Makefile index b4fb9a1d1594..8b7237a0d66a 100644 --- a/Makefile +++ b/Makefile @@ -1467,7 +1467,7 @@ $(help-board-dirs): help-%: # Documentation targets # --------------------------------------------------------------------------- -DOC_TARGETS := xmldocs sgmldocs psdocs latexdocs pdfdocs htmldocs mandocs installmandocs epubdocs cleandocs linkcheckdocs +DOC_TARGETS := xmldocs latexdocs pdfdocs htmldocs epubdocs cleandocs linkcheckdocs PHONY += $(DOC_TARGETS) $(DOC_TARGETS): scripts_basic FORCE $(Q)$(MAKE) $(build)=Documentation $@ -- cgit v1.2.3 From 24071ac1a6176dda89b441cc4c757c51e2f523e0 Mon Sep 17 00:00:00 2001 From: Mauro Carvalho Chehab Date: Mon, 17 Jul 2017 18:46:36 -0300 Subject: scripts/sphinx-pre-install: add a script to check Sphinx install Solving Sphinx dependencies can be painful. Add a script to check if everything is ok. Tested on: - Fedora 25 and 26; - Ubuntu 17.04; - OpenSuse Tumbleweed; - Arch Linux; - Gentoo. Signed-off-by: Mauro Carvalho Chehab Signed-off-by: Jonathan Corbet --- scripts/sphinx-pre-install | 517 +++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 517 insertions(+) create mode 100755 scripts/sphinx-pre-install diff --git a/scripts/sphinx-pre-install b/scripts/sphinx-pre-install new file mode 100755 index 000000000000..5fe3b4a86174 --- /dev/null +++ b/scripts/sphinx-pre-install @@ -0,0 +1,517 @@ +#!/usr/bin/perl +use strict; + +# Copyright (c) 2017 Mauro Carvalho Chehab +# +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU General Public License +# as published by the Free Software Foundation; either version 2 +# of the License, or (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. + +# +# Static vars +# + +my %missing; +my $system_release; +my $need = 0; +my $optional = 0; +my $need_symlink = 0; +my $need_sphinx = 0; +my $install = ""; + +# +# Command line arguments +# + +my $pdf = 1; +my $virtualenv = 1; + +# +# List of required texlive packages on Fedora and OpenSuse +# + +my %texlive = ( + 'adjustbox.sty' => 'texlive-adjustbox', + 'amsfonts.sty' => 'texlive-amsfonts', + 'amsmath.sty' => 'texlive-amsmath', + 'amssymb.sty' => 'texlive-amsfonts', + 'amsthm.sty' => 'texlive-amscls', + 'anyfontsize.sty' => 'texlive-anyfontsize', + 'atbegshi.sty' => 'texlive-oberdiek', + 'bm.sty' => 'texlive-tools', + 'capt-of.sty' => 'texlive-capt-of', + 'cmap.sty' => 'texlive-cmap', + 'ecrm1000.tfm' => 'texlive-ec', + 'eqparbox.sty' => 'texlive-eqparbox', + 'eu1enc.def' => 'texlive-euenc', + 'fancybox.sty' => 'texlive-fancybox', + 'fancyvrb.sty' => 'texlive-fancyvrb', + 'float.sty' => 'texlive-float', + 'fncychap.sty' => 'texlive-fncychap', + 'footnote.sty' => 'texlive-mdwtools', + 'framed.sty' => 'texlive-framed', + 'luatex85.sty' => 'texlive-luatex85', + 'multirow.sty' => 'texlive-multirow', + 'needspace.sty' => 'texlive-needspace', + 'palatino.sty' => 'texlive-psnfss', + 'parskip.sty' => 'texlive-parskip', + 'polyglossia.sty' => 'texlive-polyglossia', + 'tabulary.sty' => 'texlive-tabulary', + 'threeparttable.sty' => 'texlive-threeparttable', + 'titlesec.sty' => 'texlive-titlesec', + 'ucs.sty' => 'texlive-ucs', + 'upquote.sty' => 'texlive-upquote', + 'wrapfig.sty' => 'texlive-wrapfig', +); + +# +# Subroutines that checks if a feature exists +# + +sub check_missing(%) +{ + my %map = %{$_[0]}; + + foreach my $prog (sort keys %missing) { + my $is_optional = $missing{$prog}; + + if ($is_optional) { + print "Warning: better to also install \"$prog\".\n"; + } else { + print "ERROR: please install \"$prog\", otherwise, build won't work.\n"; + } + if (defined($map{$prog})) { + $install .= " " . $map{$prog}; + } else { + $install .= " " . $prog; + } + } + + $install =~ s/^\s//; +} + +sub add_package($$) +{ + my $package = shift; + my $is_optional = shift; + + $missing{$package} = $is_optional; + if ($is_optional) { + $optional++; + } else { + $need++; + } +} + +sub check_missing_file($$$) +{ + my $file = shift; + my $package = shift; + my $is_optional = shift; + + return if(-e $file); + + add_package($package, $is_optional); +} + +sub findprog($) +{ + foreach(split(/:/, $ENV{PATH})) { + return "$_/$_[0]" if(-x "$_/$_[0]"); + } +} + +sub check_program($$) +{ + my $prog = shift; + my $is_optional = shift; + + return if findprog($prog); + + add_package($prog, $is_optional); +} + +sub check_perl_module($$) +{ + my $prog = shift; + my $is_optional = shift; + + my $err = system("perl -M$prog -e 1 2>/dev/null /dev/null"); + return if ($err == 0); + + add_package($prog, $is_optional); +} + +sub check_python_module($$) +{ + my $prog = shift; + my $is_optional = shift; + + my $err = system("python3 -c 'import $prog' 2>/dev/null /dev/null"); + return if ($err == 0); + my $err = system("python -c 'import $prog' 2>/dev/null /dev/null"); + return if ($err == 0); + + add_package($prog, $is_optional); +} + +sub check_rpm_missing($$) +{ + my @pkgs = @{$_[0]}; + my $is_optional = $_[1]; + + foreach my $prog(@pkgs) { + my $err = system("rpm -q '$prog' 2>/dev/null >/dev/null"); + add_package($prog, $is_optional) if ($err); + } +} + +sub check_pacman_missing($$) +{ + my @pkgs = @{$_[0]}; + my $is_optional = $_[1]; + + foreach my $prog(@pkgs) { + my $err = system("pacman -Q '$prog' 2>/dev/null >/dev/null"); + add_package($prog, $is_optional) if ($err); + } +} + +sub check_missing_tex($) +{ + my $is_optional = shift; + my $kpsewhich = findprog("kpsewhich"); + + foreach my $prog(keys %texlive) { + my $package = $texlive{$prog}; + if (!$kpsewhich) { + add_package($package, $is_optional); + next; + } + my $file = qx($kpsewhich $prog); + add_package($package, $is_optional) if ($file =~ /^\s*$/); + } +} + +sub check_sphinx() +{ + return if findprog("sphinx-build"); + + if (findprog("sphinx-build-3")) { + $need_symlink = 1; + return; + } + + if ($virtualenv) { + check_program("virtualenv", 0) if (!findprog("virtualenv-3")); + check_program("pip", 0) if (!findprog("pip3")); + $need_sphinx = 1; + } else { + add_package("python-sphinx", 0); + } +} + +# +# Ancillary subroutines +# + +sub catcheck($) +{ + my $res = ""; + $res = qx(cat $_[0]) if (-r $_[0]); + return $res; +} + +sub which($) +{ + my $file = shift; + my @path = split ":", $ENV{PATH}; + + foreach my $dir(@path) { + my $name = $dir.'/'.$file; + return $name if (-x $name ); + } + return undef; +} + +# +# Subroutines that check distro-specific hints +# + +sub give_debian_hints() +{ + my %map = ( + "python-sphinx" => "python3-sphinx", + "sphinx_rtd_theme" => "python3-sphinx-rtd-theme", + "virtualenv" => "virtualenv", + "pip" => "python3-pip", + "dot" => "graphviz", + "convert" => "imagemagick", + "Pod::Usage" => "perl-modules", + "xelatex" => "texlive-xetex", + ); + + if ($pdf) { + check_missing_file("/usr/share/fonts/truetype/dejavu/DejaVuSans.ttf", + "fonts-dejavu", 1); + } + + check_program("dvipng", 1) if ($pdf); + check_missing(\%map); + + return if (!$need && !$optional); + printf("You should run:\n\n\tsudo apt-get install $install\n"); +} + +sub give_redhat_hints() +{ + my %map = ( + "python-sphinx" => "python3-sphinx", + "sphinx_rtd_theme" => "python3-sphinx_rtd_theme", + "virtualenv" => "python3-virtualenv", + "pip" => "python3-pip", + "dot" => "graphviz", + "convert" => "ImageMagick", + "Pod::Usage" => "perl-Pod-Usage", + "xelatex" => "texlive-xetex-bin", + ); + + my @fedora_tex_pkgs = ( + "texlive-collection-fontsrecommended", + "texlive-collection-latex", + "dejavu-sans-fonts", + "dejavu-serif-fonts", + "dejavu-sans-mono-fonts", + ); + + check_rpm_missing(\@fedora_tex_pkgs, 1) if ($pdf); + check_missing_tex(1) if ($pdf); + check_missing(\%map); + + return if (!$need && !$optional); + printf("You should run:\n\n\tsudo dnf install -y $install\n"); +} + +sub give_opensuse_hints() +{ + my %map = ( + "python-sphinx" => "python3-sphinx", + "sphinx_rtd_theme" => "python3-sphinx_rtd_theme", + "virtualenv" => "python3-virtualenv", + "pip" => "python3-pip", + "dot" => "graphviz", + "convert" => "ImageMagick", + "Pod::Usage" => "perl-Pod-Usage", + "xelatex" => "texlive-xetex-bin", + ); + + my @suse_tex_pkgs = ( + "texlive-babel-english", + "texlive-caption", + "texlive-colortbl", + "texlive-courier", + "texlive-dvips", + "texlive-helvetic", + "texlive-makeindex", + "texlive-metafont", + "texlive-metapost", + "texlive-palatino", + "texlive-preview", + "texlive-times", + "texlive-zapfchan", + "texlive-zapfding", + ); + + check_rpm_missing(\@suse_tex_pkgs, 1) if ($pdf); + check_missing_tex(1) if ($pdf); + check_missing(\%map); + + return if (!$need && !$optional); + printf("You should run:\n\n\tsudo zypper install --no-recommends $install\n"); +} + +sub give_arch_linux_hints() +{ + my %map = ( + "sphinx_rtd_theme" => "python-sphinx_rtd_theme", + "virtualenv" => "python-virtualenv", + "pip" => "python-pip", + "dot" => "graphviz", + "convert" => "imagemagick", + "xelatex" => "texlive-bin", + ); + + my @archlinux_tex_pkgs = ( + "texlive-core", + "texlive-latexextra", + "ttf-dejavu", + ); + check_pacman_missing(\@archlinux_tex_pkgs, 1) if ($pdf); + check_missing(\%map); + + return if (!$need && !$optional); + printf("You should run:\n\n\tsudo pacman -S $install\n"); +} + +sub give_gentoo_hints() +{ + my %map = ( + "sphinx_rtd_theme" => "dev-python/sphinx_rtd_theme", + "virtualenv" => "dev-python/virtualenv", + "pip" => "dev-python/pip", + "dot" => "media-gfx/graphviz", + "convert" => "media-gfx/imagemagick", + "xelatex" => "dev-texlive/texlive-xetex media-fonts/dejavu", + ); + + check_missing_file("/usr/share/fonts/dejavu/DejaVuSans.ttf", + "media-fonts/dejavu", 1) if ($pdf); + + check_missing(\%map); + + return if (!$need && !$optional); + printf("You should run:\n\n\tsudo emerge --ask $install\n"); +} + +sub check_distros() +{ + # Distro-specific hints + if ($system_release =~ /Red Hat Enterprise Linux/) { + give_redhat_hints; + return; + } + if ($system_release =~ /Fedora/) { + give_redhat_hints; + return; + } + if ($system_release =~ /Ubuntu/) { + give_debian_hints; + return; + } + if ($system_release =~ /Debian/) { + give_debian_hints; + return; + } + if ($system_release =~ /openSUSE/) { + give_opensuse_hints; + return; + } + if ($system_release =~ /Arch Linux/) { + give_arch_linux_hints; + return; + } + if ($system_release =~ /Gentoo/) { + give_gentoo_hints; + return; + } + + # + # Fall-back to generic hint code for other distros + # That's far from ideal, specially for LaTeX dependencies. + # + my %map = ( + "sphinx-build" => "sphinx" + ); + check_missing_tex(1) if ($pdf); + check_missing(\%map); + print "I don't know distro $system_release.\n"; + print "So, I can't provide you a hint with the install procedure.\n"; + print "There are likely missing dependencies.\n"; +} + +# +# Common dependencies +# + +sub check_needs() +{ + if ($system_release) { + print "Checking if the needed tools for $system_release are available\n"; + } else { + print "Checking if the needed tools are present\n"; + } + + # Check for needed programs/tools + check_sphinx(); + check_perl_module("Pod::Usage", 0); + check_program("make", 0); + check_program("gcc", 0); + check_python_module("sphinx_rtd_theme", 1) if (!$virtualenv); + check_program("xelatex", 1) if ($pdf); + check_program("dot", 1); + check_program("convert", 1); + + check_distros(); + + if ($need_symlink) { + printf "\tsudo ln -sf %s /usr/bin/sphinx-build\n\n", + which("sphinx-build-3"); + } + if ($need_sphinx) { + my $virtualenv = findprog("virtualenv-3"); + $virtualenv = findprog("virtualenv") if (!$virtualenv); + $virtualenv = "virtualenv" if (!$virtualenv); + + printf "\t$virtualenv sphinx_1.4\n"; + printf "\t. sphinx_1.4/bin/activate\n"; + printf "\tpip install 'docutils==0.12'\n"; + printf "\tpip install 'Sphinx==1.4.9'\n"; + printf "\tpip install sphinx_rtd_theme\n"; + $need++; + } + printf "\n"; + + print "All optional dependenties are met.\n" if (!$optional); + + if ($need == 1) { + die "Can't build as $need mandatory dependency is missing"; + } elsif ($need) { + die "Can't build as $need mandatory dependencies are missing"; + } + + print "Needed package dependencies are met.\n"; +} + +# +# Main +# + +while (@ARGV) { + my $arg = shift(@ARGV); + + if ($arg eq "--no-virtualenv") { + $virtualenv = 0; + } elsif ($arg eq "--no-pdf"){ + $pdf = 0; + } else { + print "Usage:\n\t$0 <--no-virtualenv> <--no-pdf>\n\n"; + exit -1; + } +} + +# +# Determine the system type. There's no standard unique way that would +# work with all distros with a minimal package install. So, several +# methods are used here. +# +# By default, it will use lsb_release function. If not available, it will +# fail back to reading the known different places where the distro name +# is stored +# + +$system_release = qx(lsb_release -d) if which("lsb_release"); +$system_release =~ s/Description:\s*// if ($system_release); +$system_release = catcheck("/etc/system-release") if !$system_release; +$system_release = catcheck("/etc/redhat-release") if !$system_release; +$system_release = catcheck("/etc/lsb-release") if !$system_release; +$system_release = catcheck("/etc/gentoo-release") if !$system_release; +$system_release = catcheck("/etc/issue") if !$system_release; +$system_release =~ s/\s+$//; + +check_needs; -- cgit v1.2.3 From 5be33182d4e22750237281f03172e6c3456cdb42 Mon Sep 17 00:00:00 2001 From: Mauro Carvalho Chehab Date: Mon, 17 Jul 2017 18:46:37 -0300 Subject: sphinx-pre-install: detect an existing virtualenv Detect if the script runs after creating the virtualenv, printing the command line commands to enable the virtualenv. Signed-off-by: Mauro Carvalho Chehab Signed-off-by: Jonathan Corbet --- scripts/sphinx-pre-install | 28 ++++++++++++++++++---------- 1 file changed, 18 insertions(+), 10 deletions(-) diff --git a/scripts/sphinx-pre-install b/scripts/sphinx-pre-install index 5fe3b4a86174..bcd6f7978f8d 100755 --- a/scripts/sphinx-pre-install +++ b/scripts/sphinx-pre-install @@ -13,6 +13,8 @@ use strict; # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. +my $virtenv_dir = "sphinx_1.4"; + # # Static vars # @@ -454,16 +456,22 @@ sub check_needs() which("sphinx-build-3"); } if ($need_sphinx) { - my $virtualenv = findprog("virtualenv-3"); - $virtualenv = findprog("virtualenv") if (!$virtualenv); - $virtualenv = "virtualenv" if (!$virtualenv); - - printf "\t$virtualenv sphinx_1.4\n"; - printf "\t. sphinx_1.4/bin/activate\n"; - printf "\tpip install 'docutils==0.12'\n"; - printf "\tpip install 'Sphinx==1.4.9'\n"; - printf "\tpip install sphinx_rtd_theme\n"; - $need++; + my $activate = "$virtenv_dir/bin/activate"; + if (-e "$ENV{'PWD'}/$activate") { + printf "\nNeed to activate virtualenv with:\n"; + printf "\t. $activate\n"; + } else { + my $virtualenv = findprog("virtualenv-3"); + $virtualenv = findprog("virtualenv") if (!$virtualenv); + $virtualenv = "virtualenv" if (!$virtualenv); + + printf "\t$virtualenv $virtenv_dir\n"; + printf "\t. $activate\n"; + printf "\tpip install 'docutils==0.12'\n"; + printf "\tpip install 'Sphinx==1.4.9'\n"; + printf "\tpip install sphinx_rtd_theme\n"; + $need++; + } } printf "\n"; -- cgit v1.2.3 From fb947f3f472303d54759bf898cf92fd8c2dc9bdf Mon Sep 17 00:00:00 2001 From: Mauro Carvalho Chehab Date: Mon, 17 Jul 2017 18:46:38 -0300 Subject: sphinx-pre-install: use a requirements file Instead of using 3 commands to install a virtualenv, use a single one, reading the requirements from this file: Documentation/sphinx/requirements.txt Signed-off-by: Mauro Carvalho Chehab Signed-off-by: Jonathan Corbet --- Documentation/sphinx/requirements.txt | 3 +++ scripts/sphinx-pre-install | 5 ++--- 2 files changed, 5 insertions(+), 3 deletions(-) create mode 100644 Documentation/sphinx/requirements.txt diff --git a/Documentation/sphinx/requirements.txt b/Documentation/sphinx/requirements.txt new file mode 100644 index 000000000000..742be3e12619 --- /dev/null +++ b/Documentation/sphinx/requirements.txt @@ -0,0 +1,3 @@ +docutils==0.12 +Sphinx==1.4.9 +sphinx_rtd_theme diff --git a/scripts/sphinx-pre-install b/scripts/sphinx-pre-install index bcd6f7978f8d..230d9cc7ee1d 100755 --- a/scripts/sphinx-pre-install +++ b/scripts/sphinx-pre-install @@ -14,6 +14,7 @@ use strict; # GNU General Public License for more details. my $virtenv_dir = "sphinx_1.4"; +my $requirement_file = "Documentation/sphinx/requirements.txt"; # # Static vars @@ -467,9 +468,7 @@ sub check_needs() printf "\t$virtualenv $virtenv_dir\n"; printf "\t. $activate\n"; - printf "\tpip install 'docutils==0.12'\n"; - printf "\tpip install 'Sphinx==1.4.9'\n"; - printf "\tpip install sphinx_rtd_theme\n"; + printf "\tpip install -r $requirement_file\n"; $need++; } } -- cgit v1.2.3 From 5d88953c32a64737a7879d9ff1263dc622f9aee0 Mon Sep 17 00:00:00 2001 From: Mauro Carvalho Chehab Date: Mon, 17 Jul 2017 18:46:39 -0300 Subject: sphinx-pre-install: check for the need of graphviz-gd On newer versions of graphviz packaging on Fedora, it is needed to install a separate package for PDF support. Signed-off-by: Mauro Carvalho Chehab Signed-off-by: Jonathan Corbet --- scripts/sphinx-pre-install | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/scripts/sphinx-pre-install b/scripts/sphinx-pre-install index 230d9cc7ee1d..817241c674d3 100755 --- a/scripts/sphinx-pre-install +++ b/scripts/sphinx-pre-install @@ -285,6 +285,10 @@ sub give_redhat_hints() "xelatex" => "texlive-xetex-bin", ); + my @fedora26_opt_pkgs = ( + "graphviz-gd", # Fedora 26: needed for PDF support + ); + my @fedora_tex_pkgs = ( "texlive-collection-fontsrecommended", "texlive-collection-latex", @@ -293,6 +297,11 @@ sub give_redhat_hints() "dejavu-sans-mono-fonts", ); + my $release; + + $release = $1 if ($system_release =~ /Fedora\s+release\s+(\d+)/); + + check_rpm_missing(\@fedora26_opt_pkgs, 1) if ($pdf && $release >= 26); check_rpm_missing(\@fedora_tex_pkgs, 1) if ($pdf); check_missing_tex(1) if ($pdf); check_missing(\%map); -- cgit v1.2.3 From 8e7d5d15e73fde2b22649783d4f0fa4495a1582f Mon Sep 17 00:00:00 2001 From: Mauro Carvalho Chehab Date: Mon, 17 Jul 2017 18:46:40 -0300 Subject: sphinx-pre-install: add dependencies for ImageMagick to work with svg ImageMagick actually uses librsvg for conversions when converiting from SVG (actually, it uses rsvg-convert). That causes the build to fail with: WARNING: Error #1 when calling: /usr/bin/convert /home/mchehab/docs/Documentation/media/uapi/v4l/selection.svg /home/mchehab/docs/Documentation/output/latex/selection.pdf convert: delegate failed `'rsvg-convert' -o '%o' '%i'' @ error/delegate.c/InvokeDelegate/1919. convert: unable to open file `/tmp/magick-8883oOQfHzrA5trM': No such file or directory @ error/constitute.c/ReadImage/544. Add the corresponding dependencies. Signed-off-by: Mauro Carvalho Chehab Signed-off-by: Jonathan Corbet --- scripts/sphinx-pre-install | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/scripts/sphinx-pre-install b/scripts/sphinx-pre-install index 817241c674d3..fc9891ce5c85 100755 --- a/scripts/sphinx-pre-install +++ b/scripts/sphinx-pre-install @@ -258,6 +258,7 @@ sub give_debian_hints() "convert" => "imagemagick", "Pod::Usage" => "perl-modules", "xelatex" => "texlive-xetex", + "rsvg-convert" => "librsvg2-bin", ); if ($pdf) { @@ -283,6 +284,7 @@ sub give_redhat_hints() "convert" => "ImageMagick", "Pod::Usage" => "perl-Pod-Usage", "xelatex" => "texlive-xetex-bin", + "rsvg-convert" => "librsvg2-tools", ); my @fedora26_opt_pkgs = ( @@ -321,6 +323,7 @@ sub give_opensuse_hints() "convert" => "ImageMagick", "Pod::Usage" => "perl-Pod-Usage", "xelatex" => "texlive-xetex-bin", + "rsvg-convert" => "rsvg-view", ); my @suse_tex_pkgs = ( @@ -357,6 +360,7 @@ sub give_arch_linux_hints() "dot" => "graphviz", "convert" => "imagemagick", "xelatex" => "texlive-bin", + "rsvg-convert" => "extra/librsvg", ); my @archlinux_tex_pkgs = ( @@ -380,6 +384,7 @@ sub give_gentoo_hints() "dot" => "media-gfx/graphviz", "convert" => "media-gfx/imagemagick", "xelatex" => "dev-texlive/texlive-xetex media-fonts/dejavu", + "rsvg-convert" => "gnome-base/librsvg", ); check_missing_file("/usr/share/fonts/dejavu/DejaVuSans.ttf", @@ -458,6 +463,7 @@ sub check_needs() check_program("xelatex", 1) if ($pdf); check_program("dot", 1); check_program("convert", 1); + check_program("rsvg-convert", 1) if ($pdf); check_distros(); -- cgit v1.2.3 From bba1e4cbdb5aadc937d70f04d1d07752828e4b2d Mon Sep 17 00:00:00 2001 From: Mauro Carvalho Chehab Date: Mon, 17 Jul 2017 18:46:41 -0300 Subject: sphinx-pre-install: fix USE needs for GraphViz and ImageMagick Gentoo need some USE for GraphViz and ImageMagick to have the features required by kfigure.py. Output that when providing instructions for Gentoo. Signed-off-by: Mauro Carvalho Chehab Signed-off-by: Jonathan Corbet --- scripts/sphinx-pre-install | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/scripts/sphinx-pre-install b/scripts/sphinx-pre-install index fc9891ce5c85..2d18064d57b4 100755 --- a/scripts/sphinx-pre-install +++ b/scripts/sphinx-pre-install @@ -393,7 +393,12 @@ sub give_gentoo_hints() check_missing(\%map); return if (!$need && !$optional); - printf("You should run:\n\n\tsudo emerge --ask $install\n"); + + printf("You should run:\n\n"); + printf("\tsudo su -c 'echo \"media-gfx/imagemagick svg png\" > /etc/portage/package.use/imagemagick'\n"); + printf("\tsudo su -c 'echo \"media-gfx/graphviz cairo pdf\" > /etc/portage/package.use/graphviz'\n"); + printf("\tsudo emerge --ask $install\n"); + } sub check_distros() -- cgit v1.2.3 From 868366aaac009fa74e5c8a452db6d6276c605e8b Mon Sep 17 00:00:00 2001 From: Mauro Carvalho Chehab Date: Mon, 17 Jul 2017 18:46:42 -0300 Subject: sphinx.rst: document scripts/sphinx-pre-install script Now that we have a script to check for Sphinx dependencies, document it. Signed-off-by: Mauro Carvalho Chehab Signed-off-by: Jonathan Corbet --- Documentation/doc-guide/sphinx.rst | 48 ++++++++++++++++++++++++++++++++++---- 1 file changed, 44 insertions(+), 4 deletions(-) diff --git a/Documentation/doc-guide/sphinx.rst b/Documentation/doc-guide/sphinx.rst index 5521213efa68..8faafb9b2d86 100644 --- a/Documentation/doc-guide/sphinx.rst +++ b/Documentation/doc-guide/sphinx.rst @@ -28,6 +28,9 @@ The ReST markups currently used by the Documentation/ files are meant to be built with ``Sphinx`` version 1.3 or upper. If you're desiring to build PDF outputs, it is recommended to use version 1.4.6 or upper. +There's a script that checks for the Spinx requirements. Please see +:ref:`sphinx-pre-install` for further details. + Most distributions are shipped with Sphinx, but its toolchain is fragile, and it is not uncommon that upgrading it or some other Python packages on your machine would cause the documentation build to break. @@ -47,13 +50,15 @@ or ``virtualenv``, depending on how your distribution packaged Python 3. on the Sphinx version, it should be installed in separate, with ``pip install sphinx_rtd_theme``. + #) Some ReST pages contain math expressions. Due to the way Sphinx work, + those expressions are written using LaTeX notation. It needs texlive + installed with amdfonts and amsmath in order to evaluate them. + In summary, if you want to install Sphinx version 1.4.9, you should do:: $ virtualenv sphinx_1.4 $ . sphinx_1.4/bin/activate - (sphinx_1.4) $ pip install 'docutils==0.12' - (sphinx_1.4) $ pip install 'Sphinx==1.4.9' - (sphinx_1.4) $ pip install sphinx_rtd_theme + (sphinx_1.4) $ pip install -r Documentation/sphinx/requirements.txt After running ``. sphinx_1.4/bin/activate``, the prompt will change, in order to indicate that you're using the new environment. If you @@ -83,7 +88,42 @@ For PDF and LaTeX output, you'll also need ``XeLaTeX`` version 3.14159265. Depending on the distribution, you may also need to install a series of ``texlive`` packages that provide the minimal set of functionalities -required for ``XeLaTex`` to work. +required for ``XeLaTeX`` to work. + +.. _sphinx-pre-install: + +Checking for Sphinx dependencies +-------------------------------- + +There's a script that automatically check for Sphinx dependencies. If it can +recognize your distribution, it will also give a hint about the install +command line options for your distro:: + + $ ./scripts/sphinx-pre-install + Checking if the needed tools for Fedora release 26 (Twenty Six) are available + Warning: better to also install "texlive-luatex85". + You should run: + + sudo dnf install -y texlive-luatex85 + /usr/bin/virtualenv sphinx_1.4 + . sphinx_1.4/bin/activate + pip install -r Documentation/sphinx/requirements.txt + + Can't build as 1 mandatory dependency is missing at ./scripts/sphinx-pre-install line 468. + +By default, it checks all the requirements for both html and PDF, including +the requirements for images, math expressions and LaTeX build, and assumes +that a virtual Python environment will be used. The ones needed for html +builds are assumed to be mandatory; the others to be optional. + +It supports two optional parameters: + +``--no-pdf`` + Disable checks for PDF; + +``--no-virtualenv`` + Use OS packaging for Sphinx instead of Python virtual environment. + Sphinx Build ============ -- cgit v1.2.3 From 800d408a3427fd5c74d2d9f9baa6093ac0386632 Mon Sep 17 00:00:00 2001 From: Mauro Carvalho Chehab Date: Fri, 21 Jul 2017 13:20:41 -0300 Subject: sphinx-pre-install: add support for Mageia Add support for detecting and installing missing packages on Mageia. I opted to use "urpmi" at the install instructions, as this is present on Mageia since ever. Yet, if I were using Mageia 6, I would likely be using "dnf", as it is, IMHO, easier to use. Tested with Mageia 6. Signed-off-by: Mauro Carvalho Chehab Signed-off-by: Jonathan Corbet --- scripts/sphinx-pre-install | 37 ++++++++++++++++++++++++++++++++++++- 1 file changed, 36 insertions(+), 1 deletion(-) diff --git a/scripts/sphinx-pre-install b/scripts/sphinx-pre-install index 2d18064d57b4..5d2799dcfceb 100755 --- a/scripts/sphinx-pre-install +++ b/scripts/sphinx-pre-install @@ -212,7 +212,10 @@ sub check_sphinx() } if ($virtualenv) { - check_program("virtualenv", 0) if (!findprog("virtualenv-3")); + my $prog = findprog("virtualenv-3"); + $prog = findprog("virtualenv-3.5") if (!$prog); + + check_program("virtualenv", 0) if (!$prog); check_program("pip", 0) if (!findprog("pip3")); $need_sphinx = 1; } else { @@ -351,6 +354,33 @@ sub give_opensuse_hints() printf("You should run:\n\n\tsudo zypper install --no-recommends $install\n"); } +sub give_mageia_hints() +{ + my %map = ( + "python-sphinx" => "python3-sphinx", + "sphinx_rtd_theme" => "python3-sphinx_rtd_theme", + "virtualenv" => "python3-virtualenv", + "pip" => "python3-pip", + "dot" => "graphviz", + "convert" => "ImageMagick", + "Pod::Usage" => "perl-Pod-Usage", + "xelatex" => "texlive", + "rsvg-convert" => "librsvg2-tools", + ); + + my @tex_pkgs = ( + "texlive-fontsextra", + ); + + my $release; + + check_rpm_missing(\@tex_pkgs, 1) if ($pdf); + check_missing(\%map); + + return if (!$need && !$optional); + printf("You should run:\n\n\tsudo urpmi $install\n"); +} + sub give_arch_linux_hints() { my %map = ( @@ -424,6 +454,10 @@ sub check_distros() give_opensuse_hints; return; } + if ($system_release =~ /Mageia/) { + give_mageia_hints; + return; + } if ($system_release =~ /Arch Linux/) { give_arch_linux_hints; return; @@ -483,6 +517,7 @@ sub check_needs() printf "\t. $activate\n"; } else { my $virtualenv = findprog("virtualenv-3"); + $virtualenv = findprog("virtualenv-3.5") if (!$virtualenv); $virtualenv = findprog("virtualenv") if (!$virtualenv); $virtualenv = "virtualenv" if (!$virtualenv); -- cgit v1.2.3 From 23924e31037a08d89a9da25dbe5ac6c0233d86e4 Mon Sep 17 00:00:00 2001 From: Jonathan Corbet Date: Mon, 17 Jul 2017 14:42:39 -0600 Subject: docs: Use :internal: for include/drm/drm_syncobj.h Documentation/gpu/drm-mm.rst includes from include/drm/drm_syncobj.h with :export:, but this is a header file without export directives. That results in this warning: ./include/drm/drm_syncobj.h:1: warning: no structured comments found ...and a failure to obtain the documentation from that file. Switch to :internal: instead to make both problems go away. Cc: dri-devel@lists.freedesktop.org Signed-off-by: Jonathan Corbet --- Documentation/gpu/drm-mm.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Documentation/gpu/drm-mm.rst b/Documentation/gpu/drm-mm.rst index 9412798645c1..300898298bf6 100644 --- a/Documentation/gpu/drm-mm.rst +++ b/Documentation/gpu/drm-mm.rst @@ -492,7 +492,7 @@ DRM Sync Objects :doc: Overview .. kernel-doc:: include/drm/drm_syncobj.h - :export: + :internal: .. kernel-doc:: drivers/gpu/drm/drm_syncobj.c :export: -- cgit v1.2.3 From bc7938deaca7f474918c41a0372a410049bd4e13 Mon Sep 17 00:00:00 2001 From: Frank Rowand Date: Thu, 20 Jul 2017 18:30:55 -0700 Subject: docs: submitting-patches - change non-ascii character to ascii Documentation/process/submitting-patches.rst contains a non-ascii character. Change it to the ascii equivalent. Signed-off-by: Frank Rowand Signed-off-by: Jonathan Corbet --- Documentation/process/submitting-patches.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Documentation/process/submitting-patches.rst b/Documentation/process/submitting-patches.rst index 3e10719fee35..733478ade91b 100644 --- a/Documentation/process/submitting-patches.rst +++ b/Documentation/process/submitting-patches.rst @@ -413,7 +413,7 @@ e-mail discussions. -11) Sign your work — the Developer's Certificate of Origin +11) Sign your work - the Developer's Certificate of Origin ---------------------------------------------------------- To improve tracking of who did what, especially with patches that can -- cgit v1.2.3 From 02bdf9da9e6cf150a28545ae000aaf620456b449 Mon Sep 17 00:00:00 2001 From: James Hogan Date: Fri, 4 Aug 2017 11:28:39 +0100 Subject: docs/features: parisc implements tracehook Since commit 64e2a42bca12 ("parisc: Add ARCH_TRACEHOOK and regset support") in v4.7, parisc selects HAVE_ARCH_TRACEHOOK, so update its entry in Documentation/features from TODO to ok. Signed-off-by: James Hogan Cc: "James E.J. Bottomley" Cc: Helge Deller Cc: Ingo Molnar Signed-off-by: Jonathan Corbet --- Documentation/features/core/tracehook/arch-support.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Documentation/features/core/tracehook/arch-support.txt b/Documentation/features/core/tracehook/arch-support.txt index 5e97a89420ef..dfb638c2f842 100644 --- a/Documentation/features/core/tracehook/arch-support.txt +++ b/Documentation/features/core/tracehook/arch-support.txt @@ -25,7 +25,7 @@ | mn10300: | ok | | nios2: | ok | | openrisc: | ok | - | parisc: | TODO | + | parisc: | ok | | powerpc: | ok | | s390: | ok | | score: | TODO | -- cgit v1.2.3 From 3d91a353674c9cb38431d631598c9ed15d22fbc8 Mon Sep 17 00:00:00 2001 From: Sedat Dilek Date: Mon, 24 Jul 2017 17:27:05 +0200 Subject: kbuild: Update example for ccflags-y usage Update the ccflags-y example to match current usage. Signed-off-by: Sedat Dilek Signed-off-by: Jonathan Corbet --- Documentation/kbuild/makefiles.txt | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/Documentation/kbuild/makefiles.txt b/Documentation/kbuild/makefiles.txt index 7003141a6d4f..329e740adea7 100644 --- a/Documentation/kbuild/makefiles.txt +++ b/Documentation/kbuild/makefiles.txt @@ -297,9 +297,9 @@ more details, with real examples. ccflags-y specifies options for compiling with $(CC). Example: - # drivers/acpi/Makefile - ccflags-y := -Os - ccflags-$(CONFIG_ACPI_DEBUG) += -DACPI_DEBUG_OUTPUT + # drivers/acpi/acpica/Makefile + ccflags-y := -Os -D_LINUX -DBUILDING_ACPICA + ccflags-$(CONFIG_ACPI_DEBUG) += -DACPI_DEBUG_OUTPUT This variable is necessary because the top Makefile owns the variable $(KBUILD_CFLAGS) and uses it for compilation flags for the -- cgit v1.2.3 From 9b756a9d07fd34be189ccc3d3fc6dfc686e532fb Mon Sep 17 00:00:00 2001 From: Mauro Carvalho Chehab Date: Mon, 24 Jul 2017 09:09:24 -0300 Subject: scripts/sphinx-pre-install: add minimum support for RHEL RHEL 7.x and clone distros are shipped with Sphinx 1.1.x, with is incompatible with Kernel ReST markups. So, on those systems, the only alternative is to install it via a Python virtual environment. While seeking for "pip" on CentOS 7.3, I noticed that it is not really needed, as python-virtualenv has its version packaged there already. So, remove this from the list of requirements for all distributions. With regards to PDF, we need at least texlive-tabulary extension, but that is not shipped there (at least on CentOS). So, disable PDF packages as a whole. Please notice, however, that texlive + amsmath is needed for ReST to properly handle ReST ".. math::" tags. Yet, Sphinx fall back to display the LaTeX math expressions as-is, if such extension is not available. So, let's just disable all texlive packages as a whole. Signed-off-by: Mauro Carvalho Chehab Signed-off-by: Jonathan Corbet --- scripts/sphinx-pre-install | 54 +++++++++++++++++++++++++++++++++++----------- 1 file changed, 42 insertions(+), 12 deletions(-) diff --git a/scripts/sphinx-pre-install b/scripts/sphinx-pre-install index 5d2799dcfceb..677756ae34c9 100755 --- a/scripts/sphinx-pre-install +++ b/scripts/sphinx-pre-install @@ -216,7 +216,6 @@ sub check_sphinx() $prog = findprog("virtualenv-3.5") if (!$prog); check_program("virtualenv", 0) if (!$prog); - check_program("pip", 0) if (!findprog("pip3")); $need_sphinx = 1; } else { add_package("python-sphinx", 0); @@ -256,7 +255,6 @@ sub give_debian_hints() "python-sphinx" => "python3-sphinx", "sphinx_rtd_theme" => "python3-sphinx-rtd-theme", "virtualenv" => "virtualenv", - "pip" => "python3-pip", "dot" => "graphviz", "convert" => "imagemagick", "Pod::Usage" => "perl-modules", @@ -282,7 +280,6 @@ sub give_redhat_hints() "python-sphinx" => "python3-sphinx", "sphinx_rtd_theme" => "python3-sphinx_rtd_theme", "virtualenv" => "python3-virtualenv", - "pip" => "python3-pip", "dot" => "graphviz", "convert" => "ImageMagick", "Pod::Usage" => "perl-Pod-Usage", @@ -302,6 +299,13 @@ sub give_redhat_hints() "dejavu-sans-mono-fonts", ); + # + # Checks valid for RHEL/CentOS version 7.x. + # + if (! $system_release =~ /Fedora/) { + $map{"virtualenv"} = "python-virtualenv"; + } + my $release; $release = $1 if ($system_release =~ /Fedora\s+release\s+(\d+)/); @@ -312,7 +316,14 @@ sub give_redhat_hints() check_missing(\%map); return if (!$need && !$optional); - printf("You should run:\n\n\tsudo dnf install -y $install\n"); + + if ($release >= 18) { + # dnf, for Fedora 18+ + printf("You should run:\n\n\tsudo dnf install -y $install\n"); + } else { + # yum, for RHEL (and clones) or Fedora version < 18 + printf("You should run:\n\n\tsudo yum install -y $install\n"); + } } sub give_opensuse_hints() @@ -321,7 +332,6 @@ sub give_opensuse_hints() "python-sphinx" => "python3-sphinx", "sphinx_rtd_theme" => "python3-sphinx_rtd_theme", "virtualenv" => "python3-virtualenv", - "pip" => "python3-pip", "dot" => "graphviz", "convert" => "ImageMagick", "Pod::Usage" => "perl-Pod-Usage", @@ -360,7 +370,6 @@ sub give_mageia_hints() "python-sphinx" => "python3-sphinx", "sphinx_rtd_theme" => "python3-sphinx_rtd_theme", "virtualenv" => "python3-virtualenv", - "pip" => "python3-pip", "dot" => "graphviz", "convert" => "ImageMagick", "Pod::Usage" => "perl-Pod-Usage", @@ -372,8 +381,6 @@ sub give_mageia_hints() "texlive-fontsextra", ); - my $release; - check_rpm_missing(\@tex_pkgs, 1) if ($pdf); check_missing(\%map); @@ -386,7 +393,6 @@ sub give_arch_linux_hints() my %map = ( "sphinx_rtd_theme" => "python-sphinx_rtd_theme", "virtualenv" => "python-virtualenv", - "pip" => "python-pip", "dot" => "graphviz", "convert" => "imagemagick", "xelatex" => "texlive-bin", @@ -410,7 +416,6 @@ sub give_gentoo_hints() my %map = ( "sphinx_rtd_theme" => "dev-python/sphinx_rtd_theme", "virtualenv" => "dev-python/virtualenv", - "pip" => "dev-python/pip", "dot" => "media-gfx/graphviz", "convert" => "media-gfx/imagemagick", "xelatex" => "dev-texlive/texlive-xetex media-fonts/dejavu", @@ -438,6 +443,18 @@ sub check_distros() give_redhat_hints; return; } + if ($system_release =~ /CentOS/) { + give_redhat_hints; + return; + } + if ($system_release =~ /Scientific Linux/) { + give_redhat_hints; + return; + } + if ($system_release =~ /Oracle Linux Server/) { + give_redhat_hints; + return; + } if ($system_release =~ /Fedora/) { give_redhat_hints; return; @@ -488,9 +505,22 @@ sub check_distros() sub check_needs() { if ($system_release) { - print "Checking if the needed tools for $system_release are available\n"; + print "Detected OS: $system_release.\n"; } else { - print "Checking if the needed tools are present\n"; + print "Unknown OS\n"; + } + + # RHEL 7.x and clones have Sphinx version 1.1.x and incomplete texlive + if (($system_release =~ /Red Hat Enterprise Linux/) || + ($system_release =~ /CentOS/) || + ($system_release =~ /Scientific Linux/) || + ($system_release =~ /Oracle Linux Server/)) { + $virtualenv = 1; + $pdf = 0; + + printf("NOTE: On this distro, Sphinx and TexLive shipped versions are incompatible\n"); + printf("with doc build. So, use Sphinx via a Python virtual environment.\n\n"); + printf("This script can't install a TexLive version that would provide PDF.\n"); } # Check for needed programs/tools -- cgit v1.2.3 From 720688c449997cc58c027ffff2b0323a422b9ca0 Mon Sep 17 00:00:00 2001 From: Thierry Reding Date: Mon, 24 Jul 2017 11:01:31 +0200 Subject: docs: driver-api: Remove trailing blank line There's no use for this blank line at the end of the file. Remove it. Signed-off-by: Thierry Reding Signed-off-by: Jonathan Corbet --- Documentation/driver-api/miscellaneous.rst | 1 - 1 file changed, 1 deletion(-) diff --git a/Documentation/driver-api/miscellaneous.rst b/Documentation/driver-api/miscellaneous.rst index 8da7d115bafc..304ffb146cf9 100644 --- a/Documentation/driver-api/miscellaneous.rst +++ b/Documentation/driver-api/miscellaneous.rst @@ -47,4 +47,3 @@ used by one consumer at a time. .. kernel-doc:: drivers/pwm/core.c :export: - -- cgit v1.2.3 From 8ac5ac1b0ef8709b78b9242903b070f1c3b2cedb Mon Sep 17 00:00:00 2001 From: Jian-Hong Pan Date: Mon, 7 Aug 2017 00:28:29 +0800 Subject: doc: linux-wpan: Change the old function names to the lastest function names The function declaration in the lastest include/net/mac802154.h has been changed since v3.19. ieee802154_alloc_device => ieee802154_alloc_hw ieee802154_free_device => ieee802154_free_hw ieee802154_register_device => ieee802154_register_hw ieee802154_unregister_device => ieee802154_unregister_hw However, the description in the Device drivers API section of Documentation/networking/ieee802154.txt is still in the state of v3.18.63. Signed-off-by: Jian-Hong Pan Acked-by: Stefan Schmidt Signed-off-by: Jonathan Corbet --- Documentation/networking/ieee802154.txt | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/Documentation/networking/ieee802154.txt b/Documentation/networking/ieee802154.txt index c4114346f054..057e9fdbfac9 100644 --- a/Documentation/networking/ieee802154.txt +++ b/Documentation/networking/ieee802154.txt @@ -84,17 +84,17 @@ Device drivers API ================== The include/net/mac802154.h defines following functions: - - struct ieee802154_dev *ieee802154_alloc_device - (size_t priv_size, struct ieee802154_ops *ops): - allocation of IEEE 802.15.4 compatible device + - struct ieee802154_hw * + ieee802154_alloc_hw(size_t priv_data_len, const struct ieee802154_ops *ops): + allocation of IEEE 802.15.4 compatible hardware device - - void ieee802154_free_device(struct ieee802154_dev *dev): - freeing allocated device + - void ieee802154_free_hw(struct ieee802154_hw *hw): + freeing allocated hardware device - - int ieee802154_register_device(struct ieee802154_dev *dev): - register PHY in the system + - int ieee802154_register_hw(struct ieee802154_hw *hw): + register PHY which is the allocated hardware device, in the system - - void ieee802154_unregister_device(struct ieee802154_dev *dev): + - void ieee802154_unregister_hw(struct ieee802154_hw *hw): freeing registered PHY Moreover IEEE 802.15.4 device operations structure should be filled. -- cgit v1.2.3 From adf31eebd75e9e7c2eda222695637398b3872a68 Mon Sep 17 00:00:00 2001 From: Josh Holland Date: Sat, 19 Aug 2017 16:21:06 +0100 Subject: docs: Fix paths in security/keys Several paths in the security/keys documentation were incorrect. Signed-off-by: Josh Holland Signed-off-by: Jonathan Corbet --- Documentation/security/keys/core.rst | 4 ++-- Documentation/security/keys/request-key.rst | 2 +- Documentation/security/keys/trusted-encrypted.rst | 2 +- 3 files changed, 4 insertions(+), 4 deletions(-) diff --git a/Documentation/security/keys/core.rst b/Documentation/security/keys/core.rst index 1648fa80b3bf..312f981fa4a0 100644 --- a/Documentation/security/keys/core.rst +++ b/Documentation/security/keys/core.rst @@ -443,7 +443,7 @@ The main syscalls are: /sbin/request-key will be invoked in an attempt to obtain a key. The callout_info string will be passed as an argument to the program. - See also Documentation/security/keys-request-key.txt. + See also Documentation/security/keys/request-key.rst. The keyctl syscall functions are: @@ -973,7 +973,7 @@ payload contents" for more information. If successful, the key will have been attached to the default keyring for implicitly obtained request-key keys, as set by KEYCTL_SET_REQKEY_KEYRING. - See also Documentation/security/keys-request-key.txt. + See also Documentation/security/keys/request-key.rst. * To search for a key, passing auxiliary data to the upcaller, call:: diff --git a/Documentation/security/keys/request-key.rst b/Documentation/security/keys/request-key.rst index aba32784174c..b2d16abaa9e9 100644 --- a/Documentation/security/keys/request-key.rst +++ b/Documentation/security/keys/request-key.rst @@ -3,7 +3,7 @@ Key Request Service =================== The key request service is part of the key retention service (refer to -Documentation/security/keys.txt). This document explains more fully how +Documentation/security/core.rst). This document explains more fully how the requesting algorithm works. The process starts by either the kernel requesting a service by calling diff --git a/Documentation/security/keys/trusted-encrypted.rst b/Documentation/security/keys/trusted-encrypted.rst index 7b503831bdea..3bb24e09a332 100644 --- a/Documentation/security/keys/trusted-encrypted.rst +++ b/Documentation/security/keys/trusted-encrypted.rst @@ -172,4 +172,4 @@ Other uses for trusted and encrypted keys, such as for disk and file encryption are anticipated. In particular the new format 'ecryptfs' has been defined in in order to use encrypted keys to mount an eCryptfs filesystem. More details about the usage can be found in the file -``Documentation/security/keys-ecryptfs.txt``. +``Documentation/security/keys/ecryptfs.rst``. -- cgit v1.2.3 From 92a037f01ab3e7b8e5b0533faf42d4bbfce04e98 Mon Sep 17 00:00:00 2001 From: Mauro Carvalho Chehab Date: Sun, 16 Jul 2017 19:08:06 -0300 Subject: doc: Makefile: if sphinx is not found, run a check script Right now, if the building system doesn't find Sphinx, it bails out, without providing any instructions about what should be done. Instead, run a script, providing some guidance about the steps needed for Sphinx build to work. Signed-off-by: Mauro Carvalho Chehab Signed-off-by: Jonathan Corbet --- Documentation/Makefile | 2 ++ 1 file changed, 2 insertions(+) diff --git a/Documentation/Makefile b/Documentation/Makefile index d75c00e3aadb..85f7856f0092 100644 --- a/Documentation/Makefile +++ b/Documentation/Makefile @@ -22,6 +22,8 @@ ifeq ($(HAVE_SPHINX),0) .DEFAULT: $(warning The '$(SPHINXBUILD)' command was not found. Make sure you have Sphinx installed and in PATH, or set the SPHINXBUILD make variable to point to the full path of the '$(SPHINXBUILD)' executable.) + @echo + @./scripts/sphinx-pre-install @echo " SKIP Sphinx $@ target." else # HAVE_SPHINX -- cgit v1.2.3 From 5148e6ab76ea38e5c06376afb1340acdced52e4c Mon Sep 17 00:00:00 2001 From: Mauro Carvalho Chehab Date: Wed, 23 Aug 2017 04:05:43 -0300 Subject: docs-rst: pdf: use same vertical margin on all Sphinx versions Currently, on Sphinx up to version 1.4, pdf output uses a vertical margin of 1 inch. For upper versions, it uses a margin of 0.5 inches. That causes both page headers and footers to be very close to the margin of the sheet. Not all printers support writing like that. Also, there's no reason why the layout for newer versions would be different than for previous ones. So, standardize it, by always setting to 1 inch. Signed-off-by: Mauro Carvalho Chehab Signed-off-by: Jonathan Corbet --- Documentation/conf.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Documentation/conf.py b/Documentation/conf.py index 71b032bb44fd..8e74d68037a5 100644 --- a/Documentation/conf.py +++ b/Documentation/conf.py @@ -344,7 +344,7 @@ if major == 1 and minor > 3: if major == 1 and minor <= 4: latex_elements['preamble'] += '\\usepackage[margin=0.5in, top=1in, bottom=1in]{geometry}' elif major == 1 and (minor > 5 or (minor == 5 and patch >= 3)): - latex_elements['sphinxsetup'] = 'hmargin=0.5in, vmargin=0.5in' + latex_elements['sphinxsetup'] = 'hmargin=0.5in, vmargin=1in' # Grouping the document tree into LaTeX files. List of tuples -- cgit v1.2.3 From be629b441da555eb75b5d41d48caca02da52aaae Mon Sep 17 00:00:00 2001 From: Naren Date: Wed, 16 Aug 2017 15:45:57 -0700 Subject: NVMEM documentation fix: A minor typo This is a minor patch that fixes the following typo in the NVMEM documentation: When a consumers no longer needs the NVMEM to, When a consumer no longer needs the NVMEM Signed-off-by: Narendran Sankaran Signed-off-by: Jonathan Corbet --- Documentation/nvmem/nvmem.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Documentation/nvmem/nvmem.txt b/Documentation/nvmem/nvmem.txt index dbd40d879239..8d8d8f58f96f 100644 --- a/Documentation/nvmem/nvmem.txt +++ b/Documentation/nvmem/nvmem.txt @@ -112,7 +112,7 @@ take nvmem_device as parameter. 5. Releasing a reference to the NVMEM ===================================== -When a consumers no longer needs the NVMEM, it has to release the reference +When a consumer no longer needs the NVMEM, it has to release the reference to the NVMEM it has obtained using the APIs mentioned in the above section. The NVMEM framework provides 2 APIs to release a reference to the NVMEM. -- cgit v1.2.3 From 5303b8d3a2eeaa5fddd20e1061dc2ec4707aaa67 Mon Sep 17 00:00:00 2001 From: Markus Heiser Date: Mon, 14 Aug 2017 10:15:16 +0200 Subject: docs: fix nested numbering in the TOC With Sphinx 1.6 nested numbering is reported as warning:: ./input/joydev/index.rst:13: WARNING: input/joydev/joystick-api is already assigned section numbers (nested numbered toctree?) ./media/uapi/cec/cec-funcs.rst:8: WARNING: media/uapi/cec/cec-func-open is already assigned section numbers (nested numbered toctree?) ./media/uapi/cec/cec-funcs.rst:8: WARNING: media/uapi/cec/cec-func-close is already assigned section numbers (nested numbered toctree?) ./media/uapi/cec/cec-funcs.rst:8: WARNING: media/uapi/cec/cec-func-ioctl is already assigned section numbers (nested numbered toctree?) ./media/uapi/cec/cec-funcs.rst:8: WARNING: media/uapi/cec/cec-func-poll is already assigned section numbers (nested numbered toctree?) ./media/uapi/cec/cec-funcs.rst:8: WARNING: media/uapi/cec/cec-ioc-adap-g-caps is already assigned section numbers (nested numbered toctree?) ./media/uapi/cec/cec-funcs.rst:8: WARNING: media/uapi/cec/cec-ioc-adap-g-log-addrs is already assigned section numbers (nested numbered toctree?) ./media/uapi/cec/cec-funcs.rst:8: WARNING: media/uapi/cec/cec-ioc-adap-g-phys-addr is already assigned section numbers (nested numbered toctree?) ./media/uapi/cec/cec-funcs.rst:8: WARNING: media/uapi/cec/cec-ioc-dqevent is already assigned section numbers (nested numbered toctree?) ./media/uapi/cec/cec-funcs.rst:8: WARNING: media/uapi/cec/cec-ioc-g-mode is already assigned section numbers (nested numbered toctree?) ./media/uapi/cec/cec-funcs.rst:8: WARNING: media/uapi/cec/cec-ioc-receive is already assigned section numbers (nested numbered toctree?) TOC numbering is already set in:: ./input/devices/index.rst:9: ./media/uapi/cec/cec-api.rst:19: I guess the nested numbering in: ./input/joydev/index.rst ./media/uapi/cec/cec-funcs.rst is just a C&P typo, so lets remove it. Signed-off-by: Markus Heiser Signed-off-by: Jonathan Corbet --- Documentation/input/joydev/index.rst | 1 - Documentation/media/uapi/cec/cec-funcs.rst | 1 - 2 files changed, 2 deletions(-) diff --git a/Documentation/input/joydev/index.rst b/Documentation/input/joydev/index.rst index 8d9666c7561c..ebcff43056e2 100644 --- a/Documentation/input/joydev/index.rst +++ b/Documentation/input/joydev/index.rst @@ -12,7 +12,6 @@ Linux Joystick support .. toctree:: :maxdepth: 3 - :numbered: joystick joystick-api diff --git a/Documentation/media/uapi/cec/cec-funcs.rst b/Documentation/media/uapi/cec/cec-funcs.rst index 5b7630f2e076..6d696cead5cb 100644 --- a/Documentation/media/uapi/cec/cec-funcs.rst +++ b/Documentation/media/uapi/cec/cec-funcs.rst @@ -7,7 +7,6 @@ Function Reference .. toctree:: :maxdepth: 1 - :numbered: cec-func-open cec-func-close -- cgit v1.2.3 From 0e4c2b75897c6a621546b4f11a42c6072e833f81 Mon Sep 17 00:00:00 2001 From: Markus Heiser Date: Mon, 14 Aug 2017 10:53:33 +0200 Subject: docs: fix minimal sphinx version in conf.py according to what Documentation/doc-guide/sphinx.rst says:: The ReST markups currently used by the Documentation/ files are meant to be built with ``Sphinx`` version 1.3 or upper. Signed-off-by: Markus Heiser Signed-off-by: Jonathan Corbet --- Documentation/conf.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Documentation/conf.py b/Documentation/conf.py index 8e74d68037a5..9e941be37b94 100644 --- a/Documentation/conf.py +++ b/Documentation/conf.py @@ -29,7 +29,7 @@ from load_config import loadConfig # -- General configuration ------------------------------------------------ # If your documentation needs a minimal Sphinx version, state it here. -needs_sphinx = '1.2' +needs_sphinx = '1.3' # Add any Sphinx extension module names here, as strings. They can be # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom -- cgit v1.2.3 From f1824df12ecd495b25c8c116876e201ac764ecea Mon Sep 17 00:00:00 2001 From: Alex Shi Date: Mon, 31 Jul 2017 09:50:53 +0800 Subject: rtmutex: update rt-mutex-design The rt-mutex-design documents didn't gotten meaningful update from its first version. Even after owner's pending bit was removed in commit 8161239a8bcc ("rtmutex: Simplify PI algorithm and make highest prio task get lock") and priority list 'plist' changed to rbtree. And Peter Zijlstra did some clean up and fix for deadline task changes on tip tree. So update it to latest code and make it meaningful. Steven Rostedt and Sebastian Siewior gave much of comments and input in this doc. Thanks! Signed-off-by: Alex Shi Cc: Steven Rostedt Cc: Sebastian Siewior Cc: Mathieu Poirier Cc: Juri Lelli Cc: Thomas Gleixner To: linux-doc@vger.kernel.org To: linux-kernel@vger.kernel.org To: Jonathan Corbet To: Ingo Molnar To: Peter Zijlstra Signed-off-by: Jonathan Corbet --- Documentation/locking/rt-mutex-design.txt | 432 ++++++++---------------------- 1 file changed, 105 insertions(+), 327 deletions(-) diff --git a/Documentation/locking/rt-mutex-design.txt b/Documentation/locking/rt-mutex-design.txt index 8666070d3189..6c6e8c2410de 100644 --- a/Documentation/locking/rt-mutex-design.txt +++ b/Documentation/locking/rt-mutex-design.txt @@ -97,9 +97,9 @@ waiter - A waiter is a struct that is stored on the stack of a blocked a process being blocked on the mutex, it is fine to allocate the waiter on the process's stack (local variable). This structure holds a pointer to the task, as well as the mutex that - the task is blocked on. It also has the plist node structures to - place the task in the waiter_list of a mutex as well as the - pi_list of a mutex owner task (described below). + the task is blocked on. It also has rbtree node structures to + place the task in the waiters rbtree of a mutex as well as the + pi_waiters rbtree of a mutex owner task (described below). waiter is sometimes used in reference to the task that is waiting on a mutex. This is the same as waiter->task. @@ -179,53 +179,34 @@ again. | F->L5-+ +If process G has the highest priority in the chain, then all the tasks up +the chain (A and B in this example), must have their priorities increased +to that of G. -Plist ------ - -Before I go further and talk about how the PI chain is stored through lists -on both mutexes and processes, I'll explain the plist. This is similar to -the struct list_head functionality that is already in the kernel. -The implementation of plist is out of scope for this document, but it is -very important to understand what it does. - -There are a few differences between plist and list, the most important one -being that plist is a priority sorted linked list. This means that the -priorities of the plist are sorted, such that it takes O(1) to retrieve the -highest priority item in the list. Obviously this is useful to store processes -based on their priorities. - -Another difference, which is important for implementation, is that, unlike -list, the head of the list is a different element than the nodes of a list. -So the head of the list is declared as struct plist_head and nodes that will -be added to the list are declared as struct plist_node. - - -Mutex Waiter List +Mutex Waiters Tree ----------------- -Every mutex keeps track of all the waiters that are blocked on itself. The mutex -has a plist to store these waiters by priority. This list is protected by -a spin lock that is located in the struct of the mutex. This lock is called -wait_lock. Since the modification of the waiter list is never done in -interrupt context, the wait_lock can be taken without disabling interrupts. +Every mutex keeps track of all the waiters that are blocked on itself. The +mutex has a rbtree to store these waiters by priority. This tree is protected +by a spin lock that is located in the struct of the mutex. This lock is called +wait_lock. -Task PI List +Task PI Tree ------------ -To keep track of the PI chains, each process has its own PI list. This is -a list of all top waiters of the mutexes that are owned by the process. -Note that this list only holds the top waiters and not all waiters that are +To keep track of the PI chains, each process has its own PI rbtree. This is +a tree of all top waiters of the mutexes that are owned by the process. +Note that this tree only holds the top waiters and not all waiters that are blocked on mutexes owned by the process. -The top of the task's PI list is always the highest priority task that +The top of the task's PI tree is always the highest priority task that is waiting on a mutex that is owned by the task. So if the task has inherited a priority, it will always be the priority of the task that is -at the top of this list. +at the top of this tree. -This list is stored in the task structure of a process as a plist called -pi_list. This list is protected by a spin lock also in the task structure, +This tree is stored in the task structure of a process as a rbtree called +pi_waiters. It is protected by a spin lock also in the task structure, called pi_lock. This lock may also be taken in interrupt context, so when locking the pi_lock, interrupts must be disabled. @@ -312,15 +293,12 @@ Mutex owner and flags The mutex structure contains a pointer to the owner of the mutex. If the mutex is not owned, this owner is set to NULL. Since all architectures -have the task structure on at least a four byte alignment (and if this is -not true, the rtmutex.c code will be broken!), this allows for the two -least significant bits to be used as flags. This part is also described -in Documentation/rt-mutex.txt, but will also be briefly described here. - -Bit 0 is used as the "Pending Owner" flag. This is described later. -Bit 1 is used as the "Has Waiters" flags. This is also described later - in more detail, but is set whenever there are waiters on a mutex. +have the task structure on at least a two byte alignment (and if this is +not true, the rtmutex.c code will be broken!), this allows for the least +significant bit to be used as a flag. Bit 0 is used as the "Has Waiters" +flag. It's set whenever there are waiters on a mutex. +See Documentation/locking/rt-mutex.txt for further details. cmpxchg Tricks -------------- @@ -359,40 +337,31 @@ Priority adjustments -------------------- The implementation of the PI code in rtmutex.c has several places that a -process must adjust its priority. With the help of the pi_list of a +process must adjust its priority. With the help of the pi_waiters of a process this is rather easy to know what needs to be adjusted. -The functions implementing the task adjustments are rt_mutex_adjust_prio, -__rt_mutex_adjust_prio (same as the former, but expects the task pi_lock -to already be taken), rt_mutex_getprio, and rt_mutex_setprio. +The functions implementing the task adjustments are rt_mutex_adjust_prio +and rt_mutex_setprio. rt_mutex_setprio is only used in rt_mutex_adjust_prio. -rt_mutex_getprio and rt_mutex_setprio are only used in __rt_mutex_adjust_prio. +rt_mutex_adjust_prio examines the priority of the task, and the highest +priority process that is waiting any of mutexes owned by the task. Since +the pi_waiters of a task holds an order by priority of all the top waiters +of all the mutexes that the task owns, we simply need to compare the top +pi waiter to its own normal/deadline priority and take the higher one. +Then rt_mutex_setprio is called to adjust the priority of the task to the +new priority. Note that rt_mutex_setprio is defined in kernel/sched/core.c +to implement the actual change in priority. -rt_mutex_getprio returns the priority that the task should have. Either the -task's own normal priority, or if a process of a higher priority is waiting on -a mutex owned by the task, then that higher priority should be returned. -Since the pi_list of a task holds an order by priority list of all the top -waiters of all the mutexes that the task owns, rt_mutex_getprio simply needs -to compare the top pi waiter to its own normal priority, and return the higher -priority back. +(Note: For the "prio" field in task_struct, the lower the number, the + higher the priority. A "prio" of 5 is of higher priority than a + "prio" of 10.) -(Note: if looking at the code, you will notice that the lower number of - prio is returned. This is because the prio field in the task structure - is an inverse order of the actual priority. So a "prio" of 5 is - of higher priority than a "prio" of 10.) - -__rt_mutex_adjust_prio examines the result of rt_mutex_getprio, and if the -result does not equal the task's current priority, then rt_mutex_setprio -is called to adjust the priority of the task to the new priority. -Note that rt_mutex_setprio is defined in kernel/sched/core.c to implement the -actual change in priority. - -It is interesting to note that __rt_mutex_adjust_prio can either increase +It is interesting to note that rt_mutex_adjust_prio can either increase or decrease the priority of the task. In the case that a higher priority -process has just blocked on a mutex owned by the task, __rt_mutex_adjust_prio +process has just blocked on a mutex owned by the task, rt_mutex_adjust_prio would increase/boost the task's priority. But if a higher priority task were for some reason to leave the mutex (timeout or signal), this same function -would decrease/unboost the priority of the task. That is because the pi_list +would decrease/unboost the priority of the task. That is because the pi_waiters always contains the highest priority task that is waiting on a mutex owned by the task, so we only need to compare the priority of that top pi waiter to the normal priority of the given task. @@ -412,9 +381,10 @@ priorities. rt_mutex_adjust_prio_chain is called with a task to be checked for PI (de)boosting (the owner of a mutex that a process is blocking on), a flag to -check for deadlocking, the mutex that the task owns, and a pointer to a waiter +check for deadlocking, the mutex that the task owns, a pointer to a waiter that is the process's waiter struct that is blocked on the mutex (although this -parameter may be NULL for deboosting). +parameter may be NULL for deboosting), a pointer to the mutex on which the task +is blocked, and a top_task as the top waiter of the mutex. For this explanation, I will not mention deadlock detection. This explanation will try to stay at a high level. @@ -424,133 +394,14 @@ that the state of the owner and lock can change when entered into this function. Before this function is called, the task has already had rt_mutex_adjust_prio performed on it. This means that the task is set to the priority that it -should be at, but the plist nodes of the task's waiter have not been updated -with the new priorities, and that this task may not be in the proper locations -in the pi_lists and wait_lists that the task is blocked on. This function +should be at, but the rbtree nodes of the task's waiter have not been updated +with the new priorities, and this task may not be in the proper locations +in the pi_waiters and waiters trees that the task is blocked on. This function solves all that. -A loop is entered, where task is the owner to be checked for PI changes that -was passed by parameter (for the first iteration). The pi_lock of this task is -taken to prevent any more changes to the pi_list of the task. This also -prevents new tasks from completing the blocking on a mutex that is owned by this -task. - -If the task is not blocked on a mutex then the loop is exited. We are at -the top of the PI chain. - -A check is now done to see if the original waiter (the process that is blocked -on the current mutex) is the top pi waiter of the task. That is, is this -waiter on the top of the task's pi_list. If it is not, it either means that -there is another process higher in priority that is blocked on one of the -mutexes that the task owns, or that the waiter has just woken up via a signal -or timeout and has left the PI chain. In either case, the loop is exited, since -we don't need to do any more changes to the priority of the current task, or any -task that owns a mutex that this current task is waiting on. A priority chain -walk is only needed when a new top pi waiter is made to a task. - -The next check sees if the task's waiter plist node has the priority equal to -the priority the task is set at. If they are equal, then we are done with -the loop. Remember that the function started with the priority of the -task adjusted, but the plist nodes that hold the task in other processes -pi_lists have not been adjusted. - -Next, we look at the mutex that the task is blocked on. The mutex's wait_lock -is taken. This is done by a spin_trylock, because the locking order of the -pi_lock and wait_lock goes in the opposite direction. If we fail to grab the -lock, the pi_lock is released, and we restart the loop. - -Now that we have both the pi_lock of the task as well as the wait_lock of -the mutex the task is blocked on, we update the task's waiter's plist node -that is located on the mutex's wait_list. - -Now we release the pi_lock of the task. - -Next the owner of the mutex has its pi_lock taken, so we can update the -task's entry in the owner's pi_list. If the task is the highest priority -process on the mutex's wait_list, then we remove the previous top waiter -from the owner's pi_list, and replace it with the task. - -Note: It is possible that the task was the current top waiter on the mutex, - in which case the task is not yet on the pi_list of the waiter. This - is OK, since plist_del does nothing if the plist node is not on any - list. - -If the task was not the top waiter of the mutex, but it was before we -did the priority updates, that means we are deboosting/lowering the -task. In this case, the task is removed from the pi_list of the owner, -and the new top waiter is added. - -Lastly, we unlock both the pi_lock of the task, as well as the mutex's -wait_lock, and continue the loop again. On the next iteration of the -loop, the previous owner of the mutex will be the task that will be -processed. - -Note: One might think that the owner of this mutex might have changed - since we just grab the mutex's wait_lock. And one could be right. - The important thing to remember is that the owner could not have - become the task that is being processed in the PI chain, since - we have taken that task's pi_lock at the beginning of the loop. - So as long as there is an owner of this mutex that is not the same - process as the tasked being worked on, we are OK. - - Looking closely at the code, one might be confused. The check for the - end of the PI chain is when the task isn't blocked on anything or the - task's waiter structure "task" element is NULL. This check is - protected only by the task's pi_lock. But the code to unlock the mutex - sets the task's waiter structure "task" element to NULL with only - the protection of the mutex's wait_lock, which was not taken yet. - Isn't this a race condition if the task becomes the new owner? - - The answer is No! The trick is the spin_trylock of the mutex's - wait_lock. If we fail that lock, we release the pi_lock of the - task and continue the loop, doing the end of PI chain check again. - - In the code to release the lock, the wait_lock of the mutex is held - the entire time, and it is not let go when we grab the pi_lock of the - new owner of the mutex. So if the switch of a new owner were to happen - after the check for end of the PI chain and the grabbing of the - wait_lock, the unlocking code would spin on the new owner's pi_lock - but never give up the wait_lock. So the PI chain loop is guaranteed to - fail the spin_trylock on the wait_lock, release the pi_lock, and - try again. - - If you don't quite understand the above, that's OK. You don't have to, - unless you really want to make a proof out of it ;) - - -Pending Owners and Lock stealing --------------------------------- - -One of the flags in the owner field of the mutex structure is "Pending Owner". -What this means is that an owner was chosen by the process releasing the -mutex, but that owner has yet to wake up and actually take the mutex. - -Why is this important? Why can't we just give the mutex to another process -and be done with it? - -The PI code is to help with real-time processes, and to let the highest -priority process run as long as possible with little latencies and delays. -If a high priority process owns a mutex that a lower priority process is -blocked on, when the mutex is released it would be given to the lower priority -process. What if the higher priority process wants to take that mutex again. -The high priority process would fail to take that mutex that it just gave up -and it would need to boost the lower priority process to run with full -latency of that critical section (since the low priority process just entered -it). - -There's no reason a high priority process that gives up a mutex should be -penalized if it tries to take that mutex again. If the new owner of the -mutex has not woken up yet, there's no reason that the higher priority process -could not take that mutex away. - -To solve this, we introduced Pending Ownership and Lock Stealing. When a -new process is given a mutex that it was blocked on, it is only given -pending ownership. This means that it's the new owner, unless a higher -priority process comes in and tries to grab that mutex. If a higher priority -process does come along and wants that mutex, we let the higher priority -process "steal" the mutex from the pending owner (only if it is still pending) -and continue with the mutex. - +The main operation of this function is summarized by Thomas Gleixner in +rtmutex.c. See the 'Chain walk basics and protection scope' comment for further +details. Taking of a mutex (The walk through) ------------------------------------ @@ -563,14 +414,14 @@ done when we have CMPXCHG enabled (otherwise the fast taking automatically fails). Only when the owner field of the mutex is NULL can the lock be taken with the CMPXCHG and nothing else needs to be done. -If there is contention on the lock, whether it is owned or pending owner -we go about the slow path (rt_mutex_slowlock). +If there is contention on the lock, we go about the slow path +(rt_mutex_slowlock). The slow path function is where the task's waiter structure is created on the stack. This is because the waiter structure is only needed for the scope of this function. The waiter structure holds the nodes to store -the task on the wait_list of the mutex, and if need be, the pi_list of -the owner. +the task on the waiters tree of the mutex, and if need be, the pi_waiters +tree of the owner. The wait_lock of the mutex is taken since the slow path of unlocking the mutex also takes this lock. @@ -581,102 +432,45 @@ contention). try_to_take_rt_mutex is used every time the task tries to grab a mutex in the slow path. The first thing that is done here is an atomic setting of -the "Has Waiters" flag of the mutex's owner field. Yes, this could really -be false, because if the mutex has no owner, there are no waiters and -the current task also won't have any waiters. But we don't have the lock -yet, so we assume we are going to be a waiter. The reason for this is to -play nice for those architectures that do have CMPXCHG. By setting this flag -now, the owner of the mutex can't release the mutex without going into the -slow unlock path, and it would then need to grab the wait_lock, which this -code currently holds. So setting the "Has Waiters" flag forces the owner -to synchronize with this code. - -Now that we know that we can't have any races with the owner releasing the -mutex, we check to see if we can take the ownership. This is done if the -mutex doesn't have a owner, or if we can steal the mutex from a pending -owner. Let's look at the situations we have here. - - 1) Has owner that is pending - ---------------------------- - - The mutex has a owner, but it hasn't woken up and the mutex flag - "Pending Owner" is set. The first check is to see if the owner isn't the - current task. This is because this function is also used for the pending - owner to grab the mutex. When a pending owner wakes up, it checks to see - if it can take the mutex, and this is done if the owner is already set to - itself. If so, we succeed and leave the function, clearing the "Pending - Owner" bit. - - If the pending owner is not current, we check to see if the current priority is - higher than the pending owner. If not, we fail the function and return. - - There's also something special about a pending owner. That is a pending owner - is never blocked on a mutex. So there is no PI chain to worry about. It also - means that if the mutex doesn't have any waiters, there's no accounting needed - to update the pending owner's pi_list, since we only worry about processes - blocked on the current mutex. - - If there are waiters on this mutex, and we just stole the ownership, we need - to take the top waiter, remove it from the pi_list of the pending owner, and - add it to the current pi_list. Note that at this moment, the pending owner - is no longer on the list of waiters. This is fine, since the pending owner - would add itself back when it realizes that it had the ownership stolen - from itself. When the pending owner tries to grab the mutex, it will fail - in try_to_take_rt_mutex if the owner field points to another process. - - 2) No owner - ----------- - - If there is no owner (or we successfully stole the lock), we set the owner - of the mutex to current, and set the flag of "Has Waiters" if the current - mutex actually has waiters, or we clear the flag if it doesn't. See, it was - OK that we set that flag early, since now it is cleared. - - 3) Failed to grab ownership - --------------------------- - - The most interesting case is when we fail to take ownership. This means that - there exists an owner, or there's a pending owner with equal or higher - priority than the current task. - -We'll continue on the failed case. - -If the mutex has a timeout, we set up a timer to go off to break us out -of this mutex if we failed to get it after a specified amount of time. - -Now we enter a loop that will continue to try to take ownership of the mutex, or -fail from a timeout or signal. - -Once again we try to take the mutex. This will usually fail the first time -in the loop, since it had just failed to get the mutex. But the second time -in the loop, this would likely succeed, since the task would likely be -the pending owner. - -If the mutex is TASK_INTERRUPTIBLE a check for signals and timeout is done -here. - -The waiter structure has a "task" field that points to the task that is blocked -on the mutex. This field can be NULL the first time it goes through the loop -or if the task is a pending owner and had its mutex stolen. If the "task" -field is NULL then we need to set up the accounting for it. +the "Has Waiters" flag of the mutex's owner field. By setting this flag +now, the current owner of the mutex being contended for can't release the mutex +without going into the slow unlock path, and it would then need to grab the +wait_lock, which this code currently holds. So setting the "Has Waiters" flag +forces the current owner to synchronize with this code. + +The lock is taken if the following are true: + 1) The lock has no owner + 2) The current task is the highest priority against all other + waiters of the lock + +If the task succeeds to acquire the lock, then the task is set as the +owner of the lock, and if the lock still has waiters, the top_waiter +(highest priority task waiting on the lock) is added to this task's +pi_waiters tree. + +If the lock is not taken by try_to_take_rt_mutex(), then the +task_blocks_on_rt_mutex() function is called. This will add the task to +the lock's waiter tree and propagate the pi chain of the lock as well +as the lock's owner's pi_waiters tree. This is described in the next +section. Task blocks on mutex -------------------- The accounting of a mutex and process is done with the waiter structure of the process. The "task" field is set to the process, and the "lock" field -to the mutex. The plist nodes are initialized to the processes current -priority. +to the mutex. The rbtree node of waiter are initialized to the processes +current priority. Since the wait_lock was taken at the entry of the slow lock, we can safely -add the waiter to the wait_list. If the current process is the highest -priority process currently waiting on this mutex, then we remove the -previous top waiter process (if it exists) from the pi_list of the owner, -and add the current process to that list. Since the pi_list of the owner +add the waiter to the task waiter tree. If the current process is the +highest priority process currently waiting on this mutex, then we remove the +previous top waiter process (if it exists) from the pi_waiters of the owner, +and add the current process to that tree. Since the pi_waiter of the owner has changed, we call rt_mutex_adjust_prio on the owner to see if the owner should adjust its priority accordingly. -If the owner is also blocked on a lock, and had its pi_list changed +If the owner is also blocked on a lock, and had its pi_waiters changed (or deadlock checking is on), we unlock the wait_lock of the mutex and go ahead and run rt_mutex_adjust_prio_chain on the owner, as described earlier. @@ -686,30 +480,23 @@ mutex (waiter "task" field is not NULL), then we go to sleep (call schedule). Waking up in the loop --------------------- -The schedule can then wake up for a few reasons. - 1) we were given pending ownership of the mutex. - 2) we received a signal and was TASK_INTERRUPTIBLE - 3) we had a timeout and was TASK_INTERRUPTIBLE +The task can then wake up for a couple of reasons: + 1) The previous lock owner released the lock, and the task now is top_waiter + 2) we received a signal or timeout -In any of these cases, we continue the loop and once again try to grab the -ownership of the mutex. If we succeed, we exit the loop, otherwise we continue -and on signal and timeout, will exit the loop, or if we had the mutex stolen -we just simply add ourselves back on the lists and go back to sleep. +In both cases, the task will try again to acquire the lock. If it +does, then it will take itself off the waiters tree and set itself back +to the TASK_RUNNING state. -Note: For various reasons, because of timeout and signals, the steal mutex - algorithm needs to be careful. This is because the current process is - still on the wait_list. And because of dynamic changing of priorities, - especially on SCHED_OTHER tasks, the current process can be the - highest priority task on the wait_list. - -Failed to get mutex on Timeout or Signal ----------------------------------------- +In first case, if the lock was acquired by another task before this task +could get the lock, then it will go back to sleep and wait to be woken again. -If a timeout or signal occurred, the waiter's "task" field would not be -NULL and the task needs to be taken off the wait_list of the mutex and perhaps -pi_list of the owner. If this process was a high priority process, then -the rt_mutex_adjust_prio_chain needs to be executed again on the owner, -but this time it will be lowering the priorities. +The second case is only applicable for tasks that are grabbing a mutex +that can wake up before getting the lock, either due to a signal or +a timeout (i.e. rt_mutex_timed_futex_lock()). When woken, it will try to +take the lock again, if it succeeds, then the task will return with the +lock held, otherwise it will return with -EINTR if the task was woken +by a signal, or -ETIMEDOUT if it timed out. Unlocking the Mutex @@ -739,25 +526,12 @@ owner still needs to make this check. If there are no waiters then the mutex owner field is set to NULL, the wait_lock is released and nothing more is needed. -If there are waiters, then we need to wake one up and give that waiter -pending ownership. +If there are waiters, then we need to wake one up. On the wake up code, the pi_lock of the current owner is taken. The top -waiter of the lock is found and removed from the wait_list of the mutex -as well as the pi_list of the current owner. The task field of the new -pending owner's waiter structure is set to NULL, and the owner field of the -mutex is set to the new owner with the "Pending Owner" bit set, as well -as the "Has Waiters" bit if there still are other processes blocked on the -mutex. - -The pi_lock of the previous owner is released, and the new pending owner's -pi_lock is taken. Remember that this is the trick to prevent the race -condition in rt_mutex_adjust_prio_chain from adding itself as a waiter -on the mutex. - -We now clear the "pi_blocked_on" field of the new pending owner, and if -the mutex still has waiters pending, we add the new top waiter to the pi_list -of the pending owner. +waiter of the lock is found and removed from the waiters tree of the mutex +as well as the pi_waiters tree of the current owner. The "Has Waiters" bit is +marked to prevent lower priority tasks from stealing the lock. Finally we unlock the pi_lock of the pending owner and wake it up. @@ -772,10 +546,14 @@ Credits ------- Author: Steven Rostedt +Updated: Alex Shi - 7/6/2017 -Reviewers: Ingo Molnar, Thomas Gleixner, Thomas Duetsch, and Randy Dunlap +Original Reviewers: Ingo Molnar, Thomas Gleixner, Thomas Duetsch, and + Randy Dunlap +Update (7/6/2017) Reviewers: Steven Rostedt and Sebastian Siewior Updates ------- This document was originally written for 2.6.17-rc3-mm1 +was updated on 4.12 -- cgit v1.2.3 From 68a1e349cecf4e24531cba0f1545a436fa18ee51 Mon Sep 17 00:00:00 2001 From: Alex Shi Date: Mon, 31 Jul 2017 09:50:54 +0800 Subject: rtmutex: update rt-mutex The rtmutex remove a pending owner bit in in rt_mutex::owner, in commit 8161239a8bcc ("rtmutex: Simplify PI algorithm and make highest prio task get lock") But the document was changed accordingly. Updating it to a meaningful state. BTW, as 'Steven Rostedt' mentioned: There is still technically a "Pending Owner", it's just not called that anymore. The pending owner happens to be the top_waiter of a lock that has no owner and has been woken up to grab the lock. Signed-off-by: Alex Shi Cc: Steven Rostedt Cc: Sebastian Siewior Cc: Mathieu Poirier Cc: Juri Lelli Cc: Thomas Gleixner To: linux-doc@vger.kernel.org To: linux-kernel@vger.kernel.org To: Jonathan Corbet To: Ingo Molnar To: Peter Zijlstra Signed-off-by: Jonathan Corbet --- Documentation/locking/rt-mutex.txt | 58 +++++++++++++++++--------------------- 1 file changed, 26 insertions(+), 32 deletions(-) diff --git a/Documentation/locking/rt-mutex.txt b/Documentation/locking/rt-mutex.txt index 243393d882ee..35793e003041 100644 --- a/Documentation/locking/rt-mutex.txt +++ b/Documentation/locking/rt-mutex.txt @@ -28,14 +28,13 @@ magic bullet for poorly designed applications, but it allows well-designed applications to use userspace locks in critical parts of an high priority thread, without losing determinism. -The enqueueing of the waiters into the rtmutex waiter list is done in +The enqueueing of the waiters into the rtmutex waiter tree is done in priority order. For same priorities FIFO order is chosen. For each rtmutex, only the top priority waiter is enqueued into the owner's -priority waiters list. This list too queues in priority order. Whenever +priority waiters tree. This tree too queues in priority order. Whenever the top priority waiter of a task changes (for example it timed out or -got a signal), the priority of the owner task is readjusted. [The -priority enqueueing is handled by "plists", see include/linux/plist.h -for more details.] +got a signal), the priority of the owner task is readjusted. The +priority enqueueing is handled by "pi_waiters". RT-mutexes are optimized for fastpath operations and have no internal locking overhead when locking an uncontended mutex or unlocking a mutex @@ -46,34 +45,29 @@ is used] The state of the rt-mutex is tracked via the owner field of the rt-mutex structure: -rt_mutex->owner holds the task_struct pointer of the owner. Bit 0 and 1 -are used to keep track of the "owner is pending" and "rtmutex has -waiters" state. +lock->owner holds the task_struct pointer of the owner. Bit 0 is used to +keep track of the "lock has waiters" state. - owner bit1 bit0 - NULL 0 0 mutex is free (fast acquire possible) - NULL 0 1 invalid state - NULL 1 0 Transitional state* - NULL 1 1 invalid state - taskpointer 0 0 mutex is held (fast release possible) - taskpointer 0 1 task is pending owner - taskpointer 1 0 mutex is held and has waiters - taskpointer 1 1 task is pending owner and mutex has waiters + owner bit0 + NULL 0 lock is free (fast acquire possible) + NULL 1 lock is free and has waiters and the top waiter + is going to take the lock* + taskpointer 0 lock is held (fast release possible) + taskpointer 1 lock is held and has waiters** -Pending-ownership handling is a performance optimization: -pending-ownership is assigned to the first (highest priority) waiter of -the mutex, when the mutex is released. The thread is woken up and once -it starts executing it can acquire the mutex. Until the mutex is taken -by it (bit 0 is cleared) a competing higher priority thread can "steal" -the mutex which puts the woken up thread back on the waiters list. +The fast atomic compare exchange based acquire and release is only +possible when bit 0 of lock->owner is 0. -The pending-ownership optimization is especially important for the -uninterrupted workflow of high-prio tasks which repeatedly -takes/releases locks that have lower-prio waiters. Without this -optimization the higher-prio thread would ping-pong to the lower-prio -task [because at unlock time we always assign a new owner]. +(*) It also can be a transitional state when grabbing the lock +with ->wait_lock is held. To prevent any fast path cmpxchg to the lock, +we need to set the bit0 before looking at the lock, and the owner may be +NULL in this small time, hence this can be a transitional state. -(*) The "mutex has waiters" bit gets set to take the lock. If the lock -doesn't already have an owner, this bit is quickly cleared if there are -no waiters. So this is a transitional state to synchronize with looking -at the owner field of the mutex and the mutex owner releasing the lock. +(**) There is a small time when bit 0 is set but there are no +waiters. This can happen when grabbing the lock in the slow path. +To prevent a cmpxchg of the owner releasing the lock, we need to +set this bit before looking at the lock. + +BTW, there is still technically a "Pending Owner", it's just not called +that anymore. The pending owner happens to be the top_waiter of a lock +that has no owner and has been woken up to grab the lock. -- cgit v1.2.3 From c1aa3871fa0f676559f6d0cb6941e5f4ce7de306 Mon Sep 17 00:00:00 2001 From: Andrii Bordunov Date: Sun, 13 Aug 2017 23:43:43 +0300 Subject: Documentation: stable-kernel-rules: fix broken git urls git.kernel.org links don't work (fatal: repository ... not found). Update them with the current style from https://git.kernel.org There is no HTTP option, so also switch HTTP -> HTTPS. Signed-off-by: Andrii Bordunov Signed-off-by: Jonathan Corbet --- Documentation/process/stable-kernel-rules.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/Documentation/process/stable-kernel-rules.rst b/Documentation/process/stable-kernel-rules.rst index 61e9c78bd6d1..36a2dded525b 100644 --- a/Documentation/process/stable-kernel-rules.rst +++ b/Documentation/process/stable-kernel-rules.rst @@ -166,12 +166,12 @@ Trees - The queues of patches, for both completed versions and in progress versions can be found at: - http://git.kernel.org/?p=linux/kernel/git/stable/stable-queue.git + https://git.kernel.org/pub/scm/linux/kernel/git/stable/stable-queue.git - The finalized and tagged releases of all stable kernels can be found in separate branches per version at: - http://git.kernel.org/?p=linux/kernel/git/stable/linux-stable.git + https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git Review committee -- cgit v1.2.3 From 646056ec3b8e6f952190a7b92bd8076e1d8c60d4 Mon Sep 17 00:00:00 2001 From: Mauro Carvalho Chehab Date: Sat, 26 Aug 2017 06:28:25 -0300 Subject: docs-rst: fix verbatim font size on tables On Sphinx 1.6, fancy boxes are used for verbatim. The sphinx.sty sets verbatim font is always \small. That causes a problem inside tables that use smaller fonts, as it can be too big for the box. Signed-off-by: Mauro Carvalho Chehab Signed-off-by: Jonathan Corbet --- Documentation/conf.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Documentation/conf.py b/Documentation/conf.py index 9e941be37b94..f9054ab60cb1 100644 --- a/Documentation/conf.py +++ b/Documentation/conf.py @@ -345,7 +345,7 @@ if major == 1 and minor <= 4: latex_elements['preamble'] += '\\usepackage[margin=0.5in, top=1in, bottom=1in]{geometry}' elif major == 1 and (minor > 5 or (minor == 5 and patch >= 3)): latex_elements['sphinxsetup'] = 'hmargin=0.5in, vmargin=1in' - + latex_elements['preamble'] += '\\fvset{fontsize=auto}\n' # Grouping the document tree into LaTeX files. List of tuples # (source start file, target name, title, -- cgit v1.2.3 From 95a40b86c420ce01c5eeb3330d3583ad5a55ed24 Mon Sep 17 00:00:00 2001 From: Mauro Carvalho Chehab Date: Sat, 26 Aug 2017 06:28:27 -0300 Subject: sphinx.rst: Allow Sphinx version 1.6 at the docs Now that the PDF building issues with Sphinx 1.6 got fixed, update the documentation and scripts accordingly. Signed-off-by: Mauro Carvalho Chehab Signed-off-by: Jonathan Corbet --- Documentation/doc-guide/sphinx.rst | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/Documentation/doc-guide/sphinx.rst b/Documentation/doc-guide/sphinx.rst index 8faafb9b2d86..a2417633fdd8 100644 --- a/Documentation/doc-guide/sphinx.rst +++ b/Documentation/doc-guide/sphinx.rst @@ -80,9 +80,7 @@ output. PDF and LaTeX builds -------------------- -Such builds are currently supported only with Sphinx versions 1.4 and 1.5. - -Currently, it is not possible to do pdf builds with Sphinx version 1.6. +Such builds are currently supported only with Sphinx versions 1.4 and upper. For PDF and LaTeX output, you'll also need ``XeLaTeX`` version 3.14159265. -- cgit v1.2.3 From cc4bbaae54cbd6758038f2701cbf27ea222d0fb4 Mon Sep 17 00:00:00 2001 From: Nikolay Borisov Date: Fri, 25 Aug 2017 14:29:00 +0300 Subject: swap: Remove obsolete sentence Currently there are no ->swap_{in,out} method in address_space_operations sructure definition, so the statement that anything is going to be proxied through them is wrong. Signed-off-by: Nikolay Borisov Signed-off-by: Jonathan Corbet --- Documentation/filesystems/vfs.txt | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/Documentation/filesystems/vfs.txt b/Documentation/filesystems/vfs.txt index 73e7d91f03dc..405a3df759b3 100644 --- a/Documentation/filesystems/vfs.txt +++ b/Documentation/filesystems/vfs.txt @@ -829,9 +829,7 @@ struct address_space_operations { swap_activate: Called when swapon is used on a file to allocate space if necessary and pin the block lookup information in memory. A return value of zero indicates success, - in which case this file can be used to back swapspace. The - swapspace operations will be proxied to this address space's - ->swap_{out,in} methods. + in which case this file can be used to back swapspace. swap_deactivate: Called during swapoff on files where swap_activate was successful. -- cgit v1.2.3 From 7dfedbac8b9e264711904fa5168a31ea8bc8ccce Mon Sep 17 00:00:00 2001 From: John de la Garza Date: Tue, 29 Aug 2017 18:48:54 -0700 Subject: Documentation:input: fix typo Signed-off-by: John de la Garza Signed-off-by: Jonathan Corbet --- Documentation/input/input.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Documentation/input/input.rst b/Documentation/input/input.rst index 3b3a22975106..47f86a4bf16c 100644 --- a/Documentation/input/input.rst +++ b/Documentation/input/input.rst @@ -109,7 +109,7 @@ evdev nodes are created with minors starting with 256. keyboard ~~~~~~~~ -``keyboard`` is in-kernel input handler ad is a part of VT code. It +``keyboard`` is in-kernel input handler and is a part of VT code. It consumes keyboard keystrokes and handles user input for VT consoles. mousedev -- cgit v1.2.3 From f7a6dd84b47d34666e5dc394b50ed50fa1571765 Mon Sep 17 00:00:00 2001 From: Randy Dunlap Date: Mon, 28 Aug 2017 17:40:39 -0700 Subject: docs: process: drop git snapshots from applying-patches.rst Drop all references to git daily snapshots of Linux mainline git tree since they are no longer generated. Drop the "Last update" info since 'git log' is a better source of that info and since the Last update date is not being updated. Yes, I read that this file is obsolete, but it still has some useful information in it. Signed-off-by: Randy Dunlap Signed-off-by: Jonathan Corbet --- Documentation/process/applying-patches.rst | 43 +----------------------------- 1 file changed, 1 insertion(+), 42 deletions(-) diff --git a/Documentation/process/applying-patches.rst b/Documentation/process/applying-patches.rst index a0d058cc6d25..dc2ddc345044 100644 --- a/Documentation/process/applying-patches.rst +++ b/Documentation/process/applying-patches.rst @@ -6,9 +6,6 @@ Applying Patches To The Linux Kernel Original by: Jesper Juhl, August 2005 -Last update: - 2016-09-14 - .. note:: This document is obsolete. In most cases, rather than using ``patch`` @@ -344,7 +341,7 @@ possible. This is a good branch to run for people who want to help out testing development kernels but do not want to run some of the really experimental -stuff (such people should see the sections about -git and -mm kernels below). +stuff (such people should see the sections about -next and -mm kernels below). The -rc patches are not incremental, they apply to a base 4.x kernel, just like the 4.x.y patches described above. The kernel version before the -rcN @@ -380,44 +377,6 @@ Here are 3 examples of how to apply these patches:: $ mv linux-4.7.3 linux-4.8-rc5 # rename the kernel source dir -The -git kernels -================ - -These are daily snapshots of Linus' kernel tree (managed in a git -repository, hence the name). - -These patches are usually released daily and represent the current state of -Linus's tree. They are more experimental than -rc kernels since they are -generated automatically without even a cursory glance to see if they are -sane. - --git patches are not incremental and apply either to a base 4.x kernel or -a base 4.x-rc kernel -- you can see which from their name. -A patch named 4.7-git1 applies to the 4.7 kernel source and a patch -named 4.8-rc3-git2 applies to the source of the 4.8-rc3 kernel. - -Here are some examples of how to apply these patches:: - - # moving from 4.7 to 4.7-git1 - - $ cd ~/linux-4.7 # change to the kernel source dir - $ patch -p1 < ../patch-4.7-git1 # apply the 4.7-git1 patch - $ cd .. - $ mv linux-4.7 linux-4.7-git1 # rename the kernel source dir - - # moving from 4.7-git1 to 4.8-rc2-git3 - - $ cd ~/linux-4.7-git1 # change to the kernel source dir - $ patch -p1 -R < ../patch-4.7-git1 # revert the 4.7-git1 patch - # we now have a 4.7 kernel - $ patch -p1 < ../patch-4.8-rc2 # apply the 4.8-rc2 patch - # the kernel is now 4.8-rc2 - $ patch -p1 < ../patch-4.8-rc2-git3 # apply the 4.8-rc2-git3 patch - # the kernel is now 4.8-rc2-git3 - $ cd .. - $ mv linux-4.7-git1 linux-4.8-rc2-git3 # rename source dir - - The -mm patches and the linux-next tree ======================================= -- cgit v1.2.3 From 33c2f4ec984d8a78f15b9d989968733606512bb2 Mon Sep 17 00:00:00 2001 From: Josh Holland Date: Wed, 23 Aug 2017 21:42:32 +0100 Subject: docs: ReSTify table of contents in core.rst Sphinx will now generate the table of contents automatically, which avoids having the ToC getting out of sync with the rest of the document. Signed-off-by: Josh Holland Signed-off-by: Jonathan Corbet --- Documentation/security/keys/core.rst | 12 +----------- 1 file changed, 1 insertion(+), 11 deletions(-) diff --git a/Documentation/security/keys/core.rst b/Documentation/security/keys/core.rst index 312f981fa4a0..1266eeae45f6 100644 --- a/Documentation/security/keys/core.rst +++ b/Documentation/security/keys/core.rst @@ -16,17 +16,7 @@ The key service can be configured on by enabling: This document has the following sections: - - Key overview - - Key service overview - - Key access permissions - - SELinux support - - New procfs files - - Userspace system call interface - - Kernel services - - Notes on accessing payload contents - - Defining a key type - - Request-key callback service - - Garbage collection +.. contents:: :local: Key Overview -- cgit v1.2.3 From 463a0fdc3e800c04ec62b31627909baf4611716a Mon Sep 17 00:00:00 2001 From: Markus Heiser Date: Fri, 16 Jun 2017 21:27:48 +0200 Subject: kernel-doc parser mishandles declarations split into lines Reported by Johannes Berg [1]. Problem here: function process_proto_type() concatenates the striped lines of declaration without any whitespace. A one-liner of:: struct something { struct foo bar; }; has to be:: struct something {struct foo bar;}; Without the patching process_proto_type(), the result missed the space between 'foo' and 'bar':: struct something {struct foobar;}; Bugfix of process_proto_type() brings next error when blank lines between enum declaration:: warning: Enum value ' ' not described in enum 'foo' Problem here: dump_enum() does not strip leading whitespaces from the concatenated string (with the new additional space from process_proto_type). [1] https://www.mail-archive.com/linux-doc@vger.kernel.org/msg12410.html Signed-off-by: Markus Heiser Signed-off-by: Jonathan Corbet --- scripts/kernel-doc | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/scripts/kernel-doc b/scripts/kernel-doc index 6e36b7889001..9d3eafea58f0 100755 --- a/scripts/kernel-doc +++ b/scripts/kernel-doc @@ -2226,6 +2226,7 @@ sub dump_enum($$) { if ($x =~ /enum\s+(\w+)\s*{(.*)}/) { $declaration_name = $1; my $members = $2; + $members =~ s/\s+$//; foreach my $arg (split ',', $members) { $arg =~ s/^\s*(\w+).*/$1/; @@ -2766,6 +2767,9 @@ sub process_proto_type($$) { while (1) { if ( $x =~ /([^{};]*)([{};])(.*)/ ) { + if( length $prototype ) { + $prototype .= " " + } $prototype .= $1 . $2; ($2 eq '{') && $brcount++; ($2 eq '}') && $brcount--; -- cgit v1.2.3 From 48c40c26fc0103d40eb402fcd8fcf800302c83ca Mon Sep 17 00:00:00 2001 From: Alexander Kuleshov Date: Wed, 23 Aug 2017 00:39:13 +0600 Subject: assoc_array: fix path to assoc_array documentation Signed-off-by: Alexander Kuleshov Signed-off-by: Jonathan Corbet --- lib/assoc_array.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/assoc_array.c b/lib/assoc_array.c index 59fd7c0b119c..155c55d8db5f 100644 --- a/lib/assoc_array.c +++ b/lib/assoc_array.c @@ -1,6 +1,6 @@ /* Generic associative array implementation. * - * See Documentation/assoc_array.txt for information. + * See Documentation/core-api/assoc_array.rst for information. * * Copyright (C) 2013 Red Hat, Inc. All Rights Reserved. * Written by David Howells (dhowells@redhat.com) -- cgit v1.2.3 From 7ee8adcf271acd3baa5ab12cb804ab36fda16b3f Mon Sep 17 00:00:00 2001 From: Jonathan Corbet Date: Thu, 24 Aug 2017 14:27:05 -0600 Subject: doc: Add documentation for the genalloc subsystem Genalloc/genpool has kerneldoc comments, but nothing has ever been pulled into the docs themselves. Here's a first attempt, repurposed from an article I wrote at https://lwn.net/Articles/729653/. Signed-off-by: Jonathan Corbet --- Documentation/core-api/genalloc.rst | 144 ++++++++++++++++++++++++++++++++++++ Documentation/core-api/index.rst | 1 + 2 files changed, 145 insertions(+) create mode 100644 Documentation/core-api/genalloc.rst diff --git a/Documentation/core-api/genalloc.rst b/Documentation/core-api/genalloc.rst new file mode 100644 index 000000000000..6b38a39fab24 --- /dev/null +++ b/Documentation/core-api/genalloc.rst @@ -0,0 +1,144 @@ +The genalloc/genpool subsystem +============================== + +There are a number of memory-allocation subsystems in the kernel, each +aimed at a specific need. Sometimes, however, a kernel developer needs to +implement a new allocator for a specific range of special-purpose memory; +often that memory is located on a device somewhere. The author of the +driver for that device can certainly write a little allocator to get the +job done, but that is the way to fill the kernel with dozens of poorly +tested allocators. Back in 2005, Jes Sorensen lifted one of those +allocators from the sym53c8xx_2 driver and posted_ it as a generic module +for the creation of ad hoc memory allocators. This code was merged +for the 2.6.13 release; it has been modified considerably since then. + +.. _posted: https://lwn.net/Articles/125842/ + +Code using this allocator should include . The action +begins with the creation of a pool using one of: + +.. kernel-doc:: lib/genalloc.c + :functions: gen_pool_create + +.. kernel-doc:: lib/genalloc.c + :functions: devm_gen_pool_create + +A call to :c:func:`gen_pool_create` will create a pool. The granularity of +allocations is set with min_alloc_order; it is a log-base-2 number like +those used by the page allocator, but it refers to bytes rather than pages. +So, if min_alloc_order is passed as 3, then all allocations will be a +multiple of eight bytes. Increasing min_alloc_order decreases the memory +required to track the memory in the pool. The nid parameter specifies +which NUMA node should be used for the allocation of the housekeeping +structures; it can be -1 if the caller doesn't care. + +The "managed" interface :c:func:`devm_gen_pool_create` ties the pool to a +specific device. Among other things, it will automatically clean up the +pool when the given device is destroyed. + +A pool is shut down with: + +.. kernel-doc:: lib/genalloc.c + :functions: gen_pool_destroy + +It's worth noting that, if there are still allocations outstanding from the +given pool, this function will take the rather extreme step of invoking +BUG(), crashing the entire system. You have been warned. + +A freshly created pool has no memory to allocate. It is fairly useless in +that state, so one of the first orders of business is usually to add memory +to the pool. That can be done with one of: + +.. kernel-doc:: include/linux/genalloc.h + :functions: gen_pool_add + +.. kernel-doc:: lib/genalloc.c + :functions: gen_pool_add_virt + +A call to :c:func:`gen_pool_add` will place the size bytes of memory +starting at addr (in the kernel's virtual address space) into the given +pool, once again using nid as the node ID for ancillary memory allocations. +The :c:func:`gen_pool_add_virt` variant associates an explicit physical +address with the memory; this is only necessary if the pool will be used +for DMA allocations. + +The functions for allocating memory from the pool (and putting it back) +are: + +.. kernel-doc:: lib/genalloc.c + :functions: gen_pool_alloc + +.. kernel-doc:: lib/genalloc.c + :functions: gen_pool_dma_alloc + +.. kernel-doc:: lib/genalloc.c + :functions: gen_pool_free + +As one would expect, :c:func:`gen_pool_alloc` will allocate size< bytes +from the given pool. The :c:func:`gen_pool_dma_alloc` variant allocates +memory for use with DMA operations, returning the associated physical +address in the space pointed to by dma. This will only work if the memory +was added with :c:func:`gen_pool_add_virt`. Note that this function +departs from the usual genpool pattern of using unsigned long values to +represent kernel addresses; it returns a void * instead. + +That all seems relatively simple; indeed, some developers clearly found it +to be too simple. After all, the interface above provides no control over +how the allocation functions choose which specific piece of memory to +return. If that sort of control is needed, the following functions will be +of interest: + +.. kernel-doc:: lib/genalloc.c + :functions: gen_pool_alloc_algo + +.. kernel-doc:: lib/genalloc.c + :functions: gen_pool_set_algo + +Allocations with :c:func:`gen_pool_alloc_algo` specify an algorithm to be +used to choose the memory to be allocated; the default algorithm can be set +with :c:func:`gen_pool_set_algo`. The data value is passed to the +algorithm; most ignore it, but it is occasionally needed. One can, +naturally, write a special-purpose algorithm, but there is a fair set +already available: + +- gen_pool_first_fit is a simple first-fit allocator; this is the default + algorithm if none other has been specified. + +- gen_pool_first_fit_align forces the allocation to have a specific + alignment (passed via data in a genpool_data_align structure). + +- gen_pool_first_fit_order_align aligns the allocation to the order of the + size. A 60-byte allocation will thus be 64-byte aligned, for example. + +- gen_pool_best_fit, as one would expect, is a simple best-fit allocator. + +- gen_pool_fixed_alloc allocates at a specific offset (passed in a + genpool_data_fixed structure via the data parameter) within the pool. + If the indicated memory is not available the allocation fails. + +There is a handful of other functions, mostly for purposes like querying +the space available in the pool or iterating through chunks of memory. +Most users, however, should not need much beyond what has been described +above. With luck, wider awareness of this module will help to prevent the +writing of special-purpose memory allocators in the future. + +.. kernel-doc:: lib/genalloc.c + :functions: gen_pool_virt_to_phys + +.. kernel-doc:: lib/genalloc.c + :functions: gen_pool_for_each_chunk + +.. kernel-doc:: lib/genalloc.c + :functions: addr_in_gen_pool + +.. kernel-doc:: lib/genalloc.c + :functions: gen_pool_avail + +.. kernel-doc:: lib/genalloc.c + :functions: gen_pool_size + +.. kernel-doc:: lib/genalloc.c + :functions: gen_pool_get + +.. kernel-doc:: lib/genalloc.c + :functions: of_gen_pool_get diff --git a/Documentation/core-api/index.rst b/Documentation/core-api/index.rst index 0606be3a3111..d5bbe035316d 100644 --- a/Documentation/core-api/index.rst +++ b/Documentation/core-api/index.rst @@ -20,6 +20,7 @@ Core utilities genericirq flexible-arrays librs + genalloc Interfaces for kernel debugging =============================== -- cgit v1.2.3 From a27bfcab5c1c3a0df61b68e85fc5e4cade83559f Mon Sep 17 00:00:00 2001 From: Jonathan Corbet Date: Thu, 31 Aug 2017 09:47:22 -0600 Subject: genalloc: Fix an incorrect kerneldoc comment The kerneldoc comment for the genpool_algo_t typedef was incomplete and incorrectly formatted, leading to a raft of warnings during the docs build. Fix it appropriately. Signed-off-by: Jonathan Corbet --- include/linux/genalloc.h | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/include/linux/genalloc.h b/include/linux/genalloc.h index 29d4385903d4..6dfec4d638df 100644 --- a/include/linux/genalloc.h +++ b/include/linux/genalloc.h @@ -38,12 +38,13 @@ struct device_node; struct gen_pool; /** - * Allocation callback function type definition + * typedef genpool_algo_t: Allocation callback function type definition * @map: Pointer to bitmap * @size: The bitmap size in bits * @start: The bitnumber to start searching at * @nr: The number of zeroed bits we're looking for - * @data: optional additional data used by @genpool_algo_t + * @data: optional additional data used by the callback + * @pool: the pool being allocated from */ typedef unsigned long (*genpool_algo_t)(unsigned long *map, unsigned long size, -- cgit v1.2.3 From 86c0f046a8b0c23fca65f77333c233a06c25ef9a Mon Sep 17 00:00:00 2001 From: Jani Nikula Date: Thu, 31 Aug 2017 22:21:29 +0300 Subject: Documentation/sphinx: fix kernel-doc decode for non-utf-8 locale On python3, Popen() universal_newlines=True converts the subprocess stdout to unicode text using a codec based on user preferences. Given LANG indicating ascii and utf-8 stdout from the subprocess, you'd get: WARNING: kernel-doc '../scripts/kernel-doc -rst -enable-lineno ../drivers/media/dvb-core/demux.h' processing failed with: 'ascii' codec can't decode byte 0xe2 in position 6368: ordinal not in range(128) Fix this by dropping universal_newlines=True and replacing the implicit LANG specific decode with an explicit utf-8 decode. This also gets rid of the annoying conditional code for python 2 vs. 3. Fixes: ba3501859354 ("Documentation/sphinx: fix kernel-doc extension on python3") Reference: http://mid.mail-archive.com/54c23e8e-89c0-5cea-0dcc-e938952c5642@infradead.org Reported-and-tested-by: Randy Dunlap Cc: Jonathan Corbet Cc: Mauro Carvalho Chehab Signed-off-by: Jani Nikula Signed-off-by: Jonathan Corbet --- Documentation/sphinx/kerneldoc.py | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-) diff --git a/Documentation/sphinx/kerneldoc.py b/Documentation/sphinx/kerneldoc.py index d15e07f36881..39aa9e8697cc 100644 --- a/Documentation/sphinx/kerneldoc.py +++ b/Documentation/sphinx/kerneldoc.py @@ -27,6 +27,7 @@ # Please make sure this works on both python2 and python3. # +import codecs import os import subprocess import sys @@ -88,13 +89,10 @@ class KernelDocDirective(Directive): try: env.app.verbose('calling kernel-doc \'%s\'' % (" ".join(cmd))) - p = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, universal_newlines=True) + p = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE) out, err = p.communicate() - # python2 needs conversion to unicode. - # python3 with universal_newlines=True returns strings. - if sys.version_info.major < 3: - out, err = unicode(out, 'utf-8'), unicode(err, 'utf-8') + out, err = codecs.decode(out, 'utf-8'), codecs.decode(err, 'utf-8') if p.returncode != 0: sys.stderr.write(err) -- cgit v1.2.3