kernel/linux.git/scripts, branch v5.0.15

fs: stream_open - opener for stream-like files so that read and write can run simultaneously without deadlock

2019-05-08T05:22:55+00:00

[ Upstream commit 10dce8af34226d90fa56746a934f8da5dcdba3df ] Commit 9c225f2655e3 ("vfs: atomic f_pos accesses as per POSIX") added locking for file.f_pos access and in particular made concurrent read and write not possible - now both those functions take f_pos lock for the whole run, and so if e.g. a read is blocked waiting for data, write will deadlock waiting for that read to complete. This caused regression for stream-like files where previously read and write could run simultaneously, but after that patch could not do so anymore. See e.g. commit 581d21a2d02a ("xenbus: fix deadlock on writes to /proc/xen/xenbus") which fixes such regression for particular case of /proc/xen/xenbus. The patch that added f_pos lock in 2014 did so to guarantee POSIX thread safety for read/write/lseek and added the locking to file descriptors of all regular files. In 2014 that thread-safety problem was not new as it was already discussed earlier in 2006. However even though 2006'th version of Linus's patch was adding f_pos locking "only for files that are marked seekable with FMODE_LSEEK (thus avoiding the stream-like objects like pipes and sockets)", the 2014 version - the one that actually made it into the tree as 9c225f2655e3 - is doing so irregardless of whether a file is seekable or not. See https://lore.kernel.org/lkml/53022DB1.4070805@gmail.com/ https://lwn.net/Articles/180387 https://lwn.net/Articles/180396 for historic context. The reason that it did so is, probably, that there are many files that are marked non-seekable, but e.g. their read implementation actually depends on knowing current position to correctly handle the read. Some examples: kernel/power/user.c snapshot_read fs/debugfs/file.c u32_array_read fs/fuse/control.c fuse_conn_waiting_read + ... drivers/hwmon/asus_atk0110.c atk_debugfs_ggrp_read arch/s390/hypfs/inode.c hypfs_read_iter ... Despite that, many nonseekable_open users implement read and write with pure stream semantics - they don't depend on passed ppos at all. And for those cases where read could wait for something inside, it creates a situation similar to xenbus - the write could be never made to go until read is done, and read is waiting for some, potentially external, event, for potentially unbounded time -> deadlock. Besides xenbus, there are 14 such places in the kernel that I've found with semantic patch (see below): drivers/xen/evtchn.c:667:8-24: ERROR: evtchn_fops: .read() can deadlock .write() drivers/isdn/capi/capi.c:963:8-24: ERROR: capi_fops: .read() can deadlock .write() drivers/input/evdev.c:527:1-17: ERROR: evdev_fops: .read() can deadlock .write() drivers/char/pcmcia/cm4000_cs.c:1685:7-23: ERROR: cm4000_fops: .read() can deadlock .write() net/rfkill/core.c:1146:8-24: ERROR: rfkill_fops: .read() can deadlock .write() drivers/s390/char/fs3270.c:488:1-17: ERROR: fs3270_fops: .read() can deadlock .write() drivers/usb/misc/ldusb.c:310:1-17: ERROR: ld_usb_fops: .read() can deadlock .write() drivers/hid/uhid.c:635:1-17: ERROR: uhid_fops: .read() can deadlock .write() net/batman-adv/icmp_socket.c:80:1-17: ERROR: batadv_fops: .read() can deadlock .write() drivers/media/rc/lirc_dev.c:198:1-17: ERROR: lirc_fops: .read() can deadlock .write() drivers/leds/uleds.c:77:1-17: ERROR: uleds_fops: .read() can deadlock .write() drivers/input/misc/uinput.c:400:1-17: ERROR: uinput_fops: .read() can deadlock .write() drivers/infiniband/core/user_mad.c:985:7-23: ERROR: umad_fops: .read() can deadlock .write() drivers/gnss/core.c:45:1-17: ERROR: gnss_fops: .read() can deadlock .write() In addition to the cases above another regression caused by f_pos locking is that now FUSE filesystems that implement open with FOPEN_NONSEEKABLE flag, can no longer implement bidirectional stream-like files - for the same reason as above e.g. read can deadlock write locking on file.f_pos in the kernel. FUSE's FOPEN_NONSEEKABLE was added in 2008 in a7c1b990f715 ("fuse: implement nonseekable open") to support OSSPD. OSSPD implements /dev/dsp in userspace with FOPEN_NONSEEKABLE flag, with corresponding read and write routines not depending on current position at all, and with both read and write being potentially blocking operations: See https://github.com/libfuse/osspd https://lwn.net/Articles/308445 https://github.com/libfuse/osspd/blob/14a9cff0/osspd.c#L1406 https://github.com/libfuse/osspd/blob/14a9cff0/osspd.c#L1438-L1477 https://github.com/libfuse/osspd/blob/14a9cff0/osspd.c#L1479-L1510 Corresponding libfuse example/test also describes FOPEN_NONSEEKABLE as "somewhat pipe-like files ..." with read handler not using offset. However that test implements only read without write and cannot exercise the deadlock scenario: https://github.com/libfuse/libfuse/blob/fuse-3.4.2-3-ga1bff7d/example/poll.c#L124-L131 https://github.com/libfuse/libfuse/blob/fuse-3.4.2-3-ga1bff7d/example/poll.c#L146-L163 https://github.com/libfuse/libfuse/blob/fuse-3.4.2-3-ga1bff7d/example/poll.c#L209-L216 I've actually hit the read vs write deadlock for real while implementing my FUSE filesystem where there is /head/watch file, for which open creates separate bidirectional socket-like stream in between filesystem and its user with both read and write being later performed simultaneously. And there it is semantically not easy to split the stream into two separate read-only and write-only channels: https://lab.nexedi.com/kirr/wendelin.core/blob/f13aa600/wcfs/wcfs.go#L88-169 Let's fix this regression. The plan is: 1. We can't change nonseekable_open to include &~FMODE_ATOMIC_POS - doing so would break many in-kernel nonseekable_open users which actually use ppos in read/write handlers. 2. Add stream_open() to kernel to open stream-like non-seekable file descriptors. Read and write on such file descriptors would never use nor change ppos. And with that property on stream-like files read and write will be running without taking f_pos lock - i.e. read and write could be running simultaneously. 3. With semantic patch search and convert to stream_open all in-kernel nonseekable_open users for which read and write actually do not depend on ppos and where there is no other methods in file_operations which assume @offset access. 4. Add FOPEN_STREAM to fs/fuse/ and open in-kernel file-descriptors via steam_open if that bit is present in filesystem open reply. It was tempting to change fs/fuse/ open handler to use stream_open instead of nonseekable_open on just FOPEN_NONSEEKABLE flags, but grepping through Debian codesearch shows users of FOPEN_NONSEEKABLE, and in particular GVFS which actually uses offset in its read and write handlers https://codesearch.debian.net/search?q=-%3Enonseekable+%3D https://gitlab.gnome.org/GNOME/gvfs/blob/1.40.0-6-gcbc54396/client/gvfsfusedaemon.c#L1080 https://gitlab.gnome.org/GNOME/gvfs/blob/1.40.0-6-gcbc54396/client/gvfsfusedaemon.c#L1247-1346 https://gitlab.gnome.org/GNOME/gvfs/blob/1.40.0-6-gcbc54396/client/gvfsfusedaemon.c#L1399-1481 so if we would do such a change it will break a real user. 5. Add stream_open and FOPEN_STREAM handling to stable kernels starting from v3.14+ (the kernel where 9c225f2655 first appeared). This will allow to patch OSSPD and other FUSE filesystems that provide stream-like files to return FOPEN_STREAM | FOPEN_NONSEEKABLE in their open handler and this way avoid the deadlock on all kernel versions. This should work because fs/fuse/ ignores unknown open flags returned from a filesystem and so passing FOPEN_STREAM to a kernel that is not aware of this flag cannot hurt. In turn the kernel that is not aware of FOPEN_STREAM will be < v3.14 where just FOPEN_NONSEEKABLE is sufficient to implement streams without read vs write deadlock. This patch adds stream_open, converts /proc/xen/xenbus to it and adds semantic patch to automatically locate in-kernel places that are either required to be converted due to read vs write deadlock, or that are just safe to be converted because read and write do not use ppos and there are no other funky methods in file_operations. Regarding semantic patch I've verified each generated change manually - that it is correct to convert - and each other nonseekable_open instance left - that it is either not correct to convert there, or that it is not converted due to current stream_open.cocci limitations. The script also does not convert files that should be valid to convert, but that currently have .llseek = noop_llseek or generic_file_llseek for unknown reason despite file being opened with nonseekable_open (e.g. drivers/input/mousedev.c) Cc: Michael Kerrisk Cc: Yongzhi Pan Cc: Jonathan Corbet Cc: David Vrabel Cc: Juergen Gross Cc: Miklos Szeredi Cc: Tejun Heo Cc: Kirill Tkhai Cc: Arnd Bergmann Cc: Christoph Hellwig Cc: Greg Kroah-Hartman Cc: Julia Lawall Cc: Nikolaus Rath Cc: Han-Wen Nienhuys Signed-off-by: Kirill Smelkov Signed-off-by: Linus Torvalds Signed-off-by: Sasha Levin (Microsoft)

kconfig/[mn]conf: handle backspace (^H) key

2019-05-04T07:21:22+00:00

[ Upstream commit 9c38f1f044080392603c497ecca4d7d09876ff99 ] Backspace is not working on some terminal emulators which do not send the key code defined by terminfo. Terminals either send '^H' (8) or '^?' (127). But currently only '^?' is handled. Let's also handle '^H' for those terminals. Signed-off-by: Changbin Du Signed-off-by: Masahiro Yamada Signed-off-by: Sasha Levin (Microsoft)

selinux: use kernel linux/socket.h for genheaders and mdp

2019-05-04T07:21:10+00:00

commit dfbd199a7cfe3e3cd8531e1353cdbd7175bfbc5e upstream. When compiling genheaders and mdp from a newer host kernel, the following error happens: In file included from scripts/selinux/genheaders/genheaders.c:18: ./security/selinux/include/classmap.h:238:2: error: #error New address family defined, please update secclass_map. #error New address family defined, please update secclass_map. ^~~~~ make[3]: *** [scripts/Makefile.host:107: scripts/selinux/genheaders/genheaders] Error 1 make[2]: *** [scripts/Makefile.build:599: scripts/selinux/genheaders] Error 2 make[1]: *** [scripts/Makefile.build:599: scripts/selinux] Error 2 make[1]: *** Waiting for unfinished jobs.... Instead of relying on the host definition, include linux/socket.h in classmap.h to have PF_MAX. Cc: stable@vger.kernel.org Signed-off-by: Paulo Alcantara Acked-by: Stephen Smalley [PM: manually merge in mdp.c, subject line tweaks] Signed-off-by: Paul Moore Signed-off-by: Greg Kroah-Hartman

kbuild: pkg: use -f $(srctree)/Makefile to recurse to top Makefile

2019-04-17T06:39:36+00:00

[ Upstream commit 175209cce23d6b0669ed5366add2517e26cd75cd ] '$(MAKE) KBUILD_SRC=' changes the working directory back and forth between objtree and srctree. It is better to recurse to the top-level Makefile directly. Signed-off-by: Masahiro Yamada Signed-off-by: Sasha Levin

kbuild: modversions: Fix relative CRC byte order interpretation

2019-04-03T04:27:30+00:00

commit 54a7151b1496cddbb7a83546b7998103e98edc88 upstream. Fix commit 56067812d5b0 ("kbuild: modversions: add infrastructure for emitting relative CRCs") where CRCs are interpreted in host byte order rather than proper kernel byte order. The bug is conditional on CONFIG_MODULE_REL_CRCS. For example, when loading a BE module into a BE kernel compiled with a LE system, the error "disagrees about version of symbol module_layout" is produced. A message such as "Found checksum D7FA6856 vs module 5668FAD7" will be given with debug enabled, which indicates an obvious endian problem within __kcrctab within the kernel image. The general solution is to use the macro TO_NATIVE, as is done in similar cases throughout modpost.c. With this correction it has been verified that a BE kernel compiled with a LE system accepts BE modules. This change has also been verified with a LE kernel compiled with a LE system, in which case TO_NATIVE returns its value unmodified since the byte orders match. This is by far the common case. Fixes: 56067812d5b0 ("kbuild: modversions: add infrastructure for emitting relative CRCs") Signed-off-by: Fredrik Noring Cc: stable@vger.kernel.org Signed-off-by: Masahiro Yamada Signed-off-by: Greg Kroah-Hartman

scripts/gdb: replace flags (MS_xyz -> SB_xyz)

2019-03-13T21:01:31+00:00

commit 663cb6340c6e84fe29aa6d0fa63d85ea6bd6cd19 upstream. Since commit 1751e8a6cb93 ("Rename superblock flags (MS_xyz -> SB_xyz)"), scripts/gdb should be updated to replace MS_xyz with SB_xyz. This change didn't directly affect the running operation of scripts/gdb until commit e262e32d6bde "vfs: Suppress MS_* flag defs within the kernel unless explicitly enabled" removed the definitions used by constants.py. Update constants.py.in to utilise the new internal flags, matching the implementation at fs/proc_namespace.c::show_sb_opts. Note to stable, e262e32d6bde landed in v5.0-rc1 (which was just released), so we'll want this picked back to 5.0 stable once this patch hits mainline (akpm just picked it up). Without this, debugging a kernel a kernel via GDB+QEMU is broken in the 5.0 release. [kieran.bingham@ideasonboard.com: add fixes tag, reword commit message] Link: http://lkml.kernel.org/r/20190305103014.25847-1-kieran.bingham@ideasonboard.com Fixes: e262e32d6bde "vfs: Suppress MS_* flag defs within the kernel unless explicitly enabled" Signed-off-by: Jackie Liu Signed-off-by: Kieran Bingham Tested-by: Nick Desaulniers Tested-by: Kieran Bingham Cc: Felipe Balbi Cc: Dan Robertson Cc: Jan Kiszka Cc: David Howells Cc: Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds Signed-off-by: Greg Kroah-Hartman

kasan: turn off asan-stack for clang-8 and earlier

2019-03-01T17:02:33+00:00

Building an arm64 allmodconfig kernel with clang results in over 140 warnings about overly large stack frames, the worst ones being: drivers/gpu/drm/panel/panel-sitronix-st7789v.c:196:12: error: stack frame size of 20224 bytes in function 'st7789v_prepare' drivers/video/fbdev/omap2/omapfb/displays/panel-tpo-td028ttec1.c:196:12: error: stack frame size of 13120 bytes in function 'td028ttec1_panel_enable' drivers/usb/host/max3421-hcd.c:1395:1: error: stack frame size of 10048 bytes in function 'max3421_spi_thread' drivers/net/wan/slic_ds26522.c:209:12: error: stack frame size of 9664 bytes in function 'slic_ds26522_probe' drivers/crypto/ccp/ccp-ops.c:2434:5: error: stack frame size of 8832 bytes in function 'ccp_run_cmd' drivers/media/dvb-frontends/stv0367.c:1005:12: error: stack frame size of 7840 bytes in function 'stv0367ter_algo' None of these happen with gcc today, and almost all of these are the result of a single known issue in llvm. Hopefully it will eventually get fixed with the clang-9 release. In the meantime, the best idea I have is to turn off asan-stack for clang-8 and earlier, so we can produce a kernel that is safe to run. I have posted three patches that address the frame overflow warnings that are not addressed by turning off asan-stack, so in combination with this change, we get much closer to a clean allmodconfig build, which in turn is necessary to do meaningful build regression testing. It is still possible to turn on the CONFIG_ASAN_STACK option on all versions of clang, and it's always enabled for gcc, but when CONFIG_COMPILE_TEST is set, the option remains invisible, so allmodconfig and randconfig builds (which are normally done with a forced CONFIG_COMPILE_TEST) will still result in a mostly clean build. Link: http://lkml.kernel.org/r/20190222222950.3997333-1-arnd@arndb.de Link: https://bugs.llvm.org/show_bug.cgi?id=38809 Signed-off-by: Arnd Bergmann Reviewed-by: Qian Cai Reviewed-by: Mark Brown Acked-by: Andrey Ryabinin Cc: Dmitry Vyukov Cc: Nick Desaulniers Cc: Kostya Serebryany Cc: Andrey Konovalov Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds

kallsyms: Handle too long symbols in kallsyms.c

2019-01-28T04:02:09+00:00

When checking for symbols with excessively long names, account for null terminating character. Fixes: f3462aa952cf ("Kbuild: Handle longer symbols in kallsyms.c") Signed-off-by: Eugene Loh Acked-by: Ard Biesheuvel Signed-off-by: Masahiro Yamada

Merge tag 'gcc-plugins-v5.0-rc4' of git://git.kernel.org/pub/scm/linux/kernel/git/kees/linux

2019-01-21T00:07:03+00:00

Pull gcc-plugins fixes from Kees Cook: "Fix ARM per-task stack protector plugin under GCC 9 (Ard Biesheuvel)" * tag 'gcc-plugins-v5.0-rc4' of git://git.kernel.org/pub/scm/linux/kernel/git/kees/linux: gcc-plugins: arm_ssp_per_task_plugin: fix for GCC 9+ gcc-plugins: arm_ssp_per_task_plugin: sign extend the SP mask

gcc-plugins: arm_ssp_per_task_plugin: fix for GCC 9+

2019-01-20T22:06:40+00:00

GCC 9 reworks the way the references to the stack canary are emitted, to prevent the value from being spilled to the stack before the final comparison in the epilogue, defeating the purpose, given that the spill slot is under control of the attacker that we are protecting ourselves from. Since our canary value address is obtained without accessing memory (as opposed to pre-v7 code that will obtain it from a literal pool), it is unlikely (although not guaranteed) that the compiler will spill the canary value in the same way, so let's just disable this improvement when building with GCC9+. Signed-off-by: Ard Biesheuvel Signed-off-by: Kees Cook