kernel/linux.git/scripts, branch v3.16.76

fs: stream_open - opener for stream-like files so that read and write can run simultaneously without deadlock

2019-07-09T21:04:19+00:00

commit 10dce8af34226d90fa56746a934f8da5dcdba3df upstream. [ while porting to 3.16 xenbus conflict was trivially resolved in a way that actually fixes /proc/xen/xenbus deadlock introduced in 3.14, because original upstream commit 581d21a2d02a to fix xenbus deadlock was not included into 3.16 . ] Commit 9c225f2655e3 ("vfs: atomic f_pos accesses as per POSIX") added locking for file.f_pos access and in particular made concurrent read and write not possible - now both those functions take f_pos lock for the whole run, and so if e.g. a read is blocked waiting for data, write will deadlock waiting for that read to complete. This caused regression for stream-like files where previously read and write could run simultaneously, but after that patch could not do so anymore. See e.g. commit 581d21a2d02a ("xenbus: fix deadlock on writes to /proc/xen/xenbus") which fixes such regression for particular case of /proc/xen/xenbus. The patch that added f_pos lock in 2014 did so to guarantee POSIX thread safety for read/write/lseek and added the locking to file descriptors of all regular files. In 2014 that thread-safety problem was not new as it was already discussed earlier in 2006. However even though 2006'th version of Linus's patch was adding f_pos locking "only for files that are marked seekable with FMODE_LSEEK (thus avoiding the stream-like objects like pipes and sockets)", the 2014 version - the one that actually made it into the tree as 9c225f2655e3 - is doing so irregardless of whether a file is seekable or not. See https://lore.kernel.org/lkml/53022DB1.4070805@gmail.com/ https://lwn.net/Articles/180387 https://lwn.net/Articles/180396 for historic context. The reason that it did so is, probably, that there are many files that are marked non-seekable, but e.g. their read implementation actually depends on knowing current position to correctly handle the read. Some examples: kernel/power/user.c snapshot_read fs/debugfs/file.c u32_array_read fs/fuse/control.c fuse_conn_waiting_read + ... drivers/hwmon/asus_atk0110.c atk_debugfs_ggrp_read arch/s390/hypfs/inode.c hypfs_read_iter ... Despite that, many nonseekable_open users implement read and write with pure stream semantics - they don't depend on passed ppos at all. And for those cases where read could wait for something inside, it creates a situation similar to xenbus - the write could be never made to go until read is done, and read is waiting for some, potentially external, event, for potentially unbounded time -> deadlock. Besides xenbus, there are 14 such places in the kernel that I've found with semantic patch (see below): drivers/xen/evtchn.c:667:8-24: ERROR: evtchn_fops: .read() can deadlock .write() drivers/isdn/capi/capi.c:963:8-24: ERROR: capi_fops: .read() can deadlock .write() drivers/input/evdev.c:527:1-17: ERROR: evdev_fops: .read() can deadlock .write() drivers/char/pcmcia/cm4000_cs.c:1685:7-23: ERROR: cm4000_fops: .read() can deadlock .write() net/rfkill/core.c:1146:8-24: ERROR: rfkill_fops: .read() can deadlock .write() drivers/s390/char/fs3270.c:488:1-17: ERROR: fs3270_fops: .read() can deadlock .write() drivers/usb/misc/ldusb.c:310:1-17: ERROR: ld_usb_fops: .read() can deadlock .write() drivers/hid/uhid.c:635:1-17: ERROR: uhid_fops: .read() can deadlock .write() net/batman-adv/icmp_socket.c:80:1-17: ERROR: batadv_fops: .read() can deadlock .write() drivers/media/rc/lirc_dev.c:198:1-17: ERROR: lirc_fops: .read() can deadlock .write() drivers/leds/uleds.c:77:1-17: ERROR: uleds_fops: .read() can deadlock .write() drivers/input/misc/uinput.c:400:1-17: ERROR: uinput_fops: .read() can deadlock .write() drivers/infiniband/core/user_mad.c:985:7-23: ERROR: umad_fops: .read() can deadlock .write() drivers/gnss/core.c:45:1-17: ERROR: gnss_fops: .read() can deadlock .write() In addition to the cases above another regression caused by f_pos locking is that now FUSE filesystems that implement open with FOPEN_NONSEEKABLE flag, can no longer implement bidirectional stream-like files - for the same reason as above e.g. read can deadlock write locking on file.f_pos in the kernel. FUSE's FOPEN_NONSEEKABLE was added in 2008 in a7c1b990f715 ("fuse: implement nonseekable open") to support OSSPD. OSSPD implements /dev/dsp in userspace with FOPEN_NONSEEKABLE flag, with corresponding read and write routines not depending on current position at all, and with both read and write being potentially blocking operations: See https://github.com/libfuse/osspd https://lwn.net/Articles/308445 https://github.com/libfuse/osspd/blob/14a9cff0/osspd.c#L1406 https://github.com/libfuse/osspd/blob/14a9cff0/osspd.c#L1438-L1477 https://github.com/libfuse/osspd/blob/14a9cff0/osspd.c#L1479-L1510 Corresponding libfuse example/test also describes FOPEN_NONSEEKABLE as "somewhat pipe-like files ..." with read handler not using offset. However that test implements only read without write and cannot exercise the deadlock scenario: https://github.com/libfuse/libfuse/blob/fuse-3.4.2-3-ga1bff7d/example/poll.c#L124-L131 https://github.com/libfuse/libfuse/blob/fuse-3.4.2-3-ga1bff7d/example/poll.c#L146-L163 https://github.com/libfuse/libfuse/blob/fuse-3.4.2-3-ga1bff7d/example/poll.c#L209-L216 I've actually hit the read vs write deadlock for real while implementing my FUSE filesystem where there is /head/watch file, for which open creates separate bidirectional socket-like stream in between filesystem and its user with both read and write being later performed simultaneously. And there it is semantically not easy to split the stream into two separate read-only and write-only channels: https://lab.nexedi.com/kirr/wendelin.core/blob/f13aa600/wcfs/wcfs.go#L88-169 Let's fix this regression. The plan is: 1. We can't change nonseekable_open to include &~FMODE_ATOMIC_POS - doing so would break many in-kernel nonseekable_open users which actually use ppos in read/write handlers. 2. Add stream_open() to kernel to open stream-like non-seekable file descriptors. Read and write on such file descriptors would never use nor change ppos. And with that property on stream-like files read and write will be running without taking f_pos lock - i.e. read and write could be running simultaneously. 3. With semantic patch search and convert to stream_open all in-kernel nonseekable_open users for which read and write actually do not depend on ppos and where there is no other methods in file_operations which assume @offset access. 4. Add FOPEN_STREAM to fs/fuse/ and open in-kernel file-descriptors via steam_open if that bit is present in filesystem open reply. It was tempting to change fs/fuse/ open handler to use stream_open instead of nonseekable_open on just FOPEN_NONSEEKABLE flags, but grepping through Debian codesearch shows users of FOPEN_NONSEEKABLE, and in particular GVFS which actually uses offset in its read and write handlers https://codesearch.debian.net/search?q=-%3Enonseekable+%3D https://gitlab.gnome.org/GNOME/gvfs/blob/1.40.0-6-gcbc54396/client/gvfsfusedaemon.c#L1080 https://gitlab.gnome.org/GNOME/gvfs/blob/1.40.0-6-gcbc54396/client/gvfsfusedaemon.c#L1247-1346 https://gitlab.gnome.org/GNOME/gvfs/blob/1.40.0-6-gcbc54396/client/gvfsfusedaemon.c#L1399-1481 so if we would do such a change it will break a real user. 5. Add stream_open and FOPEN_STREAM handling to stable kernels starting from v3.14+ (the kernel where 9c225f2655 first appeared). This will allow to patch OSSPD and other FUSE filesystems that provide stream-like files to return FOPEN_STREAM | FOPEN_NONSEEKABLE in their open handler and this way avoid the deadlock on all kernel versions. This should work because fs/fuse/ ignores unknown open flags returned from a filesystem and so passing FOPEN_STREAM to a kernel that is not aware of this flag cannot hurt. In turn the kernel that is not aware of FOPEN_STREAM will be < v3.14 where just FOPEN_NONSEEKABLE is sufficient to implement streams without read vs write deadlock. This patch adds stream_open, converts /proc/xen/xenbus to it and adds semantic patch to automatically locate in-kernel places that are either required to be converted due to read vs write deadlock, or that are just safe to be converted because read and write do not use ppos and there are no other funky methods in file_operations. Regarding semantic patch I've verified each generated change manually - that it is correct to convert - and each other nonseekable_open instance left - that it is either not correct to convert there, or that it is not converted due to current stream_open.cocci limitations. The script also does not convert files that should be valid to convert, but that currently have .llseek = noop_llseek or generic_file_llseek for unknown reason despite file being opened with nonseekable_open (e.g. drivers/input/mousedev.c) Cc: Michael Kerrisk Cc: Yongzhi Pan Cc: Jonathan Corbet Cc: David Vrabel Cc: Juergen Gross Cc: Miklos Szeredi Cc: Tejun Heo Cc: Kirill Tkhai Cc: Arnd Bergmann Cc: Christoph Hellwig Cc: Greg Kroah-Hartman Cc: Julia Lawall Cc: Nikolaus Rath Cc: Han-Wen Nienhuys Signed-off-by: Kirill Smelkov Signed-off-by: Linus Torvalds [ backport to 3.16: actually fixed deadlock on /proc/xen/xenbus as 581d21a2d02a was not backported to 3.16 ] Signed-off-by: Kirill Smelkov Signed-off-by: Ben Hutchings

kallsyms: Handle too long symbols in kallsyms.c

2019-05-02T20:41:30+00:00

commit 6db2983cd8064808141ccefd75218f5b4345ffae upstream. When checking for symbols with excessively long names, account for null terminating character. Fixes: f3462aa952cf ("Kbuild: Handle longer symbols in kallsyms.c") Signed-off-by: Eugene Loh Acked-by: Ard Biesheuvel Signed-off-by: Masahiro Yamada [bwh: Backported to 3.16: adjust context] Signed-off-by: Ben Hutchings

kconfig: Avoid format overflow warning from GCC 8.1

2018-11-20T18:05:09+00:00

commit 2ae89c7a82ea9d81a19b4fc2df23bef4b112f24e upstream. In file included from scripts/kconfig/zconf.tab.c:2485: scripts/kconfig/confdata.c: In function ‘conf_write’: scripts/kconfig/confdata.c:773:22: warning: ‘%s’ directive writing likely 7 or more bytes into a region of size between 1 and 4097 [-Wformat-overflow=] sprintf(newname, "%s%s", dirname, basename); ^~ scripts/kconfig/confdata.c:773:19: note: assuming directive output of 7 bytes sprintf(newname, "%s%s", dirname, basename); ^~~~~~ scripts/kconfig/confdata.c:773:2: note: ‘sprintf’ output 1 or more bytes (assuming 4104) into a destination of size 4097 sprintf(newname, "%s%s", dirname, basename); ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ scripts/kconfig/confdata.c:776:23: warning: ‘.tmpconfig.’ directive writing 11 bytes into a region of size between 1 and 4097 [-Wformat-overflow=] sprintf(tmpname, "%s.tmpconfig.%d", dirname, (int)getpid()); ^~~~~~~~~~~ scripts/kconfig/confdata.c:776:3: note: ‘sprintf’ output between 13 and 4119 bytes into a destination of size 4097 sprintf(tmpname, "%s.tmpconfig.%d", dirname, (int)getpid()); ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Increase the size of tmpname and newname to make GCC happy. Signed-off-by: Nathan Chancellor Signed-off-by: Masahiro Yamada Signed-off-by: Ben Hutchings

rpmsg: Correct support for MODULE_DEVICE_TABLE()

2018-11-20T18:04:55+00:00

commit 5b7d127726de6eed4b900bc3bbb167837690818f upstream. Due to missing a missing entry in file2alias.c MODULE_DEVICE_TABLE() are not generating the proper module aliases. Add the needed entry here. Fixes: bcabbccabffe ("rpmsg: add virtio-based remote processor messaging bus") Reported-by: Suman Anna Signed-off-by: Andrew F. Davis Signed-off-by: Greg Kroah-Hartman Signed-off-by: Ben Hutchings

module/retpoline: Warn about missing retpoline in module

2018-03-19T18:59:11+00:00

commit caf7501a1b4ec964190f31f9c3f163de252273b8 upstream. There's a risk that a kernel which has full retpoline mitigations becomes vulnerable when a module gets loaded that hasn't been compiled with the right compiler or the right option. To enable detection of that mismatch at module load time, add a module info string "retpoline" at build time when the module was compiled with retpoline support. This only covers compiled C source, but assembler source or prebuilt object files are not checked. If a retpoline enabled kernel detects a non retpoline protected module at load time, print a warning and report it in the sysfs vulnerability file. [ tglx: Massaged changelog ] Signed-off-by: Andi Kleen Signed-off-by: Thomas Gleixner Cc: David Woodhouse Cc: gregkh@linuxfoundation.org Cc: torvalds@linux-foundation.org Cc: jeyu@kernel.org Cc: arjan@linux.intel.com Link: https://lkml.kernel.org/r/20180125235028.31211-1-andi@firstfloor.org Signed-off-by: Ben Hutchings

modpost: don't emit section mismatch warnings for compiler optimizations

2017-11-11T13:33:42+00:00

commit 4a3893d069b788f3570c19c12d9e986e8e15870f upstream. Currently an allyesconfig build [gcc-4.9.1] can generate the following: WARNING: vmlinux.o(.text.unlikely+0x3864): Section mismatch in reference from the function cpumask_empty.constprop.3() to the variable .init.data:nmi_ipi_mask which comes from the cpumask_empty usage in arch/x86/kernel/nmi_selftest.c. Normally we would not see a symbol entry for cpumask_empty since it is: static inline bool cpumask_empty(const struct cpumask *srcp) however in this case, the variant of the symbol gets emitted when GCC does constant propagation optimization. Fix things up so that any locally optimized constprop variants don't warn when accessing variables that live in the __init sections. [arnd: adapted text_sections definition to 3.18] Signed-off-by: Paul Gortmaker Signed-off-by: Rusty Russell Signed-off-by: Arnd Bergmann Signed-off-by: Ben Hutchings

modpost: expand pattern matching to support substring matches

2017-11-11T13:33:42+00:00

commit 09c20c032b0f753969ae778d9783d946f054d7fe upstream. Currently the match() function supports a leading * to match any prefix and a trailing * to match any suffix. However there currently is not a combination of both that can be used to target matches of whole families of functions that share a common substring. Here we expand the *foo and foo* match to also support *foo* with the goal of targeting compiler generated symbol names that contain strings like ".constprop." and ".isra." Signed-off-by: Paul Gortmaker Signed-off-by: Rusty Russell Signed-off-by: Arnd Bergmann Signed-off-by: Ben Hutchings

sched: add macros to define bitops for task atomic flags

2017-10-12T14:28:22+00:00

commit e0e5070b20e01f0321f97db4e4e174f3f6b49e50 upstream. This will simplify code when we add new flags. v3: - Kees pointed out that no_new_privs should never be cleared, so we shouldn't define task_clear_no_new_privs(). we define 3 macros instead of a single one. v2: - updated scripts/tags.sh, suggested by Peter Cc: Ingo Molnar Cc: Miao Xie Cc: Tetsuo Handa Acked-by: Peter Zijlstra (Intel) Acked-by: Kees Cook Signed-off-by: Zefan Li Signed-off-by: Tejun Heo Signed-off-by: Ben Hutchings

tags: honor COMPILED_SOURCE with apart output directory

2017-09-15T17:29:57+00:00

commit cbf52a3e6a8a92beec6e0c70abf4111cd8f8faf7 upstream. When the kernel is compiled with an "O=" argument, the object files are not in the source tree, but in the build tree. This patch fixes O= build by looking for object files in the build tree. Fixes: 923e02ecf3f8 ("scripts/tags.sh: Support compiled source") Signed-off-by: Robert Jarzmik Signed-off-by: Masahiro Yamada Signed-off-by: Ben Hutchings

kconfig/nconf: Fix hang when editing symbol with a long prompt

2017-03-16T02:26:33+00:00

commit 79e51b5c2deea542b3bb8c66e0d502230b017dde upstream. Currently it is impossible to edit the value of a config symbol with a prompt longer than (terminal width - 2) characters. dialog_inputbox() calculates a negative x-offset for the input window and newwin() fails as this is invalid. It also doesn't check for this failure, so it busy-loops calling wgetch(NULL) which immediately returns -1. The additions in the offset calculations also don't match the intended size of the window. Limit the window size and calculate the offset similarly to show_scroll_win(). Fixes: 692d97c380c6 ("kconfig: new configuration interface (nconfig)") Signed-off-by: Ben Hutchings Signed-off-by: Ben Hutchings