summaryrefslogtreecommitdiff
path: root/fs
AgeCommit message (Collapse)AuthorFilesLines
2021-04-12io_uring: provide io_resubmit_prep() stub for !CONFIG_BLOCKJens Axboe1-8/+6
Randy reports the following error on CONFIG_BLOCK not being set: ../fs/io_uring.c: In function ‘kiocb_done’: ../fs/io_uring.c:2766:7: error: implicit declaration of function ‘io_resubmit_prep’; did you mean ‘io_put_req’? [-Werror=implicit-function-declaration] if (io_resubmit_prep(req)) { Provide a dummy stub for io_resubmit_prep() like we do for io_rw_should_reissue(), which also helps remove an ifdef sequence from io_complete_rw_iopoll() as well. Fixes: 8c130827f417 ("io_uring: don't alter iopoll reissue fail ret code") Reported-by: Randy Dunlap <rdunlap@infradead.org> Signed-off-by: Jens Axboe <axboe@kernel.dk>
2021-04-12ovl: add debug print to ovl_do_getxattr()Amir Goldstein1-1/+6
It was the only ovl_do helper missing it. Signed-off-by: Amir Goldstein <amir73il@gmail.com> Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2021-04-12ovl: invalidate readdir cache on changes to dir with originAmir Goldstein3-37/+36
The test in ovl_dentry_version_inc() was out-dated and did not include the case where readdir cache is used on a non-merge dir that has origin xattr, indicating that it may contain leftover whiteouts. To make the code more robust, use the same helper ovl_dir_is_real() to determine if readdir cache should be used and if readdir cache should be invalidated. Fixes: b79e05aaa166 ("ovl: no direct iteration for dir with origin xattr") Link: https://lore.kernel.org/linux-unionfs/CAOQ4uxht70nODhNHNwGFMSqDyOKLXOKrY0H6g849os4BQ7cokA@mail.gmail.com/ Cc: Chris Murphy <lists@colorremedies.com> Signed-off-by: Amir Goldstein <amir73il@gmail.com> Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2021-04-12ovl: allow upperdir inside lowerdirMiklos Szeredi1-5/+7
Commit 146d62e5a586 ("ovl: detect overlapping layers") made sure we don't have overlapping layers, but it also broke the arguably valid use case of mount -olowerdir=/,upperdir=/subdir,.. where upperdir overlaps lowerdir on the same filesystem. This has been causing regressions. Revert the check, but only for the specific case where upperdir and/or workdir are subdirectories of lowerdir. Any other overlap (e.g. lowerdir is subdirectory of upperdir, etc) case is crazy, so leave the check in place for those. Overlaps are detected at lookup time too, so reverting the mount time check should be safe. Fixes: 146d62e5a586 ("ovl: detect overlapping layers") Cc: <stable@vger.kernel.org> # v5.2 Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2021-04-12ovl: show "userxattr" in the mount dataGiuseppe Scrivano1-0/+2
This was missed when adding the option. Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com> Reviewed-by: Vivek Goyal <vgoyal@redhat.com> Fixes: 2d2f2d7322ff ("ovl: user xattr") Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2021-04-12ovl: trivial typo fixes in the file inode.cBhaskar Chowdhury1-3/+3
s/peresistent/persistent/ s/xatts/xattrs/ s/annotaion/annotation/ Signed-off-by: Bhaskar Chowdhury <unixbhaskar@gmail.com> Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2021-04-12ovl: fix misspellings using codespell toolXiong Zhenwu1-1/+1
A typo is found out by codespell tool: $ codespell ./fs/overlayfs/ ./fs/overlayfs/util.c:217: dependig ==> depending Fix a typo found by codespell. Signed-off-by: Xiong Zhenwu <xiong.zhenwu@zte.com.cn> Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2021-04-12ovl: do not copy attr several timesChengguang Xu1-3/+0
In ovl_xattr_set() we have already copied attr of real inode so no need to copy it again in ovl_posix_acl_xattr_set(). Signed-off-by: Chengguang Xu <cgxu519@mykernel.net> Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2021-04-12ovl: remove ovl_map_dev_ino() return valueyoungjun1-8/+4
ovl_map_dev_ino() always returns success. Remove unnecessary return value. Signed-off-by: youngjun <her0gyugyu@gmail.com> Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2021-04-12ovl: fix error for ovl_fill_super()Chengguang Xu1-0/+4
There are some places should return -EINVAL instead of -ENOMEM in ovl_fill_super(). [Amir] Consistently set error before checking the error condition. Signed-off-by: Chengguang Xu <cgxu519@mykernel.net> Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2021-04-12ovl: fix missing revert_creds() on error pathDan Carpenter1-1/+2
Smatch complains about missing that the ovl_override_creds() doesn't have a matching revert_creds() if the dentry is disconnected. Fix this by moving the ovl_override_creds() until after the disconnected check. Fixes: aa3ff3c152ff ("ovl: copy up of disconnected dentries") Signed-off-by: Dan Carpenter <dan.carpenter@oracle.com> Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2021-04-12ovl: fix leaked dentryMickaël Salaün1-0/+1
Since commit 6815f479ca90 ("ovl: use only uppermetacopy state in ovl_lookup()"), overlayfs doesn't put temporary dentry when there is a metacopy error, which leads to dentry leaks when shutting down the related superblock: overlayfs: refusing to follow metacopy origin for (/file0) ... BUG: Dentry (____ptrval____){i=3f33,n=file3} still in use (1) [unmount of overlay overlay] ... WARNING: CPU: 1 PID: 432 at umount_check.cold+0x107/0x14d CPU: 1 PID: 432 Comm: unmount-overlay Not tainted 5.12.0-rc5 #1 ... RIP: 0010:umount_check.cold+0x107/0x14d ... Call Trace: d_walk+0x28c/0x950 ? dentry_lru_isolate+0x2b0/0x2b0 ? __kasan_slab_free+0x12/0x20 do_one_tree+0x33/0x60 shrink_dcache_for_umount+0x78/0x1d0 generic_shutdown_super+0x70/0x440 kill_anon_super+0x3e/0x70 deactivate_locked_super+0xc4/0x160 deactivate_super+0xfa/0x140 cleanup_mnt+0x22e/0x370 __cleanup_mnt+0x1a/0x30 task_work_run+0x139/0x210 do_exit+0xb0c/0x2820 ? __kasan_check_read+0x1d/0x30 ? find_held_lock+0x35/0x160 ? lock_release+0x1b6/0x660 ? mm_update_next_owner+0xa20/0xa20 ? reacquire_held_locks+0x3f0/0x3f0 ? __sanitizer_cov_trace_const_cmp4+0x22/0x30 do_group_exit+0x135/0x380 __do_sys_exit_group.isra.0+0x20/0x20 __x64_sys_exit_group+0x3c/0x50 do_syscall_64+0x45/0x70 entry_SYSCALL_64_after_hwframe+0x44/0xae ... VFS: Busy inodes after unmount of overlay. Self-destruct in 5 seconds. Have a nice day... This fix has been tested with a syzkaller reproducer. Cc: Amir Goldstein <amir73il@gmail.com> Cc: <stable@vger.kernel.org> # v5.8+ Reported-by: syzbot <syzkaller@googlegroups.com> Fixes: 6815f479ca90 ("ovl: use only uppermetacopy state in ovl_lookup()") Signed-off-by: Mickaël Salaün <mic@linux.microsoft.com> Link: https://lore.kernel.org/r/20210329164907.2133175-1-mic@digikod.net Reviewed-by: Vivek Goyal <vgoyal@redhat.com> Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2021-04-12ovl: restrict lower null uuid for "xino=auto"Amir Goldstein1-6/+35
Commit a888db310195 ("ovl: fix regression with re-formatted lower squashfs") attempted to fix a regression with existing setups that use a practice that we are trying to discourage. The discourage part was described this way in the commit message: "To avoid the reported regression while still allowing the new features with single lower squashfs, do not allow decoding origin with lower null uuid unless user opted-in to one of the new features that require following the lower inode of non-dir upper (index, xino, metacopy)." The three mentioned features are disabled by default in Kconfig, so it was assumed that if they are enabled, the user opted-in for them. Apparently, distros started to configure CONFIG_OVERLAY_FS_XINO_AUTO=y some time ago, so users upgrading their kernels can still be affected by said regression even though they never opted-in for any new feature. To fix this, treat "xino=on" as "user opted-in", but not "xino=auto". Since we are changing the behavior of "xino=auto" to no longer follow to lower origin with null uuid, take this one step further and disable xino in that corner case. To be consistent, disable xino also in cases of lower fs without file handle support and upper fs without xattr support. Update documentation w.r.t the new "xino=auto" behavior and fix the out dated bits of documentation regarding "xino" and regarding offline modifications to lower layers. Link: https://lore.kernel.org/linux-unionfs/b36a429d7c563730c28d763d4d57a6fc30508a4f.1615216996.git.kevin@kevinlocke.name/ Signed-off-by: Amir Goldstein <amir73il@gmail.com> Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2021-04-12ovl: check that upperdir path is not on a read-only mountAmir Goldstein1-2/+2
So far we only checked that sb is not read-only. Suggested-by: Christian Brauner <christian.brauner@ubuntu.com> Signed-off-by: Amir Goldstein <amir73il@gmail.com> Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2021-04-12ovl: plumb through flush methodSargun Dhillon1-0/+21
Filesystems can implement their own flush method that release resources, or manipulate caches. Currently if one of these filesystems is used with overlayfs, the flush method is not called. [Amir: fix fd leak in ovl_flush()] Signed-off-by: Sargun Dhillon <sargun@sargun.me> Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2021-04-12io_uring: optimise fill_event() by inliningPavel Begunkov1-21/+36
There are three cases where we much care about performance of io_cqring_fill_event() -- flushing inline completions, iopoll and io_req_complete_post(). Inline a hot part of fill_event() into them. All others are not as important and we don't want to bloat binary for them, so add a noinline version of the function for all other use use cases. nops test(batch=32): 16.932 vs 17.822 KIOPS Signed-off-by: Pavel Begunkov <asml.silence@gmail.com> Link: https://lore.kernel.org/r/a11d59424bf4417aca33f5ec21008bb3b0ebd11e.1618101759.git.asml.silence@gmail.com Signed-off-by: Jens Axboe <axboe@kernel.dk>
2021-04-12io_uring: always pass cflags into fill_event()Pavel Begunkov1-21/+16
A simple preparation patch inlining io_cqring_fill_event(), which only role was to pass cflags=0 into an actual fill event. It helps to keep number of related helpers sane in following patches. Signed-off-by: Pavel Begunkov <asml.silence@gmail.com> Link: https://lore.kernel.org/r/704f9c85b7d9843e4ad50a9f057200c58f5adc6e.1618101759.git.asml.silence@gmail.com Signed-off-by: Jens Axboe <axboe@kernel.dk>
2021-04-12io_uring: optimise non-eventfd post-eventPavel Begunkov1-4/+2
Eventfd is not the canonical way of using io_uring, annotate io_should_trigger_evfd() with likely so it improves code generation for non-eventfd branch. Signed-off-by: Pavel Begunkov <asml.silence@gmail.com> Link: https://lore.kernel.org/r/42fdaa51c68d39479f02cef4fe5bcb24624d60fa.1618101759.git.asml.silence@gmail.com Signed-off-by: Jens Axboe <axboe@kernel.dk>
2021-04-12io_uring: refactor compat_msghdr importPavel Begunkov1-6/+5
Add an entry for user pointer to compat_msghdr into io_connect, so it's explicit that we may use it as this, and removes annoying casts. Signed-off-by: Pavel Begunkov <asml.silence@gmail.com> Link: https://lore.kernel.org/r/73fd644dea1518f528d3648981cf777ce6e537e9.1618101759.git.asml.silence@gmail.com Signed-off-by: Jens Axboe <axboe@kernel.dk>
2021-04-12io_uring: enable inline completion for more casesPavel Begunkov1-3/+3
Take advantage of delayed/inline completion flushing and pass right issue flags for completion of open, open2, fadvise and poll remove opcodes. All others either already use it or always punted and never executed inline. Signed-off-by: Pavel Begunkov <asml.silence@gmail.com> Link: https://lore.kernel.org/r/0badc7512e82f7350b73bb09abbebbecbdd5dab8.1618101759.git.asml.silence@gmail.com Signed-off-by: Jens Axboe <axboe@kernel.dk>
2021-04-12io_uring: refactor io_closePavel Begunkov1-10/+3
A small refactoring shrinking it and making easier to read. Signed-off-by: Pavel Begunkov <asml.silence@gmail.com> Link: https://lore.kernel.org/r/19b24eed7cd491a0243b50366dd2a23b558e2665.1618101759.git.asml.silence@gmail.com Signed-off-by: Jens Axboe <axboe@kernel.dk>
2021-04-12io_uring: unify files and task cancelPavel Begunkov1-42/+14
Now __io_uring_cancel() and __io_uring_files_cancel() are very similar and mostly differ by how we count requests, merge them and allow tctx_inflight() to handle counting. Signed-off-by: Pavel Begunkov <asml.silence@gmail.com> Link: https://lore.kernel.org/r/1a5986a97df4dc1378f3fe0ca1eb483dbcf42112.1618101759.git.asml.silence@gmail.com Signed-off-by: Jens Axboe <axboe@kernel.dk>
2021-04-12io_uring: track inflight requests through counterPavel Begunkov1-46/+10
Instead of keeping requests in a inflight_list, just track them with a per tctx atomic counter. Apart from it being much easier and more consistent with task cancel, it frees ->inflight_entry from being shared between iopoll and cancel-track, so less headache for us. Signed-off-by: Pavel Begunkov <asml.silence@gmail.com> Link: https://lore.kernel.org/r/3c2ee0863cd7eeefa605f3eaff4c1c461a6f1157.1618101759.git.asml.silence@gmail.com Signed-off-by: Jens Axboe <axboe@kernel.dk>
2021-04-12io_uring: unify task and files cancel loopsPavel Begunkov1-33/+41
Move tracked inflight number check up the stack into __io_uring_files_cancel() so it's similar to task cancel. Will be used for further cleaning. Signed-off-by: Pavel Begunkov <asml.silence@gmail.com> Link: https://lore.kernel.org/r/dca5a395efebd1e3e0f3bbc6b9640c5e8aa7e468.1618101759.git.asml.silence@gmail.com Signed-off-by: Jens Axboe <axboe@kernel.dk>
2021-04-12io_uring: simplify apoll hash removalPavel Begunkov1-4/+1
hash_del() works well with non-hashed nodes, there's no need to check if it is hashed first. Signed-off-by: Pavel Begunkov <asml.silence@gmail.com> Signed-off-by: Jens Axboe <axboe@kernel.dk>
2021-04-12io_uring: refactor io_poll_complete()Pavel Begunkov1-6/+7
Remove error parameter from io_poll_complete(), 0 is always passed, and do a bit of cleaning on top. Signed-off-by: Pavel Begunkov <asml.silence@gmail.com> Signed-off-by: Jens Axboe <axboe@kernel.dk>
2021-04-12io_uring: clean up io_poll_task_func()Pavel Begunkov1-6/+4
io_poll_complete() always fills an event (even an overflowed one), so we always should do io_cqring_ev_posted() afterwards. And that's what is currently happening, because second EPOLLONESHOT check is always true, it can't return !done for oneshots. Remove those branching, it's much easier to read. Signed-off-by: Pavel Begunkov <asml.silence@gmail.com> Signed-off-by: Jens Axboe <axboe@kernel.dk>
2021-04-12io-wq: Fix io_wq_worker_affinity()Peter Zijlstra1-9/+2
Do not include private headers and do not frob in internals. On top of that, while the previous code restores the affinity, it doesn't ensure the task actually moves there if it was running, leading to the fun situation that it can be observed running outside of its allowed mask for potentially significant time. Use the proper API instead. Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org> Link: https://lore.kernel.org/r/YG7QkiUzlEbW85TU@hirez.programming.kicks-ass.net Signed-off-by: Jens Axboe <axboe@kernel.dk>
2021-04-12io_uring: don't attempt re-add of multishot poll request if racingJens Axboe1-10/+20
We currently allow racy updates to multishot requests, but we can end up double adding the poll request if both completion and update does it. Ensure that we skip re-add on the update side if someone else is completing it. Fixes: b69de288e913 ("io_uring: allow events and user_data update of running poll requests") Reported-by: Joakim Hassila <joj@mac.com> Signed-off-by: Jens Axboe <axboe@kernel.dk>
2021-04-12io-wq: simplify code in __io_worker_busy()Hao Xu1-9/+6
Leverage XOR to simplify the code in __io_worker_busy. Signed-off-by: Hao Xu <haoxu@linux.alibaba.com> Link: https://lore.kernel.org/r/1617678525-3129-1-git-send-email-haoxu@linux.alibaba.com Signed-off-by: Jens Axboe <axboe@kernel.dk>
2021-04-12io_uring: kill outdated comment about splice puntPavel Begunkov1-4/+0
The splice/tee comment in io_prep_async_work() isn't relevant since the section was moved, delete it. Signed-off-by: Pavel Begunkov <asml.silence@gmail.com> Link: https://lore.kernel.org/r/892a549c89c3d422b679677b8e68ffd3fcb736b6.1617287883.git.asml.silence@gmail.com Signed-off-by: Jens Axboe <axboe@kernel.dk>
2021-04-12io_uring: encapsulate fixed files into structPavel Begunkov1-13/+19
Add struct io_fixed_file representing a single registered file, first to hide ugly struct file **, which may be misleading, and secondly to retype it to unsigned long as conversions to it and back to file * for handling and masking FFS_* flags are getting nasty. Signed-off-by: Pavel Begunkov <asml.silence@gmail.com> Link: https://lore.kernel.org/r/78669731a605a7614c577c3de552631cfaf0869a.1617287883.git.asml.silence@gmail.com Signed-off-by: Jens Axboe <axboe@kernel.dk>
2021-04-12io_uring: refactor file tables alloc/freePavel Begunkov1-26/+26
Introduce a heler io_free_file_tables() doing all the cleaning, there are several places where it's hand coded. Also move all allocations into io_sqe_alloc_file_tables() and rename it, so all of it is in one place. Signed-off-by: Pavel Begunkov <asml.silence@gmail.com> Link: https://lore.kernel.org/r/502a84ebf41ff119b095e59661e678eacb752bf8.1617287883.git.asml.silence@gmail.com Signed-off-by: Jens Axboe <axboe@kernel.dk>
2021-04-12io_uring: don't quiesce intial files registerPavel Begunkov1-0/+1
There is no reason why we would want to fully quiesce ring on IORING_REGISTER_FILES, if it's already registered we fail. Signed-off-by: Pavel Begunkov <asml.silence@gmail.com> Link: https://lore.kernel.org/r/563bb8060bb2d3efbc32fce6101678281c574d2a.1617287883.git.asml.silence@gmail.com Signed-off-by: Jens Axboe <axboe@kernel.dk>
2021-04-12io_uring: set proper FFS* flags on reg file updatePavel Begunkov1-11/+15
Set FFS_* flags (e.g. FFS_ASYNC_READ) not only in initial registration but also on registered files update. Not a bug, but may miss getting profit out of the feature. Signed-off-by: Pavel Begunkov <asml.silence@gmail.com> Link: https://lore.kernel.org/r/df29a841a2d3d3695b509cdffce5070777d9d942.1617287883.git.asml.silence@gmail.com Signed-off-by: Jens Axboe <axboe@kernel.dk>
2021-04-12io_uring: deduplicate NOSIGNAL settingPavel Begunkov1-22/+14
Set MSG_NOSIGNAL and REQ_F_NOWAIT in send/recv prep routines and don't duplicate it in all four send/recv handlers. Signed-off-by: Pavel Begunkov <asml.silence@gmail.com> Link: https://lore.kernel.org/r/e1133a3ed1c0e192975b7341ea4b0bf91f63b132.1617287883.git.asml.silence@gmail.com Signed-off-by: Jens Axboe <axboe@kernel.dk>
2021-04-12io_uring: put link timeout req consistentlyPavel Begunkov1-7/+3
Don't put linked timeout req in io_async_find_and_cancel() but do it in io_link_timeout_fn(), so we have only one point for that and won't have to do it differently as it's now (put vs put_deferred). Btw, improve a bit io_async_find_and_cancel()'s locking. Signed-off-by: Pavel Begunkov <asml.silence@gmail.com> Link: https://lore.kernel.org/r/d75b70957f245275ab7cba83e0ac9c1b86aae78a.1617287883.git.asml.silence@gmail.com Signed-off-by: Jens Axboe <axboe@kernel.dk>
2021-04-12io_uring: simplify overflow handlingPavel Begunkov1-5/+3
Overflowed CQEs doesn't lock requests anymore, so we don't care so much about cancelling them, so kill cq_overflow_flushed and simplify the code. Signed-off-by: Pavel Begunkov <asml.silence@gmail.com> Link: https://lore.kernel.org/r/5799867aeba9e713c32f49aef78e5e1aef9fbc43.1617287883.git.asml.silence@gmail.com Signed-off-by: Jens Axboe <axboe@kernel.dk>
2021-04-12io_uring: lock annotate timeouts and pollPavel Begunkov1-0/+10
Add timeout and poll ->comletion_lock annotations for Sparse, makes life easier while looking at the functions. Signed-off-by: Pavel Begunkov <asml.silence@gmail.com> Link: https://lore.kernel.org/r/2345325643093d41543383ba985a735aeb899eac.1617287883.git.asml.silence@gmail.com Signed-off-by: Jens Axboe <axboe@kernel.dk>
2021-04-12io_uring: kill unused forward declsPavel Begunkov1-4/+2
Kill unused forward declarations for io_ring_file_put() and io_queue_next(). Also btw rename the first one. Signed-off-by: Pavel Begunkov <asml.silence@gmail.com> Link: https://lore.kernel.org/r/64aa27c3f9662e14615cc119189f5eaf12989671.1617287883.git.asml.silence@gmail.com Signed-off-by: Jens Axboe <axboe@kernel.dk>
2021-04-12io_uring: store reg buffer end instead of lengthPavel Begunkov1-5/+5
It's a bit more convenient for us to store a registered buffer end address instead of length, see struct io_mapped_ubuf, as it allow to not recompute it every time. Signed-off-by: Pavel Begunkov <asml.silence@gmail.com> Link: https://lore.kernel.org/r/39164403fe92f1dc437af134adeec2423cdf9395.1617287883.git.asml.silence@gmail.com Signed-off-by: Jens Axboe <axboe@kernel.dk>
2021-04-12io_uring: improve import_fixed overflow checksPavel Begunkov1-4/+3
Replace a hand-coded overflow check with a specialised function. Even though compilers are smart enough to generate identical binary (i.e. check carry bit), but it's more foolproof and conveys the intention better. Signed-off-by: Pavel Begunkov <asml.silence@gmail.com> Link: https://lore.kernel.org/r/e437dcdc929bacbb6f11a4824ecbbf17225cb82a.1617287883.git.asml.silence@gmail.com Signed-off-by: Jens Axboe <axboe@kernel.dk>
2021-04-12io_uring: refactor io_async_cancel()Pavel Begunkov1-2/+0
Remove extra tctx==NULL checks that are already done by io_async_cancel_one(). Signed-off-by: Pavel Begunkov <asml.silence@gmail.com> Link: https://lore.kernel.org/r/70c2a8b958d942e86958a28af0452966ce1095b0.1617287883.git.asml.silence@gmail.com Signed-off-by: Jens Axboe <axboe@kernel.dk>
2021-04-12io_uring: remove unused hash_waitPavel Begunkov1-2/+0
No users of io_uring_ctx::hash_wait left, kill it. Signed-off-by: Pavel Begunkov <asml.silence@gmail.com> Link: https://lore.kernel.org/r/e25cb83c233a5f75f15275596b49fbafbea606fa.1617287883.git.asml.silence@gmail.com Signed-off-by: Jens Axboe <axboe@kernel.dk>
2021-04-12io_uring: better ref handling in poll_remove_onePavel Begunkov1-1/+1
Instead of io_put_req() to drop not a final ref, use req_ref_put(), which is slimmer and will also check the invariant. Signed-off-by: Pavel Begunkov <asml.silence@gmail.com> Link: https://lore.kernel.org/r/85b5774ce13ae55cc2e705abdc8cbafe1212f1bd.1617287883.git.asml.silence@gmail.com Signed-off-by: Jens Axboe <axboe@kernel.dk>
2021-04-12io_uring: combine lock/unlock sections on exitPavel Begunkov1-10/+8
io_ring_exit_work() already does uring_lock lock/unlock, no need to repeat it for lock waiting trick in io_ring_ctx_free(). Move the waiting with comments and spinlocking into io_ring_exit_work. Signed-off-by: Pavel Begunkov <asml.silence@gmail.com> Link: https://lore.kernel.org/r/a8ae0589b0ea64ad4791e2c282e4e9b713dd7024.1617287883.git.asml.silence@gmail.com Signed-off-by: Jens Axboe <axboe@kernel.dk>
2021-04-12io_uring: remove useless is_dying check on quiescePavel Begunkov1-6/+2
rsrc_data refs should always be valid for potential submitters, io_rsrc_ref_quiesce() restores it before unlocking, so percpu_ref_is_dying() check in io_sqe_files_unregister() does nothing and misleading. Concurrent quiesce is prevented with struct io_rsrc_data::quiesce. Signed-off-by: Pavel Begunkov <asml.silence@gmail.com> Link: https://lore.kernel.org/r/bf97055e1748ee3a382e66daf384a469eb90b931.1617287883.git.asml.silence@gmail.com Signed-off-by: Jens Axboe <axboe@kernel.dk>
2021-04-12io_uring: reuse io_rsrc_node_destroy()Pavel Begunkov1-9/+7
Reuse io_rsrc_node_destroy() in __io_rsrc_put_work(). Also move it to a more appropriate place -- to the other node routines, and remove forward declaration. Signed-off-by: Pavel Begunkov <asml.silence@gmail.com> Link: https://lore.kernel.org/r/cccafba41aee1e5bb59988704885b1340aef3a27.1617287883.git.asml.silence@gmail.com Signed-off-by: Jens Axboe <axboe@kernel.dk>
2021-04-12io_uring: ctx-wide rsrc nodesPavel Begunkov1-36/+37
If we're going to ever support multiple types of resources we need shared rsrc nodes to not bloat requests, that is implemented in this patch. It also gives a nicer API and saves one pointer dereference in io_req_set_rsrc_node(). We may say that all requests bound to a resource belong to one and only one rsrc node, and considering that nodes are removed and recycled strictly in-order, this separates requests into generations, where generation are changed on each node switch (i.e. io_rsrc_node_switch()). The API is simple, io_rsrc_node_switch() switches to a new generation if needed, and also optionally kills a passed in io_rsrc_data. Each call to io_rsrc_node_switch() have to be preceded with io_rsrc_node_switch_start(). The start function is idempotent and should not necessarily be followed by switch. One difference is that once a node was set it will always retain a valid rsrc node, even on unregister. It may be a nuisance at the moment, but makes much sense for multiple types of resources. Another thing changed is that nodes are bound to/associated with a io_rsrc_data later just before killing (i.e. switching). Signed-off-by: Pavel Begunkov <asml.silence@gmail.com> Link: https://lore.kernel.org/r/7e9c693b4b9a2f47aa784b616ce29843021bb65a.1617287883.git.asml.silence@gmail.com Signed-off-by: Jens Axboe <axboe@kernel.dk>
2021-04-12io_uring: refactor io_queue_rsrc_removal()Pavel Begunkov1-11/+4
Pass rsrc_node into io_queue_rsrc_removal() explicitly. Just a simple preparation patch, makes following changes nicer. Signed-off-by: Pavel Begunkov <asml.silence@gmail.com> Link: https://lore.kernel.org/r/002889ce4de7baf287f2b010eef86ffe889174c6.1617287883.git.asml.silence@gmail.com Signed-off-by: Jens Axboe <axboe@kernel.dk>
generated by cgit v1.2.3 (git 2.39.0) at 2025-09-12 00:44:41 +0300