kernel/linux.git/include/linux/kernfs.h, branch v6.12.80

kernfs: RCU protect kernfs_nodes and avoid kernfs_idr_lock in kernfs_find_and_get_node_by_id()

2024-01-30T23:54:25+00:00

The BPF helper bpf_cgroup_from_id() calls kernfs_find_and_get_node_by_id() which acquires kernfs_idr_lock, which is an non-raw non-IRQ-safe lock. This can lead to deadlocks as bpf_cgroup_from_id() can be called from any BPF programs including e.g. the ones that attach to functions which are holding the scheduler rq lock. Consider the following BPF program: SEC("fentry/__set_cpus_allowed_ptr_locked") int BPF_PROG(__set_cpus_allowed_ptr_locked, struct task_struct *p, struct affinity_context *affn_ctx, struct rq *rq, struct rq_flags *rf) { struct cgroup *cgrp = bpf_cgroup_from_id(p->cgroups->dfl_cgrp->kn->id); if (cgrp) { bpf_printk("%d[%s] in %s", p->pid, p->comm, cgrp->kn->name); bpf_cgroup_release(cgrp); } return 0; } __set_cpus_allowed_ptr_locked() is called with rq lock held and the above BPF program calls bpf_cgroup_from_id() within leading to the following lockdep warning: ===================================================== WARNING: HARDIRQ-safe -> HARDIRQ-unsafe lock order detected 6.7.0-rc3-work-00053-g07124366a1d7-dirty #147 Not tainted ----------------------------------------------------- repro/1620 [HC0[0]:SC0[0]:HE0:SE1] is trying to acquire: ffffffff833b3688 (kernfs_idr_lock){+.+.}-{2:2}, at: kernfs_find_and_get_node_by_id+0x1e/0x70 and this task is already holding: ffff888237ced698 (&rq->__lock){-.-.}-{2:2}, at: task_rq_lock+0x4e/0xf0 which would create a new lock dependency: (&rq->__lock){-.-.}-{2:2} -> (kernfs_idr_lock){+.+.}-{2:2} ... Possible interrupt unsafe locking scenario: CPU0 CPU1 ---- ---- lock(kernfs_idr_lock); local_irq_disable(); lock(&rq->__lock); lock(kernfs_idr_lock); lock(&rq->__lock); *** DEADLOCK *** ... Call Trace: dump_stack_lvl+0x55/0x70 dump_stack+0x10/0x20 __lock_acquire+0x781/0x2a40 lock_acquire+0xbf/0x1f0 _raw_spin_lock+0x2f/0x40 kernfs_find_and_get_node_by_id+0x1e/0x70 cgroup_get_from_id+0x21/0x240 bpf_cgroup_from_id+0xe/0x20 bpf_prog_98652316e9337a5a___set_cpus_allowed_ptr_locked+0x96/0x11a bpf_trampoline_6442545632+0x4f/0x1000 __set_cpus_allowed_ptr_locked+0x5/0x5a0 sched_setaffinity+0x1b3/0x290 __x64_sys_sched_setaffinity+0x4f/0x60 do_syscall_64+0x40/0xe0 entry_SYSCALL_64_after_hwframe+0x46/0x4e Let's fix it by protecting kernfs_node and kernfs_root with RCU and making kernfs_find_and_get_node_by_id() acquire rcu_read_lock() instead of kernfs_idr_lock. This adds an rcu_head to kernfs_node making it larger by 16 bytes on 64bit. Combined with the preceding rearrange patch, the net increase is 8 bytes. Signed-off-by: Tejun Heo Cc: Andrea Righi Cc: Geert Uytterhoeven Link: https://lore.kernel.org/r/20240109214828.252092-4-tj@kernel.org Signed-off-by: Greg Kroah-Hartman

kernfs: Rearrange kernfs_node fields to reduce its size on 64bit

2024-01-30T23:53:59+00:00

Moving .flags and .mode right below .hash makes kernfs_node smaller by 8 bytes on 64bit. To avoid creating a hole from 8 bytes alignment on 32bit archs, .priv is moved below so that there are two 32bit pointers after the 64bit .id field. v2: Updated to avoid size increase on 32bit noticed by Geert. Signed-off-by: Tejun Heo Cc: Geert Uytterhoeven Link: https://lore.kernel.org/r/ZZ7hwA18nfmFjYpj@slm.duckdns.org Signed-off-by: Greg Kroah-Hartman

kernfs: sysfs: support custom llseek method for sysfs entries

2023-10-05T11:42:11+00:00

As of now, seeking in sysfs files is handled by generic_file_llseek(). There are situations where one may want to customize seeking logic: - Many sysfs entries are fixed files while generic_file_llseek() accepts past-the-end positions. Not only being useless by itself, this also means a bug in userspace code will trigger not at lseek(), but at some later point making debugging harder. - generic_file_llseek() relies on f_mapping->host to get the file size which might not be correct for all sysfs entries. See commit 636b21b50152 ("PCI: Revoke mappings like devmem") as an example. Implement llseek method to override this behavior at sysfs attribute level. The method is optional, and if it is absent, generic_file_llseek() is called to preserve backwards compatibility. Signed-off-by: Valentine Sinitsyn Link: https://lore.kernel.org/r/20230925084013.309399-1-valesini@yandex-team.ru Signed-off-by: Greg Kroah-Hartman

kernfs: add stub helper for kernfs_generic_poll()

2023-08-05T06:31:42+00:00

In some randconfig builds, kernfs ends up being disabled, so there is no prototype for kernfs_generic_poll() In file included from kernel/sched/build_utility.c:97: kernel/sched/psi.c:1479:3: error: implicit declaration of function 'kernfs_generic_poll' is invalid in C99 [-Werror,-Wimplicit-function-declaration] kernfs_generic_poll(t->of, wait); ^ Add a stub helper for it, as we have it for other kernfs functions. Fixes: aff037078ecae ("sched/psi: use kernfs polling functions for PSI trigger polling") Fixes: 147e1a97c4a0b ("fs: kernfs: add poll file operation") Signed-off-by: Arnd Bergmann Reviewed-by: Chengming Zhou Link: https://lore.kernel.org/r/20230724121823.1357562-1-arnd@kernel.org Signed-off-by: Greg Kroah-Hartman

kernfs: Implement kernfs_show()

2022-09-01T16:08:44+00:00

Currently, kernfs nodes can be created hidden and activated later by calling kernfs_activate() to allow creation of multiple nodes to succeed or fail as a unit. This is an one-way one-time-only transition. This patch introduces kernfs_show() which can toggle visibility dynamically. As the currently proposed use - toggling the cgroup pressure files - only requires operating on leaf nodes, for the sake of simplicity, restrict it as such for now. Hiding uses the same mechanism as deactivation and likewise guarantees that there are no in-flight operations on completion. KERNFS_ACTIVATED and KERNFS_HIDDEN are used to manage the interactions between activations and show/hide operations. A node is visible iff both activated & !hidden. Cc: Chengming Zhou Cc: Johannes Weiner Tested-by: Chengming Zhou Reviewed-by: Chengming Zhou Signed-off-by: Tejun Heo Link: https://lore.kernel.org/r/20220828050440.734579-9-tj@kernel.org Signed-off-by: Greg Kroah-Hartman

kernfs: Add KERNFS_REMOVING flags

2022-09-01T16:08:44+00:00

KERNFS_ACTIVATED tracks whether a given node has ever been activated. As a node was only deactivated on removal, this was used for 1. Drain optimization (removed by the previous patch). 2. To hide !activated nodes 3. To avoid double activations 4. Reject adding children to a node being removed 5. Skip activaing a node which is being removed. We want to decouple deactivation from removal so that nodes can be deactivated and hidden dynamically, which makes KERNFS_ACTIVATED useless for all of the above purposes. #1 is already gone. #2 and #3 can instead test whether the node is currently active. A new flag KERNFS_REMOVING is added to explicitly mark nodes which are being removed for #4 and #5. While this leaves KERNFS_ACTIVATED with no users, leave it be as it will be used in a following patch. Cc: Chengming Zhou Tested-by: Chengming Zhou Reviewed-by: Chengming Zhou Signed-off-by: Tejun Heo Link: https://lore.kernel.org/r/20220828050440.734579-7-tj@kernel.org Signed-off-by: Greg Kroah-Hartman

Revert "kernfs: Change kernfs_notify_list to llist."

2022-07-06T12:20:22+00:00

This reverts commit b8f35fa1188b84035c59d4842826c4e93a1b1c9f. This is causing regression due to same kernfs_node getting added multiple times in kernfs_notify_list so revert it until safe way of using llist in this context is found. Reported-by: Nathan Chancellor Reported-by: Michael Walle Reported-by: Marek Szyprowski Signed-off-by: Imran Khan Cc: Tejun Heo Link: https://lore.kernel.org/r/20220705201026.2487665-1-imran.f.khan@oracle.com Signed-off-by: Greg Kroah-Hartman

kernfs: Replace global kernfs_open_file_mutex with hashed mutexes.

2022-06-27T14:46:15+00:00

In current kernfs design a single mutex, kernfs_open_file_mutex, protects the list of kernfs_open_file instances corresponding to a sysfs attribute. So even if different tasks are opening or closing different sysfs files they can contend on osq_lock of this mutex. The contention is more apparent in large scale systems with few hundred CPUs where most of the CPUs have running tasks that are opening, accessing or closing sysfs files at any point of time. Using hashed mutexes in place of a single global mutex, can significantly reduce contention around global mutex and hence can provide better scalability. Moreover as these hashed mutexes are not part of kernfs_node objects we will not see any singnificant change in memory utilization of kernfs based file systems like sysfs, cgroupfs etc. Modify interface introduced in previous patch to make use of hashed mutexes. Use kernfs_node address as hashing key. Acked-by: Tejun Heo Signed-off-by: Imran Khan Link: https://lore.kernel.org/r/20220615021059.862643-5-imran.f.khan@oracle.com Signed-off-by: Greg Kroah-Hartman

kernfs: Change kernfs_notify_list to llist.

2022-06-27T14:46:15+00:00

At present kernfs_notify_list is implemented as a singly linked list of kernfs_node(s), where last element points to itself and value of ->attr.next tells if node is present on the list or not. Both addition and deletion to list happen under kernfs_notify_lock. Change kernfs_notify_list to llist so that addition to list can heppen locklessly. Suggested by: Al Viro Acked-by: Tejun Heo Signed-off-by: Imran Khan Link: https://lore.kernel.org/r/20220615021059.862643-3-imran.f.khan@oracle.com Signed-off-by: Greg Kroah-Hartman

kernfs: make ->attr.open RCU protected.

2022-06-27T14:46:14+00:00

After removal of kernfs_open_node->refcnt in the previous patch, kernfs_open_node_lock can be removed as well by making ->attr.open RCU protected. kernfs_put_open_node can delegate freeing to ->attr.open to RCU and other readers of ->attr.open can do so under rcu_read_(un)lock. Suggested by: Al Viro Acked-by: Tejun Heo Signed-off-by: Imran Khan Link: https://lore.kernel.org/r/20220615021059.862643-2-imran.f.khan@oracle.com Signed-off-by: Greg Kroah-Hartman