kernel/linux.git/arch/arm/vfp, branch v6.19.11

ARM: 9446/1: Disallow kernel mode NEON when IRQs are disabled

2025-05-29T10:22:25+00:00

Commit c79f81631142 ("ARM: 9283/1: permit non-nested kernel mode NEON in softirq context") relaxed the rules around the use of SIMD instructions in kernel mode on ARM, to allow such use when serving a softirq. To avoid having to preserve/restore kernel mode NEON state when such a softirq is taken, softirqs are now disabled when using the NEON from task context. However, the fact that the softirq API does not allow unmasking of softirqs with interrupts disabled was overlooked, resulting in a WARN() in some cases, as reported by Guenter: WARNING: CPU: 0 PID: 1145 at kernel/softirq.c:369 __local_bh_enable_ip+0x118/0x194 Call trace: unwind_backtrace from show_stack+0x10/0x14 show_stack from dump_stack_lvl+0x7c/0xac dump_stack_lvl from __warn+0x7c/0x1b8 __warn from warn_slowpath_fmt+0x19c/0x1a4 warn_slowpath_fmt from __local_bh_enable_ip+0x118/0x194 __local_bh_enable_ip from crc_t10dif_arch+0xd4/0xe8 crc_t10dif_arch from crc_t10dif_wrapper+0x14/0x1c crc_t10dif_wrapper from crc_main_test+0x178/0x360 crc_main_test from kunit_try_run_case+0x78/0x1e0 kunit_try_run_case from kunit_generic_run_threadfn_adapter+0x1c/0x34 kunit_generic_run_threadfn_adapter from kthread+0x118/0x254 kthread from ret_from_fork+0x14/0x28 While disabling softirqs is not really needed when running with IRQs disabled (given that the only way a softirq can be delivered asynchrously is over the back of an IRQ), let's not complicate this logic more than needed, and simply disallow use of the NEON in kernel mode when IRQs are disabled. Another approach might be to only disable and re-enable softirqs if IRQs are enabled, but other than the test case above, there are no clear use cases for doing non-trivial arithmetic processing (hence using an accelerated SIMD implementation) with IRQs disabled. Reported-by: Guenter Roeck Tested-by: Guenter Roeck Link: https://lore.kernel.org/all/389b899f-893c-4855-9e30-d8920a5d6f91@roeck-us.net Reviewed-by: Eric Biggers Signed-off-by: Ard Biesheuvel Signed-off-by: Russell King (Oracle)

ARM: 9438/1: assembler: Drop obsolete VFP accessor fallback

2025-01-14T12:29:25+00:00

Now that the minimum supported binutils version is 2.25, we no longer need a workaround for binutils older than 2.24 for accessing VFP control registers from assembler. Reviewed-by: Linus Walleij Reviewed-by: Nathan Chancellor Signed-off-by: Ard Biesheuvel Signed-off-by: Russell King (Oracle)

ARM: 9437/1: vfp: Fix typographical errors in vfpmodule.c

2025-01-14T12:29:23+00:00

Correct the misspellings of "noftify" (should be "notify") and "swtich" (should be "switch"). Link: https://lore.kernel.org/r/20241114011939.296230-1-luoyifan@cmss.chinamobile.com Signed-off-by: Luo Yifan Reviewed-by: Linus Walleij Signed-off-by: Russell King (Oracle)

ARM: 9426/1: vfp: Move sending signals outside of vfp_state_hold()ed section.

2024-11-12T16:43:09+00:00

VFP_bounce() is invoked from within vfp_support_entry() and may send a signal. Sending a signal uses spinlock_t which becomes a sleeping lock on PREEMPT_RT and must not be acquired within a preempt-disabled section. Move the vfp_raise_sigfpe() block outside of the vfp_state_hold() section. Reviewed-by: Ard Biesheuvel Signed-off-by: Sebastian Andrzej Siewior Signed-off-by: Russell King (Oracle)

ARM: 9425/1: vfp: Use vfp_state_hold() in vfp_support_entry().

2024-11-12T16:42:30+00:00

vfp_entry() is invoked from exception handler and is fully preemptible. It uses local_bh_disable() to remain uninterrupted while checking the VFP state. This is not working on PREEMPT_RT because local_bh_disable() synchronizes the relevant section but the context remains fully preemptible. Use vfp_state_hold() for uninterrupted access. Reviewed-by: Ard Biesheuvel Signed-off-by: Sebastian Andrzej Siewior Signed-off-by: Russell King (Oracle)

ARM: 9424/1: vfp: Use vfp_state_hold() in vfp_sync_hwstate().

2024-11-12T16:41:47+00:00

vfp_sync_hwstate() uses preempt_disable() followed by local_bh_disable() to ensure that it won't get interrupted while checking the VFP state. This harms PREEMPT_RT because softirq handling can get preempted and local_bh_disable() synchronizes the related section with a sleeping lock which does not work with disabled preemption. Use the vfp_state_hold() to synchronize the access. Reviewed-by: Ard Biesheuvel Signed-off-by: Sebastian Andrzej Siewior Signed-off-by: Russell King (Oracle)

ARM: 9423/1: vfp: Provide vfp_state_hold() for VFP locking.

2024-11-12T16:41:46+00:00

kernel_neon_begin() uses local_bh_disable() to ensure exclusive access to the VFP unit. This is broken on PREEMPT_RT because a BH disabled section remains preemptible on PREEMPT_RT. Introduce vfp_state_hold() which uses local_bh_disable() and preempt_disable() on PREEMPT_RT. Since softirqs are processed always in thread context, disabling preemption is enough to ensure that the current context won't get interrupted by something that is using the VFP. Use it in kernel_neon_begin(). Reviewed-by: Ard Biesheuvel Signed-off-by: Sebastian Andrzej Siewior Signed-off-by: Russell King (Oracle)

ARM: 9410/1: vfp: Use asm volatile in fmrx/fmxr macros

2024-08-20T10:18:47+00:00

Floating point instructions in userspace can crash some arm kernels built with clang/LLD 17.0.6: BUG: unsupported FP instruction in kernel mode FPEXC == 0xc0000780 Internal error: Oops - undefined instruction: 0 [#1] ARM CPU: 0 PID: 196 Comm: vfp-reproducer Not tainted 6.10.0 #1 Hardware name: BCM2835 PC is at vfp_support_entry+0xc8/0x2cc LR is at do_undefinstr+0xa8/0x250 pc : [] lr : [] psr: a0000013 sp : dc8d1f68 ip : 60000013 fp : bedea19c r10: ec532b17 r9 : 00000010 r8 : 0044766c r7 : c0000780 r6 : ec532b17 r5 : c1c13800 r4 : dc8d1fb0 r3 : c10072c4 r2 : c0101c88 r1 : ec532b17 r0 : 0044766c Flags: NzCv IRQs on FIQs on Mode SVC_32 ISA ARM Segment none Control: 00c5387d Table: 0251c008 DAC: 00000051 Register r0 information: non-paged memory Register r1 information: vmalloc memory Register r2 information: non-slab/vmalloc memory Register r3 information: non-slab/vmalloc memory Register r4 information: 2-page vmalloc region Register r5 information: slab kmalloc-cg-2k Register r6 information: vmalloc memory Register r7 information: non-slab/vmalloc memory Register r8 information: non-paged memory Register r9 information: zero-size pointer Register r10 information: vmalloc memory Register r11 information: non-paged memory Register r12 information: non-paged memory Process vfp-reproducer (pid: 196, stack limit = 0x61aaaf8b) Stack: (0xdc8d1f68 to 0xdc8d2000) 1f60: 0000081f b6f69300 0000000f c10073f4 c10072c4 dc8d1fb0 1f80: ec532b17 0c532b17 0044766c b6f9ccd8 00000000 c010a80c 00447670 60000010 1fa0: ffffffff c1c13800 00c5387d c0100f10 b6f68af8 00448fc0 00000000 bedea188 1fc0: bedea314 00000001 00448ebc b6f9d000 00447608 b6f9ccd8 00000000 bedea19c 1fe0: bede9198 bedea188 b6e1061c 0044766c 60000010 ffffffff 00000000 00000000 Call trace: [] (vfp_support_entry) from [] (do_undefinstr+0xa8/0x250) [] (do_undefinstr) from [] (__und_usr+0x70/0x80) Exception stack(0xdc8d1fb0 to 0xdc8d1ff8) 1fa0: b6f68af8 00448fc0 00000000 bedea188 1fc0: bedea314 00000001 00448ebc b6f9d000 00447608 b6f9ccd8 00000000 bedea19c 1fe0: bede9198 bedea188 b6e1061c 0044766c 60000010 ffffffff Code: 0a000061 e3877202 e594003c e3a09010 (eef16a10) ---[ end trace 0000000000000000 ]--- Kernel panic - not syncing: Fatal exception in interrupt ---[ end Kernel panic - not syncing: Fatal exception in interrupt ]--- This is a minimal userspace reproducer on a Raspberry Pi Zero W: #include #include int main(void) { double v = 1.0; printf("%fn", NAN + *(volatile double *)&v); return 0; } Another way to consistently trigger the oops is: calvin@raspberry-pi-zero-w ~$ python -c "import json" The bug reproduces only when the kernel is built with DYNAMIC_DEBUG=n, because the pr_debug() calls act as barriers even when not activated. This is the output from the same kernel source built with the same compiler and DYNAMIC_DEBUG=y, where the userspace reproducer works as expected: VFP: bounce: trigger ec532b17 fpexc c0000780 VFP: emulate: INST=0xee377b06 SCR=0x00000000 VFP: bounce: trigger eef1fa10 fpexc c0000780 VFP: emulate: INST=0xeeb40b40 SCR=0x00000000 VFP: raising exceptions 30000000 calvin@raspberry-pi-zero-w ~$ ./vfp-reproducer nan Crudely grepping for vmsr/vmrs instructions in the otherwise nearly idential text for vfp_support_entry() makes the problem obvious: vmlinux.llvm.good [0xc0101cb8] <+48>: vmrs r7, fpexc vmlinux.llvm.good [0xc0101cd8] <+80>: vmsr fpexc, r0 vmlinux.llvm.good [0xc0101d20] <+152>: vmsr fpexc, r7 vmlinux.llvm.good [0xc0101d38] <+176>: vmrs r4, fpexc vmlinux.llvm.good [0xc0101d6c] <+228>: vmrs r0, fpscr vmlinux.llvm.good [0xc0101dc4] <+316>: vmsr fpexc, r0 vmlinux.llvm.good [0xc0101dc8] <+320>: vmrs r0, fpsid vmlinux.llvm.good [0xc0101dcc] <+324>: vmrs r6, fpscr vmlinux.llvm.good [0xc0101e10] <+392>: vmrs r10, fpinst vmlinux.llvm.good [0xc0101eb8] <+560>: vmrs r10, fpinst2 vmlinux.llvm.bad [0xc0101cb8] <+48>: vmrs r7, fpexc vmlinux.llvm.bad [0xc0101cd8] <+80>: vmsr fpexc, r0 vmlinux.llvm.bad [0xc0101d20] <+152>: vmsr fpexc, r7 vmlinux.llvm.bad [0xc0101d30] <+168>: vmrs r0, fpscr vmlinux.llvm.bad [0xc0101d50] <+200>: vmrs r6, fpscr <== BOOM! vmlinux.llvm.bad [0xc0101d6c] <+228>: vmsr fpexc, r0 vmlinux.llvm.bad [0xc0101d70] <+232>: vmrs r0, fpsid vmlinux.llvm.bad [0xc0101da4] <+284>: vmrs r10, fpinst vmlinux.llvm.bad [0xc0101df8] <+368>: vmrs r4, fpexc vmlinux.llvm.bad [0xc0101e5c] <+468>: vmrs r10, fpinst2 I think LLVM's reordering is valid as the code is currently written: the compiler doesn't know the instructions have side effects in hardware. Fix by using "asm volatile" in fmxr() and fmrx(), so they cannot be reordered with respect to each other. The original compiler now produces working kernels on my hardware with DYNAMIC_DEBUG=n. This is the relevant piece of the diff of the vfp_support_entry() text, from the original oopsing kernel to a working kernel with this patch: vmrs r0, fpscr tst r0, #4096 bne 0xc0101d48 tst r0, #458752 beq 0xc0101ecc orr r7, r7, #536870912 ldr r0, [r4, #0x3c] mov r9, #16 -vmrs r6, fpscr orr r9, r9, #251658240 add r0, r0, #4 str r0, [r4, #0x3c] mvn r0, #159 sub r0, r0, #-1207959552 and r0, r7, r0 vmsr fpexc, r0 vmrs r0, fpsid +vmrs r6, fpscr and r0, r0, #983040 cmp r0, #65536 bne 0xc0101d88 Fixes: 4708fb041346 ("ARM: vfp: Reimplement VFP exception entry in C code") Signed-off-by: Calvin Owens Signed-off-by: Russell King (Oracle)

ARM: 9327/1: vfp: Add missing VFP instructions to neon_support_hook

2023-12-05T11:40:27+00:00

Add the missing "Unconditional Advanced SIMD and floating-point instructions" in [1] to the VFP undef hook. This commit addresses the issue reported in [2], where executing the vudot instruction on a platform with FEAT_DotProd support resulted in an undefined instruction error. Link: https://developer.arm.com/documentation/ddi0597/2023-06/?lang=en [1] Link: https://lore.kernel.org/lkml/20230920083907.30479-1-mark-pk.tsai@mediatek.com/ [2] Signed-off-by: Mark-PK Tsai Tested-by: Xuewen Yan Signed-off-by: Russell King (Oracle)

Merge branch 'devel-stable' into for-next

2023-08-14T11:18:06+00:00