kernel/linux.git/arch/s390/include/asm/asm-extable.h, branch linux-7.0.y

s390/uaccess: Shorten raw_copy_from_user() / raw_copy_to_user() inline assemblies

2025-03-04T16:18:03+00:00

Add specific exception handler for copy_to_user() / copy_from_user() mvcos fault handling, which allows to shorten the inline assemblies to three instructions. On fault the exception handler adjusts the length used by the mvcos instruction in a way that the instruction completes with condition code zero, indicating the number of bytes copied with the input/output operand 'size'. This allows to calculate and return the number of bytes not copied, if any, like required. Loop and return value handling is changed to C so that the compiler may optimize the code. Signed-off-by: Heiko Carstens Signed-off-by: Vasily Gorbik

s390/uaccess: Replace EX_TABLE_UA_LOAD_MEM exception handling

2025-01-26T16:24:06+00:00

Remove EX_TABLE_UA_LOAD_MEM exception handling and replace it with EX_TABLE_UA_FAULT. Open code the return code check, and also open code setting of the destination to zero in case of an error. In almost all cases the compiler is able to optimize the open coded checks away, since all users of get_users() must check the return code, and are not supposed to use the result in case of an error. In addition this allows to change the get_user() inline assembly so that the "Q" constraint can be used for the destination, instead of only an "a" constraint. This generates slightly better code. Acked-by: Alexander Gordeev Signed-off-by: Heiko Carstens Signed-off-by: Alexander Gordeev

s390/extable: Rename EX_TABLE_UA_STORE to EX_TABLE_UA_FAULT

2025-01-26T16:24:05+00:00

Rename EX_TABLE_UA_STORE to a more generic EX_TABLE_UA_FAULT name. This allows to use the extable type also for uaccess inline assemblies which read from userspace, without causing confusion. Reviewed-by: Alexander Gordeev Signed-off-by: Heiko Carstens Signed-off-by: Alexander Gordeev

s390/fpu: Add fpc exception handler / remove fixup section again

2025-01-26T16:24:04+00:00

The fixup section was added again by mistake when test_fp_ctl() was removed. The reason for the removal of the fixup section is described in commit 484a8ed8b7d1 ("s390/extable: add dedicated uaccess handler"). Remove it again for the same reason. Add an exception handler which handles exceptions when the floating point control register is attempted to be set to invalid values. The exception handler sets the floating point control register to zero and continues execution at the specified address. The new sfpc inline assembly is open-coded to make back porting a bit easier. Fixes: 702644249d3e ("s390/fpu: get rid of test_fp_ctl()") Cc: stable@vger.kernel.org Reviewed-by: Alexander Gordeev Signed-off-by: Heiko Carstens Signed-off-by: Alexander Gordeev

s390: add support for DCACHE_WORD_ACCESS

2023-10-16T11:04:09+00:00

Implement load_unaligned_zeropad() and enable DCACHE_WORD_ACCESS to speed up string operations in fs/dcache.c and fs/namei.c. Reviewed-by: Sven Schnelle Signed-off-by: Heiko Carstens Signed-off-by: Vasily Gorbik

s390/extable: reduce number of extable macros

2023-10-16T11:04:09+00:00

Get rid of __EX_TABLE() macro, rename __EX_TABLE_UA() to __EX_TABLE() and convert users of old __EX_TABLE() macro so they pass more parameters to the changed __EX_TABLE() semantics. Reviewed-by: Sven Schnelle Signed-off-by: Heiko Carstens Signed-off-by: Vasily Gorbik

s390: consistently use .balign instead of .align

2023-06-28T11:57:09+00:00

The .align directive has inconsistent behavior across architectures. Use .balign instead everywhere. This is a no-op for s390, but with this there is no mix in using .align and .balign anymore. Future code is supposed to use only .balign. Reviewed-by: Alexander Gordeev Signed-off-by: Heiko Carstens Signed-off-by: Alexander Gordeev

s390/extable: add EX_TABLE_UA_LOAD_REGPAIR() macro

2022-11-21T12:36:15+00:00

Add new exception table type which is able to handle register pairs. If an exception is recognized on such an instruction the specified register pair will be zeroed, and the specified error register will be modified so it contains -EFAULT, similar to the existing EX_TABLE_UA_LOAD_REG() macro. Link: https://lore.kernel.org/r/Y2J8RSW2khWLgpPo@osiris Signed-off-by: Heiko Carstens

s390/uaccess: use exception handler to zero result on get_user() failure

2022-06-01T10:03:17+00:00

Historically the uaccess code pre-initializes the result of get_user() (and now also __get_kernel_nofault()) to zero and uses the result as input parameter for inline assemblies. This is different to what most, if not all, other architectures are doing, which set the result to zero within the exception handler in case of a fault. Use the new extable mechanism and handle zeroing of the result within the exception handler in case of a fault. Signed-off-by: Heiko Carstens

s390/extable: prefer local labels in .set directives

2022-05-06T18:45:14+00:00

Use local labels in .set directives to avoid potential compile errors with LTO + clang. See commit 334865b2915c ("x86/extable: Prefer local labels in .set directives") for further details. Since s390 doesn't support LTO currently this doesn't fix a real bug for now, but helps to avoid problems as soon as required pieces have been added to llvm. Signed-off-by: Heiko Carstens