kernel/linux.git/arch/arm64/include/asm/pointer_auth.h, branch v6.1.168

arm64: move preemption disablement to prctl handlers

2021-07-28T17:33:49+00:00

In the next patch, we will start reading sctlr_user from mte_update_sctlr_user and subsequently writing a new value based on the task's TCF setting and potentially the per-CPU TCF preference. This means that we need to be careful to disable preemption around any code sequences that read from sctlr_user and subsequently write to sctlr_user and/or SCTLR_EL1, so that we don't end up writing a stale value (based on the previous CPU's TCF preference) to either of them. We currently have four such sequences, in the prctl handlers for PR_SET_TAGGED_ADDR_CTRL and PR_PAC_SET_ENABLED_KEYS, as well as in the task initialization code that resets the prctl settings. Change the prctl handlers to disable preemption in the handlers themselves rather than the functions that they call, and change the task initialization code to call the respective prctl handlers instead of setting sctlr_user directly. As a result of this change, we no longer need the helper function set_task_sctlr_el1, nor does its behavior make sense any more, so remove it. Signed-off-by: Peter Collingbourne Link: https://linux-review.googlesource.com/id/Ic0e8a0c00bb47d786c1e8011df0b7fe99bee4bb5 Link: https://lore.kernel.org/r/20210727205300.2554659-4-pcc@google.com Acked-by: Will Deacon Signed-off-by: Catalin Marinas

arm64: Conditionally configure PTR_AUTH key of the kernel.

2021-06-15T10:32:31+00:00

If the kernel is not compiled with CONFIG_ARM64_PTR_AUTH_KERNEL=y, then no PACI/AUTI instructions are expected while the kernel is running so the kernel's key will not be used. Write of a system registers is expensive therefore avoid if not required. Signed-off-by: Daniel Kiss Reviewed-by: Catalin Marinas Link: https://lore.kernel.org/r/20210613092632.93591-3-daniel.kiss@arm.com Signed-off-by: Will Deacon

arm64: pac: Optimize kernel entry/exit key installation code paths

2021-04-13T16:31:44+00:00

The kernel does not use any keys besides IA so we don't need to install IB/DA/DB/GA on kernel exit if we arrange to install them on task switch instead, which we can expect to happen an order of magnitude less often. Furthermore we can avoid installing the user IA in the case where the user task has IA disabled and just leave the kernel IA installed. This also lets us avoid needing to install IA on kernel entry. On an Apple M1 under a hypervisor, the overhead of kernel entry/exit has been measured to be reduced by 15.6ns in the case where IA is enabled, and 31.9ns in the case where IA is disabled. Signed-off-by: Peter Collingbourne Link: https://linux-review.googlesource.com/id/Ieddf6b580d23c9e0bed45a822dabe72d2ffc9a8e Link: https://lore.kernel.org/r/2d653d055f38f779937f2b92f8ddd5cf9e4af4f4.1616123271.git.pcc@google.com Signed-off-by: Catalin Marinas

arm64: Introduce prctl(PR_PAC_{SET,GET}_ENABLED_KEYS)

2021-04-13T16:31:44+00:00

This change introduces a prctl that allows the user program to control which PAC keys are enabled in a particular task. The main reason why this is useful is to enable a userspace ABI that uses PAC to sign and authenticate function pointers and other pointers exposed outside of the function, while still allowing binaries conforming to the ABI to interoperate with legacy binaries that do not sign or authenticate pointers. The idea is that a dynamic loader or early startup code would issue this prctl very early after establishing that a process may load legacy binaries, but before executing any PAC instructions. This change adds a small amount of overhead to kernel entry and exit due to additional required instruction sequences. On a DragonBoard 845c (Cortex-A75) with the powersave governor, the overhead of similar instruction sequences was measured as 4.9ns when simulating the common case where IA is left enabled, or 43.7ns when simulating the uncommon case where IA is disabled. These numbers can be seen as the worst case scenario, since in more realistic scenarios a better performing governor would be used and a newer chip would be used that would support PAC unlike Cortex-A75 and would be expected to be faster than Cortex-A75. On an Apple M1 under a hypervisor, the overhead of the entry/exit instruction sequences introduced by this patch was measured as 0.3ns in the case where IA is left enabled, and 33.0ns in the case where IA is disabled. Signed-off-by: Peter Collingbourne Reviewed-by: Dave Martin Link: https://linux-review.googlesource.com/id/Ibc41a5e6a76b275efbaa126b31119dc197b927a5 Link: https://lore.kernel.org/r/d6609065f8f40397a4124654eb68c9f490b4d477.1616123271.git.pcc@google.com Signed-off-by: Catalin Marinas

arm64: Defer enabling pointer authentication on boot core

2021-02-09T13:50:57+00:00

Defer enabling pointer authentication on boot core until after its required to be enabled by cpufeature framework. This will help in controlling the feature dynamically with a boot parameter. Signed-off-by: Ajay Patil Signed-off-by: Prasad Sodagudi Signed-off-by: Srinivas Ramana Signed-off-by: Marc Zyngier Link: https://lore.kernel.org/r/1610152163-16554-2-git-send-email-sramana@codeaurora.org Reviewed-by: Catalin Marinas Acked-by: David Brazdil Link: https://lore.kernel.org/r/20210208095732.3267263-22-maz@kernel.org Signed-off-by: Will Deacon

arm64: sync kernel APIAKey when installing

2020-04-21T14:52:56+00:00

A direct write to a APxxKey_EL1 register requires a context synchronization event to ensure that indirect reads made by subsequent instructions (e.g. AUTIASP, PACIASP) observe the new value. When we initialize the boot task's APIAKey in boot_init_stack_canary() via ptrauth_keys_switch_kernel() we miss the necessary ISB, and so there is a window where instructions are not guaranteed to use the new APIAKey value. This has been observed to result in boot-time crashes where PACIASP and AUTIASP within a function used a mixture of the old and new key values. Fix this by having ptrauth_keys_switch_kernel() synchronize the new key value with an ISB. At the same time, __ptrauth_key_install() is renamed to __ptrauth_key_install_nosync() so that it is obvious that this performs no synchronization itself. Fixes: 28321582334c261c ("arm64: initialize ptrauth keys for kernel booting task") Signed-off-by: Mark Rutland Reported-by: Will Deacon Cc: Amit Daniel Kachhap Cc: Marc Zyngier Signed-off-by: Catalin Marinas Tested-by: Will Deacon

arm64: mask PAC bits of __builtin_return_address

2020-03-18T09:50:20+00:00

Functions like vmap() record how much memory has been allocated by their callers, and callers are identified using __builtin_return_address(). Once the kernel is using pointer-auth the return address will be signed. This means it will not match any kernel symbol, and will vary between threads even for the same caller. The output of /proc/vmallocinfo in this case may look like, 0x(____ptrval____)-0x(____ptrval____) 20480 0x86e28000100e7c60 pages=4 vmalloc N0=4 0x(____ptrval____)-0x(____ptrval____) 20480 0x86e28000100e7c60 pages=4 vmalloc N0=4 0x(____ptrval____)-0x(____ptrval____) 20480 0xc5c78000100e7c60 pages=4 vmalloc N0=4 The above three 64bit values should be the same symbol name and not different LR values. Use the pre-processor to add logic to clear the PAC to __builtin_return_address() callers. This patch adds a new file asm/compiler.h and is transitively included via include/compiler_types.h on the compiler command line so it is guaranteed to be loaded and the users of this macro will not find a wrong version. Helper macros ptrauth_kernel_pac_mask/ptrauth_clear_pac are created for this purpose and added in this file. Existing macro ptrauth_user_pac_mask moved from asm/pointer_auth.h. Signed-off-by: Amit Daniel Kachhap Reviewed-by: James Morse Signed-off-by: Catalin Marinas

arm64: initialize ptrauth keys for kernel booting task

2020-03-18T09:50:20+00:00

This patch uses the existing boot_init_stack_canary arch function to initialize the ptrauth keys for the booting task in the primary core. The requirement here is that it should be always inline and the caller must never return. As pointer authentication too detects a subset of stack corruption so it makes sense to place this code here. Both pointer authentication and stack canary codes are protected by their respective config option. Suggested-by: Ard Biesheuvel Signed-off-by: Amit Daniel Kachhap Reviewed-by: Vincenzo Frascino Reviewed-by: Catalin Marinas Signed-off-by: Catalin Marinas

arm64: initialize and switch ptrauth kernel keys

2020-03-18T09:50:20+00:00

Set up keys to use pointer authentication within the kernel. The kernel will be compiled with APIAKey instructions, the other keys are currently unused. Each task is given its own APIAKey, which is initialized during fork. The key is changed during context switch and on kernel entry from EL0. The keys for idle threads need to be set before calling any C functions, because it is not possible to enter and exit a function with different keys. Reviewed-by: Kees Cook Reviewed-by: Catalin Marinas Reviewed-by: Vincenzo Frascino Signed-off-by: Kristina Martsenko [Amit: Modified secondary cores key structure, comments] Signed-off-by: Amit Daniel Kachhap Signed-off-by: Catalin Marinas

arm64: install user ptrauth keys at kernel exit time

2020-03-18T09:50:19+00:00

As we're going to enable pointer auth within the kernel and use a different APIAKey for the kernel itself, so move the user APIAKey switch to EL0 exception return. The other 4 keys could remain switched during task switch, but are also moved to keep things consistent. Reviewed-by: Kees Cook Reviewed-by: James Morse Reviewed-by: Vincenzo Frascino Signed-off-by: Kristina Martsenko [Amit: commit msg, re-positioned the patch, comments] Signed-off-by: Amit Daniel Kachhap Signed-off-by: Catalin Marinas