diff options
| author | Denys Vlasenko <dvlasenk@redhat.com> | 2015-03-09 21:39:21 +0300 |
|---|---|---|
| committer | Ingo Molnar <mingo@kernel.org> | 2015-03-10 15:56:10 +0300 |
| commit | 29722cd4ef666705b2eda1c3ba44435488e509eb (patch) | |
| tree | 27fcc3f8f267ae426ce9e383422828cea79c6f59 /arch/x86/include/asm/calling.h | |
| parent | 3e1aa7cb59aff4b245b45e326fcdba1bf7f105c6 (diff) | |
| download | linux-29722cd4ef666705b2eda1c3ba44435488e509eb.tar.xz | |
x86/asm/entry/64: Save R11 into pt_regs->flags on SYSCALL64 fastpath
Before this patch, R11 was saved in pt_regs->r11.
Which looks natural, but requires messy shuffling to/from iret
frame whenever ptrace or e.g. sys_iopl() wants to modify flags -
because that's how this register is used by SYSCALL/SYSRET.
This patch saves R11 in pt_regs->flags, and uses that value for
the SYSRET64 instruction. Shuffling is eliminated.
FIXUP/RESTORE_TOP_OF_STACK are simplified.
stub_iopl is no longer needed: pt_regs->flags needs no fixing up.
Testing shows that syscall fast path is ~54.3 ns before
and after the patch (on 2.7 GHz Sandy Bridge CPU).
Signed-off-by: Denys Vlasenko <dvlasenk@redhat.com>
Cc: Alexei Starovoitov <ast@plumgrid.com>
Cc: Andy Lutomirski <luto@amacapital.net>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: H. Peter Anvin <hpa@zytor.com>
Cc: Kees Cook <keescook@chromium.org>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Oleg Nesterov <oleg@redhat.com>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Will Drewry <wad@chromium.org>
Link: http://lkml.kernel.org/r/1425926364-9526-2-git-send-email-dvlasenk@redhat.com
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Diffstat (limited to 'arch/x86/include/asm/calling.h')
| -rw-r--r-- | arch/x86/include/asm/calling.h | 20 |
1 files changed, 14 insertions, 6 deletions
diff --git a/arch/x86/include/asm/calling.h b/arch/x86/include/asm/calling.h index f1a962ff7ddf..4b5f7bf2b780 100644 --- a/arch/x86/include/asm/calling.h +++ b/arch/x86/include/asm/calling.h @@ -95,9 +95,11 @@ For 32-bit we have the following conventions - kernel is built with CFI_ADJUST_CFA_OFFSET 15*8+\addskip .endm - .macro SAVE_C_REGS_HELPER offset=0 rax=1 rcx=1 r8plus=1 - .if \r8plus + .macro SAVE_C_REGS_HELPER offset=0 rax=1 rcx=1 r8910=1 r11=1 + .if \r11 movq_cfi r11, 6*8+\offset + .endif + .if \r8910 movq_cfi r10, 7*8+\offset movq_cfi r9, 8*8+\offset movq_cfi r8, 9*8+\offset @@ -113,16 +115,19 @@ For 32-bit we have the following conventions - kernel is built with movq_cfi rdi, 14*8+\offset .endm .macro SAVE_C_REGS offset=0 - SAVE_C_REGS_HELPER \offset, 1, 1, 1 + SAVE_C_REGS_HELPER \offset, 1, 1, 1, 1 .endm .macro SAVE_C_REGS_EXCEPT_RAX_RCX offset=0 - SAVE_C_REGS_HELPER \offset, 0, 0, 1 + SAVE_C_REGS_HELPER \offset, 0, 0, 1, 1 .endm .macro SAVE_C_REGS_EXCEPT_R891011 - SAVE_C_REGS_HELPER 0, 1, 1, 0 + SAVE_C_REGS_HELPER 0, 1, 1, 0, 0 .endm .macro SAVE_C_REGS_EXCEPT_RCX_R891011 - SAVE_C_REGS_HELPER 0, 1, 0, 0 + SAVE_C_REGS_HELPER 0, 1, 0, 0, 0 + .endm + .macro SAVE_C_REGS_EXCEPT_RAX_RCX_R11 + SAVE_C_REGS_HELPER 0, 0, 0, 1, 0 .endm .macro SAVE_EXTRA_REGS offset=0 @@ -179,6 +184,9 @@ For 32-bit we have the following conventions - kernel is built with .macro RESTORE_C_REGS_EXCEPT_R11 RESTORE_C_REGS_HELPER 1,1,0,1,1 .endm + .macro RESTORE_C_REGS_EXCEPT_RCX_R11 + RESTORE_C_REGS_HELPER 1,0,0,1,1 + .endm .macro RESTORE_RSI_RDI RESTORE_C_REGS_HELPER 0,0,0,0,0 .endm |