diff options
| author | Atish Patra <atish.patra@wdc.com> | 2019-08-22 10:51:50 +0300 |
|---|---|---|
| committer | Paul Walmsley <paul.walmsley@sifive.com> | 2019-10-29 21:32:18 +0300 |
| commit | 31738ede9b339c90216b8fd10da0b1567a041f06 (patch) | |
| tree | 7dc501804afb05a31fbf29f90091a33cf75b14af | |
| parent | 6384423f49c804dbca02add1945e0ccf922cb522 (diff) | |
| download | linux-31738ede9b339c90216b8fd10da0b1567a041f06.tar.xz | |
RISC-V: Issue a local tlbflush if possible.
In RISC-V, tlb flush happens via SBI which is expensive. If the local
cpu is the only cpu in cpumask, there is no need to invoke a SBI call.
Just do a local flush and return.
Signed-off-by: Atish Patra <atish.patra@wdc.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com>
| -rw-r--r-- | arch/riscv/mm/tlbflush.c | 19 |
1 files changed, 17 insertions, 2 deletions
diff --git a/arch/riscv/mm/tlbflush.c b/arch/riscv/mm/tlbflush.c index 9ecaf73cd051..3531d4647323 100644 --- a/arch/riscv/mm/tlbflush.c +++ b/arch/riscv/mm/tlbflush.c @@ -2,6 +2,7 @@ #include <linux/mm.h> #include <linux/smp.h> +#include <linux/sched.h> #include <asm/sbi.h> void flush_tlb_all(void) @@ -9,16 +10,30 @@ void flush_tlb_all(void) sbi_remote_sfence_vma(NULL, 0, -1); } +/* + * This function must not be called with cmask being null. + * Kernel may panic if cmask is NULL. + */ static void __sbi_tlb_flush_range(struct cpumask *cmask, unsigned long start, unsigned long size) { struct cpumask hmask; + unsigned int cpuid; if (cpumask_empty(cmask)) return; - riscv_cpuid_to_hartid_mask(cmask, &hmask); - sbi_remote_sfence_vma(hmask.bits, start, size); + cpuid = get_cpu(); + + if (cpumask_any_but(cmask, cpuid) >= nr_cpu_ids) { + /* local cpu is the only cpu present in cpumask */ + local_flush_tlb_all(); + } else { + riscv_cpuid_to_hartid_mask(cmask, &hmask); + sbi_remote_sfence_vma(cpumask_bits(&hmask), start, size); + } + + put_cpu(); } void flush_tlb_mm(struct mm_struct *mm) |