BMC/Intel-BMC/linux.git/arch/mips/mm/cache.c, branch dev-4.6

MIPS: Sync icache & dcache in set_pte_at

2016-06-08T01:23:33+00:00

commit 37d22a0d798b5c938b277d32cfd86dc231381342 upstream. It's possible for pages to become visible prior to update_mmu_cache running if a thread within the same address space preempts the current thread or runs simultaneously on another CPU. That is, the following scenario is possible: CPU0 CPU1 write to page flush_dcache_page flush_icache_page set_pte_at map page update_mmu_cache If CPU1 maps the page in between CPU0's set_pte_at, which marks it valid & visible, and update_mmu_cache where the dcache flush occurs then CPU1s icache will fill from stale data (unless it fills from the dcache, in which case all is good, but most MIPS CPUs don't have this property). Commit 4d46a67a3eb8 ("MIPS: Fix race condition in lazy cache flushing.") attempted to fix that by performing the dcache flush in flush_icache_page such that it occurs before the set_pte_at call makes the page visible. However it has the problem that not all code that writes to pages exposed to userland call flush_icache_page. There are many callers of set_pte_at under mm/ and only 2 of them do call flush_icache_page. Thus the race window between a page becoming visible & being coherent between the icache & dcache remains open in some cases. To illustrate some of the cases, a WARN was added to __update_cache with this patch applied that triggered in cases where a page about to be flushed from the dcache was not the last page provided to flush_icache_page. That is, backtraces were obtained for cases in which the race window is left open without this patch. The 2 standout examples follow. When forking a process: [ 15.271842] [<80417630>] __update_cache+0xcc/0x188 [ 15.277274] [<80530394>] copy_page_range+0x56c/0x6ac [ 15.282861] [<8042936c>] copy_process.part.54+0xd40/0x17ac [ 15.289028] [<80429f80>] do_fork+0xe4/0x420 [ 15.293747] [<80413808>] handle_sys+0x128/0x14c When exec'ing an ELF binary: [ 14.445964] [<80417630>] __update_cache+0xcc/0x188 [ 14.451369] [<80538d88>] move_page_tables+0x414/0x498 [ 14.457075] [<8055d848>] setup_arg_pages+0x220/0x318 [ 14.462685] [<805b0f38>] load_elf_binary+0x530/0x12a0 [ 14.468374] [<8055ec3c>] search_binary_handler+0xbc/0x214 [ 14.474444] [<8055f6c0>] do_execveat_common+0x43c/0x67c [ 14.480324] [<8055f938>] do_execve+0x38/0x44 [ 14.485137] [<80413808>] handle_sys+0x128/0x14c These code paths write into a page, call flush_dcache_page then call set_pte_at without flush_icache_page inbetween. The end result is that the icache can become corrupted & userland processes may execute unexpected or invalid code, typically resulting in a reserved instruction exception, a trap or a segfault. Fix this race condition fully by performing any cache maintenance required to keep the icache & dcache in sync in set_pte_at, before the page is made valid. This has the added bonus of ensuring the cache maintenance always happens in one location, rather than being duplicated in flush_icache_page & update_mmu_cache. It also matches the way other architectures solve the same problem (see arm, ia64 & powerpc). Signed-off-by: Paul Burton Reported-by: Ionela Voinescu Cc: Lars Persson Fixes: 4d46a67a3eb8 ("MIPS: Fix race condition in lazy cache flushing.") Cc: Steven J. Hill Cc: David Daney Cc: Huacai Chen Cc: Aneesh Kumar K.V Cc: Andrew Morton Cc: Jerome Marchand Cc: Kirill A. Shutemov Cc: linux-mips@linux-mips.org Cc: linux-kernel@vger.kernel.org Patchwork: https://patchwork.linux-mips.org/patch/12722/ Signed-off-by: Ralf Baechle Signed-off-by: Greg Kroah-Hartman

MIPS: Handle highmem pages in __update_cache

2016-06-08T01:23:33+00:00

commit f4281bba818105c7c91799abe40bc05c0dbdaa25 upstream. The following patch will expose __update_cache to highmem pages. Handle them by mapping them in for the duration of the cache maintenance, just like in __flush_dcache_page. The code for that isn't shared because we need the page address in __update_cache so sharing became messy. Given that the entirity is an extra 5 lines, just duplicate it. Signed-off-by: Paul Burton Cc: Lars Persson Cc: Andrew Morton Cc: Jerome Marchand Cc: Kirill A. Shutemov Cc: linux-mips@linux-mips.org Cc: linux-kernel@vger.kernel.org Patchwork: https://patchwork.linux-mips.org/patch/12721/ Signed-off-by: Ralf Baechle Signed-off-by: Greg Kroah-Hartman

MIPS: Flush highmem pages in __flush_dcache_page

2016-06-08T01:23:33+00:00

commit 234859e49a15323cf1b2331bdde7f658c4cb45fb upstream. When flush_dcache_page is called on an executable page, that page is about to be provided to userland & we can presume that the icache contains no valid entries for its address range. However if the icache does not fill from the dcache then we cannot presume that the pages content has been written back as far as the memories that the dcache will fill from (ie. L2 or further out). This was being done for lowmem pages, but not for highmem which can lead to icache corruption. Fix this by mapping highmem pages & flushing their content from the dcache in __flush_dcache_page before providing the page to userland, just as is done for lowmem pages. Signed-off-by: Paul Burton Cc: Lars Persson Cc: Andrew Morton Cc: Kirill A. Shutemov Cc: linux-mips@linux-mips.org Cc: linux-kernel@vger.kernel.org Patchwork: https://patchwork.linux-mips.org/patch/12720/ Signed-off-by: Ralf Baechle Signed-off-by: Greg Kroah-Hartman

mm: differentiate page_mapped() from page_mapcount() for compound pages

2016-01-16T01:56:32+00:00

Let's define page_mapped() to be true for compound pages if any sub-pages of the compound page is mapped (with PMD or PTE). On other hand page_mapcount() return mapcount for this particular small page. This will make cases like page_get_anon_vma() behave correctly once we allow huge pages to be mapped with PTE. Most users outside core-mm should use page_mapcount() instead of page_mapped(). Signed-off-by: Kirill A. Shutemov Tested-by: Sasha Levin Tested-by: Aneesh Kumar K.V Acked-by: Jerome Marchand Cc: Vlastimil Babka Cc: Andrea Arcangeli Cc: Hugh Dickins Cc: Dave Hansen Cc: Mel Gorman Cc: Rik van Riel Cc: Naoya Horiguchi Cc: Steve Capper Cc: Johannes Weiner Cc: Michal Hocko Cc: Christoph Lameter Cc: David Rientjes Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds

MIPS: Partially disable RIXI support.

2015-08-03T07:25:11+00:00

Execution of break instruction, trap instructions, emulation of unaligned loads or floating point instructions - anything that tries to read the instruction's opcode from userspace - needs read access to a page. RIXI (Read Inhibit / Execute Inhibit) support however allows the creation of pags that are executable but not readable. On such a mapping the attempted load of the opcode by the kernel is going to cause an endless loop of page faults. The quick workaround for this is to disable the combinations that the kernel currently isn't able to handle which are executable mappings. Signed-off-by: Ralf Baechle

MIPS: Fix race condition in lazy cache flushing.

2015-03-25T12:48:00+00:00

The lazy cache flushing implemented in the MIPS kernel suffers from a race condition that is exposed by do_set_pte() in mm/memory.c. A pre-condition is a file-system that writes to the page from the CPU in its readpage method and then calls flush_dcache_page(). One example is ubifs. Another pre-condition is that the dcache flush is postponed in __flush_dcache_page(). Upon a page fault for an executable mapping not existing in the page-cache, the following will happen: 1. Write to the page 2. flush_dcache_page 3. flush_icache_page 4. set_pte_at 5. update_mmu_cache (commits the flush of a dcache-dirty page) Between steps 4 and 5 another thread can hit the same page and it will encounter a valid pte. Because the data still is in the L1 dcache the CPU will fetch stale data from L2 into the icache and execute garbage. This fix moves the commit of the cache flush to step 3 to close the race window. It also reduces the amount of flushes on non-executable mappings because we never enter __flush_dcache_page() for non-aliasing CPUs. Regressions can occur in drivers that mistakenly relies on the flush_dcache_page() in get_user_pages() for DMA operations. [ralf@linux-mips.org: Folded in patch 9346 to fix highmem issue.] Signed-off-by: Lars Persson Cc: linux-mips@linux-mips.org Cc: paul.burton@imgtec.com Cc: linux-kernel@vger.kernel.org Patchwork: https://patchwork.linux-mips.org/patch/9346/ Patchwork: https://patchwork.linux-mips.org/patch/9738/ Signed-off-by: Ralf Baechle

Revert "MIPS: Remove race window in page fault handling"

2015-03-25T12:48:00+00:00

Revert commit 2a4a8b1e5d9d ("MIPS: Remove race window in page fault handling") because it increased the number of flushed dcache pages and became a performance problem for some workloads. Signed-off-by: Lars Persson Cc: linux-mips@linux-mips.org Cc: paul.burton@imgtec.com Cc: linux-kernel@vger.kernel.org Patchwork: https://patchwork.linux-mips.org/patch/9345/ Signed-off-by: Ralf Baechle

MIPS: Remove race window in page fault handling

2014-08-19T11:31:09+00:00

Multicore MIPSes without I/D hardware coherency suffered from a race condition in the page fault handler. The page table entry was published before any pending lazy D-cache flush was committed, hence it allowed execution of stale page cache data by other VPEs in the system. To make the cache handling safe we need to perform flushing already in the set_pte_at function. MIPSes without coherent I-caches can get a small increase in flushes due to the unavailability of the execute flag in set_pte_at. [ralf@linux-mips.org: outlining set_pte_at() saves a good k in a test build, so I moved its definition from pgtable.h to cache.c.] Signed-off-by: Lars Persson Cc: linux-mips@linux-mips.org Patchwork: https://patchwork.linux-mips.org/patch/7511/ Signed-off-by: Ralf Baechle

Merge tag 'for-linus' of git://git.kernel.org/pub/scm/virt/kvm/kvm into next

2014-06-04T15:47:12+00:00

Pull KVM updates from Paolo Bonzini: "At over 200 commits, covering almost all supported architectures, this was a pretty active cycle for KVM. Changes include: - a lot of s390 changes: optimizations, support for migration, GDB support and more - ARM changes are pretty small: support for the PSCI 0.2 hypercall interface on both the guest and the host (the latter acked by Catalin) - initial POWER8 and little-endian host support - support for running u-boot on embedded POWER targets - pretty large changes to MIPS too, completing the userspace interface and improving the handling of virtualized timer hardware - for x86, a larger set of changes is scheduled for 3.17. Still, we have a few emulator bugfixes and support for running nested fully-virtualized Xen guests (para-virtualized Xen guests have always worked). And some optimizations too. The only missing architecture here is ia64. It's not a coincidence that support for KVM on ia64 is scheduled for removal in 3.17" * tag 'for-linus' of git://git.kernel.org/pub/scm/virt/kvm/kvm: (203 commits) KVM: add missing cleanup_srcu_struct KVM: PPC: Book3S PR: Rework SLB switching code KVM: PPC: Book3S PR: Use SLB entry 0 KVM: PPC: Book3S HV: Fix machine check delivery to guest KVM: PPC: Book3S HV: Work around POWER8 performance monitor bugs KVM: PPC: Book3S HV: Make sure we don't miss dirty pages KVM: PPC: Book3S HV: Fix dirty map for hugepages KVM: PPC: Book3S HV: Put huge-page HPTEs in rmap chain for base address KVM: PPC: Book3S HV: Fix check for running inside guest in global_invalidates() KVM: PPC: Book3S: Move KVM_REG_PPC_WORT to an unused register number KVM: PPC: Book3S: Add ONE_REG register names that were missed KVM: PPC: Add CAP to indicate hcall fixes KVM: PPC: MPIC: Reset IRQ source private members KVM: PPC: Graciously fail broken LE hypercalls PPC: ePAPR: Fix hypercall on LE guest KVM: PPC: BOOK3S: Remove open coded make_dsisr in alignment handler KVM: PPC: BOOK3S: Always use the saved DAR value PPC: KVM: Make NX bit available with magic page KVM: PPC: Disable NX for old magic page using guests KVM: PPC: BOOK3S: HV: Add mixed page-size support for guest ...

MIPS: Export local_flush_icache_range for KVM

2014-05-30T10:59:28+00:00

Export the local_flush_icache_range function pointer for GPL modules so that it can be used by KVM for syncing the icache after binary translation of trapping instructions. Signed-off-by: James Hogan Cc: Paolo Bonzini Cc: Gleb Natapov Cc: kvm@vger.kernel.org Cc: linux-mips@linux-mips.org Cc: Sanjay Lal Acked-by: Ralf Baechle Signed-off-by: Paolo Bonzini