summaryrefslogtreecommitdiff
path: root/arch/arm/mm
AgeCommit message (Collapse)AuthorFilesLines
2023-01-11ARM: 9284/1: include <asm/pgtable.h> from proc-macros.S to fix -Wundef warningsMasahiro Yamada1-0/+1
Since commit 80b6093b55e3 ("kbuild: add -Wundef to KBUILD_CPPFLAGS for W=1 builds"), building with W=1 detects -Wundef warnings for assembly code. $ make W=1 ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- arch/arm/mm/ [snip] AS arch/arm/mm/cache-v7.o In file included from arch/arm/mm/cache-v7.S:17: arch/arm/mm/proc-macros.S:109:5: warning: "L_PTE_SHARED" is not defined, evaluates to 0 [-Wundef] 109 | #if L_PTE_SHARED != PTE_EXT_SHARED | ^~~~~~~~~~~~ arch/arm/mm/proc-macros.S:109:21: warning: "PTE_EXT_SHARED" is not defined, evaluates to 0 [-Wundef] 109 | #if L_PTE_SHARED != PTE_EXT_SHARED | ^~~~~~~~~~~~~~ arch/arm/mm/proc-macros.S:113:10: warning: "L_PTE_XN" is not defined, evaluates to 0 [-Wundef] 113 | (L_PTE_XN+L_PTE_USER+L_PTE_RDONLY+L_PTE_DIRTY+L_PTE_YOUNG+\ | ^~~~~~~~ arch/arm/mm/proc-macros.S:113:19: warning: "L_PTE_USER" is not defined, evaluates to 0 [-Wundef] 113 | (L_PTE_XN+L_PTE_USER+L_PTE_RDONLY+L_PTE_DIRTY+L_PTE_YOUNG+\ | ^~~~~~~~~~ arch/arm/mm/proc-macros.S:113:30: warning: "L_PTE_RDONLY" is not defined, evaluates to 0 [-Wundef] 113 | (L_PTE_XN+L_PTE_USER+L_PTE_RDONLY+L_PTE_DIRTY+L_PTE_YOUNG+\ | ^~~~~~~~~~~~ arch/arm/mm/proc-macros.S:113:43: warning: "L_PTE_DIRTY" is not defined, evaluates to 0 [-Wundef] 113 | (L_PTE_XN+L_PTE_USER+L_PTE_RDONLY+L_PTE_DIRTY+L_PTE_YOUNG+\ | ^~~~~~~~~~~ arch/arm/mm/proc-macros.S:113:55: warning: "L_PTE_YOUNG" is not defined, evaluates to 0 [-Wundef] 113 | (L_PTE_XN+L_PTE_USER+L_PTE_RDONLY+L_PTE_DIRTY+L_PTE_YOUNG+\ | ^~~~~~~~~~~ arch/arm/mm/proc-macros.S:114:10: warning: "L_PTE_PRESENT" is not defined, evaluates to 0 [-Wundef] 114 | L_PTE_PRESENT) > L_PTE_SHARED | ^~~~~~~~~~~~~ arch/arm/mm/proc-macros.S:114:27: warning: "L_PTE_SHARED" is not defined, evaluates to 0 [-Wundef] 114 | L_PTE_PRESENT) > L_PTE_SHARED | ^~~~~~~~~~~~ Include <asm/pgtable.h> from proc-macros.S to fix the warnings. Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Signed-off-by: Russell King (Oracle) <rmk+kernel@armlinux.org.uk>
2023-01-11ARM: 9280/1: mm: fix warning on phys_addr_t to void pointer assignmentGiulio Benetti1-1/+1
zero_page is a void* pointer but memblock_alloc() returns phys_addr_t type so this generates a warning while using clang and with -Wint-error enabled that becomes and error. So let's cast the return of memblock_alloc() to (void *). Cc: <stable@vger.kernel.org> # 4.14.x + Fixes: 340a982825f7 ("ARM: 9266/1: mm: fix no-MMU ZERO_PAGE() implementation") Signed-off-by: Giulio Benetti <giulio.benetti@benettiengineering.com> Signed-off-by: Russell King (Oracle) <rmk+kernel@armlinux.org.uk>
2022-12-14Merge tag 'for-linus' of git://git.armlinux.org.uk/~rmk/linux-armLinus Torvalds13-17/+22
Pull ARM updates from Russell King: - update unwinder to cope with module PLTs - enable UBSAN on ARM - improve kernel fault message - update UEFI runtime page tables dump - avoid clang's __aeabi_uldivmod generated in NWFPE code - disable FIQs on CPU shutdown paths - update XOR register usage - a number of build updates (using .arch, thread pointer, removal of lazy evaluation in Makefile) - conversion of stacktrace code to stackwalk - findbit assembly updates - hwcap feature updates for ARMv8 CPUs - instruction dump updates for big-endian platforms - support for function error injection * tag 'for-linus' of git://git.armlinux.org.uk/~rmk/linux-arm: (31 commits) ARM: 9279/1: support function error injection ARM: 9277/1: Make the dumped instructions are consistent with the disassembled ones ARM: 9276/1: Refactor dump_instr() ARM: 9275/1: Drop '-mthumb' from AFLAGS_ISA ARM: 9274/1: Add hwcap for Speculative Store Bypassing Safe ARM: 9273/1: Add hwcap for Speculation Barrier(SB) ARM: 9272/1: vfp: Add hwcap for FEAT_AA32I8MM ARM: 9271/1: vfp: Add hwcap for FEAT_AA32BF16 ARM: 9270/1: vfp: Add hwcap for FEAT_FHM ARM: 9269/1: vfp: Add hwcap for FEAT_DotProd ARM: 9268/1: vfp: Add hwcap FPHP and ASIMDHP for FEAT_FP16 ARM: 9267/1: Define Armv8 registers in AArch32 state ARM: findbit: add unwinder information ARM: findbit: operate by words ARM: findbit: convert to macros ARM: findbit: provide more efficient ARMv7 implementation ARM: findbit: document ARMv5 bit offset calculation ARM: 9259/1: stacktrace: Convert stacktrace to generic ARCH_STACKWALK ARM: 9258/1: stacktrace: Make stack walk callback consistent with generic code ARM: 9265/1: pass -march= only to compiler ...
2022-12-13Merge tag 'dma-mapping-6.2-2022-12-13' of ↵Linus Torvalds1-17/+0
git://git.infradead.org/users/hch/dma-mapping Pull dma-mapping updates from Christoph Hellwig: - reduce the swiotlb buffer size on allocation failure (Alexey Kardashevskiy) - clean up passing of bogus GFP flags to the dma-coherent allocator (Christoph Hellwig) * tag 'dma-mapping-6.2-2022-12-13' of git://git.infradead.org/users/hch/dma-mapping: dma-mapping: reject __GFP_COMP in dma_alloc_attrs ALSA: memalloc: don't pass bogus GFP_ flags to dma_alloc_* s390/ism: don't pass bogus GFP_ flags to dma_alloc_coherent cnic: don't pass bogus GFP_ flags to dma_alloc_coherent RDMA/qib: don't pass bogus GFP_ flags to dma_alloc_coherent RDMA/hfi1: don't pass bogus GFP_ flags to dma_alloc_coherent media: videobuf-dma-contig: use dma_mmap_coherent swiotlb: reduce the swiotlb buffer size on allocation failure
2022-12-10Merge tag 'for-linus' of git://git.armlinux.org.uk/~rmk/linux-armLinus Torvalds2-5/+22
Pull ARM fix from Russell King: "One further ARM fix for 6.1 from Wang Kefeng, fixing up the handling for kfence faults" * tag 'for-linus' of git://git.armlinux.org.uk/~rmk/linux-arm: ARM: 9278/1: kfence: only handle translation faults
2022-12-07ARM: 9278/1: kfence: only handle translation faultsWang Kefeng2-5/+22
This is a similar fixup like arm64 does, only handle translation faults in case of unexpected kfence report when alignment faults on ARM, see more from commit 0bb1fbffc631 ("arm64: mm: kfence: only handle translation faults"). Fixes: 75969686ec0d ("ARM: 9166/1: Support KFENCE for ARM") Signed-off-by: Kefeng Wang <wangkefeng.wang@huawei.com> Signed-off-by: Russell King (Oracle) <rmk+kernel@armlinux.org.uk>
2022-11-24Merge tag 'for-linus' of git://git.armlinux.org.uk/~rmk/linux-armLinus Torvalds1-0/+19
Pull ARM fixes from Russell King: "Two fixes for 6.1: - fix stacktraces for tracepoint events in Thumb2 mode - fix for noMMU ZERO_PAGE() implementation" * tag 'for-linus' of git://git.armlinux.org.uk/~rmk/linux-arm: ARM: 9266/1: mm: fix no-MMU ZERO_PAGE() implementation ARM: 9251/1: perf: Fix stacktraces for tracepoint events in THUMB2 kernels
2022-11-21dma-mapping: reject __GFP_COMP in dma_alloc_attrsChristoph Hellwig1-17/+0
DMA allocations can never be turned back into a page pointer, so requesting compound pages doesn't make sense and it can't even be supported at all by various backends. Reject __GFP_COMP with a warning in dma_alloc_attrs, and stop clearing the flag in the arm dma ops and dma-iommu. Signed-off-by: Christoph Hellwig <hch@lst.de> Acked-by: Marek Szyprowski <m.szyprowski@samsung.com>
2022-11-08ARM: 9263/1: use .arch directives instead of assembler command line flagsNick Desaulniers12-15/+19
Similar to commit a6c30873ee4a ("ARM: 8989/1: use .fpu assembler directives instead of assembler arguments"). GCC and GNU binutils support setting the "sub arch" via -march=, -Wa,-march, target function attribute, and .arch assembler directive. Clang was missing support for -Wa,-march=, but this was implemented in clang-13. The behavior of both GCC and Clang is to prefer -Wa,-march= over -march= for assembler and assembler-with-cpp sources, but Clang will warn about the -march= being unused. clang: warning: argument unused during compilation: '-march=armv6k' [-Wunused-command-line-argument] Since most assembler is non-conditionally assembled with one sub arch (modulo arch/arm/delay-loop.S which conditionally is assembled as armv4 based on CONFIG_ARCH_RPC, and arch/arm/mach-at91/pm-suspend.S which is conditionally assembled as armv7-a based on CONFIG_CPU_V7), prefer the .arch assembler directive. Add a few more instances found in compile testing as found by Arnd and Nathan. Link: https://github.com/llvm/llvm-project/commit/1d51c699b9e2ebc5bcfdbe85c74cc871426333d4 Link: https://bugs.llvm.org/show_bug.cgi?id=48894 Link: https://github.com/ClangBuiltLinux/linux/issues/1195 Link: https://github.com/ClangBuiltLinux/linux/issues/1315 Suggested-by: Arnd Bergmann <arnd@arndb.de> Suggested-by: Nathan Chancellor <nathan@kernel.org> Signed-off-by: Arnd Bergmann <arnd@arndb.de> Tested-by: Nathan Chancellor <nathan@kernel.org> Signed-off-by: Nick Desaulniers <ndesaulniers@google.com> Signed-off-by: Russell King (Oracle) <rmk+kernel@armlinux.org.uk>
2022-11-07ARM: 9254/1: mm: Provide better message when kernel faultWang Kefeng1-2/+3
If there is a kernel fault, see do_kernel_fault(), we only print the generic "paging request" or "NULL pointer dereference" message which don't show read, write or excute information, let's provide better fault message for them. Signed-off-by: Kefeng Wang <wangkefeng.wang@huawei.com> Signed-off-by: Russell King (Oracle) <rmk+kernel@armlinux.org.uk>
2022-11-07ARM: 9266/1: mm: fix no-MMU ZERO_PAGE() implementationGiulio Benetti1-0/+19
Actually in no-MMU SoCs(i.e. i.MXRT) ZERO_PAGE(vaddr) expands to ``` virt_to_page(0) ``` that in order expands to: ``` pfn_to_page(virt_to_pfn(0)) ``` and then virt_to_pfn(0) to: ``` ((((unsigned long)(0) - PAGE_OFFSET) >> PAGE_SHIFT) + PHYS_PFN_OFFSET) ``` where PAGE_OFFSET and PHYS_PFN_OFFSET are the DRAM offset(0x80000000) and PAGE_SHIFT is 12. This way we obtain 16MB(0x01000000) summed to the base of DRAM(0x80000000). When ZERO_PAGE(0) is then used, for example in bio_add_page(), the page gets an address that is out of DRAM bounds. So instead of using fake virtual page 0 let's allocate a dedicated zero_page during paging_init() and assign it to a global 'struct page * empty_zero_page' the same way mmu.c does and it's the same approach used in m68k with commit dc068f462179 as discussed here[0]. Then let's move ZERO_PAGE() definition to the top of pgtable.h to be in common between mmu.c and nommu.c. [0]: https://lore.kernel.org/linux-m68k/2a462b23-5b8e-bbf4-ec7d-778434a3b9d7@google.com/T/#m1266ceb63 ad140743174d6b3070364d3c9a5179b Signed-off-by: Giulio Benetti <giulio.benetti@benettiengineering.com> Reviewed-by: Arnd Bergmann <arnd@arndb.de> Signed-off-by: Russell King (Oracle) <rmk+kernel@armlinux.org.uk>
2022-10-10Merge tag 'dma-mapping-6.1-2022-10-10' of ↵Linus Torvalds1-2/+8
git://git.infradead.org/users/hch/dma-mapping Pull dma-mapping updates from Christoph Hellwig: - fix a regression in the ARM dma-direct conversion (Christoph Hellwig) - use memcpy_{from,to}_page (Fabio M. De Francesco) - cleanup the swiotlb MAINTAINERS entry (Lukas Bulwahn) - make SG table pool allocation less fragile (Masahiro Yamada) - don't panic on swiotlb initialization failure (Robin Murphy) * tag 'dma-mapping-6.1-2022-10-10' of git://git.infradead.org/users/hch/dma-mapping: ARM/dma-mapping: remove the dma_coherent member of struct dev_archdata ARM/dma-mappіng: don't override ->dma_coherent when set from a bus notifier lib/sg_pool: change module_init(sg_pool_init) to subsys_initcall MAINTAINERS: merge SWIOTLB SUBSYSTEM into DMA MAPPING HELPERS swiotlb: don't panic! swiotlb: replace kmap_atomic() with memcpy_{from,to}_page()
2022-10-06Merge tag 'for-linus' of git://git.armlinux.org.uk/~rmk/linux-armLinus Torvalds3-5/+14
Pull ARM updates from Russell King: - Print an un-hashed userspace PC on undefined instruction exception - Disable FDPIC ABI - Remove redundant vfp_flush/release_thread functions - Use raw_cpu_* rather than this_cpu_* in handle_bad_stack() - Avoid needlessly long backtraces when show_regs() is called - Fix an issue with stack traces through call_with_stack() - Avoid stack traces saving a duplicate exception PC value - Pass a void pointer to virt_to_page() in DMA mapping code - Fix kasan maps for modules when CONFIG_KASAN_VMALLOC=n - Show FDT region and page table level names in kernel page tables dump * tag 'for-linus' of git://git.armlinux.org.uk/~rmk/linux-arm: ARM: 9246/1: dump: show page table level name ARM: 9245/1: dump: show FDT region ARM: 9242/1: kasan: Only map modules if CONFIG_KASAN_VMALLOC=n ARM: 9240/1: dma-mapping: Pass (void *) to virt_to_page() ARM: 9234/1: stacktrace: Avoid duplicate saving of exception PC value ARM: 9233/1: stacktrace: Skip frame pointer boundary check for call_with_stack() ARM: 9224/1: Dump the stack traces based on the parameter 'regs' of show_regs() ARM: 9232/1: Replace this_cpu_* with raw_cpu_* in handle_bad_stack() ARM: 9228/1: vfp: kill vfp_flush/release_thread() ARM: 9226/1: disable FDPIC ABI ARM: 9221/1: traps: print un-hashed user pc on undefined instruction
2022-10-06ARM/dma-mapping: remove the dma_coherent member of struct dev_archdataChristoph Hellwig1-3/+1
Since commit ae626eb97376 ("ARM/dma-mapping: use dma-direct unconditionally") only the dma_coherent flag in struct device is used, so remove the now write only flag in struct dev_archdata. Signed-off-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Russell King (Oracle) <rmk+kernel@armlinux.org.uk>
2022-10-06ARM/dma-mappіng: don't override ->dma_coherent when set from a bus notifierChristoph Hellwig1-2/+10
Commit ae626eb97376 ("ARM/dma-mapping: use dma-direct unconditionally") caused a regression on the mvebu platform, wherein devices that are dma-coherent are marked as dma-noncoherent, because although mvebu_hwcc_notifier() after that commit still marks then as coherent, the arm_coherent_dma_ops() function, which is called later, overwrites this setting, since it is being called from drivers/of/device.c with coherency parameter determined by of_dma_is_coherent(), and the device-trees do not declare the 'dma-coherent' property. Fix this by defaulting never clearing the dma_coherent flag in arm_coherent_dma_ops(). Fixes: ae626eb97376 ("ARM/dma-mapping: use dma-direct unconditionally") Reported-by: Marek Behún <kabel@kernel.org> Signed-off-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Russell King (Oracle) <rmk+kernel@armlinux.org.uk> Tested-by: Marek Behún <kabel@kernel.org>
2022-10-04ARM: 9246/1: dump: show page table level nameWang Kefeng1-1/+5
ARM could have 3 page table level if ARM_LPAE enabled, or only 2 page table level, let's show the page table level name when dump. Signed-off-by: Kefeng Wang <wangkefeng.wang@huawei.com> Signed-off-by: Russell King (Oracle) <rmk+kernel@armlinux.org.uk>
2022-10-04ARM: 9245/1: dump: show FDT regionWang Kefeng1-1/+1
Since commit 7a1be318f579 ("ARM: 9012/1: move device tree mapping out of linear region"), FDT is placed between the end of the vmalloc region and the start of the fixmap region, let's show it in dump. Signed-off-by: Kefeng Wang <wangkefeng.wang@huawei.com> Signed-off-by: Russell King (Oracle) <rmk+kernel@armlinux.org.uk>
2022-10-04ARM: 9242/1: kasan: Only map modules if CONFIG_KASAN_VMALLOC=nAlex Sverdlin1-2/+7
In case CONFIG_KASAN_VMALLOC=y kasan_populate_vmalloc() allocates the shadow pages dynamically. But even worse is that kasan_release_vmalloc() releases them, which is not compatible with create_mapping() of MODULES_VADDR..MODULES_END range: BUG: Bad page state in process kworker/9:1 pfn:2068b page:e5e06160 refcount:0 mapcount:0 mapping:00000000 index:0x0 flags: 0x1000(reserved) raw: 00001000 e5e06164 e5e06164 00000000 00000000 00000000 ffffffff 00000000 page dumped because: PAGE_FLAGS_CHECK_AT_FREE flag(s) set bad because of flags: 0x1000(reserved) Modules linked in: ip_tables CPU: 9 PID: 154 Comm: kworker/9:1 Not tainted 5.4.188-... #1 Hardware name: LSI Axxia AXM55XX Workqueue: events do_free_init unwind_backtrace show_stack dump_stack bad_page free_pcp_prepare free_unref_page kasan_depopulate_vmalloc_pte __apply_to_page_range apply_to_existing_page_range kasan_release_vmalloc __purge_vmap_area_lazy _vm_unmap_aliases.part.0 __vunmap do_free_init process_one_work worker_thread kthread Reviewed-by: Linus Walleij <linus.walleij@linaro.org> Signed-off-by: Alexander Sverdlin <alexander.sverdlin@nokia.com> Signed-off-by: Russell King (Oracle) <rmk+kernel@armlinux.org.uk>
2022-10-04ARM: 9240/1: dma-mapping: Pass (void *) to virt_to_page()Linus Walleij1-1/+1
Pointers to virtual memory functions are (void *) but the __dma_update_pte() function is passing an unsigned long. Fix this up by explicit cast. Signed-off-by: Linus Walleij <linus.walleij@linaro.org> Signed-off-by: Russell King (Oracle) <rmk+kernel@armlinux.org.uk>
2022-09-22ARM: 9247/1: mm: set readonly for MT_MEMORY_RO with ARM_LPAEWang Kefeng1-0/+4
MT_MEMORY_RO is introduced by commit 598f0a99fa8a ("ARM: 9210/1: Mark the FDT_FIXED sections as shareable"), which is a readonly memory type for FDT area, but there are some different between ARM_LPAE and non-ARM_LPAE, we need to setup PMD_SECT_AP2 and L_PMD_SECT_RDONLY for MT_MEMORY_RO when ARM_LAPE enabled. non-ARM_LPAE 0xff800000-0xffa00000 2M PGD KERNEL ro NX SHD ARM_LPAE 0xff800000-0xffc00000 4M PMD RW NX SHD ARM_LPAE+fix 0xff800000-0xffc00000 4M PMD ro NX SHD Fixes: 598f0a99fa8a ("ARM: 9210/1: Mark the FDT_FIXED sections as shareable") Signed-off-by: Kefeng Wang <wangkefeng.wang@huawei.com> Signed-off-by: Russell King (Oracle) <rmk+kernel@armlinux.org.uk>
2022-09-22ARM: 9244/1: dump: Fix wrong pg_level in walk_pmd()Wang Kefeng1-1/+1
After ARM supports p4d page tables, the pg_level for note_page() in walk_pmd() should be 4, not 3, fix it. Fixes: 84e6ffb2c49c ("arm: add support for folded p4d page tables") Signed-off-by: Kefeng Wang <wangkefeng.wang@huawei.com> Signed-off-by: Russell King (Oracle) <rmk+kernel@armlinux.org.uk>
2022-08-06Merge tag 'dma-mapping-5.20-2022-08-06' of ↵Linus Torvalds1-579/+73
git://git.infradead.org/users/hch/dma-mapping Pull dma-mapping updates from Christoph Hellwig: - convert arm32 to the common dma-direct code (Arnd Bergmann, Robin Murphy, Christoph Hellwig) - restructure the PCIe peer to peer mapping support (Logan Gunthorpe) - allow the IOMMU code to communicate an optional DMA mapping length and use that in scsi and libata (John Garry) - split the global swiotlb lock (Tianyu Lan) - various fixes and cleanup (Chao Gao, Dan Carpenter, Dongli Zhang, Lukas Bulwahn, Robin Murphy) * tag 'dma-mapping-5.20-2022-08-06' of git://git.infradead.org/users/hch/dma-mapping: (45 commits) swiotlb: fix passing local variable to debugfs_create_ulong() dma-mapping: reformat comment to suppress htmldoc warning PCI/P2PDMA: Remove pci_p2pdma_[un]map_sg() RDMA/rw: drop pci_p2pdma_[un]map_sg() RDMA/core: introduce ib_dma_pci_p2p_dma_supported() nvme-pci: convert to using dma_map_sgtable() nvme-pci: check DMA ops when indicating support for PCI P2PDMA iommu/dma: support PCI P2PDMA pages in dma-iommu map_sg iommu: Explicitly skip bus address marked segments in __iommu_map_sg() dma-mapping: add flags to dma_map_ops to indicate PCI P2PDMA support dma-direct: support PCI P2PDMA pages in dma-direct map_sg dma-mapping: allow EREMOTEIO return code for P2PDMA transfers PCI/P2PDMA: Introduce helpers for dma_map_sg implementations PCI/P2PDMA: Attempt to set map_type if it has not been set lib/scatterlist: add flag for indicating P2PDMA segments in an SGL swiotlb: clean up some coding style and minor issues dma-mapping: update comment after dmabounce removal scsi: sd: Add a comment about limiting max_sectors to shost optimal limit ata: libata-scsi: cap ata_device->max_sectors according to shost->max_sectors scsi: scsi_transport_sas: cap shost opt_sectors according to DMA optimal limit ...
2022-08-06Merge tag 'mm-stable-2022-08-03' of ↵Linus Torvalds2-0/+24
git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm Pull MM updates from Andrew Morton: "Most of the MM queue. A few things are still pending. Liam's maple tree rework didn't make it. This has resulted in a few other minor patch series being held over for next time. Multi-gen LRU still isn't merged as we were waiting for mapletree to stabilize. The current plan is to merge MGLRU into -mm soon and to later reintroduce mapletree, with a view to hopefully getting both into 6.1-rc1. Summary: - The usual batches of cleanups from Baoquan He, Muchun Song, Miaohe Lin, Yang Shi, Anshuman Khandual and Mike Rapoport - Some kmemleak fixes from Patrick Wang and Waiman Long - DAMON updates from SeongJae Park - memcg debug/visibility work from Roman Gushchin - vmalloc speedup from Uladzislau Rezki - more folio conversion work from Matthew Wilcox - enhancements for coherent device memory mapping from Alex Sierra - addition of shared pages tracking and CoW support for fsdax, from Shiyang Ruan - hugetlb optimizations from Mike Kravetz - Mel Gorman has contributed some pagealloc changes to improve latency and realtime behaviour. - mprotect soft-dirty checking has been improved by Peter Xu - Many other singleton patches all over the place" [ XFS merge from hell as per Darrick Wong in https://lore.kernel.org/all/YshKnxb4VwXycPO8@magnolia/ ] * tag 'mm-stable-2022-08-03' of git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm: (282 commits) tools/testing/selftests/vm/hmm-tests.c: fix build mm: Kconfig: fix typo mm: memory-failure: convert to pr_fmt() mm: use is_zone_movable_page() helper hugetlbfs: fix inaccurate comment in hugetlbfs_statfs() hugetlbfs: cleanup some comments in inode.c hugetlbfs: remove unneeded header file hugetlbfs: remove unneeded hugetlbfs_ops forward declaration hugetlbfs: use helper macro SZ_1{K,M} mm: cleanup is_highmem() mm/hmm: add a test for cross device private faults selftests: add soft-dirty into run_vmtests.sh selftests: soft-dirty: add test for mprotect mm/mprotect: fix soft-dirty check in can_change_pte_writable() mm: memcontrol: fix potential oom_lock recursion deadlock mm/gup.c: fix formatting in check_and_migrate_movable_page() xfs: fail dax mount if reflink is enabled on a partition mm/memcontrol.c: remove the redundant updating of stats_flush_threshold userfaultfd: don't fail on unrecognized features hugetlb_cgroup: fix wrong hugetlb cgroup numa stat ...
2022-08-05Merge tag 'for-linus' of git://git.armlinux.org.uk/~rmk/linux-armLinus Torvalds2-2/+6
Pull ARM updates from Russell King: "Not much this time around, the 5.20-rc1 development updates for arm are: - add KASAN support for vmalloc space on arm - some sparse fixes from Ben Dooks - rework amba device handling (so device addition isn't deferred)" * tag 'for-linus' of git://git.armlinux.org.uk/~rmk/linux-arm: ARM: 9220/1: amba: Remove deferred device addition ARM: 9219/1: fix undeclared soft_restart ARM: 9218/1: dma-mapping: fix pointer/integer warning ARM: 9217/1: add definition of arch_irq_work_raise() ARM: 9203/1: kconfig: fix MODULE_PLTS for KASAN with KASAN_VMALLOC ARM: 9202/1: kasan: support CONFIG_KASAN_VMALLOC
2022-08-04Merge tag 'spdx-6.0-rc1' of ↵Linus Torvalds2-8/+2
git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/spdx Pull SPDX updates from Greg KH: "Here is the set of SPDX comment updates for 6.0-rc1. Nothing huge here, just a number of updated SPDX license tags and cleanups based on the review of a number of common patterns in GPLv2 boilerplate text. Also included in here are a few other minor updates, two USB files, and one Documentation file update to get the SPDX lines correct. All of these have been in the linux-next tree for a very long time" * tag 'spdx-6.0-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/spdx: (28 commits) Documentation: samsung-s3c24xx: Add blank line after SPDX directive x86/crypto: Remove stray comment terminator treewide: Replace GPLv2 boilerplate/reference with SPDX - gpl-2.0_406.RULE treewide: Replace GPLv2 boilerplate/reference with SPDX - gpl-2.0_398.RULE treewide: Replace GPLv2 boilerplate/reference with SPDX - gpl-2.0_391.RULE treewide: Replace GPLv2 boilerplate/reference with SPDX - gpl-2.0_390.RULE treewide: Replace GPLv2 boilerplate/reference with SPDX - gpl-2.0_385.RULE treewide: Replace GPLv2 boilerplate/reference with SPDX - gpl-2.0_320.RULE treewide: Replace GPLv2 boilerplate/reference with SPDX - gpl-2.0_319.RULE treewide: Replace GPLv2 boilerplate/reference with SPDX - gpl-2.0_318.RULE treewide: Replace GPLv2 boilerplate/reference with SPDX - gpl-2.0_298.RULE treewide: Replace GPLv2 boilerplate/reference with SPDX - gpl-2.0_292.RULE treewide: Replace GPLv2 boilerplate/reference with SPDX - gpl-2.0_179.RULE treewide: Replace GPLv2 boilerplate/reference with SPDX - gpl-2.0_168.RULE (part 2) treewide: Replace GPLv2 boilerplate/reference with SPDX - gpl-2.0_168.RULE (part 1) treewide: Replace GPLv2 boilerplate/reference with SPDX - gpl-2.0_160.RULE treewide: Replace GPLv2 boilerplate/reference with SPDX - gpl-2.0_152.RULE treewide: Replace GPLv2 boilerplate/reference with SPDX - gpl-2.0_149.RULE treewide: Replace GPLv2 boilerplate/reference with SPDX - gpl-2.0_147.RULE treewide: Replace GPLv2 boilerplate/reference with SPDX - gpl-2.0_133.RULE ...
2022-08-01Merge tag 'arm64-upstream' of ↵Linus Torvalds2-16/+2
git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux Pull arm64 updates from Will Deacon: "Highlights include a major rework of our kPTI page-table rewriting code (which makes it both more maintainable and considerably faster in the cases where it is required) as well as significant changes to our early boot code to reduce the need for data cache maintenance and greatly simplify the KASLR relocation dance. Summary: - Remove unused generic cpuidle support (replaced by PSCI version) - Fix documentation describing the kernel virtual address space - Handling of some new CPU errata in Arm implementations - Rework of our exception table code in preparation for handling machine checks (i.e. RAS errors) more gracefully - Switch over to the generic implementation of ioremap() - Fix lockdep tracking in NMI context - Instrument our memory barrier macros for KCSAN - Rework of the kPTI G->nG page-table repainting so that the MMU remains enabled and the boot time is no longer slowed to a crawl for systems which require the late remapping - Enable support for direct swapping of 2MiB transparent huge-pages on systems without MTE - Fix handling of MTE tags with allocating new pages with HW KASAN - Expose the SMIDR register to userspace via sysfs - Continued rework of the stack unwinder, particularly improving the behaviour under KASAN - More repainting of our system register definitions to match the architectural terminology - Improvements to the layout of the vDSO objects - Support for allocating additional bits of HWCAP2 and exposing FEAT_EBF16 to userspace on CPUs that support it - Considerable rework and optimisation of our early boot code to reduce the need for cache maintenance and avoid jumping in and out of the kernel when handling relocation under KASLR - Support for disabling SVE and SME support on the kernel command-line - Support for the Hisilicon HNS3 PMU - Miscellanous cleanups, trivial updates and minor fixes" * tag 'arm64-upstream' of git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux: (136 commits) arm64: Delay initialisation of cpuinfo_arm64::reg_{zcr,smcr} arm64: fix KASAN_INLINE arm64/hwcap: Support FEAT_EBF16 arm64/cpufeature: Store elf_hwcaps as a bitmap rather than unsigned long arm64/hwcap: Document allocation of upper bits of AT_HWCAP arm64: enable THP_SWAP for arm64 arm64/mm: use GENMASK_ULL for TTBR_BADDR_MASK_52 arm64: errata: Remove AES hwcap for COMPAT tasks arm64: numa: Don't check node against MAX_NUMNODES drivers/perf: arm_spe: Fix consistency of SYS_PMSCR_EL1.CX perf: RISC-V: Add of_node_put() when breaking out of for_each_of_cpu_node() docs: perf: Include hns3-pmu.rst in toctree to fix 'htmldocs' WARNING arm64: kasan: Revert "arm64: mte: reset the page tag in page->flags" mm: kasan: Skip page unpoisoning only if __GFP_SKIP_KASAN_UNPOISON mm: kasan: Skip unpoisoning of user pages mm: kasan: Ensure the tags are visible before the tag in page->flags drivers/perf: hisi: add driver for HNS3 PMU drivers/perf: hisi: Add description for HNS3 PMU driver drivers/perf: riscv_pmu_sbi: perf format perf/arm-cci: Use the bitmap API to allocate bitmaps ...
2022-07-28ARM: 9218/1: dma-mapping: fix pointer/integer warningBen Dooks1-1/+1
Fix the use of a pointer assignment from integer where false is being used instead of NULL. Fix the following warning by usign NULL: arch/arm/mm/dma-mapping.c:712:52: warning: Using plain integer as NULL pointer Signed-off-by: Ben Dooks <ben-linux@fluff.org> Signed-off-by: Russell King (Oracle) <rmk+kernel@armlinux.org.uk>
2022-07-28ARM: 9202/1: kasan: support CONFIG_KASAN_VMALLOCLecopzer Chen1-1/+5
Simply make shadow of vmalloc area mapped on demand. Since the virtual address of vmalloc for Arm is also between MODULE_VADDR and 0x100000000 (ZONE_HIGHMEM), which means the shadow address has already included between KASAN_SHADOW_START and KASAN_SHADOW_END. Thus we need to change nothing for memory map of Arm. This can fix ARM_MODULE_PLTS with KASan, support KASan for higmem and support CONFIG_VMAP_STACK with KASan. Signed-off-by: Lecopzer Chen <lecopzer.chen@mediatek.com> Tested-by: Linus Walleij <linus.walleij@linaro.org> Reviewed-by: Linus Walleij <linus.walleij@linaro.org> Signed-off-by: Russell King (Oracle) <rmk+kernel@armlinux.org.uk>
2022-07-22dma-mapping: update comment after dmabounce removalLukas Bulwahn1-2/+1
Commit e3217540c271 ("ARM/dma-mapping: remove dmabounce") removes the config DMABOUNCE. A comment to the function __dma_page_cpu_to_dev() refers to this removed config DMABOUNCE. Remove the obsolete explanation, but keep the recommendation not to use __dma_page_cpu_to_dev() and use dma_sync_* functions instead. Signed-off-by: Lukas Bulwahn <lukas.bulwahn@gmail.com> Acked-by: Arnd Bergmann <arnd@arndb.de> Signed-off-by: Christoph Hellwig <hch@lst.de>
2022-07-18arm/mm: enable ARCH_HAS_VM_GET_PAGE_PROTAnshuman Khandual1-0/+20
This enables ARCH_HAS_VM_GET_PAGE_PROT on the platform and exports standard vm_get_page_prot() implementation via DECLARE_VM_GET_PAGE_PROT, which looks up a private and static protection_map[] array. Subsequently all __SXXX and __PXXX macros can be dropped which are no longer needed. Link: https://lkml.kernel.org/r/20220711070600.2378316-24-anshuman.khandual@arm.com Signed-off-by: Anshuman Khandual <anshuman.khandual@arm.com> Cc: Russell King <linux@armlinux.org.uk> Cc: Arnd Bergmann <arnd@arndb.de> Cc: Brian Cain <bcain@quicinc.com> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Christophe Leroy <christophe.leroy@csgroup.eu> Cc: Christoph Hellwig <hch@infradead.org> Cc: Christoph Hellwig <hch@lst.de> Cc: Chris Zankel <chris@zankel.net> Cc: "David S. Miller" <davem@davemloft.net> Cc: Dinh Nguyen <dinguyen@kernel.org> Cc: Geert Uytterhoeven <geert@linux-m68k.org> Cc: Guo Ren <guoren@kernel.org> Cc: Heiko Carstens <hca@linux.ibm.com> Cc: Huacai Chen <chenhuacai@kernel.org> Cc: Ingo Molnar <mingo@redhat.com> Cc: "James E.J. Bottomley" <James.Bottomley@HansenPartnership.com> Cc: Jeff Dike <jdike@addtoit.com> Cc: Jonas Bonn <jonas@southpole.se> Cc: Michael Ellerman <mpe@ellerman.id.au> Cc: Michal Simek <monstr@monstr.eu> Cc: Nicholas Piggin <npiggin@gmail.com> Cc: Palmer Dabbelt <palmer@dabbelt.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Paul Walmsley <paul.walmsley@sifive.com> Cc: Richard Henderson <rth@twiddle.net> Cc: Rich Felker <dalias@libc.org> Cc: Sam Ravnborg <sam@ravnborg.org> Cc: Stafford Horne <shorne@gmail.com> Cc: Thomas Bogendoerfer <tsbogend@alpha.franken.de> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: Vasily Gorbik <gor@linux.ibm.com> Cc: Vineet Gupta <vgupta@kernel.org> Cc: WANG Xuerui <kernel@xen0n.name> Cc: Will Deacon <will@kernel.org> Cc: Yoshinori Sato <ysato@users.sourceforge.jp> Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
2022-07-14Merge tag 'for-linus' of git://git.armlinux.org.uk/~rmk/linux-armLinus Torvalds4-7/+26
Pull ARM fixes from Russell King: - quieten the spectre-bhb prints - mark flattened device tree sections as shareable - remove some obsolete CPU domain code and help text - fix thumb unaligned access abort emulation - fix amba_device_add() refcount underflow - fix literal placement * tag 'for-linus' of git://git.armlinux.org.uk/~rmk/linux-arm: ARM: 9208/1: entry: add .ltorg directive to keep literals in range ARM: 9207/1: amba: fix refcount underflow if amba_device_add() fails ARM: 9214/1: alignment: advance IT state after emulating Thumb instruction ARM: 9213/1: Print message about disabled Spectre workarounds only once ARM: 9212/1: domain: Modify Kconfig help text ARM: 9211/1: domain: drop modify_domain() ARM: 9210/1: Mark the FDT_FIXED sections as shareable ARM: 9209/1: Spectre-BHB: avoid pr_info() every time a CPU comes out of idle
2022-07-07ARM/dma-mapping: merge IOMMU opsRobin Murphy1-24/+13
The dma_sync_* operations are now the only difference between the coherent and non-coherent IOMMU ops. Some minor tweaks to make those safe for coherent devices with minimal overhead, and we can condense down to a single set of DMA ops. Signed-off-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Christoph Hellwig <hch@lst.de> Tested-by: Marc Zyngier <maz@kernel.org>
2022-07-07ARM/dma-mapping: consolidate IOMMU ops callbacksRobin Murphy1-183/+55
Merge the coherent and non-coherent callbacks down to a single implementation each, relying on the generic dev->dma_coherent flag at the points where the difference matters. Signed-off-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Christoph Hellwig <hch@lst.de> Tested-by: Marc Zyngier <maz@kernel.org>
2022-07-07ARM/dma-mapping: drop .dma_supported for IOMMU opsRobin Murphy1-23/+0
When an IOMMU is present, we trust that it should be capable of remapping any physical memory, and since the device masks represent the input (virtual) addresses to the IOMMU it makes no sense to validate them against physical PFNs anyway. Signed-off-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Christoph Hellwig <hch@lst.de> Tested-by: Marc Zyngier <maz@kernel.org>
2022-07-07ARM/dma-mapping: use dma-direct unconditionallyChristoph Hellwig1-350/+15
Use dma-direct unconditionally on arm. It has already been used for some time for LPAE and nommu configurations. This mostly changes the streaming mapping implementation and the (simple) coherent allocator for device that are DMA coherent. The existing complex allocator for uncached mappings for non-coherent devices is still used as is using the arch_dma_alloc/arch_dma_free hooks. Signed-off-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Arnd Bergmann <arnd@arndb.de> Acked-by: Andre Przywara <andre.przywara@arm.com> [highbank] Tested-by: Marc Zyngier <maz@kernel.org>
2022-07-07ARM/dma-mapping: use dma_to_phys/phys_to_dma in the dma-mapping codeChristoph Hellwig1-12/+12
Use the helpers as expected by the dma-direct code in the old arm dma-mapping code to ease a gradual switch to the common DMA code. Signed-off-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Arnd Bergmann <arnd@arndb.de> Tested-by: Marc Zyngier <maz@kernel.org>
2022-07-07ARM/dma-mapping: mark various dma-mapping routines static in dma-mapping.cChristoph Hellwig1-54/+46
With the dmabounce removal these aren't used outside of dma-mapping.c, so mark them static. Move the dma_map_ops declarations down a bit to avoid lots of forward declarations. Signed-off-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Arnd Bergmann <arnd@arndb.de> Tested-by: Marc Zyngier <maz@kernel.org>
2022-07-07ARM: 9214/1: alignment: advance IT state after emulating Thumb instructionArd Biesheuvel1-0/+3
After emulating a misaligned load or store issued in Thumb mode, we have to advance the IT state by hand, or it will get out of sync with the actual instruction stream, which means we'll end up applying the wrong condition code to subsequent instructions. This might corrupt the program state rather catastrophically. So borrow the it_advance() helper from the probing code, and use it on CPSR if the emulated instruction is Thumb. Cc: <stable@vger.kernel.org> Reviewed-by: Linus Walleij <linus.walleij@linaro.org> Signed-off-by: Ard Biesheuvel <ardb@kernel.org> Signed-off-by: Russell King (Oracle) <rmk+kernel@armlinux.org.uk>
2022-07-07ARM: 9213/1: Print message about disabled Spectre workarounds only onceDmitry Osipenko1-2/+1
Print the message about disabled Spectre workarounds only once. The message is printed each time CPU goes out from idling state on NVIDIA Tegra boards, causing storm in KMSG that makes system unusable. Cc: stable@vger.kernel.org Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com> Signed-off-by: Russell King (Oracle) <rmk+kernel@armlinux.org.uk>
2022-07-07ARM: 9212/1: domain: Modify Kconfig help textLinus Walleij1-1/+5
After the removal of set_fs() the reference to set_fs() is stale. Alter the helptext to reflect what the config option really does. Fixes: 8ac6f5d7f84b ("ARM: 9113/1: uaccess: remove set_fs() implementation") Acked-by: Arnd Bergmann <arnd@arndb.de> Signed-off-by: Linus Walleij <linus.walleij@linaro.org> Signed-off-by: Russell King (Oracle) <rmk+kernel@armlinux.org.uk>
2022-07-07ARM: 9210/1: Mark the FDT_FIXED sections as shareableZhen Lei1-1/+14
commit 7a1be318f579 ("ARM: 9012/1: move device tree mapping out of linear region") use FDT_FIXED_BASE to map the whole FDT_FIXED_SIZE memory area which contains fdt. But it only reserves the exact physical memory that fdt occupied. Unfortunately, this mapping is non-shareable. An illegal or speculative read access can bring the RAM content from non-fdt zone into cache, PIPT makes it to be hit by subsequently read access through shareable mapping(such as linear mapping), and the cache consistency between cores is lost due to non-shareable property. |<---------FDT_FIXED_SIZE------>| | | ------------------------------- | <non-fdt> | <fdt> | <non-fdt> | ------------------------------- 1. CoreA read <non-fdt> through MT_ROM mapping, the old data is loaded into the cache. 2. CoreB write <non-fdt> to update data through linear mapping. CoreA received the notification to invalid the corresponding cachelines, but the property non-shareable makes it to be ignored. 3. CoreA read <non-fdt> through linear mapping, cache hit, the old data is read. To eliminate this risk, add a new memory type MT_MEMORY_RO. Compared to MT_ROM, it is shareable and non-executable. Here's an example: list_del corruption. prev->next should be c0ecbf74, but was c08410dc kernel BUG at lib/list_debug.c:53! ... ... PC is at __list_del_entry_valid+0x58/0x98 LR is at __list_del_entry_valid+0x58/0x98 psr: 60000093 sp : c0ecbf30 ip : 00000000 fp : 00000001 r10: c08410d0 r9 : 00000001 r8 : c0825e0c r7 : 20000013 r6 : c08410d0 r5 : c0ecbf74 r4 : c0ecbf74 r3 : c0825d08 r2 : 00000000 r1 : df7ce6f4 r0 : 00000044 ... ... Stack: (0xc0ecbf30 to 0xc0ecc000) bf20: c0ecbf74 c0164fd0 c0ecbf70 c0165170 bf40: c0eca000 c0840c00 c0840c00 c0824500 c0825e0c c0189bbc c088f404 60000013 bf60: 60000013 c0e85100 000004ec 00000000 c0ebcdc0 c0ecbf74 c0ecbf74 c0825d08 ... ... < next prev > (__list_del_entry_valid) from (__list_del_entry+0xc/0x20) (__list_del_entry) from (finish_swait+0x60/0x7c) (finish_swait) from (rcu_gp_kthread+0x560/0xa20) (rcu_gp_kthread) from (kthread+0x14c/0x15c) (kthread) from (ret_from_fork+0x14/0x24) The faulty list node to be deleted is a local variable, its address is c0ecbf74. The dumped stack shows that 'prev' = c0ecbf74, but its value before lib/list_debug.c:53 is c08410dc. A large amount of printing results in swapping out the cacheline containing the old data(MT_ROM mapping is read only, so the cacheline cannot be dirty), and the subsequent dump operation obtains new data from the DDR. Fixes: 7a1be318f579 ("ARM: 9012/1: move device tree mapping out of linear region") Suggested-by: Ard Biesheuvel <ardb@kernel.org> Signed-off-by: Zhen Lei <thunder.leizhen@huawei.com> Reviewed-by: Ard Biesheuvel <ardb@kernel.org> Reviewed-by: Kefeng Wang <wangkefeng.wang@huawei.com> Signed-off-by: Russell King (Oracle) <rmk+kernel@armlinux.org.uk>
2022-07-07ARM: 9209/1: Spectre-BHB: avoid pr_info() every time a CPU comes out of idleArd Biesheuvel1-3/+3
Jon reports that the Spectre-BHB init code is filling up the kernel log with spurious notifications about which mitigation has been enabled, every time any CPU comes out of a low power state. Given that Spectre-BHB mitigations are system wide, only a single mitigation can be enabled, and we already print an error if two types of CPUs coexist in a single system that require different Spectre-BHB mitigations. This means that the pr_info() that describes the selected mitigation does not need to be emitted for each CPU anyway, and so we can simply emit it only once. In order to clarify the above in the log message, update it to describe that the selected mitigation will be enabled on all CPUs, including ones that are unaffected. If another CPU comes up later that is affected and requires a different mitigation, we report an error as before. Fixes: b9baf5c8c5c3 ("ARM: Spectre-BHB workaround") Tested-by: Jon Hunter <jonathanh@nvidia.com> Signed-off-by: Ard Biesheuvel <ardb@kernel.org> Signed-off-by: Russell King (Oracle) <rmk+kernel@armlinux.org.uk>
2022-06-27ARM: mm: kill unused runtime hook arch_iounmap()Kefeng Wang2-16/+2
Since the following commits, v5.4 commit 59d3ae9a5bf6 ("ARM: remove Intel iop33x and iop13xx support") v5.11 commit 3e3f354bc383 ("ARM: remove ebsa110 platform") The runtime hook arch_iounmap() on ARM is useless, kill arch_iounmap() and __iounmap(). Cc: Russell King <linux@armlinux.org.uk> Reviewed-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Arnd Bergmann <arnd@arndb.de> Signed-off-by: Kefeng Wang <wangkefeng.wang@huawei.com> Reviewed-by: Russell King (Oracle) <rmk+kernel@armlinux.org.uk> Reported-by: kernel test robot <lkp@intel.com> Link: https://lore.kernel.org/r/20220607125027.44946-2-wangkefeng.wang@huawei.com Signed-off-by: Will Deacon <will@kernel.org>
2022-06-17mm: avoid unnecessary page fault retires on shared memory typesPeter Xu1-0/+4
I observed that for each of the shared file-backed page faults, we're very likely to retry one more time for the 1st write fault upon no page. It's because we'll need to release the mmap lock for dirty rate limit purpose with balance_dirty_pages_ratelimited() (in fault_dirty_shared_page()). Then after that throttling we return VM_FAULT_RETRY. We did that probably because VM_FAULT_RETRY is the only way we can return to the fault handler at that time telling it we've released the mmap lock. However that's not ideal because it's very likely the fault does not need to be retried at all since the pgtable was well installed before the throttling, so the next continuous fault (including taking mmap read lock, walk the pgtable, etc.) could be in most cases unnecessary. It's not only slowing down page faults for shared file-backed, but also add more mmap lock contention which is in most cases not needed at all. To observe this, one could try to write to some shmem page and look at "pgfault" value in /proc/vmstat, then we should expect 2 counts for each shmem write simply because we retried, and vm event "pgfault" will capture that. To make it more efficient, add a new VM_FAULT_COMPLETED return code just to show that we've completed the whole fault and released the lock. It's also a hint that we should very possibly not need another fault immediately on this page because we've just completed it. This patch provides a ~12% perf boost on my aarch64 test VM with a simple program sequentially dirtying 400MB shmem file being mmap()ed and these are the time it needs: Before: 650.980 ms (+-1.94%) After: 569.396 ms (+-1.38%) I believe it could help more than that. We need some special care on GUP and the s390 pgfault handler (for gmap code before returning from pgfault), the rest changes in the page fault handlers should be relatively straightforward. Another thing to mention is that mm_account_fault() does take this new fault as a generic fault to be accounted, unlike VM_FAULT_RETRY. I explicitly didn't touch hmm_vma_fault() and break_ksm() because they do not handle VM_FAULT_RETRY even with existing code, so I'm literally keeping them as-is. Link: https://lkml.kernel.org/r/20220530183450.42886-1-peterx@redhat.com Signed-off-by: Peter Xu <peterx@redhat.com> Acked-by: Geert Uytterhoeven <geert@linux-m68k.org> Acked-by: Peter Zijlstra (Intel) <peterz@infradead.org> Acked-by: Johannes Weiner <hannes@cmpxchg.org> Acked-by: Vineet Gupta <vgupta@kernel.org> Acked-by: Guo Ren <guoren@kernel.org> Acked-by: Max Filippov <jcmvbkbc@gmail.com> Acked-by: Christian Borntraeger <borntraeger@linux.ibm.com> Acked-by: Michael Ellerman <mpe@ellerman.id.au> (powerpc) Acked-by: Catalin Marinas <catalin.marinas@arm.com> Reviewed-by: Alistair Popple <apopple@nvidia.com> Reviewed-by: Ingo Molnar <mingo@kernel.org> Acked-by: Russell King (Oracle) <rmk+kernel@armlinux.org.uk> [arm part] Acked-by: Heiko Carstens <hca@linux.ibm.com> Cc: Vasily Gorbik <gor@linux.ibm.com> Cc: Stafford Horne <shorne@gmail.com> Cc: David S. Miller <davem@davemloft.net> Cc: Johannes Berg <johannes@sipsolutions.net> Cc: Brian Cain <bcain@quicinc.com> Cc: Richard Henderson <rth@twiddle.net> Cc: Richard Weinberger <richard@nod.at> Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: Janosch Frank <frankja@linux.ibm.com> Cc: Albert Ou <aou@eecs.berkeley.edu> Cc: Anton Ivanov <anton.ivanov@cambridgegreys.com> Cc: Dave Hansen <dave.hansen@linux.intel.com> Cc: Borislav Petkov <bp@alien8.de> Cc: Sven Schnelle <svens@linux.ibm.com> Cc: Andrea Arcangeli <aarcange@redhat.com> Cc: James Bottomley <James.Bottomley@HansenPartnership.com> Cc: Al Viro <viro@zeniv.linux.org.uk> Cc: Alexander Gordeev <agordeev@linux.ibm.com> Cc: Jonas Bonn <jonas@southpole.se> Cc: Will Deacon <will@kernel.org> Cc: Vlastimil Babka <vbabka@suse.cz> Cc: Michal Simek <monstr@monstr.eu> Cc: Matt Turner <mattst88@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: David Hildenbrand <david@redhat.com> Cc: Nicholas Piggin <npiggin@gmail.com> Cc: Palmer Dabbelt <palmer@dabbelt.com> Cc: Stefan Kristiansson <stefan.kristiansson@saunalahti.fi> Cc: Paul Walmsley <paul.walmsley@sifive.com> Cc: Ivan Kokshaysky <ink@jurassic.park.msu.ru> Cc: Chris Zankel <chris@zankel.net> Cc: Hugh Dickins <hughd@google.com> Cc: Dinh Nguyen <dinguyen@kernel.org> Cc: Rich Felker <dalias@libc.org> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Andy Lutomirski <luto@kernel.org> Cc: Thomas Bogendoerfer <tsbogend@alpha.franken.de> Cc: Helge Deller <deller@gmx.de> Cc: Yoshinori Sato <ysato@users.osdn.me> Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
2022-06-10treewide: Replace GPLv2 boilerplate/reference with SPDX - gpl-2.0_56.RULE ↵Thomas Gleixner2-8/+2
(part 1) Based on the normalized pattern: this file is licensed under the terms of the gnu general public license version 2 this program is licensed as is without any warranty of any kind whether express or implied extracted by the scancode license scanner the SPDX license identifier GPL-2.0-only has been chosen to replace the boilerplate/reference. Reviewed-by: Allison Randal <allison@lohutok.net> Signed-off-by: Thomas Gleixner <tglx@linutronix.de> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2022-06-06arm/xen: Introduce xen_setup_dma_ops()Oleksandr Tyshchenko1-5/+2
This patch introduces new helper and places it in new header. The helper's purpose is to assign any Xen specific DMA ops in a single place. For now, we deal with xen-swiotlb DMA ops only. The one of the subsequent commits in current series will add xen-grant DMA ops case. Also re-use the xen_swiotlb_detect() check on Arm32. Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com> Reviewed-by: Stefano Stabellini <sstabellini@kernel.org> [For arm64] Acked-by: Catalin Marinas <catalin.marinas@arm.com> Link: https://lore.kernel.org/r/1654197833-25362-2-git-send-email-olekstysh@gmail.com Signed-off-by: Juergen Gross <jgross@suse.com>
2022-06-03Merge tag 'arm-multiplatform-5.19-2' of ↵Linus Torvalds2-1/+3
git://git.kernel.org/pub/scm/linux/kernel/git/soc/soc Pull more ARM multiplatform updates from Arnd Bergmann: "The second part of the multiplatform changes now converts the Intel/Marvell PXA platform along with the rest. The patches went through several rebases before the merge window as bugs were found, so they remained separate. This has to touch a lot of drivers, in particular the touchscreen, pcmcia, sound and clk bits, to detach the driver files from the platform and board specific header files" * tag 'arm-multiplatform-5.19-2' of git://git.kernel.org/pub/scm/linux/kernel/git/soc/soc: (48 commits) ARM: pxa/mmp: remove traces of plat-pxa ARM: pxa: convert to multiplatform ARM: pxa/sa1100: move I/O space to PCI_IOBASE ARM: pxa: remove support for MTD_XIP ARM: pxa: move mach/*.h to mach-pxa/ ARM: PXA: fix multi-cpu build of xsc3 ARM: pxa: move plat-pxa to drivers/soc/ ARM: mmp: rename pxa_register_device ARM: mmp: remove tavorevb board support ARM: pxa: remove unused mach/bitfield.h ARM: pxa: move clk register definitions to driver ARM: pxa: move smemc register access from clk to platform cpufreq: pxa3: move clk register access to clk driver ARM: pxa: remove get_clk_frequency_khz() ARM: pxa: pcmcia: move smemc configuration back to arch ASoC: pxa: i2s: use normal MMIO accessors ASoC: pxa: ac97: use normal MMIO accessors ASoC: pxa: use pdev resource for FIFO regs Input: wm97xx - get rid of irq_enable method in wm97xx_mach_ops Input: wm97xx - switch to using threaded IRQ ...
2022-05-26Merge tag 'arm-multiplatform-5.19-1' of ↵Linus Torvalds1-10/+18
git://git.kernel.org/pub/scm/linux/kernel/git/soc/soc Pull ARMv4T/v5 multiplatform support from Arnd Bergmann: "This series has been 12 years in the making, it mostly finishes the work that was started with the founding of Linaro to clean up platform support in the kernel. The largest change here is a cleanup of the omap1 platform, which is the final ARM machine type to get converted to the common-clk subsystem. All the omap1 specific drivers are now made independent of the mach/*.h headers to allow the platform to be part of a generic ARMv4/v5 multiplatform kernel. The last bit that enables this support is still missing here while we wait for some last dependencies to make it into the mainline kernel through other subsystems. The s3c24xx, ixp4xx, iop32x, ep93xx and dove platforms were all almost at the point of allowing multiplatform kernels, this work gets completed here along with a few additional cleanup. At the same time, the s3c24xx and s3c64xx are now deprecated and expected to get removed in the future. The PXA and OMAP1 bits are in a separate branch because of dependencies. Once both branches are merged, only the three Intel StrongARM platforms (RiscPC, Footbridge/NetWinder and StrongARM1100) need separate kernels, and there are no plans to include these" * tag 'arm-multiplatform-5.19-1' of git://git.kernel.org/pub/scm/linux/kernel/git/soc/soc: (61 commits) ARM: ixp4xx: Consolidate Kconfig fixing issue ARM: versatile: Add missing of_node_put in dcscb_init ARM: config: Refresh IXP4xx config after multiplatform ARM: omap1: add back omap_set_dma_priority() stub ARM: omap: fix missing declaration warnings ARM: omap: fix address space warnings from sparse ARM: spear: remove include/mach/ subdirectory ARM: davinci: remove include/mach/ subdirectory ARM: omap2: remove include/mach/ subdirectory integrator: remove empty ap_init_early() ARM: s3c: fix include path MAINTAINERS: omap1: Add Janusz as an additional maintainer ARM: omap1: htc_herald: fix typos in comments ARM: OMAP1: fix typos in comments ARM: OMAP1: clock: Remove noop code ARM: OMAP1: clock: Remove unused code ARM: OMAP1: clock: Fix UART rate reporting algorithm ARM: OMAP1: clock: Fix early UART rate issues ARM: OMAP1: Prepare for conversion of OMAP1 clocks to CCF ARM: omap1: fix build with no SoC selected ...
2022-05-26Merge tag 'arm-soc-5.19' of ↵Linus Torvalds1-1/+1
git://git.kernel.org/pub/scm/linux/kernel/git/soc/soc Pull 32-bit ARM SoC updates from Arnd Bergmann: "These updates are for platform specific code in arch/arm/, mostly fixing minor issues. The at91 platform gains support for better power management on the lan966 platform and new firmware on the sama5 platform. The mediatek soc drivers in turn are enabled for the new mt8195 SoC" * tag 'arm-soc-5.19' of git://git.kernel.org/pub/scm/linux/kernel/git/soc/soc: (34 commits) ARM: at91: debug: add lan966 support ARM: at91: pm: add support for sama5d2 secure suspend ARM: at91: add code to handle secure calls ARM: at91: Kconfig: implement PIT64B selection ARM: at91: pm: add quirks for pm ARM: at91: pm: use kernel documentation style ARM: at91: pm: introduce macros for pm mode replacement ARM: at91: pm: keep documentation inline with structure members orion5x: fix typos in comments ARM: hisi: Add missing of_node_put after of_find_compatible_node ARM: shmobile: rcar-gen2: Drop comma after OF match table sentinel ARM: shmobile: Drop commas after dt_compat sentinels soc: mediatek: mutex: remove mt8195 MOD0 and SOF0 definition MAINTAINERS: Add Broadcom BCMBCA entry arm: bcmbca: add arch bcmbca machine entry MAINTAINERS: Broadcom internal lists aren't maintainers dt-bindings: pwrap: mediatek: Update pwrap document for mt8195 soc: mediatek: add DDP_DOMPONENT_DITHER0 enum for mt8195 vdosys0 soc: mediatek: add mtk-mutex support for mt8195 vdosys0 soc: mediatek: add mtk-mmsys support for mt8195 vdosys0 ...
2022-05-26Merge tag 'dma-mapping-5.19-2022-05-25' of ↵Linus Torvalds1-5/+1
git://git.infradead.org/users/hch/dma-mapping Pull dma-mapping updates from Christoph Hellwig: - don't over-decrypt memory (Robin Murphy) - takes min align mask into account for the swiotlb max mapping size (Tianyu Lan) - use GFP_ATOMIC in dma-debug (Mikulas Patocka) - fix DMA_ATTR_NO_KERNEL_MAPPING on xen/arm (me) - don't fail on highmem CMA pages in dma_direct_alloc_pages (me) - cleanup swiotlb initialization and share more code with swiotlb-xen (me, Stefano Stabellini) * tag 'dma-mapping-5.19-2022-05-25' of git://git.infradead.org/users/hch/dma-mapping: (23 commits) dma-direct: don't over-decrypt memory swiotlb: max mapping size takes min align mask into account swiotlb: use the right nslabs-derived sizes in swiotlb_init_late swiotlb: use the right nslabs value in swiotlb_init_remap swiotlb: don't panic when the swiotlb buffer can't be allocated dma-debug: change allocation mode from GFP_NOWAIT to GFP_ATIOMIC dma-direct: don't fail on highmem CMA pages in dma_direct_alloc_pages swiotlb-xen: fix DMA_ATTR_NO_KERNEL_MAPPING on arm x86: remove cruft from <asm/dma-mapping.h> swiotlb: remove swiotlb_init_with_tbl and swiotlb_init_late_with_tbl swiotlb: merge swiotlb-xen initialization into swiotlb swiotlb: provide swiotlb_init variants that remap the buffer swiotlb: pass a gfp_mask argument to swiotlb_init_late swiotlb: add a SWIOTLB_ANY flag to lift the low memory restriction swiotlb: make the swiotlb_init interface more useful x86: centralize setting SWIOTLB_FORCE when guest memory encryption is enabled x86: remove the IOMMU table infrastructure MIPS/octeon: use swiotlb_init instead of open coding it arm/xen: don't check for xen_initial_domain() in xen_create_contiguous_region swiotlb: rename swiotlb_late_init_with_default_size ...
generated by cgit v1.2.3 (git 2.39.0) at 2024-12-28 22:24:56 +0300