From 6eea69dd8befeabd3d0f217400f54b157dd91fe9 Mon Sep 17 00:00:00 2001 From: Andrew Morton Date: Mon, 31 Oct 2011 17:06:29 -0700 Subject: include/linux/dmar.h: forward-declare struct acpi_dmar_header x86_64 allnoconfig: In file included from arch/x86/kernel/pci-dma.c:3: include/linux/dmar.h:248: warning: 'struct acpi_dmar_header' declared inside parameter list include/linux/dmar.h:248: warning: its scope is only this definition or declaration, which is probably not what you want Cc: Suresh Siddha Cc: Ingo Molnar Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds --- include/linux/dmar.h | 2 ++ 1 file changed, 2 insertions(+) (limited to 'include/linux') diff --git a/include/linux/dmar.h b/include/linux/dmar.h index a8b1a847c103..731a60975101 100644 --- a/include/linux/dmar.h +++ b/include/linux/dmar.h @@ -26,6 +26,8 @@ #include #include +struct acpi_dmar_header; + /* DMAR Flags */ #define DMAR_INTR_REMAP 0x1 #define DMAR_X2APIC_OPT_OUT 0x2 -- cgit v1.2.3 From fcf634098c00dd9cd247447368495f0b79be12d1 Mon Sep 17 00:00:00 2001 From: Christopher Yeoh Date: Mon, 31 Oct 2011 17:06:39 -0700 Subject: Cross Memory Attach The basic idea behind cross memory attach is to allow MPI programs doing intra-node communication to do a single copy of the message rather than a double copy of the message via shared memory. The following patch attempts to achieve this by allowing a destination process, given an address and size from a source process, to copy memory directly from the source process into its own address space via a system call. There is also a symmetrical ability to copy from the current process's address space into a destination process's address space. - Use of /proc/pid/mem has been considered, but there are issues with using it: - Does not allow for specifying iovecs for both src and dest, assuming preadv or pwritev was implemented either the area read from or written to would need to be contiguous. - Currently mem_read allows only processes who are currently ptrace'ing the target and are still able to ptrace the target to read from the target. This check could possibly be moved to the open call, but its not clear exactly what race this restriction is stopping (reason appears to have been lost) - Having to send the fd of /proc/self/mem via SCM_RIGHTS on unix domain socket is a bit ugly from a userspace point of view, especially when you may have hundreds if not (eventually) thousands of processes that all need to do this with each other - Doesn't allow for some future use of the interface we would like to consider adding in the future (see below) - Interestingly reading from /proc/pid/mem currently actually involves two copies! (But this could be fixed pretty easily) As mentioned previously use of vmsplice instead was considered, but has problems. Since you need the reader and writer working co-operatively if the pipe is not drained then you block. Which requires some wrapping to do non blocking on the send side or polling on the receive. In all to all communication it requires ordering otherwise you can deadlock. And in the example of many MPI tasks writing to one MPI task vmsplice serialises the copying. There are some cases of MPI collectives where even a single copy interface does not get us the performance gain we could. For example in an MPI_Reduce rather than copy the data from the source we would like to instead use it directly in a mathops (say the reduce is doing a sum) as this would save us doing a copy. We don't need to keep a copy of the data from the source. I haven't implemented this, but I think this interface could in the future do all this through the use of the flags - eg could specify the math operation and type and the kernel rather than just copying the data would apply the specified operation between the source and destination and store it in the destination. Although we don't have a "second user" of the interface (though I've had some nibbles from people who may be interested in using it for intra process messaging which is not MPI). This interface is something which hardware vendors are already doing for their custom drivers to implement fast local communication. And so in addition to this being useful for OpenMPI it would mean the driver maintainers don't have to fix things up when the mm changes. There was some discussion about how much faster a true zero copy would go. Here's a link back to the email with some testing I did on that: http://marc.info/?l=linux-mm&m=130105930902915&w=2 There is a basic man page for the proposed interface here: http://ozlabs.org/~cyeoh/cma/process_vm_readv.txt This has been implemented for x86 and powerpc, other architecture should mainly (I think) just need to add syscall numbers for the process_vm_readv and process_vm_writev. There are 32 bit compatibility versions for 64-bit kernels. For arch maintainers there are some simple tests to be able to quickly verify that the syscalls are working correctly here: http://ozlabs.org/~cyeoh/cma/cma-test-20110718.tgz Signed-off-by: Chris Yeoh Cc: Ingo Molnar Cc: "H. Peter Anvin" Cc: Thomas Gleixner Cc: Arnd Bergmann Cc: Paul Mackerras Cc: Benjamin Herrenschmidt Cc: David Howells Cc: James Morris Cc: Cc: Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds --- arch/powerpc/include/asm/systbl.h | 2 + arch/powerpc/include/asm/unistd.h | 4 +- arch/x86/ia32/ia32entry.S | 2 + arch/x86/include/asm/unistd_32.h | 4 +- arch/x86/include/asm/unistd_64.h | 4 + arch/x86/kernel/syscall_table_32.S | 2 + fs/aio.c | 4 +- fs/compat.c | 7 +- fs/read_write.c | 8 +- include/linux/compat.h | 3 +- include/linux/fs.h | 7 +- include/linux/syscalls.h | 13 + kernel/sys_ni.c | 4 + mm/Makefile | 3 +- mm/process_vm_access.c | 496 +++++++++++++++++++++++++++++++++++++ security/keys/compat.c | 2 +- security/keys/keyctl.c | 2 +- 17 files changed, 550 insertions(+), 17 deletions(-) create mode 100644 mm/process_vm_access.c (limited to 'include/linux') diff --git a/arch/powerpc/include/asm/systbl.h b/arch/powerpc/include/asm/systbl.h index fa0d27a400de..559ae1ee6706 100644 --- a/arch/powerpc/include/asm/systbl.h +++ b/arch/powerpc/include/asm/systbl.h @@ -354,3 +354,5 @@ COMPAT_SYS_SPU(clock_adjtime) SYSCALL_SPU(syncfs) COMPAT_SYS_SPU(sendmmsg) SYSCALL_SPU(setns) +COMPAT_SYS(process_vm_readv) +COMPAT_SYS(process_vm_writev) diff --git a/arch/powerpc/include/asm/unistd.h b/arch/powerpc/include/asm/unistd.h index b8b3f599362b..d3d1b5efd7eb 100644 --- a/arch/powerpc/include/asm/unistd.h +++ b/arch/powerpc/include/asm/unistd.h @@ -373,10 +373,12 @@ #define __NR_syncfs 348 #define __NR_sendmmsg 349 #define __NR_setns 350 +#define __NR_process_vm_readv 351 +#define __NR_process_vm_writev 352 #ifdef __KERNEL__ -#define __NR_syscalls 351 +#define __NR_syscalls 353 #define __NR__exit __NR_exit #define NR_syscalls __NR_syscalls diff --git a/arch/x86/ia32/ia32entry.S b/arch/x86/ia32/ia32entry.S index 54edb207ff3a..a6253ec1b284 100644 --- a/arch/x86/ia32/ia32entry.S +++ b/arch/x86/ia32/ia32entry.S @@ -850,4 +850,6 @@ ia32_sys_call_table: .quad sys_syncfs .quad compat_sys_sendmmsg /* 345 */ .quad sys_setns + .quad compat_sys_process_vm_readv + .quad compat_sys_process_vm_writev ia32_syscall_end: diff --git a/arch/x86/include/asm/unistd_32.h b/arch/x86/include/asm/unistd_32.h index 593485b38ab3..599c77d38f33 100644 --- a/arch/x86/include/asm/unistd_32.h +++ b/arch/x86/include/asm/unistd_32.h @@ -352,10 +352,12 @@ #define __NR_syncfs 344 #define __NR_sendmmsg 345 #define __NR_setns 346 +#define __NR_process_vm_readv 347 +#define __NR_process_vm_writev 348 #ifdef __KERNEL__ -#define NR_syscalls 347 +#define NR_syscalls 349 #define __ARCH_WANT_IPC_PARSE_VERSION #define __ARCH_WANT_OLD_READDIR diff --git a/arch/x86/include/asm/unistd_64.h b/arch/x86/include/asm/unistd_64.h index 0a6ba337a2eb..0431f193c3f2 100644 --- a/arch/x86/include/asm/unistd_64.h +++ b/arch/x86/include/asm/unistd_64.h @@ -682,6 +682,10 @@ __SYSCALL(__NR_sendmmsg, sys_sendmmsg) __SYSCALL(__NR_setns, sys_setns) #define __NR_getcpu 309 __SYSCALL(__NR_getcpu, sys_getcpu) +#define __NR_process_vm_readv 310 +__SYSCALL(__NR_process_vm_readv, sys_process_vm_readv) +#define __NR_process_vm_writev 311 +__SYSCALL(__NR_process_vm_writev, sys_process_vm_writev) #ifndef __NO_STUBS #define __ARCH_WANT_OLD_READDIR diff --git a/arch/x86/kernel/syscall_table_32.S b/arch/x86/kernel/syscall_table_32.S index bc19be332bc9..9a0e31293920 100644 --- a/arch/x86/kernel/syscall_table_32.S +++ b/arch/x86/kernel/syscall_table_32.S @@ -346,3 +346,5 @@ ENTRY(sys_call_table) .long sys_syncfs .long sys_sendmmsg /* 345 */ .long sys_setns + .long sys_process_vm_readv + .long sys_process_vm_writev diff --git a/fs/aio.c b/fs/aio.c index e29ec485af25..632b235f4fbe 100644 --- a/fs/aio.c +++ b/fs/aio.c @@ -1387,13 +1387,13 @@ static ssize_t aio_setup_vectored_rw(int type, struct kiocb *kiocb, bool compat) ret = compat_rw_copy_check_uvector(type, (struct compat_iovec __user *)kiocb->ki_buf, kiocb->ki_nbytes, 1, &kiocb->ki_inline_vec, - &kiocb->ki_iovec); + &kiocb->ki_iovec, 1); else #endif ret = rw_copy_check_uvector(type, (struct iovec __user *)kiocb->ki_buf, kiocb->ki_nbytes, 1, &kiocb->ki_inline_vec, - &kiocb->ki_iovec); + &kiocb->ki_iovec, 1); if (ret < 0) goto out; diff --git a/fs/compat.c b/fs/compat.c index 302e761bd0aa..c98787536bb8 100644 --- a/fs/compat.c +++ b/fs/compat.c @@ -546,7 +546,7 @@ out: ssize_t compat_rw_copy_check_uvector(int type, const struct compat_iovec __user *uvector, unsigned long nr_segs, unsigned long fast_segs, struct iovec *fast_pointer, - struct iovec **ret_pointer) + struct iovec **ret_pointer, int check_access) { compat_ssize_t tot_len; struct iovec *iov = *ret_pointer = fast_pointer; @@ -593,7 +593,8 @@ ssize_t compat_rw_copy_check_uvector(int type, } if (len < 0) /* size_t not fitting in compat_ssize_t .. */ goto out; - if (!access_ok(vrfy_dir(type), compat_ptr(buf), len)) { + if (check_access && + !access_ok(vrfy_dir(type), compat_ptr(buf), len)) { ret = -EFAULT; goto out; } @@ -1107,7 +1108,7 @@ static ssize_t compat_do_readv_writev(int type, struct file *file, goto out; tot_len = compat_rw_copy_check_uvector(type, uvector, nr_segs, - UIO_FASTIOV, iovstack, &iov); + UIO_FASTIOV, iovstack, &iov, 1); if (tot_len == 0) { ret = 0; goto out; diff --git a/fs/read_write.c b/fs/read_write.c index dfd125798791..5ad4248b0cd8 100644 --- a/fs/read_write.c +++ b/fs/read_write.c @@ -633,7 +633,8 @@ ssize_t do_loop_readv_writev(struct file *filp, struct iovec *iov, ssize_t rw_copy_check_uvector(int type, const struct iovec __user * uvector, unsigned long nr_segs, unsigned long fast_segs, struct iovec *fast_pointer, - struct iovec **ret_pointer) + struct iovec **ret_pointer, + int check_access) { unsigned long seg; ssize_t ret; @@ -689,7 +690,8 @@ ssize_t rw_copy_check_uvector(int type, const struct iovec __user * uvector, ret = -EINVAL; goto out; } - if (unlikely(!access_ok(vrfy_dir(type), buf, len))) { + if (check_access + && unlikely(!access_ok(vrfy_dir(type), buf, len))) { ret = -EFAULT; goto out; } @@ -721,7 +723,7 @@ static ssize_t do_readv_writev(int type, struct file *file, } ret = rw_copy_check_uvector(type, uvector, nr_segs, - ARRAY_SIZE(iovstack), iovstack, &iov); + ARRAY_SIZE(iovstack), iovstack, &iov, 1); if (ret <= 0) goto out; diff --git a/include/linux/compat.h b/include/linux/compat.h index c6e7523bf765..154bf5683015 100644 --- a/include/linux/compat.h +++ b/include/linux/compat.h @@ -547,7 +547,8 @@ extern ssize_t compat_rw_copy_check_uvector(int type, const struct compat_iovec __user *uvector, unsigned long nr_segs, unsigned long fast_segs, struct iovec *fast_pointer, - struct iovec **ret_pointer); + struct iovec **ret_pointer, + int check_access); extern void __user *compat_alloc_user_space(unsigned long len); diff --git a/include/linux/fs.h b/include/linux/fs.h index 14493a2d5a03..87b4c6b9692d 100644 --- a/include/linux/fs.h +++ b/include/linux/fs.h @@ -1633,9 +1633,10 @@ struct inode_operations { struct seq_file; ssize_t rw_copy_check_uvector(int type, const struct iovec __user * uvector, - unsigned long nr_segs, unsigned long fast_segs, - struct iovec *fast_pointer, - struct iovec **ret_pointer); + unsigned long nr_segs, unsigned long fast_segs, + struct iovec *fast_pointer, + struct iovec **ret_pointer, + int check_access); extern ssize_t vfs_read(struct file *, char __user *, size_t, loff_t *); extern ssize_t vfs_write(struct file *, const char __user *, size_t, loff_t *); diff --git a/include/linux/syscalls.h b/include/linux/syscalls.h index 1ff0ec2a5e8d..86a24b1166d1 100644 --- a/include/linux/syscalls.h +++ b/include/linux/syscalls.h @@ -844,4 +844,17 @@ asmlinkage long sys_open_by_handle_at(int mountdirfd, struct file_handle __user *handle, int flags); asmlinkage long sys_setns(int fd, int nstype); +asmlinkage long sys_process_vm_readv(pid_t pid, + const struct iovec __user *lvec, + unsigned long liovcnt, + const struct iovec __user *rvec, + unsigned long riovcnt, + unsigned long flags); +asmlinkage long sys_process_vm_writev(pid_t pid, + const struct iovec __user *lvec, + unsigned long liovcnt, + const struct iovec __user *rvec, + unsigned long riovcnt, + unsigned long flags); + #endif diff --git a/kernel/sys_ni.c b/kernel/sys_ni.c index a9a5de07c4f1..47bfa16430d7 100644 --- a/kernel/sys_ni.c +++ b/kernel/sys_ni.c @@ -145,6 +145,10 @@ cond_syscall(sys_io_submit); cond_syscall(sys_io_cancel); cond_syscall(sys_io_getevents); cond_syscall(sys_syslog); +cond_syscall(sys_process_vm_readv); +cond_syscall(sys_process_vm_writev); +cond_syscall(compat_sys_process_vm_readv); +cond_syscall(compat_sys_process_vm_writev); /* arch-specific weak syscall entries */ cond_syscall(sys_pciconfig_read); diff --git a/mm/Makefile b/mm/Makefile index 836e4163c1bf..50ec00ef2a0e 100644 --- a/mm/Makefile +++ b/mm/Makefile @@ -5,7 +5,8 @@ mmu-y := nommu.o mmu-$(CONFIG_MMU) := fremap.o highmem.o madvise.o memory.o mincore.o \ mlock.o mmap.o mprotect.o mremap.o msync.o rmap.o \ - vmalloc.o pagewalk.o pgtable-generic.o + vmalloc.o pagewalk.o pgtable-generic.o \ + process_vm_access.o obj-y := filemap.o mempool.o oom_kill.o fadvise.o \ maccess.o page_alloc.o page-writeback.o \ diff --git a/mm/process_vm_access.c b/mm/process_vm_access.c new file mode 100644 index 000000000000..e920aa3ce104 --- /dev/null +++ b/mm/process_vm_access.c @@ -0,0 +1,496 @@ +/* + * linux/mm/process_vm_access.c + * + * Copyright (C) 2010-2011 Christopher Yeoh , IBM Corp. + * + * This program is free software; you can redistribute it and/or + * modify it under the terms of the GNU General Public License + * as published by the Free Software Foundation; either version + * 2 of the License, or (at your option) any later version. + */ + +#include +#include +#include +#include +#include +#include +#include + +#ifdef CONFIG_COMPAT +#include +#endif + +/** + * process_vm_rw_pages - read/write pages from task specified + * @task: task to read/write from + * @mm: mm for task + * @process_pages: struct pages area that can store at least + * nr_pages_to_copy struct page pointers + * @pa: address of page in task to start copying from/to + * @start_offset: offset in page to start copying from/to + * @len: number of bytes to copy + * @lvec: iovec array specifying where to copy to/from + * @lvec_cnt: number of elements in iovec array + * @lvec_current: index in iovec array we are up to + * @lvec_offset: offset in bytes from current iovec iov_base we are up to + * @vm_write: 0 means copy from, 1 means copy to + * @nr_pages_to_copy: number of pages to copy + * @bytes_copied: returns number of bytes successfully copied + * Returns 0 on success, error code otherwise + */ +static int process_vm_rw_pages(struct task_struct *task, + struct mm_struct *mm, + struct page **process_pages, + unsigned long pa, + unsigned long start_offset, + unsigned long len, + const struct iovec *lvec, + unsigned long lvec_cnt, + unsigned long *lvec_current, + size_t *lvec_offset, + int vm_write, + unsigned int nr_pages_to_copy, + ssize_t *bytes_copied) +{ + int pages_pinned; + void *target_kaddr; + int pgs_copied = 0; + int j; + int ret; + ssize_t bytes_to_copy; + ssize_t rc = 0; + + *bytes_copied = 0; + + /* Get the pages we're interested in */ + down_read(&mm->mmap_sem); + pages_pinned = get_user_pages(task, mm, pa, + nr_pages_to_copy, + vm_write, 0, process_pages, NULL); + up_read(&mm->mmap_sem); + + if (pages_pinned != nr_pages_to_copy) { + rc = -EFAULT; + goto end; + } + + /* Do the copy for each page */ + for (pgs_copied = 0; + (pgs_copied < nr_pages_to_copy) && (*lvec_current < lvec_cnt); + pgs_copied++) { + /* Make sure we have a non zero length iovec */ + while (*lvec_current < lvec_cnt + && lvec[*lvec_current].iov_len == 0) + (*lvec_current)++; + if (*lvec_current == lvec_cnt) + break; + + /* + * Will copy smallest of: + * - bytes remaining in page + * - bytes remaining in destination iovec + */ + bytes_to_copy = min_t(ssize_t, PAGE_SIZE - start_offset, + len - *bytes_copied); + bytes_to_copy = min_t(ssize_t, bytes_to_copy, + lvec[*lvec_current].iov_len + - *lvec_offset); + + target_kaddr = kmap(process_pages[pgs_copied]) + start_offset; + + if (vm_write) + ret = copy_from_user(target_kaddr, + lvec[*lvec_current].iov_base + + *lvec_offset, + bytes_to_copy); + else + ret = copy_to_user(lvec[*lvec_current].iov_base + + *lvec_offset, + target_kaddr, bytes_to_copy); + kunmap(process_pages[pgs_copied]); + if (ret) { + *bytes_copied += bytes_to_copy - ret; + pgs_copied++; + rc = -EFAULT; + goto end; + } + *bytes_copied += bytes_to_copy; + *lvec_offset += bytes_to_copy; + if (*lvec_offset == lvec[*lvec_current].iov_len) { + /* + * Need to copy remaining part of page into the + * next iovec if there are any bytes left in page + */ + (*lvec_current)++; + *lvec_offset = 0; + start_offset = (start_offset + bytes_to_copy) + % PAGE_SIZE; + if (start_offset) + pgs_copied--; + } else { + start_offset = 0; + } + } + +end: + if (vm_write) { + for (j = 0; j < pages_pinned; j++) { + if (j < pgs_copied) + set_page_dirty_lock(process_pages[j]); + put_page(process_pages[j]); + } + } else { + for (j = 0; j < pages_pinned; j++) + put_page(process_pages[j]); + } + + return rc; +} + +/* Maximum number of pages kmalloc'd to hold struct page's during copy */ +#define PVM_MAX_KMALLOC_PAGES (PAGE_SIZE * 2) + +/** + * process_vm_rw_single_vec - read/write pages from task specified + * @addr: start memory address of target process + * @len: size of area to copy to/from + * @lvec: iovec array specifying where to copy to/from locally + * @lvec_cnt: number of elements in iovec array + * @lvec_current: index in iovec array we are up to + * @lvec_offset: offset in bytes from current iovec iov_base we are up to + * @process_pages: struct pages area that can store at least + * nr_pages_to_copy struct page pointers + * @mm: mm for task + * @task: task to read/write from + * @vm_write: 0 means copy from, 1 means copy to + * @bytes_copied: returns number of bytes successfully copied + * Returns 0 on success or on failure error code + */ +static int process_vm_rw_single_vec(unsigned long addr, + unsigned long len, + const struct iovec *lvec, + unsigned long lvec_cnt, + unsigned long *lvec_current, + size_t *lvec_offset, + struct page **process_pages, + struct mm_struct *mm, + struct task_struct *task, + int vm_write, + ssize_t *bytes_copied) +{ + unsigned long pa = addr & PAGE_MASK; + unsigned long start_offset = addr - pa; + unsigned long nr_pages; + ssize_t bytes_copied_loop; + ssize_t rc = 0; + unsigned long nr_pages_copied = 0; + unsigned long nr_pages_to_copy; + unsigned long max_pages_per_loop = PVM_MAX_KMALLOC_PAGES + / sizeof(struct pages *); + + *bytes_copied = 0; + + /* Work out address and page range required */ + if (len == 0) + return 0; + nr_pages = (addr + len - 1) / PAGE_SIZE - addr / PAGE_SIZE + 1; + + while ((nr_pages_copied < nr_pages) && (*lvec_current < lvec_cnt)) { + nr_pages_to_copy = min(nr_pages - nr_pages_copied, + max_pages_per_loop); + + rc = process_vm_rw_pages(task, mm, process_pages, pa, + start_offset, len, + lvec, lvec_cnt, + lvec_current, lvec_offset, + vm_write, nr_pages_to_copy, + &bytes_copied_loop); + start_offset = 0; + *bytes_copied += bytes_copied_loop; + + if (rc < 0) { + return rc; + } else { + len -= bytes_copied_loop; + nr_pages_copied += nr_pages_to_copy; + pa += nr_pages_to_copy * PAGE_SIZE; + } + } + + return rc; +} + +/* Maximum number of entries for process pages array + which lives on stack */ +#define PVM_MAX_PP_ARRAY_COUNT 16 + +/** + * process_vm_rw_core - core of reading/writing pages from task specified + * @pid: PID of process to read/write from/to + * @lvec: iovec array specifying where to copy to/from locally + * @liovcnt: size of lvec array + * @rvec: iovec array specifying where to copy to/from in the other process + * @riovcnt: size of rvec array + * @flags: currently unused + * @vm_write: 0 if reading from other process, 1 if writing to other process + * Returns the number of bytes read/written or error code. May + * return less bytes than expected if an error occurs during the copying + * process. + */ +static ssize_t process_vm_rw_core(pid_t pid, const struct iovec *lvec, + unsigned long liovcnt, + const struct iovec *rvec, + unsigned long riovcnt, + unsigned long flags, int vm_write) +{ + struct task_struct *task; + struct page *pp_stack[PVM_MAX_PP_ARRAY_COUNT]; + struct page **process_pages = pp_stack; + struct mm_struct *mm; + unsigned long i; + ssize_t rc = 0; + ssize_t bytes_copied_loop; + ssize_t bytes_copied = 0; + unsigned long nr_pages = 0; + unsigned long nr_pages_iov; + unsigned long iov_l_curr_idx = 0; + size_t iov_l_curr_offset = 0; + ssize_t iov_len; + + /* + * Work out how many pages of struct pages we're going to need + * when eventually calling get_user_pages + */ + for (i = 0; i < riovcnt; i++) { + iov_len = rvec[i].iov_len; + if (iov_len > 0) { + nr_pages_iov = ((unsigned long)rvec[i].iov_base + + iov_len) + / PAGE_SIZE - (unsigned long)rvec[i].iov_base + / PAGE_SIZE + 1; + nr_pages = max(nr_pages, nr_pages_iov); + } + } + + if (nr_pages == 0) + return 0; + + if (nr_pages > PVM_MAX_PP_ARRAY_COUNT) { + /* For reliability don't try to kmalloc more than + 2 pages worth */ + process_pages = kmalloc(min_t(size_t, PVM_MAX_KMALLOC_PAGES, + sizeof(struct pages *)*nr_pages), + GFP_KERNEL); + + if (!process_pages) + return -ENOMEM; + } + + /* Get process information */ + rcu_read_lock(); + task = find_task_by_vpid(pid); + if (task) + get_task_struct(task); + rcu_read_unlock(); + if (!task) { + rc = -ESRCH; + goto free_proc_pages; + } + + task_lock(task); + if (__ptrace_may_access(task, PTRACE_MODE_ATTACH)) { + task_unlock(task); + rc = -EPERM; + goto put_task_struct; + } + mm = task->mm; + + if (!mm || (task->flags & PF_KTHREAD)) { + task_unlock(task); + rc = -EINVAL; + goto put_task_struct; + } + + atomic_inc(&mm->mm_users); + task_unlock(task); + + for (i = 0; i < riovcnt && iov_l_curr_idx < liovcnt; i++) { + rc = process_vm_rw_single_vec( + (unsigned long)rvec[i].iov_base, rvec[i].iov_len, + lvec, liovcnt, &iov_l_curr_idx, &iov_l_curr_offset, + process_pages, mm, task, vm_write, &bytes_copied_loop); + bytes_copied += bytes_copied_loop; + if (rc != 0) { + /* If we have managed to copy any data at all then + we return the number of bytes copied. Otherwise + we return the error code */ + if (bytes_copied) + rc = bytes_copied; + goto put_mm; + } + } + + rc = bytes_copied; +put_mm: + mmput(mm); + +put_task_struct: + put_task_struct(task); + +free_proc_pages: + if (process_pages != pp_stack) + kfree(process_pages); + return rc; +} + +/** + * process_vm_rw - check iovecs before calling core routine + * @pid: PID of process to read/write from/to + * @lvec: iovec array specifying where to copy to/from locally + * @liovcnt: size of lvec array + * @rvec: iovec array specifying where to copy to/from in the other process + * @riovcnt: size of rvec array + * @flags: currently unused + * @vm_write: 0 if reading from other process, 1 if writing to other process + * Returns the number of bytes read/written or error code. May + * return less bytes than expected if an error occurs during the copying + * process. + */ +static ssize_t process_vm_rw(pid_t pid, + const struct iovec __user *lvec, + unsigned long liovcnt, + const struct iovec __user *rvec, + unsigned long riovcnt, + unsigned long flags, int vm_write) +{ + struct iovec iovstack_l[UIO_FASTIOV]; + struct iovec iovstack_r[UIO_FASTIOV]; + struct iovec *iov_l = iovstack_l; + struct iovec *iov_r = iovstack_r; + ssize_t rc; + + if (flags != 0) + return -EINVAL; + + /* Check iovecs */ + if (vm_write) + rc = rw_copy_check_uvector(WRITE, lvec, liovcnt, UIO_FASTIOV, + iovstack_l, &iov_l, 1); + else + rc = rw_copy_check_uvector(READ, lvec, liovcnt, UIO_FASTIOV, + iovstack_l, &iov_l, 1); + if (rc <= 0) + goto free_iovecs; + + rc = rw_copy_check_uvector(READ, rvec, riovcnt, UIO_FASTIOV, + iovstack_r, &iov_r, 0); + if (rc <= 0) + goto free_iovecs; + + rc = process_vm_rw_core(pid, iov_l, liovcnt, iov_r, riovcnt, flags, + vm_write); + +free_iovecs: + if (iov_r != iovstack_r) + kfree(iov_r); + if (iov_l != iovstack_l) + kfree(iov_l); + + return rc; +} + +SYSCALL_DEFINE6(process_vm_readv, pid_t, pid, const struct iovec __user *, lvec, + unsigned long, liovcnt, const struct iovec __user *, rvec, + unsigned long, riovcnt, unsigned long, flags) +{ + return process_vm_rw(pid, lvec, liovcnt, rvec, riovcnt, flags, 0); +} + +SYSCALL_DEFINE6(process_vm_writev, pid_t, pid, + const struct iovec __user *, lvec, + unsigned long, liovcnt, const struct iovec __user *, rvec, + unsigned long, riovcnt, unsigned long, flags) +{ + return process_vm_rw(pid, lvec, liovcnt, rvec, riovcnt, flags, 1); +} + +#ifdef CONFIG_COMPAT + +asmlinkage ssize_t +compat_process_vm_rw(compat_pid_t pid, + const struct compat_iovec __user *lvec, + unsigned long liovcnt, + const struct compat_iovec __user *rvec, + unsigned long riovcnt, + unsigned long flags, int vm_write) +{ + struct iovec iovstack_l[UIO_FASTIOV]; + struct iovec iovstack_r[UIO_FASTIOV]; + struct iovec *iov_l = iovstack_l; + struct iovec *iov_r = iovstack_r; + ssize_t rc = -EFAULT; + + if (flags != 0) + return -EINVAL; + + if (!access_ok(VERIFY_READ, lvec, liovcnt * sizeof(*lvec))) + goto out; + + if (!access_ok(VERIFY_READ, rvec, riovcnt * sizeof(*rvec))) + goto out; + + if (vm_write) + rc = compat_rw_copy_check_uvector(WRITE, lvec, liovcnt, + UIO_FASTIOV, iovstack_l, + &iov_l, 1); + else + rc = compat_rw_copy_check_uvector(READ, lvec, liovcnt, + UIO_FASTIOV, iovstack_l, + &iov_l, 1); + if (rc <= 0) + goto free_iovecs; + rc = compat_rw_copy_check_uvector(READ, rvec, riovcnt, + UIO_FASTIOV, iovstack_r, + &iov_r, 0); + if (rc <= 0) + goto free_iovecs; + + rc = process_vm_rw_core(pid, iov_l, liovcnt, iov_r, riovcnt, flags, + vm_write); + +free_iovecs: + if (iov_r != iovstack_r) + kfree(iov_r); + if (iov_l != iovstack_l) + kfree(iov_l); + +out: + return rc; +} + +asmlinkage ssize_t +compat_sys_process_vm_readv(compat_pid_t pid, + const struct compat_iovec __user *lvec, + unsigned long liovcnt, + const struct compat_iovec __user *rvec, + unsigned long riovcnt, + unsigned long flags) +{ + return compat_process_vm_rw(pid, lvec, liovcnt, rvec, + riovcnt, flags, 0); +} + +asmlinkage ssize_t +compat_sys_process_vm_writev(compat_pid_t pid, + const struct compat_iovec __user *lvec, + unsigned long liovcnt, + const struct compat_iovec __user *rvec, + unsigned long riovcnt, + unsigned long flags) +{ + return compat_process_vm_rw(pid, lvec, liovcnt, rvec, + riovcnt, flags, 1); +} + +#endif diff --git a/security/keys/compat.c b/security/keys/compat.c index 338b510e9027..4c48e13448f8 100644 --- a/security/keys/compat.c +++ b/security/keys/compat.c @@ -38,7 +38,7 @@ long compat_keyctl_instantiate_key_iov( ret = compat_rw_copy_check_uvector(WRITE, _payload_iov, ioc, ARRAY_SIZE(iovstack), - iovstack, &iov); + iovstack, &iov, 1); if (ret < 0) return ret; if (ret == 0) diff --git a/security/keys/keyctl.c b/security/keys/keyctl.c index eca51918c951..0b3f5d72af1c 100644 --- a/security/keys/keyctl.c +++ b/security/keys/keyctl.c @@ -1065,7 +1065,7 @@ long keyctl_instantiate_key_iov(key_serial_t id, goto no_payload; ret = rw_copy_check_uvector(WRITE, _payload_iov, ioc, - ARRAY_SIZE(iovstack), iovstack, &iov); + ARRAY_SIZE(iovstack), iovstack, &iov, 1); if (ret < 0) return ret; if (ret == 0) -- cgit v1.2.3 From 4356f21d09283dc6d39a6f7287a65ddab61e2808 Mon Sep 17 00:00:00 2001 From: Minchan Kim Date: Mon, 31 Oct 2011 17:06:47 -0700 Subject: mm: change isolate mode from #define to bitwise type Change ISOLATE_XXX macro with bitwise isolate_mode_t type. Normally, macro isn't recommended as it's type-unsafe and making debugging harder as symbol cannot be passed throught to the debugger. Quote from Johannes " Hmm, it would probably be cleaner to fully convert the isolation mode into independent flags. INACTIVE, ACTIVE, BOTH is currently a tri-state among flags, which is a bit ugly." This patch moves isolate mode from swap.h to mmzone.h by memcontrol.h Signed-off-by: Minchan Kim Cc: Johannes Weiner Cc: KAMEZAWA Hiroyuki Cc: KOSAKI Motohiro Cc: Mel Gorman Cc: Rik van Riel Cc: Michal Hocko Cc: Andrea Arcangeli Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds --- .../trace/postprocess/trace-vmscan-postprocess.pl | 8 ++--- include/linux/memcontrol.h | 3 +- include/linux/mmzone.h | 8 +++++ include/linux/swap.h | 7 +--- include/trace/events/vmscan.h | 8 ++--- mm/compaction.c | 3 +- mm/memcontrol.c | 3 +- mm/vmscan.c | 37 ++++++++++++---------- 8 files changed, 43 insertions(+), 34 deletions(-) (limited to 'include/linux') diff --git a/Documentation/trace/postprocess/trace-vmscan-postprocess.pl b/Documentation/trace/postprocess/trace-vmscan-postprocess.pl index 12cecc83cd91..4a37c4759cd2 100644 --- a/Documentation/trace/postprocess/trace-vmscan-postprocess.pl +++ b/Documentation/trace/postprocess/trace-vmscan-postprocess.pl @@ -379,10 +379,10 @@ EVENT_PROCESS: # To closer match vmstat scanning statistics, only count isolate_both # and isolate_inactive as scanning. isolate_active is rotation - # isolate_inactive == 0 - # isolate_active == 1 - # isolate_both == 2 - if ($isolate_mode != 1) { + # isolate_inactive == 1 + # isolate_active == 2 + # isolate_both == 3 + if ($isolate_mode != 2) { $perprocesspid{$process_pid}->{HIGH_NR_SCANNED} += $nr_scanned; } $perprocesspid{$process_pid}->{HIGH_NR_CONTIG_DIRTY} += $nr_contig_dirty; diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index 343bd7661f2a..ac797fa03ef8 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -35,7 +35,8 @@ enum mem_cgroup_page_stat_item { extern unsigned long mem_cgroup_isolate_pages(unsigned long nr_to_scan, struct list_head *dst, unsigned long *scanned, int order, - int mode, struct zone *z, + isolate_mode_t mode, + struct zone *z, struct mem_cgroup *mem_cont, int active, int file); diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h index be1ac8d7789b..436ce6e7a446 100644 --- a/include/linux/mmzone.h +++ b/include/linux/mmzone.h @@ -164,6 +164,14 @@ static inline int is_unevictable_lru(enum lru_list l) #define LRU_ALL_EVICTABLE (LRU_ALL_FILE | LRU_ALL_ANON) #define LRU_ALL ((1 << NR_LRU_LISTS) - 1) +/* Isolate inactive pages */ +#define ISOLATE_INACTIVE ((__force isolate_mode_t)0x1) +/* Isolate active pages */ +#define ISOLATE_ACTIVE ((__force isolate_mode_t)0x2) + +/* LRU Isolation modes. */ +typedef unsigned __bitwise__ isolate_mode_t; + enum zone_watermarks { WMARK_MIN, WMARK_LOW, diff --git a/include/linux/swap.h b/include/linux/swap.h index c71f84bb62ec..1e22e126d2ac 100644 --- a/include/linux/swap.h +++ b/include/linux/swap.h @@ -243,15 +243,10 @@ static inline void lru_cache_add_file(struct page *page) __lru_cache_add(page, LRU_INACTIVE_FILE); } -/* LRU Isolation modes. */ -#define ISOLATE_INACTIVE 0 /* Isolate inactive pages. */ -#define ISOLATE_ACTIVE 1 /* Isolate active pages. */ -#define ISOLATE_BOTH 2 /* Isolate both active and inactive pages. */ - /* linux/mm/vmscan.c */ extern unsigned long try_to_free_pages(struct zonelist *zonelist, int order, gfp_t gfp_mask, nodemask_t *mask); -extern int __isolate_lru_page(struct page *page, int mode, int file); +extern int __isolate_lru_page(struct page *page, isolate_mode_t mode, int file); extern unsigned long try_to_free_mem_cgroup_pages(struct mem_cgroup *mem, gfp_t gfp_mask, bool noswap); extern unsigned long mem_cgroup_shrink_node_zone(struct mem_cgroup *mem, diff --git a/include/trace/events/vmscan.h b/include/trace/events/vmscan.h index 36851f7f13da..edc4b3d25a2d 100644 --- a/include/trace/events/vmscan.h +++ b/include/trace/events/vmscan.h @@ -266,7 +266,7 @@ DECLARE_EVENT_CLASS(mm_vmscan_lru_isolate_template, unsigned long nr_lumpy_taken, unsigned long nr_lumpy_dirty, unsigned long nr_lumpy_failed, - int isolate_mode), + isolate_mode_t isolate_mode), TP_ARGS(order, nr_requested, nr_scanned, nr_taken, nr_lumpy_taken, nr_lumpy_dirty, nr_lumpy_failed, isolate_mode), @@ -278,7 +278,7 @@ DECLARE_EVENT_CLASS(mm_vmscan_lru_isolate_template, __field(unsigned long, nr_lumpy_taken) __field(unsigned long, nr_lumpy_dirty) __field(unsigned long, nr_lumpy_failed) - __field(int, isolate_mode) + __field(isolate_mode_t, isolate_mode) ), TP_fast_assign( @@ -312,7 +312,7 @@ DEFINE_EVENT(mm_vmscan_lru_isolate_template, mm_vmscan_lru_isolate, unsigned long nr_lumpy_taken, unsigned long nr_lumpy_dirty, unsigned long nr_lumpy_failed, - int isolate_mode), + isolate_mode_t isolate_mode), TP_ARGS(order, nr_requested, nr_scanned, nr_taken, nr_lumpy_taken, nr_lumpy_dirty, nr_lumpy_failed, isolate_mode) @@ -327,7 +327,7 @@ DEFINE_EVENT(mm_vmscan_lru_isolate_template, mm_vmscan_memcg_isolate, unsigned long nr_lumpy_taken, unsigned long nr_lumpy_dirty, unsigned long nr_lumpy_failed, - int isolate_mode), + isolate_mode_t isolate_mode), TP_ARGS(order, nr_requested, nr_scanned, nr_taken, nr_lumpy_taken, nr_lumpy_dirty, nr_lumpy_failed, isolate_mode) diff --git a/mm/compaction.c b/mm/compaction.c index b2977a5d659a..47f717fa4233 100644 --- a/mm/compaction.c +++ b/mm/compaction.c @@ -349,7 +349,8 @@ static isolate_migrate_t isolate_migratepages(struct zone *zone, } /* Try isolate the page */ - if (__isolate_lru_page(page, ISOLATE_BOTH, 0) != 0) + if (__isolate_lru_page(page, + ISOLATE_ACTIVE|ISOLATE_INACTIVE, 0) != 0) continue; VM_BUG_ON(PageTransCompound(page)); diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 3508777837c7..2d5755544afe 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -1185,7 +1185,8 @@ mem_cgroup_get_reclaim_stat_from_page(struct page *page) unsigned long mem_cgroup_isolate_pages(unsigned long nr_to_scan, struct list_head *dst, unsigned long *scanned, int order, - int mode, struct zone *z, + isolate_mode_t mode, + struct zone *z, struct mem_cgroup *mem_cont, int active, int file) { diff --git a/mm/vmscan.c b/mm/vmscan.c index 9fdfce7ba403..ec6dbcb976d1 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -1012,23 +1012,27 @@ keep_lumpy: * * returns 0 on success, -ve errno on failure. */ -int __isolate_lru_page(struct page *page, int mode, int file) +int __isolate_lru_page(struct page *page, isolate_mode_t mode, int file) { + bool all_lru_mode; int ret = -EINVAL; /* Only take pages on the LRU. */ if (!PageLRU(page)) return ret; + all_lru_mode = (mode & (ISOLATE_ACTIVE|ISOLATE_INACTIVE)) == + (ISOLATE_ACTIVE|ISOLATE_INACTIVE); + /* * When checking the active state, we need to be sure we are * dealing with comparible boolean values. Take the logical not * of each. */ - if (mode != ISOLATE_BOTH && (!PageActive(page) != !mode)) + if (!all_lru_mode && !PageActive(page) != !(mode & ISOLATE_ACTIVE)) return ret; - if (mode != ISOLATE_BOTH && page_is_file_cache(page) != file) + if (!all_lru_mode && !!page_is_file_cache(page) != file) return ret; /* @@ -1076,7 +1080,8 @@ int __isolate_lru_page(struct page *page, int mode, int file) */ static unsigned long isolate_lru_pages(unsigned long nr_to_scan, struct list_head *src, struct list_head *dst, - unsigned long *scanned, int order, int mode, int file) + unsigned long *scanned, int order, isolate_mode_t mode, + int file) { unsigned long nr_taken = 0; unsigned long nr_lumpy_taken = 0; @@ -1201,8 +1206,8 @@ static unsigned long isolate_lru_pages(unsigned long nr_to_scan, static unsigned long isolate_pages_global(unsigned long nr, struct list_head *dst, unsigned long *scanned, int order, - int mode, struct zone *z, - int active, int file) + isolate_mode_t mode, + struct zone *z, int active, int file) { int lru = LRU_BASE; if (active) @@ -1448,6 +1453,7 @@ shrink_inactive_list(unsigned long nr_to_scan, struct zone *zone, unsigned long nr_taken; unsigned long nr_anon; unsigned long nr_file; + isolate_mode_t reclaim_mode = ISOLATE_INACTIVE; while (unlikely(too_many_isolated(zone, file, sc))) { congestion_wait(BLK_RW_ASYNC, HZ/10); @@ -1458,15 +1464,15 @@ shrink_inactive_list(unsigned long nr_to_scan, struct zone *zone, } set_reclaim_mode(priority, sc, false); + if (sc->reclaim_mode & RECLAIM_MODE_LUMPYRECLAIM) + reclaim_mode |= ISOLATE_ACTIVE; + lru_add_drain(); spin_lock_irq(&zone->lru_lock); if (scanning_global_lru(sc)) { - nr_taken = isolate_pages_global(nr_to_scan, - &page_list, &nr_scanned, sc->order, - sc->reclaim_mode & RECLAIM_MODE_LUMPYRECLAIM ? - ISOLATE_BOTH : ISOLATE_INACTIVE, - zone, 0, file); + nr_taken = isolate_pages_global(nr_to_scan, &page_list, + &nr_scanned, sc->order, reclaim_mode, zone, 0, file); zone->pages_scanned += nr_scanned; if (current_is_kswapd()) __count_zone_vm_events(PGSCAN_KSWAPD, zone, @@ -1475,12 +1481,9 @@ shrink_inactive_list(unsigned long nr_to_scan, struct zone *zone, __count_zone_vm_events(PGSCAN_DIRECT, zone, nr_scanned); } else { - nr_taken = mem_cgroup_isolate_pages(nr_to_scan, - &page_list, &nr_scanned, sc->order, - sc->reclaim_mode & RECLAIM_MODE_LUMPYRECLAIM ? - ISOLATE_BOTH : ISOLATE_INACTIVE, - zone, sc->mem_cgroup, - 0, file); + nr_taken = mem_cgroup_isolate_pages(nr_to_scan, &page_list, + &nr_scanned, sc->order, reclaim_mode, zone, + sc->mem_cgroup, 0, file); /* * mem_cgroup_isolate_pages() keeps track of * scanned pages on its own. -- cgit v1.2.3 From 39deaf8585152f1a35c1676d3d7dc6ae0fb65967 Mon Sep 17 00:00:00 2001 From: Minchan Kim Date: Mon, 31 Oct 2011 17:06:51 -0700 Subject: mm: compaction: make isolate_lru_page() filter-aware In async mode, compaction doesn't migrate dirty or writeback pages. So, it's meaningless to pick the page and re-add it to lru list. Of course, when we isolate the page in compaction, the page might be dirty or writeback but when we try to migrate the page, the page would be not dirty, writeback. So it could be migrated. But it's very unlikely as isolate and migration cycle is much faster than writeout. So, this patch helps cpu overhead and prevent unnecessary LRU churning. Signed-off-by: Minchan Kim Acked-by: Johannes Weiner Reviewed-by: KAMEZAWA Hiroyuki Reviewed-by: KOSAKI Motohiro Acked-by: Mel Gorman Acked-by: Rik van Riel Reviewed-by: Michal Hocko Cc: Andrea Arcangeli Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds --- include/linux/mmzone.h | 2 ++ mm/compaction.c | 7 +++++-- mm/vmscan.c | 3 +++ 3 files changed, 10 insertions(+), 2 deletions(-) (limited to 'include/linux') diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h index 436ce6e7a446..80da968798ea 100644 --- a/include/linux/mmzone.h +++ b/include/linux/mmzone.h @@ -168,6 +168,8 @@ static inline int is_unevictable_lru(enum lru_list l) #define ISOLATE_INACTIVE ((__force isolate_mode_t)0x1) /* Isolate active pages */ #define ISOLATE_ACTIVE ((__force isolate_mode_t)0x2) +/* Isolate clean file */ +#define ISOLATE_CLEAN ((__force isolate_mode_t)0x4) /* LRU Isolation modes. */ typedef unsigned __bitwise__ isolate_mode_t; diff --git a/mm/compaction.c b/mm/compaction.c index 47f717fa4233..a0e420207ebf 100644 --- a/mm/compaction.c +++ b/mm/compaction.c @@ -261,6 +261,7 @@ static isolate_migrate_t isolate_migratepages(struct zone *zone, unsigned long last_pageblock_nr = 0, pageblock_nr; unsigned long nr_scanned = 0, nr_isolated = 0; struct list_head *migratelist = &cc->migratepages; + isolate_mode_t mode = ISOLATE_ACTIVE|ISOLATE_INACTIVE; /* Do not scan outside zone boundaries */ low_pfn = max(cc->migrate_pfn, zone->zone_start_pfn); @@ -348,9 +349,11 @@ static isolate_migrate_t isolate_migratepages(struct zone *zone, continue; } + if (!cc->sync) + mode |= ISOLATE_CLEAN; + /* Try isolate the page */ - if (__isolate_lru_page(page, - ISOLATE_ACTIVE|ISOLATE_INACTIVE, 0) != 0) + if (__isolate_lru_page(page, mode, 0) != 0) continue; VM_BUG_ON(PageTransCompound(page)); diff --git a/mm/vmscan.c b/mm/vmscan.c index ec6dbcb976d1..c007e78d7078 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -1045,6 +1045,9 @@ int __isolate_lru_page(struct page *page, isolate_mode_t mode, int file) ret = -EBUSY; + if ((mode & ISOLATE_CLEAN) && (PageDirty(page) || PageWriteback(page))) + return ret; + if (likely(get_page_unless_zero(page))) { /* * Be careful not to clear PageLRU until after we're -- cgit v1.2.3 From f80c0673610e36ae29d63e3297175e22f70dde5f Mon Sep 17 00:00:00 2001 From: Minchan Kim Date: Mon, 31 Oct 2011 17:06:55 -0700 Subject: mm: zone_reclaim: make isolate_lru_page() filter-aware In __zone_reclaim case, we don't want to shrink mapped page. Nonetheless, we have isolated mapped page and re-add it into LRU's head. It's unnecessary CPU overhead and makes LRU churning. Of course, when we isolate the page, the page might be mapped but when we try to migrate the page, the page would be not mapped. So it could be migrated. But race is rare and although it happens, it's no big deal. Signed-off-by: Minchan Kim Acked-by: Johannes Weiner Reviewed-by: KAMEZAWA Hiroyuki Reviewed-by: KOSAKI Motohiro Reviewed-by: Michal Hocko Cc: Mel Gorman Cc: Rik van Riel Cc: Andrea Arcangeli Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds --- include/linux/mmzone.h | 2 ++ mm/vmscan.c | 20 ++++++++++++++++++-- 2 files changed, 20 insertions(+), 2 deletions(-) (limited to 'include/linux') diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h index 80da968798ea..ec57779c5a57 100644 --- a/include/linux/mmzone.h +++ b/include/linux/mmzone.h @@ -170,6 +170,8 @@ static inline int is_unevictable_lru(enum lru_list l) #define ISOLATE_ACTIVE ((__force isolate_mode_t)0x2) /* Isolate clean file */ #define ISOLATE_CLEAN ((__force isolate_mode_t)0x4) +/* Isolate unmapped file */ +#define ISOLATE_UNMAPPED ((__force isolate_mode_t)0x8) /* LRU Isolation modes. */ typedef unsigned __bitwise__ isolate_mode_t; diff --git a/mm/vmscan.c b/mm/vmscan.c index c007e78d7078..b68a9342d5a3 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -1048,6 +1048,9 @@ int __isolate_lru_page(struct page *page, isolate_mode_t mode, int file) if ((mode & ISOLATE_CLEAN) && (PageDirty(page) || PageWriteback(page))) return ret; + if ((mode & ISOLATE_UNMAPPED) && page_mapped(page)) + return ret; + if (likely(get_page_unless_zero(page))) { /* * Be careful not to clear PageLRU until after we're @@ -1471,6 +1474,12 @@ shrink_inactive_list(unsigned long nr_to_scan, struct zone *zone, reclaim_mode |= ISOLATE_ACTIVE; lru_add_drain(); + + if (!sc->may_unmap) + reclaim_mode |= ISOLATE_UNMAPPED; + if (!sc->may_writepage) + reclaim_mode |= ISOLATE_CLEAN; + spin_lock_irq(&zone->lru_lock); if (scanning_global_lru(sc)) { @@ -1588,19 +1597,26 @@ static void shrink_active_list(unsigned long nr_pages, struct zone *zone, struct page *page; struct zone_reclaim_stat *reclaim_stat = get_reclaim_stat(zone, sc); unsigned long nr_rotated = 0; + isolate_mode_t reclaim_mode = ISOLATE_ACTIVE; lru_add_drain(); + + if (!sc->may_unmap) + reclaim_mode |= ISOLATE_UNMAPPED; + if (!sc->may_writepage) + reclaim_mode |= ISOLATE_CLEAN; + spin_lock_irq(&zone->lru_lock); if (scanning_global_lru(sc)) { nr_taken = isolate_pages_global(nr_pages, &l_hold, &pgscanned, sc->order, - ISOLATE_ACTIVE, zone, + reclaim_mode, zone, 1, file); zone->pages_scanned += pgscanned; } else { nr_taken = mem_cgroup_isolate_pages(nr_pages, &l_hold, &pgscanned, sc->order, - ISOLATE_ACTIVE, zone, + reclaim_mode, zone, sc->mem_cgroup, 1, file); /* * mem_cgroup_isolate_pages() keeps track of -- cgit v1.2.3 From c9f01245b6a7d77d17deaa71af10f6aca14fa24e Mon Sep 17 00:00:00 2001 From: David Rientjes Date: Mon, 31 Oct 2011 17:07:15 -0700 Subject: oom: remove oom_disable_count This removes mm->oom_disable_count entirely since it's unnecessary and currently buggy. The counter was intended to be per-process but it's currently decremented in the exit path for each thread that exits, causing it to underflow. The count was originally intended to prevent oom killing threads that share memory with threads that cannot be killed since it doesn't lead to future memory freeing. The counter could be fixed to represent all threads sharing the same mm, but it's better to remove the count since: - it is possible that the OOM_DISABLE thread sharing memory with the victim is waiting on that thread to exit and will actually cause future memory freeing, and - there is no guarantee that a thread is disabled from oom killing just because another thread sharing its mm is oom disabled. Signed-off-by: David Rientjes Reported-by: Oleg Nesterov Reviewed-by: Oleg Nesterov Cc: Ying Han Cc: KOSAKI Motohiro Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds --- fs/exec.c | 4 ---- fs/proc/base.c | 13 ------------- include/linux/mm_types.h | 3 --- kernel/exit.c | 2 -- kernel/fork.c | 10 +--------- mm/oom_kill.c | 23 +++++------------------ 6 files changed, 6 insertions(+), 49 deletions(-) (limited to 'include/linux') diff --git a/fs/exec.c b/fs/exec.c index 25dcbe5fc356..36254645b7cc 100644 --- a/fs/exec.c +++ b/fs/exec.c @@ -841,10 +841,6 @@ static int exec_mmap(struct mm_struct *mm) tsk->mm = mm; tsk->active_mm = mm; activate_mm(active_mm, mm); - if (old_mm && tsk->signal->oom_score_adj == OOM_SCORE_ADJ_MIN) { - atomic_dec(&old_mm->oom_disable_count); - atomic_inc(&tsk->mm->oom_disable_count); - } task_unlock(tsk); arch_pick_mmap_layout(mm); if (old_mm) { diff --git a/fs/proc/base.c b/fs/proc/base.c index 5eb02069e1b8..8f0087e20e16 100644 --- a/fs/proc/base.c +++ b/fs/proc/base.c @@ -1107,13 +1107,6 @@ static ssize_t oom_adjust_write(struct file *file, const char __user *buf, goto err_sighand; } - if (oom_adjust != task->signal->oom_adj) { - if (oom_adjust == OOM_DISABLE) - atomic_inc(&task->mm->oom_disable_count); - if (task->signal->oom_adj == OOM_DISABLE) - atomic_dec(&task->mm->oom_disable_count); - } - /* * Warn that /proc/pid/oom_adj is deprecated, see * Documentation/feature-removal-schedule.txt. @@ -1215,12 +1208,6 @@ static ssize_t oom_score_adj_write(struct file *file, const char __user *buf, goto err_sighand; } - if (oom_score_adj != task->signal->oom_score_adj) { - if (oom_score_adj == OOM_SCORE_ADJ_MIN) - atomic_inc(&task->mm->oom_disable_count); - if (task->signal->oom_score_adj == OOM_SCORE_ADJ_MIN) - atomic_dec(&task->mm->oom_disable_count); - } task->signal->oom_score_adj = oom_score_adj; if (has_capability_noaudit(current, CAP_SYS_RESOURCE)) task->signal->oom_score_adj_min = oom_score_adj; diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h index c93d00a6e95d..6456624aa964 100644 --- a/include/linux/mm_types.h +++ b/include/linux/mm_types.h @@ -336,9 +336,6 @@ struct mm_struct { unsigned int token_priority; unsigned int last_interval; - /* How many tasks sharing this mm are OOM_DISABLE */ - atomic_t oom_disable_count; - unsigned long flags; /* Must use atomic bitops to access the bits */ struct core_state *core_state; /* coredumping support */ diff --git a/kernel/exit.c b/kernel/exit.c index 2913b3509d42..d0b7d988f873 100644 --- a/kernel/exit.c +++ b/kernel/exit.c @@ -681,8 +681,6 @@ static void exit_mm(struct task_struct * tsk) enter_lazy_tlb(mm, current); /* We don't want this task to be frozen prematurely */ clear_freeze_flag(tsk); - if (tsk->signal->oom_score_adj == OOM_SCORE_ADJ_MIN) - atomic_dec(&mm->oom_disable_count); task_unlock(tsk); mm_update_next_owner(mm); mmput(mm); diff --git a/kernel/fork.c b/kernel/fork.c index 8e6b6f4fb272..70d76191afb9 100644 --- a/kernel/fork.c +++ b/kernel/fork.c @@ -501,7 +501,6 @@ static struct mm_struct *mm_init(struct mm_struct *mm, struct task_struct *p) mm->cached_hole_size = ~0UL; mm_init_aio(mm); mm_init_owner(mm, p); - atomic_set(&mm->oom_disable_count, 0); if (likely(!mm_alloc_pgd(mm))) { mm->def_flags = 0; @@ -816,8 +815,6 @@ good_mm: /* Initializing for Swap token stuff */ mm->token_priority = 0; mm->last_interval = 0; - if (tsk->signal->oom_score_adj == OOM_SCORE_ADJ_MIN) - atomic_inc(&mm->oom_disable_count); tsk->mm = mm; tsk->active_mm = mm; @@ -1391,13 +1388,8 @@ bad_fork_cleanup_io: bad_fork_cleanup_namespaces: exit_task_namespaces(p); bad_fork_cleanup_mm: - if (p->mm) { - task_lock(p); - if (p->signal->oom_score_adj == OOM_SCORE_ADJ_MIN) - atomic_dec(&p->mm->oom_disable_count); - task_unlock(p); + if (p->mm) mmput(p->mm); - } bad_fork_cleanup_signal: if (!(clone_flags & CLONE_THREAD)) free_signal_struct(p->signal); diff --git a/mm/oom_kill.c b/mm/oom_kill.c index b0d8943bc9fd..2b97e8f04607 100644 --- a/mm/oom_kill.c +++ b/mm/oom_kill.c @@ -54,13 +54,7 @@ int test_set_oom_score_adj(int new_val) spin_lock_irq(&sighand->siglock); old_val = current->signal->oom_score_adj; - if (new_val != old_val) { - if (new_val == OOM_SCORE_ADJ_MIN) - atomic_inc(¤t->mm->oom_disable_count); - else if (old_val == OOM_SCORE_ADJ_MIN) - atomic_dec(¤t->mm->oom_disable_count); - current->signal->oom_score_adj = new_val; - } + current->signal->oom_score_adj = new_val; spin_unlock_irq(&sighand->siglock); return old_val; @@ -172,16 +166,6 @@ unsigned int oom_badness(struct task_struct *p, struct mem_cgroup *mem, if (!p) return 0; - /* - * Shortcut check for a thread sharing p->mm that is OOM_SCORE_ADJ_MIN - * so the entire heuristic doesn't need to be executed for something - * that cannot be killed. - */ - if (atomic_read(&p->mm->oom_disable_count)) { - task_unlock(p); - return 0; - } - /* * The memory controller may have a limit of 0 bytes, so avoid a divide * by zero, if necessary. @@ -451,6 +435,9 @@ static int oom_kill_task(struct task_struct *p, struct mem_cgroup *mem) for_each_process(q) if (q->mm == mm && !same_thread_group(q, p) && !(q->flags & PF_KTHREAD)) { + if (q->signal->oom_score_adj == OOM_SCORE_ADJ_MIN) + continue; + task_lock(q); /* Protect ->comm from prctl() */ pr_err("Kill process %d (%s) sharing same memory\n", task_pid_nr(q), q->comm); @@ -727,7 +714,7 @@ void out_of_memory(struct zonelist *zonelist, gfp_t gfp_mask, read_lock(&tasklist_lock); if (sysctl_oom_kill_allocating_task && !oom_unkillable_task(current, NULL, nodemask) && - current->mm && !atomic_read(¤t->mm->oom_disable_count)) { + current->mm) { /* * oom_kill_process() needs tasklist_lock held. If it returns * non-zero, current could not be killed so we must fallback to -- cgit v1.2.3 From 43362a4977e37db46f86f7e6ab935f0006956632 Mon Sep 17 00:00:00 2001 From: David Rientjes Date: Mon, 31 Oct 2011 17:07:18 -0700 Subject: oom: fix race while temporarily setting current's oom_score_adj test_set_oom_score_adj() was introduced in 72788c385604 ("oom: replace PF_OOM_ORIGIN with toggling oom_score_adj") to temporarily elevate current's oom_score_adj for ksm and swapoff without requiring an additional per-process flag. Using that function to both set oom_score_adj to OOM_SCORE_ADJ_MAX and then reinstate the previous value is racy since it's possible that userspace can set the value to something else itself before the old value is reinstated. That results in userspace setting current's oom_score_adj to a different value and then the kernel immediately setting it back to its previous value without notification. To fix this, a new compare_swap_oom_score_adj() function is introduced with the same semantics as the compare and swap CAS instruction, or CMPXCHG on x86. It is used to reinstate the previous value of oom_score_adj if and only if the present value is the same as the old value. Signed-off-by: David Rientjes Cc: Oleg Nesterov Cc: Ying Han Cc: KOSAKI Motohiro Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds --- include/linux/oom.h | 1 + mm/ksm.c | 3 ++- mm/oom_kill.c | 19 +++++++++++++++++++ mm/swapfile.c | 2 +- 4 files changed, 23 insertions(+), 2 deletions(-) (limited to 'include/linux') diff --git a/include/linux/oom.h b/include/linux/oom.h index 13b7b02e599a..6f9d04a85336 100644 --- a/include/linux/oom.h +++ b/include/linux/oom.h @@ -40,6 +40,7 @@ enum oom_constraint { CONSTRAINT_MEMCG, }; +extern void compare_swap_oom_score_adj(int old_val, int new_val); extern int test_set_oom_score_adj(int new_val); extern unsigned int oom_badness(struct task_struct *p, struct mem_cgroup *mem, diff --git a/mm/ksm.c b/mm/ksm.c index 9a68b0cf0a1c..310544a379ae 100644 --- a/mm/ksm.c +++ b/mm/ksm.c @@ -1905,7 +1905,8 @@ static ssize_t run_store(struct kobject *kobj, struct kobj_attribute *attr, oom_score_adj = test_set_oom_score_adj(OOM_SCORE_ADJ_MAX); err = unmerge_and_remove_all_rmap_items(); - test_set_oom_score_adj(oom_score_adj); + compare_swap_oom_score_adj(OOM_SCORE_ADJ_MAX, + oom_score_adj); if (err) { ksm_run = KSM_RUN_STOP; count = err; diff --git a/mm/oom_kill.c b/mm/oom_kill.c index 2b97e8f04607..e916168b6e0a 100644 --- a/mm/oom_kill.c +++ b/mm/oom_kill.c @@ -39,6 +39,25 @@ int sysctl_oom_kill_allocating_task; int sysctl_oom_dump_tasks = 1; static DEFINE_SPINLOCK(zone_scan_lock); +/* + * compare_swap_oom_score_adj() - compare and swap current's oom_score_adj + * @old_val: old oom_score_adj for compare + * @new_val: new oom_score_adj for swap + * + * Sets the oom_score_adj value for current to @new_val iff its present value is + * @old_val. Usually used to reinstate a previous value to prevent racing with + * userspacing tuning the value in the interim. + */ +void compare_swap_oom_score_adj(int old_val, int new_val) +{ + struct sighand_struct *sighand = current->sighand; + + spin_lock_irq(&sighand->siglock); + if (current->signal->oom_score_adj == old_val) + current->signal->oom_score_adj = new_val; + spin_unlock_irq(&sighand->siglock); +} + /** * test_set_oom_score_adj() - set current's oom_score_adj and return old value * @new_val: new oom_score_adj value diff --git a/mm/swapfile.c b/mm/swapfile.c index 17bc224bce68..c9d654009125 100644 --- a/mm/swapfile.c +++ b/mm/swapfile.c @@ -1617,7 +1617,7 @@ SYSCALL_DEFINE1(swapoff, const char __user *, specialfile) oom_score_adj = test_set_oom_score_adj(OOM_SCORE_ADJ_MAX); err = try_to_unuse(type); - test_set_oom_score_adj(oom_score_adj); + compare_swap_oom_score_adj(OOM_SCORE_ADJ_MAX, oom_score_adj); if (err) { /* -- cgit v1.2.3 From bc3e53f682d93df677dbd5006a404722b3adfe18 Mon Sep 17 00:00:00 2001 From: Christoph Lameter Date: Mon, 31 Oct 2011 17:07:30 -0700 Subject: mm: distinguish between mlocked and pinned pages Some kernel components pin user space memory (infiniband and perf) (by increasing the page count) and account that memory as "mlocked". The difference between mlocking and pinning is: A. mlocked pages are marked with PG_mlocked and are exempt from swapping. Page migration may move them around though. They are kept on a special LRU list. B. Pinned pages cannot be moved because something needs to directly access physical memory. They may not be on any LRU list. I recently saw an mlockalled process where mm->locked_vm became bigger than the virtual size of the process (!) because some memory was accounted for twice: Once when the page was mlocked and once when the Infiniband layer increased the refcount because it needt to pin the RDMA memory. This patch introduces a separate counter for pinned pages and accounts them seperately. Signed-off-by: Christoph Lameter Cc: Mike Marciniszyn Cc: Roland Dreier Cc: Sean Hefty Cc: Hugh Dickins Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds --- drivers/infiniband/core/umem.c | 6 +++--- drivers/infiniband/hw/ipath/ipath_user_pages.c | 6 +++--- drivers/infiniband/hw/qib/qib_user_pages.c | 4 ++-- fs/proc/task_mmu.c | 2 ++ include/linux/mm_types.h | 2 +- kernel/events/core.c | 6 +++--- 6 files changed, 14 insertions(+), 12 deletions(-) (limited to 'include/linux') diff --git a/drivers/infiniband/core/umem.c b/drivers/infiniband/core/umem.c index b645e558876f..9155f91d66bf 100644 --- a/drivers/infiniband/core/umem.c +++ b/drivers/infiniband/core/umem.c @@ -136,7 +136,7 @@ struct ib_umem *ib_umem_get(struct ib_ucontext *context, unsigned long addr, down_write(¤t->mm->mmap_sem); - locked = npages + current->mm->locked_vm; + locked = npages + current->mm->pinned_vm; lock_limit = rlimit(RLIMIT_MEMLOCK) >> PAGE_SHIFT; if ((locked > lock_limit) && !capable(CAP_IPC_LOCK)) { @@ -206,7 +206,7 @@ out: __ib_umem_release(context->device, umem, 0); kfree(umem); } else - current->mm->locked_vm = locked; + current->mm->pinned_vm = locked; up_write(¤t->mm->mmap_sem); if (vma_list) @@ -222,7 +222,7 @@ static void ib_umem_account(struct work_struct *work) struct ib_umem *umem = container_of(work, struct ib_umem, work); down_write(&umem->mm->mmap_sem); - umem->mm->locked_vm -= umem->diff; + umem->mm->pinned_vm -= umem->diff; up_write(&umem->mm->mmap_sem); mmput(umem->mm); kfree(umem); diff --git a/drivers/infiniband/hw/ipath/ipath_user_pages.c b/drivers/infiniband/hw/ipath/ipath_user_pages.c index cfed5399f074..dc66c4506916 100644 --- a/drivers/infiniband/hw/ipath/ipath_user_pages.c +++ b/drivers/infiniband/hw/ipath/ipath_user_pages.c @@ -79,7 +79,7 @@ static int __ipath_get_user_pages(unsigned long start_page, size_t num_pages, goto bail_release; } - current->mm->locked_vm += num_pages; + current->mm->pinned_vm += num_pages; ret = 0; goto bail; @@ -178,7 +178,7 @@ void ipath_release_user_pages(struct page **p, size_t num_pages) __ipath_release_user_pages(p, num_pages, 1); - current->mm->locked_vm -= num_pages; + current->mm->pinned_vm -= num_pages; up_write(¤t->mm->mmap_sem); } @@ -195,7 +195,7 @@ static void user_pages_account(struct work_struct *_work) container_of(_work, struct ipath_user_pages_work, work); down_write(&work->mm->mmap_sem); - work->mm->locked_vm -= work->num_pages; + work->mm->pinned_vm -= work->num_pages; up_write(&work->mm->mmap_sem); mmput(work->mm); kfree(work); diff --git a/drivers/infiniband/hw/qib/qib_user_pages.c b/drivers/infiniband/hw/qib/qib_user_pages.c index 7689e49c13c9..2bc1d2b96298 100644 --- a/drivers/infiniband/hw/qib/qib_user_pages.c +++ b/drivers/infiniband/hw/qib/qib_user_pages.c @@ -74,7 +74,7 @@ static int __qib_get_user_pages(unsigned long start_page, size_t num_pages, goto bail_release; } - current->mm->locked_vm += num_pages; + current->mm->pinned_vm += num_pages; ret = 0; goto bail; @@ -151,7 +151,7 @@ void qib_release_user_pages(struct page **p, size_t num_pages) __qib_release_user_pages(p, num_pages, 1); if (current->mm) { - current->mm->locked_vm -= num_pages; + current->mm->pinned_vm -= num_pages; up_write(¤t->mm->mmap_sem); } } diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c index c7d4ee663f14..e418c5abdb0e 100644 --- a/fs/proc/task_mmu.c +++ b/fs/proc/task_mmu.c @@ -44,6 +44,7 @@ void task_mem(struct seq_file *m, struct mm_struct *mm) "VmPeak:\t%8lu kB\n" "VmSize:\t%8lu kB\n" "VmLck:\t%8lu kB\n" + "VmPin:\t%8lu kB\n" "VmHWM:\t%8lu kB\n" "VmRSS:\t%8lu kB\n" "VmData:\t%8lu kB\n" @@ -55,6 +56,7 @@ void task_mem(struct seq_file *m, struct mm_struct *mm) hiwater_vm << (PAGE_SHIFT-10), (total_vm - mm->reserved_vm) << (PAGE_SHIFT-10), mm->locked_vm << (PAGE_SHIFT-10), + mm->pinned_vm << (PAGE_SHIFT-10), hiwater_rss << (PAGE_SHIFT-10), total_rss << (PAGE_SHIFT-10), data << (PAGE_SHIFT-10), diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h index 6456624aa964..f3175830cc73 100644 --- a/include/linux/mm_types.h +++ b/include/linux/mm_types.h @@ -304,7 +304,7 @@ struct mm_struct { unsigned long hiwater_rss; /* High-watermark of RSS usage */ unsigned long hiwater_vm; /* High-water virtual memory usage */ - unsigned long total_vm, locked_vm, shared_vm, exec_vm; + unsigned long total_vm, locked_vm, pinned_vm, shared_vm, exec_vm; unsigned long stack_vm, reserved_vm, def_flags, nr_ptes; unsigned long start_code, end_code, start_data, end_data; unsigned long start_brk, brk, start_stack; diff --git a/kernel/events/core.c b/kernel/events/core.c index d1a1bee35228..12a0287e0358 100644 --- a/kernel/events/core.c +++ b/kernel/events/core.c @@ -3544,7 +3544,7 @@ static void perf_mmap_close(struct vm_area_struct *vma) struct ring_buffer *rb = event->rb; atomic_long_sub((size >> PAGE_SHIFT) + 1, &user->locked_vm); - vma->vm_mm->locked_vm -= event->mmap_locked; + vma->vm_mm->pinned_vm -= event->mmap_locked; rcu_assign_pointer(event->rb, NULL); mutex_unlock(&event->mmap_mutex); @@ -3625,7 +3625,7 @@ static int perf_mmap(struct file *file, struct vm_area_struct *vma) lock_limit = rlimit(RLIMIT_MEMLOCK); lock_limit >>= PAGE_SHIFT; - locked = vma->vm_mm->locked_vm + extra; + locked = vma->vm_mm->pinned_vm + extra; if ((locked > lock_limit) && perf_paranoid_tracepoint_raw() && !capable(CAP_IPC_LOCK)) { @@ -3651,7 +3651,7 @@ static int perf_mmap(struct file *file, struct vm_area_struct *vma) atomic_long_add(user_extra, &user->locked_vm); event->mmap_locked = extra; event->mmap_user = get_current_user(); - vma->vm_mm->locked_vm += event->mmap_locked; + vma->vm_mm->pinned_vm += event->mmap_locked; unlock: if (!ret) -- cgit v1.2.3 From e10d59f2c3decaf22cc5d3de7040eba202bc2df3 Mon Sep 17 00:00:00 2001 From: Christoph Lameter Date: Mon, 31 Oct 2011 17:07:34 -0700 Subject: mm: add comments to explain mm_struct fields Add comments to explain the page statistics field in the mm_struct. [akpm@linux-foundation.org: add missing ;] Signed-off-by: Christoph Lameter Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds --- include/linux/mm_types.h | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) (limited to 'include/linux') diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h index f3175830cc73..3e01a19a91e8 100644 --- a/include/linux/mm_types.h +++ b/include/linux/mm_types.h @@ -304,8 +304,15 @@ struct mm_struct { unsigned long hiwater_rss; /* High-watermark of RSS usage */ unsigned long hiwater_vm; /* High-water virtual memory usage */ - unsigned long total_vm, locked_vm, pinned_vm, shared_vm, exec_vm; - unsigned long stack_vm, reserved_vm, def_flags, nr_ptes; + unsigned long total_vm; /* Total pages mapped */ + unsigned long locked_vm; /* Pages that have PG_mlocked set */ + unsigned long pinned_vm; /* Refcount permanently increased */ + unsigned long shared_vm; /* Shared pages (files) */ + unsigned long exec_vm; /* VM_EXEC & ~VM_WRITE */ + unsigned long stack_vm; /* VM_GROWSUP/DOWN */ + unsigned long reserved_vm; /* VM_RESERVED|VM_IO pages */ + unsigned long def_flags; + unsigned long nr_ptes; /* Page table pages */ unsigned long start_code, end_code, start_data, end_data; unsigned long start_brk, brk, start_stack; unsigned long arg_start, arg_end, env_start, env_end; -- cgit v1.2.3 From ee72886d8ed5d9de3fa0ed3b99a7ca7702576a96 Mon Sep 17 00:00:00 2001 From: Mel Gorman Date: Mon, 31 Oct 2011 17:07:38 -0700 Subject: mm: vmscan: do not writeback filesystem pages in direct reclaim Testing from the XFS folk revealed that there is still too much I/O from the end of the LRU in kswapd. Previously it was considered acceptable by VM people for a small number of pages to be written back from reclaim with testing generally showing about 0.3% of pages reclaimed were written back (higher if memory was low). That writing back a small number of pages is ok has been heavily disputed for quite some time and Dave Chinner explained it well; It doesn't have to be a very high number to be a problem. IO is orders of magnitude slower than the CPU time it takes to flush a page, so the cost of making a bad flush decision is very high. And single page writeback from the LRU is almost always a bad flush decision. To complicate matters, filesystems respond very differently to requests from reclaim according to Christoph Hellwig; xfs tries to write it back if the requester is kswapd ext4 ignores the request if it's a delayed allocation btrfs ignores the request As a result, each filesystem has different performance characteristics when under memory pressure and there are many pages being dirtied. In some cases, the request is ignored entirely so the VM cannot depend on the IO being dispatched. The objective of this series is to reduce writing of filesystem-backed pages from reclaim, play nicely with writeback that is already in progress and throttle reclaim appropriately when writeback pages are encountered. The assumption is that the flushers will always write pages faster than if reclaim issues the IO. A secondary goal is to avoid the problem whereby direct reclaim splices two potentially deep call stacks together. There is a potential new problem as reclaim has less control over how long before a page in a particularly zone or container is cleaned and direct reclaimers depend on kswapd or flusher threads to do the necessary work. However, as filesystems sometimes ignore direct reclaim requests already, it is not expected to be a serious issue. Patch 1 disables writeback of filesystem pages from direct reclaim entirely. Anonymous pages are still written. Patch 2 removes dead code in lumpy reclaim as it is no longer able to synchronously write pages. This hurts lumpy reclaim but there is an expectation that compaction is used for hugepage allocations these days and lumpy reclaim's days are numbered. Patches 3-4 add warnings to XFS and ext4 if called from direct reclaim. With patch 1, this "never happens" and is intended to catch regressions in this logic in the future. Patch 5 disables writeback of filesystem pages from kswapd unless the priority is raised to the point where kswapd is considered to be in trouble. Patch 6 throttles reclaimers if too many dirty pages are being encountered and the zones or backing devices are congested. Patch 7 invalidates dirty pages found at the end of the LRU so they are reclaimed quickly after being written back rather than waiting for a reclaimer to find them I consider this series to be orthogonal to the writeback work but it is worth noting that the writeback work affects the viability of patch 8 in particular. I tested this on ext4 and xfs using fs_mark, a simple writeback test based on dd and a micro benchmark that does a streaming write to a large mapping (exercises use-once LRU logic) followed by streaming writes to a mix of anonymous and file-backed mappings. The command line for fs_mark when botted with 512M looked something like ./fs_mark -d /tmp/fsmark-2676 -D 100 -N 150 -n 150 -L 25 -t 1 -S0 -s 10485760 The number of files was adjusted depending on the amount of available memory so that the files created was about 3xRAM. For multiple threads, the -d switch is specified multiple times. The test machine is x86-64 with an older generation of AMD processor with 4 cores. The underlying storage was 4 disks configured as RAID-0 as this was the best configuration of storage I had available. Swap is on a separate disk. Dirty ratio was tuned to 40% instead of the default of 20%. Testing was run with and without monitors to both verify that the patches were operating as expected and that any performance gain was real and not due to interference from monitors. Here is a summary of results based on testing XFS. 512M1P-xfs Files/s mean 32.69 ( 0.00%) 34.44 ( 5.08%) 512M1P-xfs Elapsed Time fsmark 51.41 48.29 512M1P-xfs Elapsed Time simple-wb 114.09 108.61 512M1P-xfs Elapsed Time mmap-strm 113.46 109.34 512M1P-xfs Kswapd efficiency fsmark 62% 63% 512M1P-xfs Kswapd efficiency simple-wb 56% 61% 512M1P-xfs Kswapd efficiency mmap-strm 44% 42% 512M-xfs Files/s mean 30.78 ( 0.00%) 35.94 (14.36%) 512M-xfs Elapsed Time fsmark 56.08 48.90 512M-xfs Elapsed Time simple-wb 112.22 98.13 512M-xfs Elapsed Time mmap-strm 219.15 196.67 512M-xfs Kswapd efficiency fsmark 54% 56% 512M-xfs Kswapd efficiency simple-wb 54% 55% 512M-xfs Kswapd efficiency mmap-strm 45% 44% 512M-4X-xfs Files/s mean 30.31 ( 0.00%) 33.33 ( 9.06%) 512M-4X-xfs Elapsed Time fsmark 63.26 55.88 512M-4X-xfs Elapsed Time simple-wb 100.90 90.25 512M-4X-xfs Elapsed Time mmap-strm 261.73 255.38 512M-4X-xfs Kswapd efficiency fsmark 49% 50% 512M-4X-xfs Kswapd efficiency simple-wb 54% 56% 512M-4X-xfs Kswapd efficiency mmap-strm 37% 36% 512M-16X-xfs Files/s mean 60.89 ( 0.00%) 65.22 ( 6.64%) 512M-16X-xfs Elapsed Time fsmark 67.47 58.25 512M-16X-xfs Elapsed Time simple-wb 103.22 90.89 512M-16X-xfs Elapsed Time mmap-strm 237.09 198.82 512M-16X-xfs Kswapd efficiency fsmark 45% 46% 512M-16X-xfs Kswapd efficiency simple-wb 53% 55% 512M-16X-xfs Kswapd efficiency mmap-strm 33% 33% Up until 512-4X, the FSmark improvements were statistically significant. For the 4X and 16X tests the results were within standard deviations but just barely. The time to completion for all tests is improved which is an important result. In general, kswapd efficiency is not affected by skipping dirty pages. 1024M1P-xfs Files/s mean 39.09 ( 0.00%) 41.15 ( 5.01%) 1024M1P-xfs Elapsed Time fsmark 84.14 80.41 1024M1P-xfs Elapsed Time simple-wb 210.77 184.78 1024M1P-xfs Elapsed Time mmap-strm 162.00 160.34 1024M1P-xfs Kswapd efficiency fsmark 69% 75% 1024M1P-xfs Kswapd efficiency simple-wb 71% 77% 1024M1P-xfs Kswapd efficiency mmap-strm 43% 44% 1024M-xfs Files/s mean 35.45 ( 0.00%) 37.00 ( 4.19%) 1024M-xfs Elapsed Time fsmark 94.59 91.00 1024M-xfs Elapsed Time simple-wb 229.84 195.08 1024M-xfs Elapsed Time mmap-strm 405.38 440.29 1024M-xfs Kswapd efficiency fsmark 79% 71% 1024M-xfs Kswapd efficiency simple-wb 74% 74% 1024M-xfs Kswapd efficiency mmap-strm 39% 42% 1024M-4X-xfs Files/s mean 32.63 ( 0.00%) 35.05 ( 6.90%) 1024M-4X-xfs Elapsed Time fsmark 103.33 97.74 1024M-4X-xfs Elapsed Time simple-wb 204.48 178.57 1024M-4X-xfs Elapsed Time mmap-strm 528.38 511.88 1024M-4X-xfs Kswapd efficiency fsmark 81% 70% 1024M-4X-xfs Kswapd efficiency simple-wb 73% 72% 1024M-4X-xfs Kswapd efficiency mmap-strm 39% 38% 1024M-16X-xfs Files/s mean 42.65 ( 0.00%) 42.97 ( 0.74%) 1024M-16X-xfs Elapsed Time fsmark 103.11 99.11 1024M-16X-xfs Elapsed Time simple-wb 200.83 178.24 1024M-16X-xfs Elapsed Time mmap-strm 397.35 459.82 1024M-16X-xfs Kswapd efficiency fsmark 84% 69% 1024M-16X-xfs Kswapd efficiency simple-wb 74% 73% 1024M-16X-xfs Kswapd efficiency mmap-strm 39% 40% All FSMark tests up to 16X had statistically significant improvements. For the most part, tests are completing faster with the exception of the streaming writes to a mixture of anonymous and file-backed mappings which were slower in two cases In the cases where the mmap-strm tests were slower, there was more swapping due to dirty pages being skipped. The number of additional pages swapped is almost identical to the fewer number of pages written from reclaim. In other words, roughly the same number of pages were reclaimed but swapping was slower. As the test is a bit unrealistic and stresses memory heavily, the small shift is acceptable. 4608M1P-xfs Files/s mean 29.75 ( 0.00%) 30.96 ( 3.91%) 4608M1P-xfs Elapsed Time fsmark 512.01 492.15 4608M1P-xfs Elapsed Time simple-wb 618.18 566.24 4608M1P-xfs Elapsed Time mmap-strm 488.05 465.07 4608M1P-xfs Kswapd efficiency fsmark 93% 86% 4608M1P-xfs Kswapd efficiency simple-wb 88% 84% 4608M1P-xfs Kswapd efficiency mmap-strm 46% 45% 4608M-xfs Files/s mean 27.60 ( 0.00%) 28.85 ( 4.33%) 4608M-xfs Elapsed Time fsmark 555.96 532.34 4608M-xfs Elapsed Time simple-wb 659.72 571.85 4608M-xfs Elapsed Time mmap-strm 1082.57 1146.38 4608M-xfs Kswapd efficiency fsmark 89% 91% 4608M-xfs Kswapd efficiency simple-wb 88% 82% 4608M-xfs Kswapd efficiency mmap-strm 48% 46% 4608M-4X-xfs Files/s mean 26.00 ( 0.00%) 27.47 ( 5.35%) 4608M-4X-xfs Elapsed Time fsmark 592.91 564.00 4608M-4X-xfs Elapsed Time simple-wb 616.65 575.07 4608M-4X-xfs Elapsed Time mmap-strm 1773.02 1631.53 4608M-4X-xfs Kswapd efficiency fsmark 90% 94% 4608M-4X-xfs Kswapd efficiency simple-wb 87% 82% 4608M-4X-xfs Kswapd efficiency mmap-strm 43% 43% 4608M-16X-xfs Files/s mean 26.07 ( 0.00%) 26.42 ( 1.32%) 4608M-16X-xfs Elapsed Time fsmark 602.69 585.78 4608M-16X-xfs Elapsed Time simple-wb 606.60 573.81 4608M-16X-xfs Elapsed Time mmap-strm 1549.75 1441.86 4608M-16X-xfs Kswapd efficiency fsmark 98% 98% 4608M-16X-xfs Kswapd efficiency simple-wb 88% 82% 4608M-16X-xfs Kswapd efficiency mmap-strm 44% 42% Unlike the other tests, the fsmark results are not statistically significant but the min and max times are both improved and for the most part, tests completed faster. There are other indications that this is an improvement as well. For example, in the vast majority of cases, there were fewer pages scanned by direct reclaim implying in many cases that stalls due to direct reclaim are reduced. KSwapd is scanning more due to skipping dirty pages which is unfortunate but the CPU usage is still acceptable In an earlier set of tests, I used blktrace and in almost all cases throughput throughout the entire test was higher. However, I ended up discarding those results as recording blktrace data was too heavy for my liking. On a laptop, I plugged in a USB stick and ran a similar tests of tests using it as backing storage. A desktop environment was running and for the entire duration of the tests, firefox and gnome terminal were launching and exiting to vaguely simulate a user. 1024M-xfs Files/s mean 0.41 ( 0.00%) 0.44 ( 6.82%) 1024M-xfs Elapsed Time fsmark 2053.52 1641.03 1024M-xfs Elapsed Time simple-wb 1229.53 768.05 1024M-xfs Elapsed Time mmap-strm 4126.44 4597.03 1024M-xfs Kswapd efficiency fsmark 84% 85% 1024M-xfs Kswapd efficiency simple-wb 92% 81% 1024M-xfs Kswapd efficiency mmap-strm 60% 51% 1024M-xfs Avg wait ms fsmark 5404.53 4473.87 1024M-xfs Avg wait ms simple-wb 2541.35 1453.54 1024M-xfs Avg wait ms mmap-strm 3400.25 3852.53 The mmap-strm results were hurt because firefox launching had a tendency to push the test out of memory. On the postive side, firefox launched marginally faster with the patches applied. Time to completion for many tests was faster but more importantly - the "Avg wait" time as measured by iostat was far lower implying the system would be more responsive. It was also the case that "Avg wait ms" on the root filesystem was lower. I tested it manually and while the system felt slightly more responsive while copying data to a USB stick, it was marginal enough that it could be my imagination. This patch: do not writeback filesystem pages in direct reclaim. When kswapd is failing to keep zones above the min watermark, a process will enter direct reclaim in the same manner kswapd does. If a dirty page is encountered during the scan, this page is written to backing storage using mapping->writepage. This causes two problems. First, it can result in very deep call stacks, particularly if the target storage or filesystem are complex. Some filesystems ignore write requests from direct reclaim as a result. The second is that a single-page flush is inefficient in terms of IO. While there is an expectation that the elevator will merge requests, this does not always happen. Quoting Christoph Hellwig; The elevator has a relatively small window it can operate on, and can never fix up a bad large scale writeback pattern. This patch prevents direct reclaim writing back filesystem pages by checking if current is kswapd. Anonymous pages are still written to swap as there is not the equivalent of a flusher thread for anonymous pages. If the dirty pages cannot be written back, they are placed back on the LRU lists. There is now a direct dependency on dirty page balancing to prevent too many pages in the system being dirtied which would prevent reclaim making forward progress. Signed-off-by: Mel Gorman Reviewed-by: Minchan Kim Cc: Dave Chinner Cc: Christoph Hellwig Cc: Johannes Weiner Cc: Wu Fengguang Cc: Jan Kara Cc: Rik van Riel Cc: Mel Gorman Cc: Alex Elder Cc: Theodore Ts'o Cc: Chris Mason Cc: Dave Hansen Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds --- include/linux/mmzone.h | 1 + mm/vmscan.c | 9 +++++++++ mm/vmstat.c | 1 + 3 files changed, 11 insertions(+) (limited to 'include/linux') diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h index ec57779c5a57..2c41b2c1943b 100644 --- a/include/linux/mmzone.h +++ b/include/linux/mmzone.h @@ -100,6 +100,7 @@ enum zone_stat_item { NR_UNSTABLE_NFS, /* NFS unstable pages */ NR_BOUNCE, NR_VMSCAN_WRITE, + NR_VMSCAN_WRITE_SKIP, NR_WRITEBACK_TEMP, /* Writeback using temporary buffers */ NR_ISOLATED_ANON, /* Temporary isolated pages from anon lru */ NR_ISOLATED_FILE, /* Temporary isolated pages from file lru */ diff --git a/mm/vmscan.c b/mm/vmscan.c index d29b2bdb9e03..10f9c59aed55 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -865,6 +865,15 @@ static unsigned long shrink_page_list(struct list_head *page_list, if (PageDirty(page)) { nr_dirty++; + /* + * Only kswapd can writeback filesystem pages to + * avoid risk of stack overflow + */ + if (page_is_file_cache(page) && !current_is_kswapd()) { + inc_zone_page_state(page, NR_VMSCAN_WRITE_SKIP); + goto keep_locked; + } + if (references == PAGEREF_RECLAIM_CLEAN) goto keep_locked; if (!may_enter_fs) diff --git a/mm/vmstat.c b/mm/vmstat.c index d52b13d28e8f..210bd8ff3a6e 100644 --- a/mm/vmstat.c +++ b/mm/vmstat.c @@ -702,6 +702,7 @@ const char * const vmstat_text[] = { "nr_unstable", "nr_bounce", "nr_vmscan_write", + "nr_vmscan_write_skip", "nr_writeback_temp", "nr_isolated_anon", "nr_isolated_file", -- cgit v1.2.3 From 49ea7eb65e7c5060807fb9312b1ad4c3eab82e2c Mon Sep 17 00:00:00 2001 From: Mel Gorman Date: Mon, 31 Oct 2011 17:07:59 -0700 Subject: mm: vmscan: immediately reclaim end-of-LRU dirty pages when writeback completes When direct reclaim encounters a dirty page, it gets recycled around the LRU for another cycle. This patch marks the page PageReclaim similar to deactivate_page() so that the page gets reclaimed almost immediately after the page gets cleaned. This is to avoid reclaiming clean pages that are younger than a dirty page encountered at the end of the LRU that might have been something like a use-once page. Signed-off-by: Mel Gorman Acked-by: Johannes Weiner Cc: Dave Chinner Cc: Christoph Hellwig Cc: Wu Fengguang Cc: Jan Kara Cc: Minchan Kim Cc: Rik van Riel Cc: Mel Gorman Cc: Alex Elder Cc: Theodore Ts'o Cc: Chris Mason Cc: Dave Hansen Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds --- include/linux/mmzone.h | 2 +- mm/vmscan.c | 10 +++++++++- mm/vmstat.c | 2 +- 3 files changed, 11 insertions(+), 3 deletions(-) (limited to 'include/linux') diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h index 2c41b2c1943b..188cb2ffe8db 100644 --- a/include/linux/mmzone.h +++ b/include/linux/mmzone.h @@ -100,7 +100,7 @@ enum zone_stat_item { NR_UNSTABLE_NFS, /* NFS unstable pages */ NR_BOUNCE, NR_VMSCAN_WRITE, - NR_VMSCAN_WRITE_SKIP, + NR_VMSCAN_IMMEDIATE, /* Prioritise for reclaim when writeback ends */ NR_WRITEBACK_TEMP, /* Writeback using temporary buffers */ NR_ISOLATED_ANON, /* Temporary isolated pages from anon lru */ NR_ISOLATED_FILE, /* Temporary isolated pages from file lru */ diff --git a/mm/vmscan.c b/mm/vmscan.c index 7b0573f33a27..a297603d35bc 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -866,7 +866,15 @@ static unsigned long shrink_page_list(struct list_head *page_list, */ if (page_is_file_cache(page) && (!current_is_kswapd() || priority >= DEF_PRIORITY - 2)) { - inc_zone_page_state(page, NR_VMSCAN_WRITE_SKIP); + /* + * Immediately reclaim when written back. + * Similar in principal to deactivate_page() + * except we already have the page isolated + * and know it's dirty + */ + inc_zone_page_state(page, NR_VMSCAN_IMMEDIATE); + SetPageReclaim(page); + goto keep_locked; } diff --git a/mm/vmstat.c b/mm/vmstat.c index 210bd8ff3a6e..56e529a40517 100644 --- a/mm/vmstat.c +++ b/mm/vmstat.c @@ -702,7 +702,7 @@ const char * const vmstat_text[] = { "nr_unstable", "nr_bounce", "nr_vmscan_write", - "nr_vmscan_write_skip", + "nr_vmscan_immediate_reclaim", "nr_writeback_temp", "nr_isolated_anon", "nr_isolated_file", -- cgit v1.2.3 From 798248206b59acc6e1238c778281419c041891a7 Mon Sep 17 00:00:00 2001 From: Akinobu Mita Date: Mon, 31 Oct 2011 17:08:07 -0700 Subject: lib/string.c: introduce memchr_inv() memchr_inv() is mainly used to check whether the whole buffer is filled with just a specified byte. The function name and prototype are stolen from logfs and the implementation is from SLUB. Signed-off-by: Akinobu Mita Acked-by: Christoph Lameter Acked-by: Pekka Enberg Cc: Matt Mackall Acked-by: Joern Engel Cc: Marcin Slusarz Cc: Eric Dumazet Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds --- fs/logfs/logfs.h | 1 - fs/logfs/super.c | 22 -------------------- include/linux/string.h | 1 + lib/string.c | 54 ++++++++++++++++++++++++++++++++++++++++++++++++++ mm/slub.c | 47 ++----------------------------------------- 5 files changed, 57 insertions(+), 68 deletions(-) (limited to 'include/linux') diff --git a/fs/logfs/logfs.h b/fs/logfs/logfs.h index f22d108bfa5d..398ecff6e548 100644 --- a/fs/logfs/logfs.h +++ b/fs/logfs/logfs.h @@ -618,7 +618,6 @@ static inline int logfs_buf_recover(struct logfs_area *area, u64 ofs, struct page *emergency_read_begin(struct address_space *mapping, pgoff_t index); void emergency_read_end(struct page *page); void logfs_crash_dump(struct super_block *sb); -void *memchr_inv(const void *s, int c, size_t n); int logfs_statfs(struct dentry *dentry, struct kstatfs *stats); int logfs_check_ds(struct logfs_disk_super *ds); int logfs_write_sb(struct super_block *sb); diff --git a/fs/logfs/super.c b/fs/logfs/super.c index ce03a182c771..f2697e4df109 100644 --- a/fs/logfs/super.c +++ b/fs/logfs/super.c @@ -90,28 +90,6 @@ void logfs_crash_dump(struct super_block *sb) dump_segfile(sb); } -/* - * TODO: move to lib/string.c - */ -/** - * memchr_inv - Find a character in an area of memory. - * @s: The memory area - * @c: The byte to search for - * @n: The size of the area. - * - * returns the address of the first character other than @c, or %NULL - * if the whole buffer contains just @c. - */ -void *memchr_inv(const void *s, int c, size_t n) -{ - const unsigned char *p = s; - while (n-- != 0) - if ((unsigned char)c != *p++) - return (void *)(p - 1); - - return NULL; -} - /* * FIXME: There should be a reserve for root, similar to ext2. */ diff --git a/include/linux/string.h b/include/linux/string.h index a176db2f2c85..e033564f10ba 100644 --- a/include/linux/string.h +++ b/include/linux/string.h @@ -114,6 +114,7 @@ extern int memcmp(const void *,const void *,__kernel_size_t); #ifndef __HAVE_ARCH_MEMCHR extern void * memchr(const void *,int,__kernel_size_t); #endif +void *memchr_inv(const void *s, int c, size_t n); extern char *kstrdup(const char *s, gfp_t gfp); extern char *kstrndup(const char *s, size_t len, gfp_t gfp); diff --git a/lib/string.c b/lib/string.c index 01fad9b203e1..11df54325fb8 100644 --- a/lib/string.c +++ b/lib/string.c @@ -756,3 +756,57 @@ void *memchr(const void *s, int c, size_t n) } EXPORT_SYMBOL(memchr); #endif + +static void *check_bytes8(const u8 *start, u8 value, unsigned int bytes) +{ + while (bytes) { + if (*start != value) + return (void *)start; + start++; + bytes--; + } + return NULL; +} + +/** + * memchr_inv - Find an unmatching character in an area of memory. + * @start: The memory area + * @c: Find a character other than c + * @bytes: The size of the area. + * + * returns the address of the first character other than @c, or %NULL + * if the whole buffer contains just @c. + */ +void *memchr_inv(const void *start, int c, size_t bytes) +{ + u8 value = c; + u64 value64; + unsigned int words, prefix; + + if (bytes <= 16) + return check_bytes8(start, value, bytes); + + value64 = value | value << 8 | value << 16 | value << 24; + value64 = (value64 & 0xffffffff) | value64 << 32; + prefix = 8 - ((unsigned long)start) % 8; + + if (prefix) { + u8 *r = check_bytes8(start, value, prefix); + if (r) + return r; + start += prefix; + bytes -= prefix; + } + + words = bytes / 8; + + while (words) { + if (*(u64 *)start != value64) + return check_bytes8(start, value, 8); + start += 8; + words--; + } + + return check_bytes8(start, value, bytes % 8); +} +EXPORT_SYMBOL(memchr_inv); diff --git a/mm/slub.c b/mm/slub.c index 95215aa6a75e..7d2a996c307e 100644 --- a/mm/slub.c +++ b/mm/slub.c @@ -655,49 +655,6 @@ static void init_object(struct kmem_cache *s, void *object, u8 val) memset(p + s->objsize, val, s->inuse - s->objsize); } -static u8 *check_bytes8(u8 *start, u8 value, unsigned int bytes) -{ - while (bytes) { - if (*start != value) - return start; - start++; - bytes--; - } - return NULL; -} - -static u8 *check_bytes(u8 *start, u8 value, unsigned int bytes) -{ - u64 value64; - unsigned int words, prefix; - - if (bytes <= 16) - return check_bytes8(start, value, bytes); - - value64 = value | value << 8 | value << 16 | value << 24; - value64 = (value64 & 0xffffffff) | value64 << 32; - prefix = 8 - ((unsigned long)start) % 8; - - if (prefix) { - u8 *r = check_bytes8(start, value, prefix); - if (r) - return r; - start += prefix; - bytes -= prefix; - } - - words = bytes / 8; - - while (words) { - if (*(u64 *)start != value64) - return check_bytes8(start, value, 8); - start += 8; - words--; - } - - return check_bytes8(start, value, bytes % 8); -} - static void restore_bytes(struct kmem_cache *s, char *message, u8 data, void *from, void *to) { @@ -712,7 +669,7 @@ static int check_bytes_and_report(struct kmem_cache *s, struct page *page, u8 *fault; u8 *end; - fault = check_bytes(start, value, bytes); + fault = memchr_inv(start, value, bytes); if (!fault) return 1; @@ -805,7 +762,7 @@ static int slab_pad_check(struct kmem_cache *s, struct page *page) if (!remainder) return 1; - fault = check_bytes(end - remainder, POISON_INUSE, remainder); + fault = memchr_inv(end - remainder, POISON_INUSE, remainder); if (!fault) return 1; while (end > fault && end[-1] == POISON_INUSE) -- cgit v1.2.3 From f5252e009d5b87071a919221e4f6624184005368 Mon Sep 17 00:00:00 2001 From: Mitsuo Hayasaka Date: Mon, 31 Oct 2011 17:08:13 -0700 Subject: mm: avoid null pointer access in vm_struct via /proc/vmallocinfo The /proc/vmallocinfo shows information about vmalloc allocations in vmlist that is a linklist of vm_struct. It, however, may access pages field of vm_struct where a page was not allocated. This results in a null pointer access and leads to a kernel panic. Why this happens: In __vmalloc_node_range() called from vmalloc(), newly allocated vm_struct is added to vmlist at __get_vm_area_node() and then, some fields of vm_struct such as nr_pages and pages are set at __vmalloc_area_node(). In other words, it is added to vmlist before it is fully initialized. At the same time, when the /proc/vmallocinfo is read, it accesses the pages field of vm_struct according to the nr_pages field at show_numa_info(). Thus, a null pointer access happens. The patch adds the newly allocated vm_struct to the vmlist *after* it is fully initialized. So, it can avoid accessing the pages field with unallocated page when show_numa_info() is called. Signed-off-by: Mitsuo Hayasaka Cc: Andrew Morton Cc: David Rientjes Cc: Namhyung Kim Cc: "Paul E. McKenney" Cc: Jeremy Fitzhardinge Cc: Cc: Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds --- include/linux/vmalloc.h | 1 + mm/vmalloc.c | 65 ++++++++++++++++++++++++++++++++++++------------- 2 files changed, 49 insertions(+), 17 deletions(-) (limited to 'include/linux') diff --git a/include/linux/vmalloc.h b/include/linux/vmalloc.h index 9332e52ea8c2..687fb11e2010 100644 --- a/include/linux/vmalloc.h +++ b/include/linux/vmalloc.h @@ -13,6 +13,7 @@ struct vm_area_struct; /* vma defining user mapping in mm_types.h */ #define VM_MAP 0x00000004 /* vmap()ed pages */ #define VM_USERMAP 0x00000008 /* suitable for remap_vmalloc_range */ #define VM_VPAGES 0x00000010 /* buffer for pages was vmalloc'ed */ +#define VM_UNLIST 0x00000020 /* vm_struct is not listed in vmlist */ /* bits [20..32] reserved for arch specific ioremap internals */ /* diff --git a/mm/vmalloc.c b/mm/vmalloc.c index 5016f19e1661..56faf3163ee2 100644 --- a/mm/vmalloc.c +++ b/mm/vmalloc.c @@ -1253,18 +1253,22 @@ EXPORT_SYMBOL_GPL(map_vm_area); DEFINE_RWLOCK(vmlist_lock); struct vm_struct *vmlist; -static void insert_vmalloc_vm(struct vm_struct *vm, struct vmap_area *va, +static void setup_vmalloc_vm(struct vm_struct *vm, struct vmap_area *va, unsigned long flags, void *caller) { - struct vm_struct *tmp, **p; - vm->flags = flags; vm->addr = (void *)va->va_start; vm->size = va->va_end - va->va_start; vm->caller = caller; va->private = vm; va->flags |= VM_VM_AREA; +} + +static void insert_vmalloc_vmlist(struct vm_struct *vm) +{ + struct vm_struct *tmp, **p; + vm->flags &= ~VM_UNLIST; write_lock(&vmlist_lock); for (p = &vmlist; (tmp = *p) != NULL; p = &tmp->next) { if (tmp->addr >= vm->addr) @@ -1275,6 +1279,13 @@ static void insert_vmalloc_vm(struct vm_struct *vm, struct vmap_area *va, write_unlock(&vmlist_lock); } +static void insert_vmalloc_vm(struct vm_struct *vm, struct vmap_area *va, + unsigned long flags, void *caller) +{ + setup_vmalloc_vm(vm, va, flags, caller); + insert_vmalloc_vmlist(vm); +} + static struct vm_struct *__get_vm_area_node(unsigned long size, unsigned long align, unsigned long flags, unsigned long start, unsigned long end, int node, gfp_t gfp_mask, void *caller) @@ -1313,7 +1324,18 @@ static struct vm_struct *__get_vm_area_node(unsigned long size, return NULL; } - insert_vmalloc_vm(area, va, flags, caller); + /* + * When this function is called from __vmalloc_node_range, + * we do not add vm_struct to vmlist here to avoid + * accessing uninitialized members of vm_struct such as + * pages and nr_pages fields. They will be set later. + * To distinguish it from others, we use a VM_UNLIST flag. + */ + if (flags & VM_UNLIST) + setup_vmalloc_vm(area, va, flags, caller); + else + insert_vmalloc_vm(area, va, flags, caller); + return area; } @@ -1381,17 +1403,20 @@ struct vm_struct *remove_vm_area(const void *addr) va = find_vmap_area((unsigned long)addr); if (va && va->flags & VM_VM_AREA) { struct vm_struct *vm = va->private; - struct vm_struct *tmp, **p; - /* - * remove from list and disallow access to this vm_struct - * before unmap. (address range confliction is maintained by - * vmap.) - */ - write_lock(&vmlist_lock); - for (p = &vmlist; (tmp = *p) != vm; p = &tmp->next) - ; - *p = tmp->next; - write_unlock(&vmlist_lock); + + if (!(vm->flags & VM_UNLIST)) { + struct vm_struct *tmp, **p; + /* + * remove from list and disallow access to + * this vm_struct before unmap. (address range + * confliction is maintained by vmap.) + */ + write_lock(&vmlist_lock); + for (p = &vmlist; (tmp = *p) != vm; p = &tmp->next) + ; + *p = tmp->next; + write_unlock(&vmlist_lock); + } vmap_debug_free_range(va->va_start, va->va_end); free_unmap_vmap_area(va); @@ -1602,14 +1627,20 @@ void *__vmalloc_node_range(unsigned long size, unsigned long align, if (!size || (size >> PAGE_SHIFT) > totalram_pages) return NULL; - area = __get_vm_area_node(size, align, VM_ALLOC, start, end, node, - gfp_mask, caller); + area = __get_vm_area_node(size, align, VM_ALLOC | VM_UNLIST, + start, end, node, gfp_mask, caller); if (!area) return NULL; addr = __vmalloc_area_node(area, gfp_mask, prot, node, caller); + /* + * In this function, newly allocated vm_struct is not added + * to vmlist at __get_vm_area_node(). so, it is added here. + */ + insert_vmalloc_vmlist(area); + /* * A ref_count = 3 is needed because the vm_struct and vmap_area * structures allocated in the __get_vm_area_node() function contain -- cgit v1.2.3 From 0a93ebef698b08ed04af0d7d913bab8aedfdc253 Mon Sep 17 00:00:00 2001 From: Sam Ravnborg Date: Mon, 31 Oct 2011 17:08:16 -0700 Subject: memblock: add memblock_start_of_DRAM() SPARC32 require access to the start address. Add a new helper memblock_start_of_DRAM() to give access to the address of the first memblock - which contains the lowest address. The awkward name was chosen to match the already present memblock_end_of_DRAM(). Signed-off-by: Sam Ravnborg Cc: "David S. Miller" Cc: Yinghai Lu Acked-by: Tejun Heo Cc: "H. Peter Anvin" Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds --- include/linux/memblock.h | 1 + mm/memblock.c | 6 ++++++ 2 files changed, 7 insertions(+) (limited to 'include/linux') diff --git a/include/linux/memblock.h b/include/linux/memblock.h index 7525e38c434d..e6b843e16e81 100644 --- a/include/linux/memblock.h +++ b/include/linux/memblock.h @@ -80,6 +80,7 @@ extern phys_addr_t __memblock_alloc_base(phys_addr_t size, phys_addr_t align, phys_addr_t max_addr); extern phys_addr_t memblock_phys_mem_size(void); +extern phys_addr_t memblock_start_of_DRAM(void); extern phys_addr_t memblock_end_of_DRAM(void); extern void memblock_enforce_memory_limit(phys_addr_t memory_limit); extern int memblock_is_memory(phys_addr_t addr); diff --git a/mm/memblock.c b/mm/memblock.c index ccbf97339592..b7ed63633581 100644 --- a/mm/memblock.c +++ b/mm/memblock.c @@ -626,6 +626,12 @@ phys_addr_t __init memblock_phys_mem_size(void) return memblock.memory_size; } +/* lowest address */ +phys_addr_t __init_memblock memblock_start_of_DRAM(void) +{ + return memblock.memory.regions[0].base; +} + phys_addr_t __init_memblock memblock_end_of_DRAM(void) { int idx = memblock.memory.cnt - 1; -- cgit v1.2.3 From 37a1c49a91ad55f917a399ef2174b5ebda4283f9 Mon Sep 17 00:00:00 2001 From: Andrea Arcangeli Date: Mon, 31 Oct 2011 17:08:30 -0700 Subject: thp: mremap support and TLB optimization This adds THP support to mremap (decreases the number of split_huge_page() calls). Here are also some benchmarks with a proggy like this: === #define _GNU_SOURCE #include #include #include #include #include #define SIZE (5UL*1024*1024*1024) int main() { static struct timeval oldstamp, newstamp; long diffsec; char *p, *p2, *p3, *p4; if (posix_memalign((void **)&p, 2*1024*1024, SIZE)) perror("memalign"), exit(1); if (posix_memalign((void **)&p2, 2*1024*1024, SIZE)) perror("memalign"), exit(1); if (posix_memalign((void **)&p3, 2*1024*1024, 4096)) perror("memalign"), exit(1); memset(p, 0xff, SIZE); memset(p2, 0xff, SIZE); memset(p3, 0x77, 4096); gettimeofday(&oldstamp, NULL); p4 = mremap(p, SIZE, SIZE, MREMAP_FIXED|MREMAP_MAYMOVE, p3); gettimeofday(&newstamp, NULL); diffsec = newstamp.tv_sec - oldstamp.tv_sec; diffsec = newstamp.tv_usec - oldstamp.tv_usec + 1000000 * diffsec; printf("usec %ld\n", diffsec); if (p == MAP_FAILED || p4 != p3) //if (p == MAP_FAILED) perror("mremap"), exit(1); if (memcmp(p4, p2, SIZE)) printf("mremap bug\n"), exit(1); printf("ok\n"); return 0; } === THP on Performance counter stats for './largepage13' (3 runs): 69195836 dTLB-loads ( +- 3.546% ) (scaled from 50.30%) 60708 dTLB-load-misses ( +- 11.776% ) (scaled from 52.62%) 676266476 dTLB-stores ( +- 5.654% ) (scaled from 69.54%) 29856 dTLB-store-misses ( +- 4.081% ) (scaled from 89.22%) 1055848782 iTLB-loads ( +- 4.526% ) (scaled from 80.18%) 8689 iTLB-load-misses ( +- 2.987% ) (scaled from 58.20%) 7.314454164 seconds time elapsed ( +- 0.023% ) THP off Performance counter stats for './largepage13' (3 runs): 1967379311 dTLB-loads ( +- 0.506% ) (scaled from 60.59%) 9238687 dTLB-load-misses ( +- 22.547% ) (scaled from 61.87%) 2014239444 dTLB-stores ( +- 0.692% ) (scaled from 60.40%) 3312335 dTLB-store-misses ( +- 7.304% ) (scaled from 67.60%) 6764372065 iTLB-loads ( +- 0.925% ) (scaled from 79.00%) 8202 iTLB-load-misses ( +- 0.475% ) (scaled from 70.55%) 9.693655243 seconds time elapsed ( +- 0.069% ) grep thp /proc/vmstat thp_fault_alloc 35849 thp_fault_fallback 0 thp_collapse_alloc 3 thp_collapse_alloc_failed 0 thp_split 0 thp_split 0 confirms no thp split despite plenty of hugepages allocated. The measurement of only the mremap time (so excluding the 3 long memset and final long 10GB memory accessing memcmp): THP on usec 14824 usec 14862 usec 14859 THP off usec 256416 usec 255981 usec 255847 With an older kernel without the mremap optimizations (the below patch optimizes the non THP version too). THP on usec 392107 usec 390237 usec 404124 THP off usec 444294 usec 445237 usec 445820 I guess with a threaded program that sends more IPI on large SMP it'd create an even larger difference. All debug options are off except DEBUG_VM to avoid skewing the results. The only problem for native 2M mremap like it happens above both the source and destination address must be 2M aligned or the hugepmd can't be moved without a split but that is an hardware limitation. [akpm@linux-foundation.org: coding-style nitpicking] Signed-off-by: Andrea Arcangeli Acked-by: Johannes Weiner Acked-by: Mel Gorman Acked-by: Rik van Riel Cc: Hugh Dickins Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds --- include/linux/huge_mm.h | 5 +++++ mm/huge_memory.c | 45 +++++++++++++++++++++++++++++++++++++++++++++ mm/mremap.c | 22 ++++++++++++++++++---- 3 files changed, 68 insertions(+), 4 deletions(-) (limited to 'include/linux') diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h index 48c32ebf65a7..a9ace9c32507 100644 --- a/include/linux/huge_mm.h +++ b/include/linux/huge_mm.h @@ -22,6 +22,11 @@ extern int zap_huge_pmd(struct mmu_gather *tlb, extern int mincore_huge_pmd(struct vm_area_struct *vma, pmd_t *pmd, unsigned long addr, unsigned long end, unsigned char *vec); +extern int move_huge_pmd(struct vm_area_struct *vma, + struct vm_area_struct *new_vma, + unsigned long old_addr, + unsigned long new_addr, unsigned long old_end, + pmd_t *old_pmd, pmd_t *new_pmd); extern int change_huge_pmd(struct vm_area_struct *vma, pmd_t *pmd, unsigned long addr, pgprot_t newprot); diff --git a/mm/huge_memory.c b/mm/huge_memory.c index e2d1587be269..6b072bdccf81 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -1052,6 +1052,51 @@ int mincore_huge_pmd(struct vm_area_struct *vma, pmd_t *pmd, return ret; } +int move_huge_pmd(struct vm_area_struct *vma, struct vm_area_struct *new_vma, + unsigned long old_addr, + unsigned long new_addr, unsigned long old_end, + pmd_t *old_pmd, pmd_t *new_pmd) +{ + int ret = 0; + pmd_t pmd; + + struct mm_struct *mm = vma->vm_mm; + + if ((old_addr & ~HPAGE_PMD_MASK) || + (new_addr & ~HPAGE_PMD_MASK) || + old_end - old_addr < HPAGE_PMD_SIZE || + (new_vma->vm_flags & VM_NOHUGEPAGE)) + goto out; + + /* + * The destination pmd shouldn't be established, free_pgtables() + * should have release it. + */ + if (WARN_ON(!pmd_none(*new_pmd))) { + VM_BUG_ON(pmd_trans_huge(*new_pmd)); + goto out; + } + + spin_lock(&mm->page_table_lock); + if (likely(pmd_trans_huge(*old_pmd))) { + if (pmd_trans_splitting(*old_pmd)) { + spin_unlock(&mm->page_table_lock); + wait_split_huge_page(vma->anon_vma, old_pmd); + ret = -1; + } else { + pmd = pmdp_get_and_clear(mm, old_addr, old_pmd); + VM_BUG_ON(!pmd_none(*new_pmd)); + set_pmd_at(mm, new_addr, new_pmd, pmd); + spin_unlock(&mm->page_table_lock); + ret = 1; + } + } else { + spin_unlock(&mm->page_table_lock); + } +out: + return ret; +} + int change_huge_pmd(struct vm_area_struct *vma, pmd_t *pmd, unsigned long addr, pgprot_t newprot) { diff --git a/mm/mremap.c b/mm/mremap.c index a184f3732e1e..d6959cb4df58 100644 --- a/mm/mremap.c +++ b/mm/mremap.c @@ -41,8 +41,7 @@ static pmd_t *get_old_pmd(struct mm_struct *mm, unsigned long addr) return NULL; pmd = pmd_offset(pud, addr); - split_huge_page_pmd(mm, pmd); - if (pmd_none_or_clear_bad(pmd)) + if (pmd_none(*pmd)) return NULL; return pmd; @@ -65,8 +64,6 @@ static pmd_t *alloc_new_pmd(struct mm_struct *mm, struct vm_area_struct *vma, return NULL; VM_BUG_ON(pmd_trans_huge(*pmd)); - if (pmd_none(*pmd) && __pte_alloc(mm, vma, pmd, addr)) - return NULL; return pmd; } @@ -149,6 +146,23 @@ unsigned long move_page_tables(struct vm_area_struct *vma, new_pmd = alloc_new_pmd(vma->vm_mm, vma, new_addr); if (!new_pmd) break; + if (pmd_trans_huge(*old_pmd)) { + int err = 0; + if (extent == HPAGE_PMD_SIZE) + err = move_huge_pmd(vma, new_vma, old_addr, + new_addr, old_end, + old_pmd, new_pmd); + if (err > 0) { + need_flush = true; + continue; + } else if (!err) { + split_huge_page_pmd(vma->vm_mm, old_pmd); + } + VM_BUG_ON(pmd_trans_huge(*old_pmd)); + } + if (pmd_none(*new_pmd) && __pte_alloc(new_vma->vm_mm, new_vma, + new_pmd, new_addr)) + break; next = (new_addr + PMD_SIZE) & PMD_MASK; if (extent > next - new_addr) extent = next - new_addr; -- cgit v1.2.3 From 3ee9a4f086716d792219c021e8509f91165a4128 Mon Sep 17 00:00:00 2001 From: Joe Perches Date: Mon, 31 Oct 2011 17:08:35 -0700 Subject: mm: neaten warn_alloc_failed Add __attribute__((format (printf...) to the function to validate format and arguments. Use vsprintf extension %pV to avoid any possible message interleaving. Coalesce format string. Convert printks/pr_warning to pr_warn. [akpm@linux-foundation.org: use the __printf() macro] Signed-off-by: Joe Perches Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds --- include/linux/mm.h | 3 ++- mm/page_alloc.c | 16 +++++++++++----- mm/vmalloc.c | 4 ++-- 3 files changed, 15 insertions(+), 8 deletions(-) (limited to 'include/linux') diff --git a/include/linux/mm.h b/include/linux/mm.h index 7438071b44aa..3b3e3b8bb706 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -1334,7 +1334,8 @@ extern void si_meminfo(struct sysinfo * val); extern void si_meminfo_node(struct sysinfo *val, int nid); extern int after_bootmem; -extern void warn_alloc_failed(gfp_t gfp_mask, int order, const char *fmt, ...); +extern __printf(3, 4) +void warn_alloc_failed(gfp_t gfp_mask, int order, const char *fmt, ...); extern void setup_per_cpu_pageset(void); diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 83a02052bce4..9dd443d89d8b 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -1754,7 +1754,6 @@ static DEFINE_RATELIMIT_STATE(nopage_rs, void warn_alloc_failed(gfp_t gfp_mask, int order, const char *fmt, ...) { - va_list args; unsigned int filter = SHOW_MEM_FILTER_NODES; if ((gfp_mask & __GFP_NOWARN) || !__ratelimit(&nopage_rs)) @@ -1773,14 +1772,21 @@ void warn_alloc_failed(gfp_t gfp_mask, int order, const char *fmt, ...) filter &= ~SHOW_MEM_FILTER_NODES; if (fmt) { - printk(KERN_WARNING); + struct va_format vaf; + va_list args; + va_start(args, fmt); - vprintk(fmt, args); + + vaf.fmt = fmt; + vaf.va = &args; + + pr_warn("%pV", &vaf); + va_end(args); } - pr_warning("%s: page allocation failure: order:%d, mode:0x%x\n", - current->comm, order, gfp_mask); + pr_warn("%s: page allocation failure: order:%d, mode:0x%x\n", + current->comm, order, gfp_mask); dump_stack(); if (!should_suppress_show_mem()) diff --git a/mm/vmalloc.c b/mm/vmalloc.c index 56faf3163ee2..08ab0aa1406c 100644 --- a/mm/vmalloc.c +++ b/mm/vmalloc.c @@ -1593,8 +1593,8 @@ static void *__vmalloc_area_node(struct vm_struct *area, gfp_t gfp_mask, return area->addr; fail: - warn_alloc_failed(gfp_mask, order, "vmalloc: allocation failure, " - "allocated %ld of %ld bytes\n", + warn_alloc_failed(gfp_mask, order, + "vmalloc: allocation failure, allocated %ld of %ld bytes\n", (area->nr_pages*PAGE_SIZE), area->size); vfree(area->addr); return NULL; -- cgit v1.2.3 From 09f363c7363eb10cfb4b82094bd7064e5608258b Mon Sep 17 00:00:00 2001 From: Mikulas Patocka Date: Mon, 31 Oct 2011 17:08:57 -0700 Subject: vmscan: fix shrinker callback bug in fs/super.c The callback must not return -1 when nr_to_scan is zero. Fix the bug in fs/super.c and add this requirement to the callback specification. Signed-off-by: Mikulas Patocka Cc: Dave Chinner Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds --- fs/super.c | 2 +- include/linux/shrinker.h | 1 + 2 files changed, 2 insertions(+), 1 deletion(-) (limited to 'include/linux') diff --git a/fs/super.c b/fs/super.c index 3f56a269a4f4..32a81f3467e0 100644 --- a/fs/super.c +++ b/fs/super.c @@ -61,7 +61,7 @@ static int prune_super(struct shrinker *shrink, struct shrink_control *sc) return -1; if (!grab_super_passive(sb)) - return -1; + return !sc->nr_to_scan ? 0 : -1; if (sb->s_op && sb->s_op->nr_cached_objects) fs_objects = sb->s_op->nr_cached_objects(sb); diff --git a/include/linux/shrinker.h b/include/linux/shrinker.h index 790651b4e5ba..a83833a1f7a2 100644 --- a/include/linux/shrinker.h +++ b/include/linux/shrinker.h @@ -20,6 +20,7 @@ struct shrink_control { * 'nr_to_scan' entries and attempt to free them up. It should return * the number of objects which remain in the cache. If it returns -1, it means * it cannot do any scanning at this time (eg. there is a risk of deadlock). + * The callback must not return -1 if nr_to_scan is zero. * * The 'gfpmask' refers to the allocation we are currently trying to * fulfil. -- cgit v1.2.3 From d43a87e68e9e71d2987a29cc239acec4e8f410c9 Mon Sep 17 00:00:00 2001 From: Kyungmin Park Date: Mon, 31 Oct 2011 17:09:08 -0700 Subject: mm: compaction: make compact_zone_order() static There's no compact_zone_order() user outside file scope, so make it static. Signed-off-by: Kyungmin Park Acked-by: David Rientjes Reviewed-by: Minchan Kim Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds --- include/linux/compaction.h | 8 -------- mm/compaction.c | 2 +- 2 files changed, 1 insertion(+), 9 deletions(-) (limited to 'include/linux') diff --git a/include/linux/compaction.h b/include/linux/compaction.h index cc9f7a428649..bb2bbdbe5464 100644 --- a/include/linux/compaction.h +++ b/include/linux/compaction.h @@ -24,8 +24,6 @@ extern unsigned long try_to_compact_pages(struct zonelist *zonelist, int order, gfp_t gfp_mask, nodemask_t *mask, bool sync); extern unsigned long compaction_suitable(struct zone *zone, int order); -extern unsigned long compact_zone_order(struct zone *zone, int order, - gfp_t gfp_mask, bool sync); /* Do not skip compaction more than 64 times */ #define COMPACT_MAX_DEFER_SHIFT 6 @@ -69,12 +67,6 @@ static inline unsigned long compaction_suitable(struct zone *zone, int order) return COMPACT_SKIPPED; } -static inline unsigned long compact_zone_order(struct zone *zone, int order, - gfp_t gfp_mask, bool sync) -{ - return COMPACT_CONTINUE; -} - static inline void defer_compaction(struct zone *zone) { } diff --git a/mm/compaction.c b/mm/compaction.c index a0e420207ebf..899d95638586 100644 --- a/mm/compaction.c +++ b/mm/compaction.c @@ -582,7 +582,7 @@ out: return ret; } -unsigned long compact_zone_order(struct zone *zone, +static unsigned long compact_zone_order(struct zone *zone, int order, gfp_t gfp_mask, bool sync) { -- cgit v1.2.3 From ec400c9fab99d16a491cea17d27d0c6a5780b97c Mon Sep 17 00:00:00 2001 From: Mark Brown Date: Mon, 31 Oct 2011 17:11:07 -0700 Subject: lis3lv02d: make regulator API usage unconditional MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The regulator API contains a range of features for stubbing itself out when not in use and for transparently restricting the actual effect of regulator API calls where they can't be supported on a particular system so that drivers don't need to individually implement this. Simplify the driver slightly by making use of this idiom. The only in tree user is ecovec24 which does not use the regulator API. Signed-off-by: Mark Brown Cc: Éric Piel Cc: Ilkka Koskinen Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds --- drivers/misc/lis3lv02d/lis3lv02d_i2c.c | 34 ++++++++++++---------------------- include/linux/lis3lv02d.h | 1 - 2 files changed, 12 insertions(+), 23 deletions(-) (limited to 'include/linux') diff --git a/drivers/misc/lis3lv02d/lis3lv02d_i2c.c b/drivers/misc/lis3lv02d/lis3lv02d_i2c.c index 6cdc38f6a9a8..c02fea029dcf 100644 --- a/drivers/misc/lis3lv02d/lis3lv02d_i2c.c +++ b/drivers/misc/lis3lv02d/lis3lv02d_i2c.c @@ -79,8 +79,7 @@ static int lis3_i2c_init(struct lis3lv02d *lis3) u8 reg; int ret; - if (lis3->reg_ctrl) - lis3_reg_ctrl(lis3, LIS3_REG_ON); + lis3_reg_ctrl(lis3, LIS3_REG_ON); lis3->read(lis3, WHO_AM_I, ®); if (reg != lis3->whoami) @@ -106,10 +105,6 @@ static int __devinit lis3lv02d_i2c_probe(struct i2c_client *client, struct lis3lv02d_platform_data *pdata = client->dev.platform_data; if (pdata) { - /* Regulator control is optional */ - if (pdata->driver_features & LIS3_USE_REGULATOR_CTRL) - lis3_dev.reg_ctrl = lis3_reg_ctrl; - if ((pdata->driver_features & LIS3_USE_BLOCK_READ) && (i2c_check_functionality(client->adapter, I2C_FUNC_SMBUS_I2C_BLOCK))) @@ -131,15 +126,13 @@ static int __devinit lis3lv02d_i2c_probe(struct i2c_client *client, goto fail; } - if (lis3_dev.reg_ctrl) { - lis3_dev.regulators[0].supply = reg_vdd; - lis3_dev.regulators[1].supply = reg_vdd_io; - ret = regulator_bulk_get(&client->dev, - ARRAY_SIZE(lis3_dev.regulators), - lis3_dev.regulators); - if (ret < 0) - goto fail; - } + lis3_dev.regulators[0].supply = reg_vdd; + lis3_dev.regulators[1].supply = reg_vdd_io; + ret = regulator_bulk_get(&client->dev, + ARRAY_SIZE(lis3_dev.regulators), + lis3_dev.regulators); + if (ret < 0) + goto fail; lis3_dev.pdata = pdata; lis3_dev.bus_priv = client; @@ -153,13 +146,11 @@ static int __devinit lis3lv02d_i2c_probe(struct i2c_client *client, i2c_set_clientdata(client, &lis3_dev); /* Provide power over the init call */ - if (lis3_dev.reg_ctrl) - lis3_reg_ctrl(&lis3_dev, LIS3_REG_ON); + lis3_reg_ctrl(&lis3_dev, LIS3_REG_ON); ret = lis3lv02d_init_device(&lis3_dev); - if (lis3_dev.reg_ctrl) - lis3_reg_ctrl(&lis3_dev, LIS3_REG_OFF); + lis3_reg_ctrl(&lis3_dev, LIS3_REG_OFF); if (ret) goto fail2; @@ -185,9 +176,8 @@ static int __devexit lis3lv02d_i2c_remove(struct i2c_client *client) lis3lv02d_joystick_disable(lis3); lis3lv02d_remove_fs(&lis3_dev); - if (lis3_dev.reg_ctrl) - regulator_bulk_free(ARRAY_SIZE(lis3->regulators), - lis3_dev.regulators); + regulator_bulk_free(ARRAY_SIZE(lis3->regulators), + lis3_dev.regulators); return 0; } diff --git a/include/linux/lis3lv02d.h b/include/linux/lis3lv02d.h index d4292c8431e0..f1664c636af0 100644 --- a/include/linux/lis3lv02d.h +++ b/include/linux/lis3lv02d.h @@ -113,7 +113,6 @@ struct lis3lv02d_platform_data { s8 axis_x; s8 axis_y; s8 axis_z; -#define LIS3_USE_REGULATOR_CTRL 0x01 #define LIS3_USE_BLOCK_READ 0x02 u16 driver_features; int default_rate; -- cgit v1.2.3 From b9075fa968a0a4347aef35e235e2995c0e57dddd Mon Sep 17 00:00:00 2001 From: Joe Perches Date: Mon, 31 Oct 2011 17:11:33 -0700 Subject: treewide: use __printf not __attribute__((format(printf,...))) Standardize the style for compiler based printf format verification. Standardized the location of __printf too. Done via script and a little typing. $ grep -rPl --include=*.[ch] -w "__attribute__" * | \ grep -vP "^(tools|scripts|include/linux/compiler-gcc.h)" | \ xargs perl -n -i -e 'local $/; while (<>) { s/\b__attribute__\s*\(\s*\(\s*format\s*\(\s*printf\s*,\s*(.+)\s*,\s*(.+)\s*\)\s*\)\s*\)/__printf($1, $2)/g ; print; }' [akpm@linux-foundation.org: revert arch bits] Signed-off-by: Joe Perches Cc: "Kirill A. Shutemov" Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds --- drivers/isdn/hisax/callc.c | 4 +- drivers/isdn/hisax/hisax.h | 4 +- drivers/isdn/hisax/isdnl1.h | 2 +- drivers/isdn/hisax/isdnl3.c | 2 +- drivers/isdn/hisax/st5481_d.c | 4 +- drivers/net/ethernet/mellanox/mlx4/mlx4_en.h | 3 +- drivers/net/wireless/ath/ath.h | 5 +- drivers/net/wireless/ath/ath5k/debug.h | 4 +- drivers/net/wireless/ath/ath6kl/debug.h | 4 +- drivers/net/wireless/b43/b43.h | 12 +-- drivers/net/wireless/b43legacy/b43legacy.h | 16 ++-- drivers/staging/iio/trigger.h | 3 +- fs/ecryptfs/ecryptfs_kernel.h | 2 +- fs/ext2/ext2.h | 8 +- fs/ext4/ext4.h | 44 +++++------ fs/fat/fat.h | 9 +-- fs/gfs2/glock.h | 2 +- fs/hpfs/hpfs_fn.h | 4 +- fs/nilfs2/nilfs.h | 8 +- fs/ntfs/debug.h | 15 ++-- fs/ocfs2/super.h | 14 ++-- fs/partitions/ldm.c | 16 ++-- fs/udf/udfdecl.h | 4 +- fs/ufs/ufs.h | 9 ++- fs/xfs/xfs_message.h | 42 +++++----- include/asm-generic/bug.h | 11 +-- include/drm/drmP.h | 10 +-- include/linux/audit.h | 11 ++- include/linux/blktrace_api.h | 2 +- include/linux/device.h | 110 +++++++++++++-------------- include/linux/dynamic_debug.h | 19 +++-- include/linux/ext3_fs.h | 16 ++-- include/linux/fs.h | 4 +- include/linux/fscache-cache.h | 8 +- include/linux/gameport.h | 8 +- include/linux/kallsyms.h | 5 +- include/linux/kdb.h | 9 +-- include/linux/kernel.h | 44 +++++------ include/linux/kexec.h | 4 +- include/linux/kmod.h | 4 +- include/linux/kobject.h | 26 +++---- include/linux/kthread.h | 4 +- include/linux/libata.h | 18 ++--- include/linux/mmiotrace.h | 8 +- include/linux/netdevice.h | 34 ++++----- include/linux/printk.h | 12 +-- include/linux/quotaops.h | 2 +- include/linux/seq_file.h | 3 +- include/linux/trace_seq.h | 8 +- include/net/bluetooth/bluetooth.h | 2 +- include/net/netfilter/nf_log.h | 3 +- include/net/sock.h | 4 +- include/sound/core.h | 4 +- include/sound/info.h | 4 +- include/sound/seq_kernel.h | 4 +- include/xen/hvc-console.h | 4 +- include/xen/xenbus.h | 12 +-- net/nfc/nfc.h | 2 +- net/rds/rds.h | 8 +- net/sunrpc/svc.c | 5 +- sound/firewire/cmp.c | 2 +- 61 files changed, 324 insertions(+), 350 deletions(-) (limited to 'include/linux') diff --git a/drivers/isdn/hisax/callc.c b/drivers/isdn/hisax/callc.c index 37e685eafd24..c4897e1075d8 100644 --- a/drivers/isdn/hisax/callc.c +++ b/drivers/isdn/hisax/callc.c @@ -65,7 +65,7 @@ hisax_findcard(int driverid) return (struct IsdnCardState *) 0; } -static __attribute__((format(printf, 3, 4))) void +static __printf(3, 4) void link_debug(struct Channel *chanp, int direction, char *fmt, ...) { va_list args; @@ -1068,7 +1068,7 @@ init_d_st(struct Channel *chanp) return 0; } -static __attribute__((format(printf, 2, 3))) void +static __printf(2, 3) void callc_debug(struct FsmInst *fi, char *fmt, ...) { va_list args; diff --git a/drivers/isdn/hisax/hisax.h b/drivers/isdn/hisax/hisax.h index 0a5c42a3f125..aff45a11a92d 100644 --- a/drivers/isdn/hisax/hisax.h +++ b/drivers/isdn/hisax/hisax.h @@ -1287,9 +1287,9 @@ int jiftime(char *s, long mark); int HiSax_command(isdn_ctrl * ic); int HiSax_writebuf_skb(int id, int chan, int ack, struct sk_buff *skb); -__attribute__((format(printf, 3, 4))) +__printf(3, 4) void HiSax_putstatus(struct IsdnCardState *cs, char *head, char *fmt, ...); -__attribute__((format(printf, 3, 0))) +__printf(3, 0) void VHiSax_putstatus(struct IsdnCardState *cs, char *head, char *fmt, va_list args); void HiSax_reportcard(int cardnr, int sel); int QuickHex(char *txt, u_char * p, int cnt); diff --git a/drivers/isdn/hisax/isdnl1.h b/drivers/isdn/hisax/isdnl1.h index 425d86116f2b..66ddcab19bba 100644 --- a/drivers/isdn/hisax/isdnl1.h +++ b/drivers/isdn/hisax/isdnl1.h @@ -21,7 +21,7 @@ #define B_XMTBUFREADY 1 #define B_ACKPENDING 2 -__attribute__((format(printf, 2, 3))) +__printf(2, 3) void debugl1(struct IsdnCardState *cs, char *fmt, ...); void DChannel_proc_xmt(struct IsdnCardState *cs); void DChannel_proc_rcv(struct IsdnCardState *cs); diff --git a/drivers/isdn/hisax/isdnl3.c b/drivers/isdn/hisax/isdnl3.c index ad291f21b201..1c24e4457b6f 100644 --- a/drivers/isdn/hisax/isdnl3.c +++ b/drivers/isdn/hisax/isdnl3.c @@ -66,7 +66,7 @@ static char *strL3Event[] = "EV_TIMEOUT", }; -static __attribute__((format(printf, 2, 3))) void +static __printf(2, 3) void l3m_debug(struct FsmInst *fi, char *fmt, ...) { va_list args; diff --git a/drivers/isdn/hisax/st5481_d.c b/drivers/isdn/hisax/st5481_d.c index 44082637a09f..db247b79e561 100644 --- a/drivers/isdn/hisax/st5481_d.c +++ b/drivers/isdn/hisax/st5481_d.c @@ -167,7 +167,7 @@ static struct FsmNode L1FnList[] __initdata = {ST_L1_F8, EV_IND_RSY, l1_ignore}, }; -static __attribute__((format(printf, 2, 3))) +static __printf(2, 3) void l1m_debug(struct FsmInst *fi, char *fmt, ...) { va_list args; @@ -270,7 +270,7 @@ static char *strDoutEvent[] = "EV_DOUT_UNDERRUN", }; -static __attribute__((format(printf, 2, 3))) +static __printf(2, 3) void dout_debug(struct FsmInst *fi, char *fmt, ...) { va_list args; diff --git a/drivers/net/ethernet/mellanox/mlx4/mlx4_en.h b/drivers/net/ethernet/mellanox/mlx4/mlx4_en.h index fca66165110e..8fda331c65df 100644 --- a/drivers/net/ethernet/mellanox/mlx4/mlx4_en.h +++ b/drivers/net/ethernet/mellanox/mlx4/mlx4_en.h @@ -581,8 +581,9 @@ extern const struct ethtool_ops mlx4_en_ethtool_ops; * printk / logging functions */ +__printf(3, 4) int en_print(const char *level, const struct mlx4_en_priv *priv, - const char *format, ...) __attribute__ ((format (printf, 3, 4))); + const char *format, ...); #define en_dbg(mlevel, priv, format, arg...) \ do { \ diff --git a/drivers/net/wireless/ath/ath.h b/drivers/net/wireless/ath/ath.h index 908fdbc3e0ee..0f9ee46cfc97 100644 --- a/drivers/net/wireless/ath/ath.h +++ b/drivers/net/wireless/ath/ath.h @@ -173,8 +173,7 @@ bool ath_hw_keyreset(struct ath_common *common, u16 entry); void ath_hw_cycle_counters_update(struct ath_common *common); int32_t ath_hw_get_listen_time(struct ath_common *common); -extern __attribute__((format (printf, 2, 3))) -void ath_printk(const char *level, const char *fmt, ...); +extern __printf(2, 3) void ath_printk(const char *level, const char *fmt, ...); #define _ath_printk(level, common, fmt, ...) \ do { \ @@ -258,7 +257,7 @@ do { \ #else -static inline __attribute__((format (printf, 3, 4))) +static inline __attribute__ ((format (printf, 3, 4))) void ath_dbg(struct ath_common *common, enum ATH_DEBUG dbg_mask, const char *fmt, ...) { diff --git a/drivers/net/wireless/ath/ath5k/debug.h b/drivers/net/wireless/ath/ath5k/debug.h index 7f37df3125fd..0a3f916a1ef3 100644 --- a/drivers/net/wireless/ath/ath5k/debug.h +++ b/drivers/net/wireless/ath/ath5k/debug.h @@ -141,10 +141,10 @@ ath5k_debug_printtxbuf(struct ath5k_hw *ah, struct ath5k_buf *bf); #include -static inline void __attribute__ ((format (printf, 3, 4))) +static inline __printf(3, 4) void ATH5K_DBG(struct ath5k_hw *ah, unsigned int m, const char *fmt, ...) {} -static inline void __attribute__ ((format (printf, 3, 4))) +static inline __printf(3, 4) void ATH5K_DBG_UNLIMIT(struct ath5k_hw *ah, unsigned int m, const char *fmt, ...) {} diff --git a/drivers/net/wireless/ath/ath6kl/debug.h b/drivers/net/wireless/ath/ath6kl/debug.h index 9288a3ce1e39..7b7675f70a10 100644 --- a/drivers/net/wireless/ath/ath6kl/debug.h +++ b/drivers/net/wireless/ath/ath6kl/debug.h @@ -44,8 +44,8 @@ enum ATH6K_DEBUG_MASK { }; extern unsigned int debug_mask; -extern int ath6kl_printk(const char *level, const char *fmt, ...) - __attribute__ ((format (printf, 2, 3))); +extern __printf(2, 3) +int ath6kl_printk(const char *level, const char *fmt, ...); #define ath6kl_info(fmt, ...) \ ath6kl_printk(KERN_INFO, fmt, ##__VA_ARGS__) diff --git a/drivers/net/wireless/b43/b43.h b/drivers/net/wireless/b43/b43.h index 447a2307c9d9..37110dfd2c96 100644 --- a/drivers/net/wireless/b43/b43.h +++ b/drivers/net/wireless/b43/b43.h @@ -1011,14 +1011,10 @@ static inline bool b43_using_pio_transfers(struct b43_wldev *dev) } /* Message printing */ -void b43info(struct b43_wl *wl, const char *fmt, ...) - __attribute__ ((format(printf, 2, 3))); -void b43err(struct b43_wl *wl, const char *fmt, ...) - __attribute__ ((format(printf, 2, 3))); -void b43warn(struct b43_wl *wl, const char *fmt, ...) - __attribute__ ((format(printf, 2, 3))); -void b43dbg(struct b43_wl *wl, const char *fmt, ...) - __attribute__ ((format(printf, 2, 3))); +__printf(2, 3) void b43info(struct b43_wl *wl, const char *fmt, ...); +__printf(2, 3) void b43err(struct b43_wl *wl, const char *fmt, ...); +__printf(2, 3) void b43warn(struct b43_wl *wl, const char *fmt, ...); +__printf(2, 3) void b43dbg(struct b43_wl *wl, const char *fmt, ...); /* A WARN_ON variant that vanishes when b43 debugging is disabled. diff --git a/drivers/net/wireless/b43legacy/b43legacy.h b/drivers/net/wireless/b43legacy/b43legacy.h index 12b518251581..1d4fc9db7f5e 100644 --- a/drivers/net/wireless/b43legacy/b43legacy.h +++ b/drivers/net/wireless/b43legacy/b43legacy.h @@ -810,15 +810,15 @@ struct b43legacy_lopair *b43legacy_get_lopair(struct b43legacy_phy *phy, /* Message printing */ -void b43legacyinfo(struct b43legacy_wl *wl, const char *fmt, ...) - __attribute__((format(printf, 2, 3))); -void b43legacyerr(struct b43legacy_wl *wl, const char *fmt, ...) - __attribute__((format(printf, 2, 3))); -void b43legacywarn(struct b43legacy_wl *wl, const char *fmt, ...) - __attribute__((format(printf, 2, 3))); +__printf(2, 3) +void b43legacyinfo(struct b43legacy_wl *wl, const char *fmt, ...); +__printf(2, 3) +void b43legacyerr(struct b43legacy_wl *wl, const char *fmt, ...); +__printf(2, 3) +void b43legacywarn(struct b43legacy_wl *wl, const char *fmt, ...); #if B43legacy_DEBUG -void b43legacydbg(struct b43legacy_wl *wl, const char *fmt, ...) - __attribute__((format(printf, 2, 3))); +__printf(2, 3) +void b43legacydbg(struct b43legacy_wl *wl, const char *fmt, ...); #else /* DEBUG */ # define b43legacydbg(wl, fmt...) do { /* nothing */ } while (0) #endif /* DEBUG */ diff --git a/drivers/staging/iio/trigger.h b/drivers/staging/iio/trigger.h index 598fcb3599f9..5cc42a655c88 100644 --- a/drivers/staging/iio/trigger.h +++ b/drivers/staging/iio/trigger.h @@ -115,8 +115,7 @@ void iio_trigger_poll_chained(struct iio_trigger *trig, s64 time); irqreturn_t iio_trigger_generic_data_rdy_poll(int irq, void *private); -struct iio_trigger *iio_allocate_trigger(const char *fmt, ...) - __attribute__((format(printf, 1, 2))); +__printf(1, 2) struct iio_trigger *iio_allocate_trigger(const char *fmt, ...); void iio_free_trigger(struct iio_trigger *trig); #endif /* _IIO_TRIGGER_H_ */ diff --git a/fs/ecryptfs/ecryptfs_kernel.h b/fs/ecryptfs/ecryptfs_kernel.h index b36c5572b3f3..54481a3b2c79 100644 --- a/fs/ecryptfs/ecryptfs_kernel.h +++ b/fs/ecryptfs/ecryptfs_kernel.h @@ -514,7 +514,7 @@ ecryptfs_set_dentry_lower_mnt(struct dentry *dentry, struct vfsmount *lower_mnt) #define ecryptfs_printk(type, fmt, arg...) \ __ecryptfs_printk(type "%s: " fmt, __func__, ## arg); -__attribute__ ((format(printf, 1, 2))) +__printf(1, 2) void __ecryptfs_printk(const char *fmt, ...); extern const struct file_operations ecryptfs_main_fops; diff --git a/fs/ext2/ext2.h b/fs/ext2/ext2.h index af9fc89b1b2d..9a4e5e206d08 100644 --- a/fs/ext2/ext2.h +++ b/fs/ext2/ext2.h @@ -135,10 +135,10 @@ extern long ext2_compat_ioctl(struct file *, unsigned int, unsigned long); struct dentry *ext2_get_parent(struct dentry *child); /* super.c */ -extern void ext2_error (struct super_block *, const char *, const char *, ...) - __attribute__ ((format (printf, 3, 4))); -extern void ext2_msg(struct super_block *, const char *, const char *, ...) - __attribute__ ((format (printf, 3, 4))); +extern __printf(3, 4) +void ext2_error(struct super_block *, const char *, const char *, ...); +extern __printf(3, 4) +void ext2_msg(struct super_block *, const char *, const char *, ...); extern void ext2_update_dynamic_rev (struct super_block *sb); extern void ext2_write_super (struct super_block *); diff --git a/fs/ext4/ext4.h b/fs/ext4/ext4.h index b7d7bd0f066e..cec3145e532c 100644 --- a/fs/ext4/ext4.h +++ b/fs/ext4/ext4.h @@ -1878,40 +1878,40 @@ extern int ext4_group_extend(struct super_block *sb, extern void *ext4_kvmalloc(size_t size, gfp_t flags); extern void *ext4_kvzalloc(size_t size, gfp_t flags); extern void ext4_kvfree(void *ptr); -extern void __ext4_error(struct super_block *, const char *, unsigned int, - const char *, ...) - __attribute__ ((format (printf, 4, 5))); +extern __printf(4, 5) +void __ext4_error(struct super_block *, const char *, unsigned int, + const char *, ...); #define ext4_error(sb, message...) __ext4_error(sb, __func__, \ __LINE__, ## message) -extern void ext4_error_inode(struct inode *, const char *, unsigned int, - ext4_fsblk_t, const char *, ...) - __attribute__ ((format (printf, 5, 6))); -extern void ext4_error_file(struct file *, const char *, unsigned int, - ext4_fsblk_t, const char *, ...) - __attribute__ ((format (printf, 5, 6))); +extern __printf(5, 6) +void ext4_error_inode(struct inode *, const char *, unsigned int, ext4_fsblk_t, + const char *, ...); +extern __printf(5, 6) +void ext4_error_file(struct file *, const char *, unsigned int, ext4_fsblk_t, + const char *, ...); extern void __ext4_std_error(struct super_block *, const char *, unsigned int, int); -extern void __ext4_abort(struct super_block *, const char *, unsigned int, - const char *, ...) - __attribute__ ((format (printf, 4, 5))); +extern __printf(4, 5) +void __ext4_abort(struct super_block *, const char *, unsigned int, + const char *, ...); #define ext4_abort(sb, message...) __ext4_abort(sb, __func__, \ __LINE__, ## message) -extern void __ext4_warning(struct super_block *, const char *, unsigned int, - const char *, ...) - __attribute__ ((format (printf, 4, 5))); +extern __printf(4, 5) +void __ext4_warning(struct super_block *, const char *, unsigned int, + const char *, ...); #define ext4_warning(sb, message...) __ext4_warning(sb, __func__, \ __LINE__, ## message) -extern void ext4_msg(struct super_block *, const char *, const char *, ...) - __attribute__ ((format (printf, 3, 4))); +extern __printf(3, 4) +void ext4_msg(struct super_block *, const char *, const char *, ...); extern void __dump_mmp_msg(struct super_block *, struct mmp_struct *mmp, const char *, unsigned int, const char *); #define dump_mmp_msg(sb, mmp, msg) __dump_mmp_msg(sb, mmp, __func__, \ __LINE__, msg) -extern void __ext4_grp_locked_error(const char *, unsigned int, \ - struct super_block *, ext4_group_t, \ - unsigned long, ext4_fsblk_t, \ - const char *, ...) - __attribute__ ((format (printf, 7, 8))); +extern __printf(7, 8) +void __ext4_grp_locked_error(const char *, unsigned int, + struct super_block *, ext4_group_t, + unsigned long, ext4_fsblk_t, + const char *, ...); #define ext4_grp_locked_error(sb, grp, message...) \ __ext4_grp_locked_error(__func__, __LINE__, (sb), (grp), ## message) extern void ext4_update_dynamic_rev(struct super_block *sb); diff --git a/fs/fat/fat.h b/fs/fat/fat.h index a5d3853822e0..1510a4d51990 100644 --- a/fs/fat/fat.h +++ b/fs/fat/fat.h @@ -326,15 +326,14 @@ extern int fat_fill_super(struct super_block *sb, void *data, int silent, extern int fat_flush_inodes(struct super_block *sb, struct inode *i1, struct inode *i2); /* fat/misc.c */ -extern void -__fat_fs_error(struct super_block *sb, int report, const char *fmt, ...) - __attribute__ ((format (printf, 3, 4))) __cold; +extern __printf(3, 4) __cold +void __fat_fs_error(struct super_block *sb, int report, const char *fmt, ...); #define fat_fs_error(sb, fmt, args...) \ __fat_fs_error(sb, 1, fmt , ## args) #define fat_fs_error_ratelimit(sb, fmt, args...) \ __fat_fs_error(sb, __ratelimit(&MSDOS_SB(sb)->ratelimit), fmt , ## args) -void fat_msg(struct super_block *sb, const char *level, const char *fmt, ...) - __attribute__ ((format (printf, 3, 4))) __cold; +__printf(3, 4) __cold +void fat_msg(struct super_block *sb, const char *level, const char *fmt, ...); extern int fat_clusters_flush(struct super_block *sb); extern int fat_chain_add(struct inode *inode, int new_dclus, int nr_cluster); extern void fat_time_fat2unix(struct msdos_sb_info *sbi, struct timespec *ts, diff --git a/fs/gfs2/glock.h b/fs/gfs2/glock.h index 66707118af25..2553b858a72e 100644 --- a/fs/gfs2/glock.h +++ b/fs/gfs2/glock.h @@ -201,7 +201,7 @@ int gfs2_glock_nq_m(unsigned int num_gh, struct gfs2_holder *ghs); void gfs2_glock_dq_m(unsigned int num_gh, struct gfs2_holder *ghs); void gfs2_glock_dq_uninit_m(unsigned int num_gh, struct gfs2_holder *ghs); -__attribute__ ((format(printf, 2, 3))) +__printf(2, 3) void gfs2_print_dbg(struct seq_file *seq, const char *fmt, ...); /** diff --git a/fs/hpfs/hpfs_fn.h b/fs/hpfs/hpfs_fn.h index 331b5e234ef3..de946170ebb1 100644 --- a/fs/hpfs/hpfs_fn.h +++ b/fs/hpfs/hpfs_fn.h @@ -311,8 +311,8 @@ static inline struct hpfs_sb_info *hpfs_sb(struct super_block *sb) /* super.c */ -void hpfs_error(struct super_block *, const char *, ...) - __attribute__((format (printf, 2, 3))); +__printf(2, 3) +void hpfs_error(struct super_block *, const char *, ...); int hpfs_stop_cycles(struct super_block *, int, int *, int *, char *); unsigned hpfs_count_one_bitmap(struct super_block *, secno); diff --git a/fs/nilfs2/nilfs.h b/fs/nilfs2/nilfs.h index 255d5e1c03b7..3777d138f895 100644 --- a/fs/nilfs2/nilfs.h +++ b/fs/nilfs2/nilfs.h @@ -276,10 +276,10 @@ int nilfs_fiemap(struct inode *inode, struct fiemap_extent_info *fieinfo, /* super.c */ extern struct inode *nilfs_alloc_inode(struct super_block *); extern void nilfs_destroy_inode(struct inode *); -extern void nilfs_error(struct super_block *, const char *, const char *, ...) - __attribute__ ((format (printf, 3, 4))); -extern void nilfs_warning(struct super_block *, const char *, const char *, ...) - __attribute__ ((format (printf, 3, 4))); +extern __printf(3, 4) +void nilfs_error(struct super_block *, const char *, const char *, ...); +extern __printf(3, 4) +void nilfs_warning(struct super_block *, const char *, const char *, ...); extern struct nilfs_super_block * nilfs_read_super_block(struct super_block *, u64, int, struct buffer_head **); extern int nilfs_store_magic_and_option(struct super_block *, diff --git a/fs/ntfs/debug.h b/fs/ntfs/debug.h index 2142b1c68b61..53c27eaf2307 100644 --- a/fs/ntfs/debug.h +++ b/fs/ntfs/debug.h @@ -30,8 +30,9 @@ extern int debug_msgs; -extern void __ntfs_debug(const char *file, int line, const char *function, - const char *format, ...) __attribute__ ((format (printf, 4, 5))); +extern __printf(4, 5) +void __ntfs_debug(const char *file, int line, const char *function, + const char *format, ...); /** * ntfs_debug - write a debug level message to syslog * @f: a printf format string containing the message @@ -52,12 +53,14 @@ extern void ntfs_debug_dump_runlist(const runlist_element *rl); #endif /* !DEBUG */ -extern void __ntfs_warning(const char *function, const struct super_block *sb, - const char *fmt, ...) __attribute__ ((format (printf, 3, 4))); +extern __printf(3, 4) +void __ntfs_warning(const char *function, const struct super_block *sb, + const char *fmt, ...); #define ntfs_warning(sb, f, a...) __ntfs_warning(__func__, sb, f, ##a) -extern void __ntfs_error(const char *function, const struct super_block *sb, - const char *fmt, ...) __attribute__ ((format (printf, 3, 4))); +extern __printf(3, 4) +void __ntfs_error(const char *function, const struct super_block *sb, + const char *fmt, ...); #define ntfs_error(sb, f, a...) __ntfs_error(__func__, sb, f, ##a) #endif /* _LINUX_NTFS_DEBUG_H */ diff --git a/fs/ocfs2/super.h b/fs/ocfs2/super.h index 40c7de084c10..74ff74cf78fe 100644 --- a/fs/ocfs2/super.h +++ b/fs/ocfs2/super.h @@ -31,17 +31,15 @@ extern struct workqueue_struct *ocfs2_wq; int ocfs2_publish_get_mount_state(struct ocfs2_super *osb, int node_num); -void __ocfs2_error(struct super_block *sb, - const char *function, - const char *fmt, ...) - __attribute__ ((format (printf, 3, 4))); +__printf(3, 4) +void __ocfs2_error(struct super_block *sb, const char *function, + const char *fmt, ...); #define ocfs2_error(sb, fmt, args...) __ocfs2_error(sb, __PRETTY_FUNCTION__, fmt, ##args) -void __ocfs2_abort(struct super_block *sb, - const char *function, - const char *fmt, ...) - __attribute__ ((format (printf, 3, 4))); +__printf(3, 4) +void __ocfs2_abort(struct super_block *sb, const char *function, + const char *fmt, ...); #define ocfs2_abort(sb, fmt, args...) __ocfs2_abort(sb, __PRETTY_FUNCTION__, fmt, ##args) diff --git a/fs/partitions/ldm.c b/fs/partitions/ldm.c index af9fdf046769..bd8ae788f689 100644 --- a/fs/partitions/ldm.c +++ b/fs/partitions/ldm.c @@ -49,18 +49,20 @@ #define ldm_error(f, a...) _ldm_printk (KERN_ERR, __func__, f, ##a) #define ldm_info(f, a...) _ldm_printk (KERN_INFO, __func__, f, ##a) -__attribute__ ((format (printf, 3, 4))) -static void _ldm_printk (const char *level, const char *function, - const char *fmt, ...) +static __printf(3, 4) +void _ldm_printk(const char *level, const char *function, const char *fmt, ...) { - static char buf[128]; + struct va_format vaf; va_list args; va_start (args, fmt); - vsnprintf (buf, sizeof (buf), fmt, args); - va_end (args); - printk ("%s%s(): %s\n", level, function, buf); + vaf.fmt = fmt; + vaf.va = &args; + + printk("%s%s(): %pV\n", level, function, &vaf); + + va_end(args); } /** diff --git a/fs/udf/udfdecl.h b/fs/udf/udfdecl.h index dbd52d4b5eed..dc8a8dcc5ae1 100644 --- a/fs/udf/udfdecl.h +++ b/fs/udf/udfdecl.h @@ -112,8 +112,8 @@ struct extent_position { /* super.c */ -__attribute__((format(printf, 3, 4))) -extern void udf_warning(struct super_block *, const char *, const char *, ...); +extern __printf(3, 4) void udf_warning(struct super_block *, const char *, + const char *, ...); static inline void udf_updated_lvid(struct super_block *sb) { struct buffer_head *bh = UDF_SB(sb)->s_lvid_bh; diff --git a/fs/ufs/ufs.h b/fs/ufs/ufs.h index 5be2755dd715..c26f2bcec264 100644 --- a/fs/ufs/ufs.h +++ b/fs/ufs/ufs.h @@ -117,9 +117,12 @@ extern int ufs_getfrag_block (struct inode *inode, sector_t fragment, struct buf extern const struct file_operations ufs_dir_operations; /* super.c */ -extern void ufs_warning (struct super_block *, const char *, const char *, ...) __attribute__ ((format (printf, 3, 4))); -extern void ufs_error (struct super_block *, const char *, const char *, ...) __attribute__ ((format (printf, 3, 4))); -extern void ufs_panic (struct super_block *, const char *, const char *, ...) __attribute__ ((format (printf, 3, 4))); +extern __printf(3, 4) +void ufs_warning(struct super_block *, const char *, const char *, ...); +extern __printf(3, 4) +void ufs_error(struct super_block *, const char *, const char *, ...); +extern __printf(3, 4) +void ufs_panic(struct super_block *, const char *, const char *, ...); /* symlink.c */ extern const struct inode_operations ufs_fast_symlink_inode_operations; diff --git a/fs/xfs/xfs_message.h b/fs/xfs/xfs_message.h index 7fb7ea007672..56dc0c17f16a 100644 --- a/fs/xfs/xfs_message.h +++ b/fs/xfs/xfs_message.h @@ -3,31 +3,29 @@ struct xfs_mount; -extern void xfs_emerg(const struct xfs_mount *mp, const char *fmt, ...) - __attribute__ ((format (printf, 2, 3))); -extern void xfs_alert(const struct xfs_mount *mp, const char *fmt, ...) - __attribute__ ((format (printf, 2, 3))); -extern void xfs_alert_tag(const struct xfs_mount *mp, int tag, - const char *fmt, ...) - __attribute__ ((format (printf, 3, 4))); -extern void xfs_crit(const struct xfs_mount *mp, const char *fmt, ...) - __attribute__ ((format (printf, 2, 3))); -extern void xfs_err(const struct xfs_mount *mp, const char *fmt, ...) - __attribute__ ((format (printf, 2, 3))); -extern void xfs_warn(const struct xfs_mount *mp, const char *fmt, ...) - __attribute__ ((format (printf, 2, 3))); -extern void xfs_notice(const struct xfs_mount *mp, const char *fmt, ...) - __attribute__ ((format (printf, 2, 3))); -extern void xfs_info(const struct xfs_mount *mp, const char *fmt, ...) - __attribute__ ((format (printf, 2, 3))); +extern __printf(2, 3) +void xfs_emerg(const struct xfs_mount *mp, const char *fmt, ...); +extern __printf(2, 3) +void xfs_alert(const struct xfs_mount *mp, const char *fmt, ...); +extern __printf(3, 4) +void xfs_alert_tag(const struct xfs_mount *mp, int tag, const char *fmt, ...); +extern __printf(2, 3) +void xfs_crit(const struct xfs_mount *mp, const char *fmt, ...); +extern __printf(2, 3) +void xfs_err(const struct xfs_mount *mp, const char *fmt, ...); +extern __printf(2, 3) +void xfs_warn(const struct xfs_mount *mp, const char *fmt, ...); +extern __printf(2, 3) +void xfs_notice(const struct xfs_mount *mp, const char *fmt, ...); +extern __printf(2, 3) +void xfs_info(const struct xfs_mount *mp, const char *fmt, ...); #ifdef DEBUG -extern void xfs_debug(const struct xfs_mount *mp, const char *fmt, ...) - __attribute__ ((format (printf, 2, 3))); +extern __printf(2, 3) +void xfs_debug(const struct xfs_mount *mp, const char *fmt, ...); #else -static inline void -__attribute__ ((format (printf, 2, 3))) -xfs_debug(const struct xfs_mount *mp, const char *fmt, ...) +static inline __printf(2, 3) +void xfs_debug(const struct xfs_mount *mp, const char *fmt, ...) { } #endif diff --git a/include/asm-generic/bug.h b/include/asm-generic/bug.h index dfb0ec666c94..84458b0c38d1 100644 --- a/include/asm-generic/bug.h +++ b/include/asm-generic/bug.h @@ -61,11 +61,12 @@ struct bug_entry { */ #ifndef __WARN_TAINT #ifndef __ASSEMBLY__ -extern void warn_slowpath_fmt(const char *file, const int line, - const char *fmt, ...) __attribute__((format(printf, 3, 4))); -extern void warn_slowpath_fmt_taint(const char *file, const int line, - unsigned taint, const char *fmt, ...) - __attribute__((format(printf, 4, 5))); +extern __printf(3, 4) +void warn_slowpath_fmt(const char *file, const int line, + const char *fmt, ...); +extern __printf(4, 5) +void warn_slowpath_fmt_taint(const char *file, const int line, unsigned taint, + const char *fmt, ...); extern void warn_slowpath_null(const char *file, const int line); #define WANT_WARN_ON_SLOWPATH #endif diff --git a/include/drm/drmP.h b/include/drm/drmP.h index 43538b643560..cf3b446139ea 100644 --- a/include/drm/drmP.h +++ b/include/drm/drmP.h @@ -122,12 +122,12 @@ struct drm_device; * using the DRM_DEBUG_KMS and DRM_DEBUG. */ -extern __attribute__((format (printf, 4, 5))) +extern __printf(4, 5) void drm_ut_debug_printk(unsigned int request_level, - const char *prefix, - const char *function_name, - const char *format, ...); -extern __attribute__((format (printf, 2, 3))) + const char *prefix, + const char *function_name, + const char *format, ...); +extern __printf(2, 3) int drm_err(const char *func, const char *format, ...); /***********************************************************************/ diff --git a/include/linux/audit.h b/include/linux/audit.h index 0c8006129fb2..2f81c6f3b630 100644 --- a/include/linux/audit.h +++ b/include/linux/audit.h @@ -584,14 +584,13 @@ extern int audit_signals; #ifdef CONFIG_AUDIT /* These are defined in audit.c */ /* Public API */ -extern void audit_log(struct audit_context *ctx, gfp_t gfp_mask, - int type, const char *fmt, ...) - __attribute__((format(printf,4,5))); +extern __printf(4, 5) +void audit_log(struct audit_context *ctx, gfp_t gfp_mask, int type, + const char *fmt, ...); extern struct audit_buffer *audit_log_start(struct audit_context *ctx, gfp_t gfp_mask, int type); -extern void audit_log_format(struct audit_buffer *ab, - const char *fmt, ...) - __attribute__((format(printf,2,3))); +extern __printf(2, 3) +void audit_log_format(struct audit_buffer *ab, const char *fmt, ...); extern void audit_log_end(struct audit_buffer *ab); extern int audit_string_contains_control(const char *string, size_t len); diff --git a/include/linux/blktrace_api.h b/include/linux/blktrace_api.h index 8e9e4bc6d73b..4d1a0748eaf8 100644 --- a/include/linux/blktrace_api.h +++ b/include/linux/blktrace_api.h @@ -170,7 +170,7 @@ extern void blk_trace_shutdown(struct request_queue *); extern int do_blk_trace_setup(struct request_queue *q, char *name, dev_t dev, struct block_device *bdev, struct blk_user_trace_setup *buts); -extern __attribute__((format(printf, 2, 3))) +extern __printf(2, 3) void __trace_note_message(struct blk_trace *, const char *fmt, ...); /** diff --git a/include/linux/device.h b/include/linux/device.h index 85e78fc7d7fd..e88abeecfadf 100644 --- a/include/linux/device.h +++ b/include/linux/device.h @@ -622,8 +622,8 @@ static inline const char *dev_name(const struct device *dev) return kobject_name(&dev->kobj); } -extern int dev_set_name(struct device *dev, const char *name, ...) - __attribute__((format(printf, 2, 3))); +extern __printf(2, 3) +int dev_set_name(struct device *dev, const char *name, ...); #ifdef CONFIG_NUMA static inline int dev_to_node(struct device *dev) @@ -753,10 +753,10 @@ extern struct device *device_create_vargs(struct class *cls, void *drvdata, const char *fmt, va_list vargs); -extern struct device *device_create(struct class *cls, struct device *parent, - dev_t devt, void *drvdata, - const char *fmt, ...) - __attribute__((format(printf, 5, 6))); +extern __printf(5, 6) +struct device *device_create(struct class *cls, struct device *parent, + dev_t devt, void *drvdata, + const char *fmt, ...); extern void device_destroy(struct class *cls, dev_t devt); /* @@ -800,64 +800,56 @@ extern const char *dev_driver_string(const struct device *dev); extern int __dev_printk(const char *level, const struct device *dev, struct va_format *vaf); -extern int dev_printk(const char *level, const struct device *dev, - const char *fmt, ...) - __attribute__ ((format (printf, 3, 4))); -extern int dev_emerg(const struct device *dev, const char *fmt, ...) - __attribute__ ((format (printf, 2, 3))); -extern int dev_alert(const struct device *dev, const char *fmt, ...) - __attribute__ ((format (printf, 2, 3))); -extern int dev_crit(const struct device *dev, const char *fmt, ...) - __attribute__ ((format (printf, 2, 3))); -extern int dev_err(const struct device *dev, const char *fmt, ...) - __attribute__ ((format (printf, 2, 3))); -extern int dev_warn(const struct device *dev, const char *fmt, ...) - __attribute__ ((format (printf, 2, 3))); -extern int dev_notice(const struct device *dev, const char *fmt, ...) - __attribute__ ((format (printf, 2, 3))); -extern int _dev_info(const struct device *dev, const char *fmt, ...) - __attribute__ ((format (printf, 2, 3))); +extern __printf(3, 4) +int dev_printk(const char *level, const struct device *dev, + const char *fmt, ...) + ; +extern __printf(2, 3) +int dev_emerg(const struct device *dev, const char *fmt, ...); +extern __printf(2, 3) +int dev_alert(const struct device *dev, const char *fmt, ...); +extern __printf(2, 3) +int dev_crit(const struct device *dev, const char *fmt, ...); +extern __printf(2, 3) +int dev_err(const struct device *dev, const char *fmt, ...); +extern __printf(2, 3) +int dev_warn(const struct device *dev, const char *fmt, ...); +extern __printf(2, 3) +int dev_notice(const struct device *dev, const char *fmt, ...); +extern __printf(2, 3) +int _dev_info(const struct device *dev, const char *fmt, ...); #else static inline int __dev_printk(const char *level, const struct device *dev, struct va_format *vaf) - { return 0; } -static inline int dev_printk(const char *level, const struct device *dev, - const char *fmt, ...) - __attribute__ ((format (printf, 3, 4))); -static inline int dev_printk(const char *level, const struct device *dev, - const char *fmt, ...) - { return 0; } - -static inline int dev_emerg(const struct device *dev, const char *fmt, ...) - __attribute__ ((format (printf, 2, 3))); -static inline int dev_emerg(const struct device *dev, const char *fmt, ...) - { return 0; } -static inline int dev_crit(const struct device *dev, const char *fmt, ...) - __attribute__ ((format (printf, 2, 3))); -static inline int dev_crit(const struct device *dev, const char *fmt, ...) - { return 0; } -static inline int dev_alert(const struct device *dev, const char *fmt, ...) - __attribute__ ((format (printf, 2, 3))); -static inline int dev_alert(const struct device *dev, const char *fmt, ...) - { return 0; } -static inline int dev_err(const struct device *dev, const char *fmt, ...) - __attribute__ ((format (printf, 2, 3))); -static inline int dev_err(const struct device *dev, const char *fmt, ...) - { return 0; } -static inline int dev_warn(const struct device *dev, const char *fmt, ...) - __attribute__ ((format (printf, 2, 3))); -static inline int dev_warn(const struct device *dev, const char *fmt, ...) - { return 0; } -static inline int dev_notice(const struct device *dev, const char *fmt, ...) - __attribute__ ((format (printf, 2, 3))); -static inline int dev_notice(const struct device *dev, const char *fmt, ...) - { return 0; } -static inline int _dev_info(const struct device *dev, const char *fmt, ...) - __attribute__ ((format (printf, 2, 3))); -static inline int _dev_info(const struct device *dev, const char *fmt, ...) - { return 0; } +{ return 0; } +static inline __printf(3, 4) +int dev_printk(const char *level, const struct device *dev, + const char *fmt, ...) +{ return 0; } + +static inline __printf(2, 3) +int dev_emerg(const struct device *dev, const char *fmt, ...) +{ return 0; } +static inline __printf(2, 3) +int dev_crit(const struct device *dev, const char *fmt, ...) +{ return 0; } +static inline __printf(2, 3) +int dev_alert(const struct device *dev, const char *fmt, ...) +{ return 0; } +static inline __printf(2, 3) +int dev_err(const struct device *dev, const char *fmt, ...) +{ return 0; } +static inline __printf(2, 3) +int dev_warn(const struct device *dev, const char *fmt, ...) +{ return 0; } +static inline __printf(2, 3) +int dev_notice(const struct device *dev, const char *fmt, ...) +{ return 0; } +static inline __printf(2, 3) +int _dev_info(const struct device *dev, const char *fmt, ...) +{ return 0; } #endif diff --git a/include/linux/dynamic_debug.h b/include/linux/dynamic_debug.h index 13aae8087b56..0564e3c39882 100644 --- a/include/linux/dynamic_debug.h +++ b/include/linux/dynamic_debug.h @@ -37,22 +37,21 @@ int ddebug_add_module(struct _ddebug *tab, unsigned int n, #if defined(CONFIG_DYNAMIC_DEBUG) extern int ddebug_remove_module(const char *mod_name); -extern int __dynamic_pr_debug(struct _ddebug *descriptor, const char *fmt, ...) - __attribute__ ((format (printf, 2, 3))); +extern __printf(2, 3) +int __dynamic_pr_debug(struct _ddebug *descriptor, const char *fmt, ...); struct device; -extern int __dynamic_dev_dbg(struct _ddebug *descriptor, - const struct device *dev, - const char *fmt, ...) - __attribute__ ((format (printf, 3, 4))); +extern __printf(3, 4) +int __dynamic_dev_dbg(struct _ddebug *descriptor, const struct device *dev, + const char *fmt, ...); struct net_device; -extern int __dynamic_netdev_dbg(struct _ddebug *descriptor, - const struct net_device *dev, - const char *fmt, ...) - __attribute__ ((format (printf, 3, 4))); +extern __printf(3, 4) +int __dynamic_netdev_dbg(struct _ddebug *descriptor, + const struct net_device *dev, + const char *fmt, ...); #define DEFINE_DYNAMIC_DEBUG_METADATA(name, fmt) \ static struct _ddebug __used __aligned(8) \ diff --git a/include/linux/ext3_fs.h b/include/linux/ext3_fs.h index 67a803aee619..81965cce6bfa 100644 --- a/include/linux/ext3_fs.h +++ b/include/linux/ext3_fs.h @@ -937,15 +937,15 @@ extern int ext3_group_extend(struct super_block *sb, ext3_fsblk_t n_blocks_count); /* super.c */ -extern void ext3_error (struct super_block *, const char *, const char *, ...) - __attribute__ ((format (printf, 3, 4))); +extern __printf(3, 4) +void ext3_error(struct super_block *, const char *, const char *, ...); extern void __ext3_std_error (struct super_block *, const char *, int); -extern void ext3_abort (struct super_block *, const char *, const char *, ...) - __attribute__ ((format (printf, 3, 4))); -extern void ext3_warning (struct super_block *, const char *, const char *, ...) - __attribute__ ((format (printf, 3, 4))); -extern void ext3_msg(struct super_block *, const char *, const char *, ...) - __attribute__ ((format (printf, 3, 4))); +extern __printf(3, 4) +void ext3_abort(struct super_block *, const char *, const char *, ...); +extern __printf(3, 4) +void ext3_warning(struct super_block *, const char *, const char *, ...); +extern __printf(3, 4) +void ext3_msg(struct super_block *, const char *, const char *, ...); extern void ext3_update_dynamic_rev (struct super_block *sb); #define ext3_std_error(sb, errno) \ diff --git a/include/linux/fs.h b/include/linux/fs.h index 87b4c6b9692d..7a049fd2aa4c 100644 --- a/include/linux/fs.h +++ b/include/linux/fs.h @@ -2635,8 +2635,8 @@ static const struct file_operations __fops = { \ .llseek = generic_file_llseek, \ }; -static inline void __attribute__((format(printf, 1, 2))) -__simple_attr_check_format(const char *fmt, ...) +static inline __printf(1, 2) +void __simple_attr_check_format(const char *fmt, ...) { /* don't do anything, just let the compiler check the arguments; */ } diff --git a/include/linux/fscache-cache.h b/include/linux/fscache-cache.h index af095b54502e..ce31408b1e47 100644 --- a/include/linux/fscache-cache.h +++ b/include/linux/fscache-cache.h @@ -492,10 +492,10 @@ static inline void fscache_end_io(struct fscache_retrieval *op, /* * out-of-line cache backend functions */ -extern void fscache_init_cache(struct fscache_cache *cache, - const struct fscache_cache_ops *ops, - const char *idfmt, - ...) __attribute__ ((format (printf, 3, 4))); +extern __printf(3, 4) +void fscache_init_cache(struct fscache_cache *cache, + const struct fscache_cache_ops *ops, + const char *idfmt, ...); extern int fscache_add_cache(struct fscache_cache *cache, struct fscache_object *fsdef, diff --git a/include/linux/gameport.h b/include/linux/gameport.h index b65a6f472775..069ee4139105 100644 --- a/include/linux/gameport.h +++ b/include/linux/gameport.h @@ -78,8 +78,8 @@ static inline void gameport_register_port(struct gameport *gameport) void gameport_unregister_port(struct gameport *gameport); -void gameport_set_phys(struct gameport *gameport, const char *fmt, ...) - __attribute__ ((format (printf, 2, 3))); +__printf(2, 3) +void gameport_set_phys(struct gameport *gameport, const char *fmt, ...); #else @@ -93,8 +93,8 @@ static inline void gameport_unregister_port(struct gameport *gameport) return; } -static inline void gameport_set_phys(struct gameport *gameport, - const char *fmt, ...) +static inline __printf(2, 3) +void gameport_set_phys(struct gameport *gameport, const char *fmt, ...) { return; } diff --git a/include/linux/kallsyms.h b/include/linux/kallsyms.h index 0df513b7a9f8..387571959dd9 100644 --- a/include/linux/kallsyms.h +++ b/include/linux/kallsyms.h @@ -101,9 +101,8 @@ static inline int lookup_symbol_attrs(unsigned long addr, unsigned long *size, u #endif /*CONFIG_KALLSYMS*/ /* This macro allows us to keep printk typechecking */ -static void __check_printsym_format(const char *fmt, ...) -__attribute__((format(printf,1,2))); -static inline void __check_printsym_format(const char *fmt, ...) +static __printf(1, 2) +void __check_printsym_format(const char *fmt, ...) { } diff --git a/include/linux/kdb.h b/include/linux/kdb.h index 529d9a0c75a5..064725854db8 100644 --- a/include/linux/kdb.h +++ b/include/linux/kdb.h @@ -114,12 +114,9 @@ typedef enum { } kdb_reason_t; extern int kdb_trap_printk; -extern int vkdb_printf(const char *fmt, va_list args) - __attribute__ ((format (printf, 1, 0))); -extern int kdb_printf(const char *, ...) - __attribute__ ((format (printf, 1, 2))); -typedef int (*kdb_printf_t)(const char *, ...) - __attribute__ ((format (printf, 1, 2))); +extern __printf(1, 0) int vkdb_printf(const char *fmt, va_list args); +extern __printf(1, 2) int kdb_printf(const char *, ...); +typedef __printf(1, 2) int (*kdb_printf_t)(const char *, ...); extern void kdb_init(int level); diff --git a/include/linux/kernel.h b/include/linux/kernel.h index 8eefcf7e95eb..e40c950e1d62 100644 --- a/include/linux/kernel.h +++ b/include/linux/kernel.h @@ -296,20 +296,18 @@ extern long long simple_strtoll(const char *,char **,unsigned int); #define strict_strtoull kstrtoull #define strict_strtoll kstrtoll -extern int sprintf(char * buf, const char * fmt, ...) - __attribute__ ((format (printf, 2, 3))); -extern int vsprintf(char *buf, const char *, va_list) - __attribute__ ((format (printf, 2, 0))); -extern int snprintf(char * buf, size_t size, const char * fmt, ...) - __attribute__ ((format (printf, 3, 4))); -extern int vsnprintf(char *buf, size_t size, const char *fmt, va_list args) - __attribute__ ((format (printf, 3, 0))); -extern int scnprintf(char * buf, size_t size, const char * fmt, ...) - __attribute__ ((format (printf, 3, 4))); -extern int vscnprintf(char *buf, size_t size, const char *fmt, va_list args) - __attribute__ ((format (printf, 3, 0))); -extern char *kasprintf(gfp_t gfp, const char *fmt, ...) - __attribute__ ((format (printf, 2, 3))); +extern __printf(2, 3) int sprintf(char *buf, const char * fmt, ...); +extern __printf(2, 0) int vsprintf(char *buf, const char *, va_list); +extern __printf(3, 4) +int snprintf(char *buf, size_t size, const char *fmt, ...); +extern __printf(3, 0) +int vsnprintf(char *buf, size_t size, const char *fmt, va_list args); +extern __printf(3, 4) +int scnprintf(char *buf, size_t size, const char *fmt, ...); +extern __printf(3, 0) +int vscnprintf(char *buf, size_t size, const char *fmt, va_list args); +extern __printf(2, 3) +char *kasprintf(gfp_t gfp, const char *fmt, ...); extern char *kvasprintf(gfp_t gfp, const char *fmt, va_list args); extern int sscanf(const char *, const char *, ...) @@ -427,8 +425,8 @@ extern void tracing_start(void); extern void tracing_stop(void); extern void ftrace_off_permanent(void); -static inline void __attribute__ ((format (printf, 1, 2))) -____trace_printk_check_format(const char *fmt, ...) +static inline __printf(1, 2) +void ____trace_printk_check_format(const char *fmt, ...) { } #define __trace_printk_check_format(fmt, args...) \ @@ -467,13 +465,11 @@ do { \ __trace_printk(_THIS_IP_, fmt, ##args); \ } while (0) -extern int -__trace_bprintk(unsigned long ip, const char *fmt, ...) - __attribute__ ((format (printf, 2, 3))); +extern __printf(2, 3) +int __trace_bprintk(unsigned long ip, const char *fmt, ...); -extern int -__trace_printk(unsigned long ip, const char *fmt, ...) - __attribute__ ((format (printf, 2, 3))); +extern __printf(2, 3) +int __trace_printk(unsigned long ip, const char *fmt, ...); extern void trace_dump_stack(void); @@ -502,8 +498,8 @@ __ftrace_vprintk(unsigned long ip, const char *fmt, va_list ap); extern void ftrace_dump(enum ftrace_dump_mode oops_dump_mode); #else -static inline int -trace_printk(const char *fmt, ...) __attribute__ ((format (printf, 1, 2))); +static inline __printf(1, 2) +int trace_printk(const char *fmt, ...); static inline void tracing_start(void) { } static inline void tracing_stop(void) { } diff --git a/include/linux/kexec.h b/include/linux/kexec.h index c2478a342cd7..8944ebe7963e 100644 --- a/include/linux/kexec.h +++ b/include/linux/kexec.h @@ -130,8 +130,8 @@ int kexec_should_crash(struct task_struct *); void crash_save_cpu(struct pt_regs *regs, int cpu); void crash_save_vmcoreinfo(void); void arch_crash_save_vmcoreinfo(void); -void vmcoreinfo_append_str(const char *fmt, ...) - __attribute__ ((format (printf, 1, 2))); +__printf(1, 2) +void vmcoreinfo_append_str(const char *fmt, ...); unsigned long paddr_vmcoreinfo_note(void); #define VMCOREINFO_OSRELEASE(value) \ diff --git a/include/linux/kmod.h b/include/linux/kmod.h index 0da38cf7db7b..b16f65390734 100644 --- a/include/linux/kmod.h +++ b/include/linux/kmod.h @@ -32,8 +32,8 @@ extern char modprobe_path[]; /* for sysctl */ /* modprobe exit status on success, -ve on error. Return value * usually useless though. */ -extern int __request_module(bool wait, const char *name, ...) \ - __attribute__((format(printf, 2, 3))); +extern __printf(2, 3) +int __request_module(bool wait, const char *name, ...); #define request_module(mod...) __request_module(true, mod) #define request_module_nowait(mod...) __request_module(false, mod) #define try_then_request_module(x, mod...) \ diff --git a/include/linux/kobject.h b/include/linux/kobject.h index 668729cc0fe9..ad81e1c51487 100644 --- a/include/linux/kobject.h +++ b/include/linux/kobject.h @@ -72,8 +72,8 @@ struct kobject { unsigned int uevent_suppress:1; }; -extern int kobject_set_name(struct kobject *kobj, const char *name, ...) - __attribute__((format(printf, 2, 3))); +extern __printf(2, 3) +int kobject_set_name(struct kobject *kobj, const char *name, ...); extern int kobject_set_name_vargs(struct kobject *kobj, const char *fmt, va_list vargs); @@ -83,15 +83,13 @@ static inline const char *kobject_name(const struct kobject *kobj) } extern void kobject_init(struct kobject *kobj, struct kobj_type *ktype); -extern int __must_check kobject_add(struct kobject *kobj, - struct kobject *parent, - const char *fmt, ...) - __attribute__((format(printf, 3, 4))); -extern int __must_check kobject_init_and_add(struct kobject *kobj, - struct kobj_type *ktype, - struct kobject *parent, - const char *fmt, ...) - __attribute__((format(printf, 4, 5))); +extern __printf(3, 4) __must_check +int kobject_add(struct kobject *kobj, struct kobject *parent, + const char *fmt, ...); +extern __printf(4, 5) __must_check +int kobject_init_and_add(struct kobject *kobj, + struct kobj_type *ktype, struct kobject *parent, + const char *fmt, ...); extern void kobject_del(struct kobject *kobj); @@ -212,8 +210,8 @@ int kobject_uevent(struct kobject *kobj, enum kobject_action action); int kobject_uevent_env(struct kobject *kobj, enum kobject_action action, char *envp[]); -int add_uevent_var(struct kobj_uevent_env *env, const char *format, ...) - __attribute__((format (printf, 2, 3))); +__printf(2, 3) +int add_uevent_var(struct kobj_uevent_env *env, const char *format, ...); int kobject_action_type(const char *buf, size_t count, enum kobject_action *type); @@ -226,7 +224,7 @@ static inline int kobject_uevent_env(struct kobject *kobj, char *envp[]) { return 0; } -static inline __attribute__((format(printf, 2, 3))) +static inline __printf(2, 3) int add_uevent_var(struct kobj_uevent_env *env, const char *format, ...) { return 0; } diff --git a/include/linux/kthread.h b/include/linux/kthread.h index 1e923e5e88e8..5cac19b3a266 100644 --- a/include/linux/kthread.h +++ b/include/linux/kthread.h @@ -4,11 +4,11 @@ #include #include +__printf(4, 5) struct task_struct *kthread_create_on_node(int (*threadfn)(void *data), void *data, int node, - const char namefmt[], ...) - __attribute__((format(printf, 4, 5))); + const char namefmt[], ...); #define kthread_create(threadfn, data, namefmt, arg...) \ kthread_create_on_node(threadfn, data, -1, namefmt, ##arg) diff --git a/include/linux/libata.h b/include/linux/libata.h index 23fa829bf7a3..cafc09a64fe4 100644 --- a/include/linux/libata.h +++ b/include/linux/libata.h @@ -1256,13 +1256,13 @@ static inline int sata_srst_pmp(struct ata_link *link) /* * printk helpers */ -__attribute__((format (printf, 3, 4))) +__printf(3, 4) int ata_port_printk(const struct ata_port *ap, const char *level, const char *fmt, ...); -__attribute__((format (printf, 3, 4))) +__printf(3, 4) int ata_link_printk(const struct ata_link *link, const char *level, const char *fmt, ...); -__attribute__((format (printf, 3, 4))) +__printf(3, 4) int ata_dev_printk(const struct ata_device *dev, const char *level, const char *fmt, ...); @@ -1304,10 +1304,10 @@ void ata_print_version(const struct device *dev, const char *version); /* * ata_eh_info helpers */ -extern void __ata_ehi_push_desc(struct ata_eh_info *ehi, const char *fmt, ...) - __attribute__ ((format (printf, 2, 3))); -extern void ata_ehi_push_desc(struct ata_eh_info *ehi, const char *fmt, ...) - __attribute__ ((format (printf, 2, 3))); +extern __printf(2, 3) +void __ata_ehi_push_desc(struct ata_eh_info *ehi, const char *fmt, ...); +extern __printf(2, 3) +void ata_ehi_push_desc(struct ata_eh_info *ehi, const char *fmt, ...); extern void ata_ehi_clear_desc(struct ata_eh_info *ehi); static inline void ata_ehi_hotplugged(struct ata_eh_info *ehi) @@ -1321,8 +1321,8 @@ static inline void ata_ehi_hotplugged(struct ata_eh_info *ehi) /* * port description helpers */ -extern void ata_port_desc(struct ata_port *ap, const char *fmt, ...) - __attribute__ ((format (printf, 2, 3))); +extern __printf(2, 3) +void ata_port_desc(struct ata_port *ap, const char *fmt, ...); #ifdef CONFIG_PCI extern void ata_port_pbar_desc(struct ata_port *ap, int bar, ssize_t offset, const char *name); diff --git a/include/linux/mmiotrace.h b/include/linux/mmiotrace.h index 97491f78b08c..c5d52780d6a0 100644 --- a/include/linux/mmiotrace.h +++ b/include/linux/mmiotrace.h @@ -49,8 +49,7 @@ extern void mmiotrace_ioremap(resource_size_t offset, unsigned long size, extern void mmiotrace_iounmap(volatile void __iomem *addr); /* For anyone to insert markers. Remember trailing newline. */ -extern int mmiotrace_printk(const char *fmt, ...) - __attribute__ ((format (printf, 1, 2))); +extern __printf(1, 2) int mmiotrace_printk(const char *fmt, ...); #else /* !CONFIG_MMIOTRACE: */ static inline int is_kmmio_active(void) { @@ -71,10 +70,7 @@ static inline void mmiotrace_iounmap(volatile void __iomem *addr) { } -static inline int mmiotrace_printk(const char *fmt, ...) - __attribute__ ((format (printf, 1, 0))); - -static inline int mmiotrace_printk(const char *fmt, ...) +static inline __printf(1, 2) int mmiotrace_printk(const char *fmt, ...) { return 0; } diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h index df1c836e6948..cbeb5867cff7 100644 --- a/include/linux/netdevice.h +++ b/include/linux/netdevice.h @@ -2622,23 +2622,23 @@ static inline const char *netdev_name(const struct net_device *dev) extern int __netdev_printk(const char *level, const struct net_device *dev, struct va_format *vaf); -extern int netdev_printk(const char *level, const struct net_device *dev, - const char *format, ...) - __attribute__ ((format (printf, 3, 4))); -extern int netdev_emerg(const struct net_device *dev, const char *format, ...) - __attribute__ ((format (printf, 2, 3))); -extern int netdev_alert(const struct net_device *dev, const char *format, ...) - __attribute__ ((format (printf, 2, 3))); -extern int netdev_crit(const struct net_device *dev, const char *format, ...) - __attribute__ ((format (printf, 2, 3))); -extern int netdev_err(const struct net_device *dev, const char *format, ...) - __attribute__ ((format (printf, 2, 3))); -extern int netdev_warn(const struct net_device *dev, const char *format, ...) - __attribute__ ((format (printf, 2, 3))); -extern int netdev_notice(const struct net_device *dev, const char *format, ...) - __attribute__ ((format (printf, 2, 3))); -extern int netdev_info(const struct net_device *dev, const char *format, ...) - __attribute__ ((format (printf, 2, 3))); +extern __printf(3, 4) +int netdev_printk(const char *level, const struct net_device *dev, + const char *format, ...); +extern __printf(2, 3) +int netdev_emerg(const struct net_device *dev, const char *format, ...); +extern __printf(2, 3) +int netdev_alert(const struct net_device *dev, const char *format, ...); +extern __printf(2, 3) +int netdev_crit(const struct net_device *dev, const char *format, ...); +extern __printf(2, 3) +int netdev_err(const struct net_device *dev, const char *format, ...); +extern __printf(2, 3) +int netdev_warn(const struct net_device *dev, const char *format, ...); +extern __printf(2, 3) +int netdev_notice(const struct net_device *dev, const char *format, ...); +extern __printf(2, 3) +int netdev_info(const struct net_device *dev, const char *format, ...); #define MODULE_ALIAS_NETDEV(device) \ MODULE_ALIAS("netdev-" device) diff --git a/include/linux/printk.h b/include/linux/printk.h index 0101d55d9651..f0e22f75143f 100644 --- a/include/linux/printk.h +++ b/include/linux/printk.h @@ -82,22 +82,22 @@ struct va_format { * Dummy printk for disabled debugging statements to use whilst maintaining * gcc's format and side-effect checking. */ -static inline __attribute__ ((format (printf, 1, 2))) +static inline __printf(1, 2) int no_printk(const char *fmt, ...) { return 0; } -extern asmlinkage __attribute__ ((format (printf, 1, 2))) +extern asmlinkage __printf(1, 2) void early_printk(const char *fmt, ...); extern int printk_needs_cpu(int cpu); extern void printk_tick(void); #ifdef CONFIG_PRINTK -asmlinkage __attribute__ ((format (printf, 1, 0))) +asmlinkage __printf(1, 0) int vprintk(const char *fmt, va_list args); -asmlinkage __attribute__ ((format (printf, 1, 2))) __cold +asmlinkage __printf(1, 2) __cold int printk(const char *fmt, ...); /* @@ -117,12 +117,12 @@ extern int kptr_restrict; void log_buf_kexec_setup(void); void __init setup_log_buf(int early); #else -static inline __attribute__ ((format (printf, 1, 0))) +static inline __printf(1, 0) int vprintk(const char *s, va_list args) { return 0; } -static inline __attribute__ ((format (printf, 1, 2))) __cold +static inline __printf(1, 2) __cold int printk(const char *s, ...) { return 0; diff --git a/include/linux/quotaops.h b/include/linux/quotaops.h index 26f9e3612e0f..d93f95e6177c 100644 --- a/include/linux/quotaops.h +++ b/include/linux/quotaops.h @@ -31,7 +31,7 @@ static inline bool is_quota_modification(struct inode *inode, struct iattr *ia) #define quota_error(sb, fmt, args...) \ __quota_error((sb), __func__, fmt , ## args) -extern __attribute__((format (printf, 3, 4))) +extern __printf(3, 4) void __quota_error(struct super_block *sb, const char *func, const char *fmt, ...); diff --git a/include/linux/seq_file.h b/include/linux/seq_file.h index be720cd2038d..0b69a4684216 100644 --- a/include/linux/seq_file.h +++ b/include/linux/seq_file.h @@ -84,8 +84,7 @@ int seq_putc(struct seq_file *m, char c); int seq_puts(struct seq_file *m, const char *s); int seq_write(struct seq_file *seq, const void *data, size_t len); -int seq_printf(struct seq_file *, const char *, ...) - __attribute__ ((format (printf,2,3))); +__printf(2, 3) int seq_printf(struct seq_file *, const char *, ...); int seq_path(struct seq_file *, struct path *, char *); int seq_dentry(struct seq_file *, struct dentry *, char *); diff --git a/include/linux/trace_seq.h b/include/linux/trace_seq.h index 5cf397ceb726..7dadc3df0c77 100644 --- a/include/linux/trace_seq.h +++ b/include/linux/trace_seq.h @@ -29,10 +29,10 @@ trace_seq_init(struct trace_seq *s) * Currently only defined when tracing is enabled. */ #ifdef CONFIG_TRACING -extern int trace_seq_printf(struct trace_seq *s, const char *fmt, ...) - __attribute__ ((format (printf, 2, 3))); -extern int trace_seq_vprintf(struct trace_seq *s, const char *fmt, va_list args) - __attribute__ ((format (printf, 2, 0))); +extern __printf(2, 3) +int trace_seq_printf(struct trace_seq *s, const char *fmt, ...); +extern __printf(2, 0) +int trace_seq_vprintf(struct trace_seq *s, const char *fmt, va_list args); extern int trace_seq_bprintf(struct trace_seq *s, const char *fmt, const u32 *binary); extern int trace_print_seq(struct seq_file *m, struct trace_seq *s); diff --git a/include/net/bluetooth/bluetooth.h b/include/net/bluetooth/bluetooth.h index e727555d4ee9..e86af08293a8 100644 --- a/include/net/bluetooth/bluetooth.h +++ b/include/net/bluetooth/bluetooth.h @@ -77,7 +77,7 @@ struct bt_power { #define BT_POWER_FORCE_ACTIVE_OFF 0 #define BT_POWER_FORCE_ACTIVE_ON 1 -__attribute__((format (printf, 2, 3))) +__printf(2, 3) int bt_printk(const char *level, const char *fmt, ...); #define BT_INFO(fmt, arg...) bt_printk(KERN_INFO, pr_fmt(fmt), ##arg) diff --git a/include/net/netfilter/nf_log.h b/include/net/netfilter/nf_log.h index 920997f1aff0..e991bd0a27af 100644 --- a/include/net/netfilter/nf_log.h +++ b/include/net/netfilter/nf_log.h @@ -53,12 +53,13 @@ int nf_log_bind_pf(u_int8_t pf, const struct nf_logger *logger); void nf_log_unbind_pf(u_int8_t pf); /* Calls the registered backend logging function */ +__printf(7, 8) void nf_log_packet(u_int8_t pf, unsigned int hooknum, const struct sk_buff *skb, const struct net_device *in, const struct net_device *out, const struct nf_loginfo *li, - const char *fmt, ...) __attribute__ ((format(printf,7,8))); + const char *fmt, ...); #endif /* _NF_LOG_H */ diff --git a/include/net/sock.h b/include/net/sock.h index 5ac682f73d63..c6658bef7f32 100644 --- a/include/net/sock.h +++ b/include/net/sock.h @@ -76,8 +76,8 @@ printk(KERN_DEBUG msg); } while (0) #else /* Validate arguments and do nothing */ -static inline void __attribute__ ((format (printf, 2, 3))) -SOCK_DEBUG(struct sock *sk, const char *msg, ...) +static inline __printf(2, 3) +void SOCK_DEBUG(struct sock *sk, const char *msg, ...) { } #endif diff --git a/include/sound/core.h b/include/sound/core.h index 1fa2407c966f..91d513879a78 100644 --- a/include/sound/core.h +++ b/include/sound/core.h @@ -326,9 +326,9 @@ void release_and_free_resource(struct resource *res); /* --- */ #if defined(CONFIG_SND_DEBUG) || defined(CONFIG_SND_VERBOSE_PRINTK) +__printf(4, 5) void __snd_printk(unsigned int level, const char *file, int line, - const char *format, ...) - __attribute__ ((format (printf, 4, 5))); + const char *format, ...); #else #define __snd_printk(level, file, line, format, args...) \ printk(format, ##args) diff --git a/include/sound/info.h b/include/sound/info.h index 4e94cf1ff762..5492cc40dc57 100644 --- a/include/sound/info.h +++ b/include/sound/info.h @@ -110,8 +110,8 @@ void snd_card_info_read_oss(struct snd_info_buffer *buffer); static inline void snd_card_info_read_oss(struct snd_info_buffer *buffer) {} #endif -int snd_iprintf(struct snd_info_buffer *buffer, const char *fmt, ...) \ - __attribute__ ((format (printf, 2, 3))); +__printf(2, 3) +int snd_iprintf(struct snd_info_buffer *buffer, const char *fmt, ...); int snd_info_init(void); int snd_info_done(void); diff --git a/include/sound/seq_kernel.h b/include/sound/seq_kernel.h index 3d9afb6a8c9c..f352a98ce4f4 100644 --- a/include/sound/seq_kernel.h +++ b/include/sound/seq_kernel.h @@ -75,9 +75,9 @@ struct snd_seq_port_callback { }; /* interface for kernel client */ +__printf(3, 4) int snd_seq_create_kernel_client(struct snd_card *card, int client_index, - const char *name_fmt, ...) - __attribute__ ((format (printf, 3, 4))); + const char *name_fmt, ...); int snd_seq_delete_kernel_client(int client); int snd_seq_kernel_client_enqueue(int client, struct snd_seq_event *ev, int atomic, int hop); int snd_seq_kernel_client_dispatch(int client, struct snd_seq_event *ev, int atomic, int hop); diff --git a/include/xen/hvc-console.h b/include/xen/hvc-console.h index 901724dc528d..b62dfef15f61 100644 --- a/include/xen/hvc-console.h +++ b/include/xen/hvc-console.h @@ -6,12 +6,12 @@ extern struct console xenboot_console; #ifdef CONFIG_HVC_XEN void xen_console_resume(void); void xen_raw_console_write(const char *str); -__attribute__((format(printf, 1, 2))) +__printf(1, 2) void xen_raw_printk(const char *fmt, ...); #else static inline void xen_console_resume(void) { } static inline void xen_raw_console_write(const char *str) { } -static inline __attribute__((format(printf, 1, 2))) +static inline __printf(1, 2) void xen_raw_printk(const char *fmt, ...) { } #endif diff --git a/include/xen/xenbus.h b/include/xen/xenbus.h index aceeca799fd7..b9f9fb5af0d8 100644 --- a/include/xen/xenbus.h +++ b/include/xen/xenbus.h @@ -156,9 +156,9 @@ int xenbus_scanf(struct xenbus_transaction t, __attribute__((format(scanf, 4, 5))); /* Single printf and write: returns -errno or 0. */ +__printf(4, 5) int xenbus_printf(struct xenbus_transaction t, - const char *dir, const char *node, const char *fmt, ...) - __attribute__((format(printf, 4, 5))); + const char *dir, const char *node, const char *fmt, ...); /* Generic read function: NULL-terminated triples of name, * sprintf-style type string, and pointer. Returns 0 or errno.*/ @@ -200,11 +200,11 @@ int xenbus_watch_path(struct xenbus_device *dev, const char *path, struct xenbus_watch *watch, void (*callback)(struct xenbus_watch *, const char **, unsigned int)); +__printf(4, 5) int xenbus_watch_pathfmt(struct xenbus_device *dev, struct xenbus_watch *watch, void (*callback)(struct xenbus_watch *, const char **, unsigned int), - const char *pathfmt, ...) - __attribute__ ((format (printf, 4, 5))); + const char *pathfmt, ...); int xenbus_switch_state(struct xenbus_device *dev, enum xenbus_state new_state); int xenbus_grant_ring(struct xenbus_device *dev, unsigned long ring_mfn); @@ -223,9 +223,9 @@ int xenbus_free_evtchn(struct xenbus_device *dev, int port); enum xenbus_state xenbus_read_driver_state(const char *path); -__attribute__((format(printf, 3, 4))) +__printf(3, 4) void xenbus_dev_error(struct xenbus_device *dev, int err, const char *fmt, ...); -__attribute__((format(printf, 3, 4))) +__printf(3, 4) void xenbus_dev_fatal(struct xenbus_device *dev, int err, const char *fmt, ...); const char *xenbus_strstate(enum xenbus_state state); diff --git a/net/nfc/nfc.h b/net/nfc/nfc.h index b6753f45624e..d86583f4831d 100644 --- a/net/nfc/nfc.h +++ b/net/nfc/nfc.h @@ -27,7 +27,7 @@ #include #include -__attribute__((format (printf, 2, 3))) +__printf(2, 3) int nfc_printk(const char *level, const char *fmt, ...); #define nfc_info(fmt, arg...) nfc_printk(KERN_INFO, fmt, ##arg) diff --git a/net/rds/rds.h b/net/rds/rds.h index da8adac2bf06..7eaba1831f0d 100644 --- a/net/rds/rds.h +++ b/net/rds/rds.h @@ -36,8 +36,8 @@ #define rdsdebug(fmt, args...) pr_debug("%s(): " fmt, __func__ , ##args) #else /* sigh, pr_debug() causes unused variable warnings */ -static inline void __attribute__ ((format (printf, 1, 2))) -rdsdebug(char *fmt, ...) +static inline __printf(1, 2) +void rdsdebug(char *fmt, ...) { } #endif @@ -625,8 +625,8 @@ void rds_for_each_conn_info(struct socket *sock, unsigned int len, struct rds_info_lengths *lens, int (*visitor)(struct rds_connection *, void *), size_t item_len); -void __rds_conn_error(struct rds_connection *conn, const char *, ...) - __attribute__ ((format (printf, 2, 3))); +__printf(2, 3) +void __rds_conn_error(struct rds_connection *conn, const char *, ...); #define rds_conn_error(conn, fmt...) \ __rds_conn_error(conn, KERN_WARNING "RDS: " fmt) diff --git a/net/sunrpc/svc.c b/net/sunrpc/svc.c index 30d70abb4e2c..dd5cc00ed559 100644 --- a/net/sunrpc/svc.c +++ b/net/sunrpc/svc.c @@ -971,9 +971,8 @@ static void svc_unregister(const struct svc_serv *serv) /* * Printk the given error with the address of the client that caused it. */ -static int -__attribute__ ((format (printf, 2, 3))) -svc_printk(struct svc_rqst *rqstp, const char *fmt, ...) +static __printf(2, 3) +int svc_printk(struct svc_rqst *rqstp, const char *fmt, ...) { va_list args; int r; diff --git a/sound/firewire/cmp.c b/sound/firewire/cmp.c index 14cacbc655dd..76294f2ae47f 100644 --- a/sound/firewire/cmp.c +++ b/sound/firewire/cmp.c @@ -32,7 +32,7 @@ enum bus_reset_handling { SUCCEED_ON_BUS_RESET, }; -static __attribute__((format(printf, 2, 3))) +static __printf(2, 3) void cmp_error(struct cmp_connection *c, const char *fmt, ...) { va_list va; -- cgit v1.2.3 From 0556dc340e5159cdff925a5ab7f3a72f49745661 Mon Sep 17 00:00:00 2001 From: Mark Brown Date: Mon, 31 Oct 2011 17:11:48 -0700 Subject: backlight: fix broken regulator API usage in l4f00242t03 The regulator support in the l4f00242t03 is very non-idiomatic. Rather than requesting the regulators based on the device name and the supply names used by the device the driver requires boards to pass system specific supply names around through platform data. The driver also conditionally requests the regulators based on this platform data, adding unneeded conditional code to the driver. Fix this by removing the platform data and converting to the standard idiom, also updating all in tree users of the driver. As no datasheet appears to be available for the LCD I'm guessing the names for the supplies based on the existing users and I've no ability to do anything more than compile test. The use of regulator_set_voltage() in the driver is also problematic, since fixed voltages are required the expectation would be that the voltages would be fixed in the constraints set by the machines rather than manually configured by the driver, but is less problematic. Signed-off-by: Mark Brown Tested-by: Fabio Estevam Cc: Richard Purdie Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds --- arch/arm/mach-imx/mach-mx27_3ds.c | 6 ++-- arch/arm/mach-imx/mach-mx31_3ds.c | 6 ++-- drivers/video/backlight/l4f00242t03.c | 57 ++++++++++++----------------------- include/linux/spi/l4f00242t03.h | 2 -- 4 files changed, 24 insertions(+), 47 deletions(-) (limited to 'include/linux') diff --git a/arch/arm/mach-imx/mach-mx27_3ds.c b/arch/arm/mach-imx/mach-mx27_3ds.c index 2eafbac2c763..377230497dcc 100644 --- a/arch/arm/mach-imx/mach-mx27_3ds.c +++ b/arch/arm/mach-imx/mach-mx27_3ds.c @@ -241,7 +241,7 @@ static struct regulator_init_data gpo_init = { }; static struct regulator_consumer_supply vmmc1_consumers[] = { - REGULATOR_SUPPLY("lcd_2v8", NULL), + REGULATOR_SUPPLY("vcore", "spi0.0"), }; static struct regulator_init_data vmmc1_init = { @@ -257,7 +257,7 @@ static struct regulator_init_data vmmc1_init = { }; static struct regulator_consumer_supply vgen_consumers[] = { - REGULATOR_SUPPLY("vdd_lcdio", NULL), + REGULATOR_SUPPLY("vdd", "spi0.0"), }; static struct regulator_init_data vgen_init = { @@ -348,8 +348,6 @@ static const struct imx_fb_platform_data mx27_3ds_fb_data __initconst = { static struct l4f00242t03_pdata mx27_3ds_lcd_pdata = { .reset_gpio = LCD_RESET, .data_enable_gpio = LCD_ENABLE, - .core_supply = "lcd_2v8", - .io_supply = "vdd_lcdio", }; static struct spi_board_info mx27_3ds_spi_devs[] __initdata = { diff --git a/arch/arm/mach-imx/mach-mx31_3ds.c b/arch/arm/mach-imx/mach-mx31_3ds.c index 589066fb3316..6484db525bd7 100644 --- a/arch/arm/mach-imx/mach-mx31_3ds.c +++ b/arch/arm/mach-imx/mach-mx31_3ds.c @@ -285,8 +285,6 @@ static struct mx3fb_platform_data mx3fb_pdata __initdata = { static struct l4f00242t03_pdata mx31_3ds_l4f00242t03_pdata = { .reset_gpio = IOMUX_TO_GPIO(MX31_PIN_LCS1), .data_enable_gpio = IOMUX_TO_GPIO(MX31_PIN_SER_RS), - .core_supply = "lcd_2v8", - .io_supply = "vdd_lcdio", }; /* @@ -411,7 +409,7 @@ static struct regulator_init_data vmmc2_init = { }; static struct regulator_consumer_supply vmmc1_consumers[] = { - REGULATOR_SUPPLY("lcd_2v8", NULL), + REGULATOR_SUPPLY("vcore", "spi0.0"), REGULATOR_SUPPLY("cmos_2v8", "soc-camera-pdrv.0"), }; @@ -428,7 +426,7 @@ static struct regulator_init_data vmmc1_init = { }; static struct regulator_consumer_supply vgen_consumers[] = { - REGULATOR_SUPPLY("vdd_lcdio", NULL), + REGULATOR_SUPPLY("vdd", "spi0.0"), }; static struct regulator_init_data vgen_init = { diff --git a/drivers/video/backlight/l4f00242t03.c b/drivers/video/backlight/l4f00242t03.c index 98ad3e5f7c85..77ec70bcb303 100644 --- a/drivers/video/backlight/l4f00242t03.c +++ b/drivers/video/backlight/l4f00242t03.c @@ -52,15 +52,11 @@ static void l4f00242t03_lcd_init(struct spi_device *spi) dev_dbg(&spi->dev, "initializing LCD\n"); - if (priv->io_reg) { - regulator_set_voltage(priv->io_reg, 1800000, 1800000); - regulator_enable(priv->io_reg); - } + regulator_set_voltage(priv->io_reg, 1800000, 1800000); + regulator_enable(priv->io_reg); - if (priv->core_reg) { - regulator_set_voltage(priv->core_reg, 2800000, 2800000); - regulator_enable(priv->core_reg); - } + regulator_set_voltage(priv->core_reg, 2800000, 2800000); + regulator_enable(priv->core_reg); l4f00242t03_reset(pdata->reset_gpio); @@ -78,11 +74,8 @@ static void l4f00242t03_lcd_powerdown(struct spi_device *spi) gpio_set_value(pdata->data_enable_gpio, 0); - if (priv->io_reg) - regulator_disable(priv->io_reg); - - if (priv->core_reg) - regulator_disable(priv->core_reg); + regulator_disable(priv->io_reg); + regulator_disable(priv->core_reg); } static int l4f00242t03_lcd_power_get(struct lcd_device *ld) @@ -201,24 +194,18 @@ static int __devinit l4f00242t03_probe(struct spi_device *spi) if (ret) goto err3; - if (pdata->io_supply) { - priv->io_reg = regulator_get(NULL, pdata->io_supply); - - if (IS_ERR(priv->io_reg)) { - pr_err("%s: Unable to get the IO regulator\n", - __func__); - goto err3; - } + priv->io_reg = regulator_get(&spi->dev, "vdd"); + if (IS_ERR(priv->io_reg)) { + dev_err(&spi->dev, "%s: Unable to get the IO regulator\n", + __func__); + goto err3; } - if (pdata->core_supply) { - priv->core_reg = regulator_get(NULL, pdata->core_supply); - - if (IS_ERR(priv->core_reg)) { - pr_err("%s: Unable to get the core regulator\n", - __func__); - goto err4; - } + priv->core_reg = regulator_get(&spi->dev, "vcore"); + if (IS_ERR(priv->core_reg)) { + dev_err(&spi->dev, "%s: Unable to get the core regulator\n", + __func__); + goto err4; } priv->ld = lcd_device_register("l4f00242t03", @@ -238,11 +225,9 @@ static int __devinit l4f00242t03_probe(struct spi_device *spi) return 0; err5: - if (priv->core_reg) - regulator_put(priv->core_reg); + regulator_put(priv->core_reg); err4: - if (priv->io_reg) - regulator_put(priv->io_reg); + regulator_put(priv->io_reg); err3: gpio_free(pdata->data_enable_gpio); err2: @@ -266,10 +251,8 @@ static int __devexit l4f00242t03_remove(struct spi_device *spi) gpio_free(pdata->data_enable_gpio); gpio_free(pdata->reset_gpio); - if (priv->io_reg) - regulator_put(priv->io_reg); - if (priv->core_reg) - regulator_put(priv->core_reg); + regulator_put(priv->io_reg); + regulator_put(priv->core_reg); kfree(priv); diff --git a/include/linux/spi/l4f00242t03.h b/include/linux/spi/l4f00242t03.h index aee1dbda4edc..bc8677c8eba9 100644 --- a/include/linux/spi/l4f00242t03.h +++ b/include/linux/spi/l4f00242t03.h @@ -24,8 +24,6 @@ struct l4f00242t03_pdata { unsigned int reset_gpio; unsigned int data_enable_gpio; - const char *io_supply; /* will be set to 1.8 V */ - const char *core_supply; /* will be set to 2.8 V */ }; #endif /* _INCLUDE_LINUX_SPI_L4F00242T03_H_ */ -- cgit v1.2.3 From f59b6f9f323ff1b4567a69f9063cdd8bb57805e6 Mon Sep 17 00:00:00 2001 From: Magnus Damm Date: Mon, 31 Oct 2011 17:11:55 -0700 Subject: leds: Renesas TPU LED driver Add V2 of the LED driver for a single timer channel for the TPU hardware block commonly found in Renesas SoCs. The driver has been written with optimal Power Management in mind, so to save power the LED is driven as a regular GPIO pin in case of maximum brightness and power off which allows the TPU hardware to be idle and which in turn allows the clocks to be stopped and the power domain to be turned off transparently. Any other brightness level requires use of the TPU hardware in PWM mode. TPU hardware device clocks and power are managed through Runtime PM. System suspend and resume is known to be working - during suspend the LED is set to off by the generic LED code. The TPU hardware timer is equipeed with a 16-bit counter together with an up-to-divide-by-64 prescaler which makes the hardware suitable for brightness control. Hardware blink is unsupported. The LED PWM waveform has been verified with a Fluke 123 Scope meter on a sh7372 Mackerel board. Tested with experimental sh7372 A3SP power domain patches. Platform device bind/unbind tested ok. V2 has been tested on the DS2 LED of the sh73a0-based AG5EVM. [axel.lin@gmail.com: include linux/module.h] Signed-off-by: Magnus Damm Cc: Paul Mundt Cc: Richard Purdie Cc: Grant Likely Signed-off-by: Axel Lin Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds --- drivers/leds/Kconfig | 12 ++ drivers/leds/Makefile | 1 + drivers/leds/leds-renesas-tpu.c | 344 +++++++++++++++++++++++++++++++++++++++ include/linux/leds-renesas-tpu.h | 14 ++ 4 files changed, 371 insertions(+) create mode 100644 drivers/leds/leds-renesas-tpu.c create mode 100644 include/linux/leds-renesas-tpu.h (limited to 'include/linux') diff --git a/drivers/leds/Kconfig b/drivers/leds/Kconfig index dc7caaddecf4..ff203a421863 100644 --- a/drivers/leds/Kconfig +++ b/drivers/leds/Kconfig @@ -375,6 +375,18 @@ config LEDS_ASIC3 cannot be used. This driver supports hardware blinking with an on+off period from 62ms to 125s. Say Y to enable LEDs on the HP iPAQ hx4700. +config LEDS_RENESAS_TPU + bool "LED support for Renesas TPU" + depends on LEDS_CLASS && HAVE_CLK && GENERIC_GPIO + help + This option enables build of the LED TPU platform driver, + suitable to drive any TPU channel on newer Renesas SoCs. + The driver controls the GPIO pin connected to the LED via + the GPIO framework and expects the LED to be connected to + a pin that can be driven in both GPIO mode and using TPU + pin function. The latter to support brightness control. + Brightness control is supported but hardware blinking is not. + config LEDS_TRIGGERS bool "LED Trigger support" depends on LEDS_CLASS diff --git a/drivers/leds/Makefile b/drivers/leds/Makefile index a0a1b89d78a8..e4f6bf568880 100644 --- a/drivers/leds/Makefile +++ b/drivers/leds/Makefile @@ -42,6 +42,7 @@ obj-$(CONFIG_LEDS_MC13783) += leds-mc13783.o obj-$(CONFIG_LEDS_NS2) += leds-ns2.o obj-$(CONFIG_LEDS_NETXBIG) += leds-netxbig.o obj-$(CONFIG_LEDS_ASIC3) += leds-asic3.o +obj-$(CONFIG_LEDS_RENESAS_TPU) += leds-renesas-tpu.o # LED SPI Drivers obj-$(CONFIG_LEDS_DAC124S085) += leds-dac124s085.o diff --git a/drivers/leds/leds-renesas-tpu.c b/drivers/leds/leds-renesas-tpu.c new file mode 100644 index 000000000000..d9a7d72a7569 --- /dev/null +++ b/drivers/leds/leds-renesas-tpu.c @@ -0,0 +1,344 @@ +/* + * LED control using Renesas TPU + * + * Copyright (C) 2011 Magnus Damm + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +enum r_tpu_pin { R_TPU_PIN_UNUSED, R_TPU_PIN_GPIO, R_TPU_PIN_GPIO_FN }; +enum r_tpu_timer { R_TPU_TIMER_UNUSED, R_TPU_TIMER_ON }; + +struct r_tpu_priv { + struct led_classdev ldev; + void __iomem *mapbase; + struct clk *clk; + struct platform_device *pdev; + enum r_tpu_pin pin_state; + enum r_tpu_timer timer_state; + unsigned long min_rate; + unsigned int refresh_rate; +}; + +static DEFINE_SPINLOCK(r_tpu_lock); + +#define TSTR -1 /* Timer start register (shared register) */ +#define TCR 0 /* Timer control register (+0x00) */ +#define TMDR 1 /* Timer mode register (+0x04) */ +#define TIOR 2 /* Timer I/O control register (+0x08) */ +#define TIER 3 /* Timer interrupt enable register (+0x0c) */ +#define TSR 4 /* Timer status register (+0x10) */ +#define TCNT 5 /* Timer counter (+0x14) */ +#define TGRA 6 /* Timer general register A (+0x18) */ +#define TGRB 7 /* Timer general register B (+0x1c) */ +#define TGRC 8 /* Timer general register C (+0x20) */ +#define TGRD 9 /* Timer general register D (+0x24) */ + +static inline unsigned short r_tpu_read(struct r_tpu_priv *p, int reg_nr) +{ + struct led_renesas_tpu_config *cfg = p->pdev->dev.platform_data; + void __iomem *base = p->mapbase; + unsigned long offs = reg_nr << 2; + + if (reg_nr == TSTR) + return ioread16(base - cfg->channel_offset); + + return ioread16(base + offs); +} + +static inline void r_tpu_write(struct r_tpu_priv *p, int reg_nr, + unsigned short value) +{ + struct led_renesas_tpu_config *cfg = p->pdev->dev.platform_data; + void __iomem *base = p->mapbase; + unsigned long offs = reg_nr << 2; + + if (reg_nr == TSTR) { + iowrite16(value, base - cfg->channel_offset); + return; + } + + iowrite16(value, base + offs); +} + +static void r_tpu_start_stop_ch(struct r_tpu_priv *p, int start) +{ + struct led_renesas_tpu_config *cfg = p->pdev->dev.platform_data; + unsigned long flags, value; + + /* start stop register shared by multiple timer channels */ + spin_lock_irqsave(&r_tpu_lock, flags); + value = r_tpu_read(p, TSTR); + + if (start) + value |= 1 << cfg->timer_bit; + else + value &= ~(1 << cfg->timer_bit); + + r_tpu_write(p, TSTR, value); + spin_unlock_irqrestore(&r_tpu_lock, flags); +} + +static int r_tpu_enable(struct r_tpu_priv *p, enum led_brightness brightness) +{ + struct led_renesas_tpu_config *cfg = p->pdev->dev.platform_data; + int prescaler[] = { 1, 4, 16, 64 }; + int k, ret; + unsigned long rate, tmp; + + if (p->timer_state == R_TPU_TIMER_ON) + return 0; + + /* wake up device and enable clock */ + pm_runtime_get_sync(&p->pdev->dev); + ret = clk_enable(p->clk); + if (ret) { + dev_err(&p->pdev->dev, "cannot enable clock\n"); + return ret; + } + + /* make sure channel is disabled */ + r_tpu_start_stop_ch(p, 0); + + /* get clock rate after enabling it */ + rate = clk_get_rate(p->clk); + + /* pick the lowest acceptable rate */ + for (k = 0; k < ARRAY_SIZE(prescaler); k++) + if ((rate / prescaler[k]) < p->min_rate) + break; + + if (!k) { + dev_err(&p->pdev->dev, "clock rate mismatch\n"); + goto err0; + } + dev_dbg(&p->pdev->dev, "rate = %lu, prescaler %u\n", + rate, prescaler[k - 1]); + + /* clear TCNT on TGRB match, count on rising edge, set prescaler */ + r_tpu_write(p, TCR, 0x0040 | (k - 1)); + + /* output 0 until TGRA, output 1 until TGRB */ + r_tpu_write(p, TIOR, 0x0002); + + rate /= prescaler[k - 1] * p->refresh_rate; + r_tpu_write(p, TGRB, rate); + dev_dbg(&p->pdev->dev, "TRGB = 0x%04lx\n", rate); + + tmp = (cfg->max_brightness - brightness) * rate; + r_tpu_write(p, TGRA, tmp / cfg->max_brightness); + dev_dbg(&p->pdev->dev, "TRGA = 0x%04lx\n", tmp / cfg->max_brightness); + + /* PWM mode */ + r_tpu_write(p, TMDR, 0x0002); + + /* enable channel */ + r_tpu_start_stop_ch(p, 1); + + p->timer_state = R_TPU_TIMER_ON; + return 0; + err0: + clk_disable(p->clk); + pm_runtime_put_sync(&p->pdev->dev); + return -ENOTSUPP; +} + +static void r_tpu_disable(struct r_tpu_priv *p) +{ + if (p->timer_state == R_TPU_TIMER_UNUSED) + return; + + /* disable channel */ + r_tpu_start_stop_ch(p, 0); + + /* stop clock and mark device as idle */ + clk_disable(p->clk); + pm_runtime_put_sync(&p->pdev->dev); + + p->timer_state = R_TPU_TIMER_UNUSED; +} + +static void r_tpu_set_pin(struct r_tpu_priv *p, enum r_tpu_pin new_state, + enum led_brightness brightness) +{ + struct led_renesas_tpu_config *cfg = p->pdev->dev.platform_data; + + if (p->pin_state == new_state) { + if (p->pin_state == R_TPU_PIN_GPIO) + gpio_set_value(cfg->pin_gpio, brightness); + return; + } + + if (p->pin_state == R_TPU_PIN_GPIO) + gpio_free(cfg->pin_gpio); + + if (p->pin_state == R_TPU_PIN_GPIO_FN) + gpio_free(cfg->pin_gpio_fn); + + if (new_state == R_TPU_PIN_GPIO) { + gpio_request(cfg->pin_gpio, cfg->name); + gpio_direction_output(cfg->pin_gpio, !!brightness); + } + if (new_state == R_TPU_PIN_GPIO_FN) + gpio_request(cfg->pin_gpio_fn, cfg->name); + + p->pin_state = new_state; +} + +static void r_tpu_set_brightness(struct led_classdev *ldev, + enum led_brightness brightness) +{ + struct r_tpu_priv *p = container_of(ldev, struct r_tpu_priv, ldev); + + r_tpu_disable(p); + + /* off and maximum are handled as GPIO pins, in between PWM */ + if ((brightness == 0) || (brightness == ldev->max_brightness)) + r_tpu_set_pin(p, R_TPU_PIN_GPIO, brightness); + else { + r_tpu_set_pin(p, R_TPU_PIN_GPIO_FN, 0); + r_tpu_enable(p, brightness); + } +} + +static int __devinit r_tpu_probe(struct platform_device *pdev) +{ + struct led_renesas_tpu_config *cfg = pdev->dev.platform_data; + struct r_tpu_priv *p; + struct resource *res; + int ret = -ENXIO; + + if (!cfg) { + dev_err(&pdev->dev, "missing platform data\n"); + goto err0; + } + + p = kzalloc(sizeof(*p), GFP_KERNEL); + if (p == NULL) { + dev_err(&pdev->dev, "failed to allocate driver data\n"); + ret = -ENOMEM; + goto err0; + } + + res = platform_get_resource(pdev, IORESOURCE_MEM, 0); + if (!res) { + dev_err(&pdev->dev, "failed to get I/O memory\n"); + goto err1; + } + + /* map memory, let mapbase point to our channel */ + p->mapbase = ioremap_nocache(res->start, resource_size(res)); + if (p->mapbase == NULL) { + dev_err(&pdev->dev, "failed to remap I/O memory\n"); + goto err1; + } + + /* get hold of clock */ + p->clk = clk_get(&pdev->dev, NULL); + if (IS_ERR(p->clk)) { + dev_err(&pdev->dev, "cannot get clock\n"); + ret = PTR_ERR(p->clk); + goto err2; + } + + p->pdev = pdev; + p->pin_state = R_TPU_PIN_UNUSED; + p->timer_state = R_TPU_TIMER_UNUSED; + p->refresh_rate = cfg->refresh_rate ? cfg->refresh_rate : 100; + r_tpu_set_pin(p, R_TPU_PIN_GPIO, LED_OFF); + platform_set_drvdata(pdev, p); + + + p->ldev.name = cfg->name; + p->ldev.brightness = LED_OFF; + p->ldev.max_brightness = cfg->max_brightness; + p->ldev.brightness_set = r_tpu_set_brightness; + p->ldev.flags |= LED_CORE_SUSPENDRESUME; + ret = led_classdev_register(&pdev->dev, &p->ldev); + if (ret < 0) + goto err3; + + /* max_brightness may be updated by the LED core code */ + p->min_rate = p->ldev.max_brightness * p->refresh_rate; + + pm_runtime_enable(&pdev->dev); + return 0; + + err3: + r_tpu_set_pin(p, R_TPU_PIN_UNUSED, LED_OFF); + clk_put(p->clk); + err2: + iounmap(p->mapbase); + err1: + kfree(p); + err0: + return ret; +} + +static int __devexit r_tpu_remove(struct platform_device *pdev) +{ + struct r_tpu_priv *p = platform_get_drvdata(pdev); + + r_tpu_set_brightness(&p->ldev, LED_OFF); + led_classdev_unregister(&p->ldev); + r_tpu_disable(p); + r_tpu_set_pin(p, R_TPU_PIN_UNUSED, LED_OFF); + + pm_runtime_disable(&pdev->dev); + clk_put(p->clk); + + iounmap(p->mapbase); + kfree(p); + return 0; +} + +static struct platform_driver r_tpu_device_driver = { + .probe = r_tpu_probe, + .remove = __devexit_p(r_tpu_remove), + .driver = { + .name = "leds-renesas-tpu", + } +}; + +static int __init r_tpu_init(void) +{ + return platform_driver_register(&r_tpu_device_driver); +} + +static void __exit r_tpu_exit(void) +{ + platform_driver_unregister(&r_tpu_device_driver); +} + +module_init(r_tpu_init); +module_exit(r_tpu_exit); + +MODULE_AUTHOR("Magnus Damm"); +MODULE_DESCRIPTION("Renesas TPU LED Driver"); +MODULE_LICENSE("GPL v2"); diff --git a/include/linux/leds-renesas-tpu.h b/include/linux/leds-renesas-tpu.h new file mode 100644 index 000000000000..055387086fc1 --- /dev/null +++ b/include/linux/leds-renesas-tpu.h @@ -0,0 +1,14 @@ +#ifndef __LEDS_RENESAS_TPU_H__ +#define __LEDS_RENESAS_TPU_H__ + +struct led_renesas_tpu_config { + char *name; + unsigned pin_gpio_fn; + unsigned pin_gpio; + unsigned int channel_offset; + unsigned int timer_bit; + unsigned int max_brightness; + unsigned int refresh_rate; +}; + +#endif /* __LEDS_RENESAS_TPU_H__ */ -- cgit v1.2.3 From 2b67c95b74f17c13c7b3a990540c9dd9b4a8480d Mon Sep 17 00:00:00 2001 From: Magnus Damm Date: Mon, 31 Oct 2011 17:12:09 -0700 Subject: drivers/leds/leds-renesas-tpu.c: move Renesas TPU LED driver platform data Use the platform_data include directory for the TPU LED driver, as suggested by Paul Mundt. Signed-off-by: Magnus Damm Cc: Paul Mundt Cc: Richard Purdie Cc: Grant Likely Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds --- drivers/leds/leds-renesas-tpu.c | 2 +- include/linux/leds-renesas-tpu.h | 14 -------------- include/linux/platform_data/leds-renesas-tpu.h | 14 ++++++++++++++ 3 files changed, 15 insertions(+), 15 deletions(-) delete mode 100644 include/linux/leds-renesas-tpu.h create mode 100644 include/linux/platform_data/leds-renesas-tpu.h (limited to 'include/linux') diff --git a/drivers/leds/leds-renesas-tpu.c b/drivers/leds/leds-renesas-tpu.c index 45dbdf58f5ee..3ee540eb127e 100644 --- a/drivers/leds/leds-renesas-tpu.c +++ b/drivers/leds/leds-renesas-tpu.c @@ -26,7 +26,7 @@ #include #include #include -#include +#include #include #include #include diff --git a/include/linux/leds-renesas-tpu.h b/include/linux/leds-renesas-tpu.h deleted file mode 100644 index 055387086fc1..000000000000 --- a/include/linux/leds-renesas-tpu.h +++ /dev/null @@ -1,14 +0,0 @@ -#ifndef __LEDS_RENESAS_TPU_H__ -#define __LEDS_RENESAS_TPU_H__ - -struct led_renesas_tpu_config { - char *name; - unsigned pin_gpio_fn; - unsigned pin_gpio; - unsigned int channel_offset; - unsigned int timer_bit; - unsigned int max_brightness; - unsigned int refresh_rate; -}; - -#endif /* __LEDS_RENESAS_TPU_H__ */ diff --git a/include/linux/platform_data/leds-renesas-tpu.h b/include/linux/platform_data/leds-renesas-tpu.h new file mode 100644 index 000000000000..055387086fc1 --- /dev/null +++ b/include/linux/platform_data/leds-renesas-tpu.h @@ -0,0 +1,14 @@ +#ifndef __LEDS_RENESAS_TPU_H__ +#define __LEDS_RENESAS_TPU_H__ + +struct led_renesas_tpu_config { + char *name; + unsigned pin_gpio_fn; + unsigned pin_gpio; + unsigned int channel_offset; + unsigned int timer_bit; + unsigned int max_brightness; + unsigned int refresh_rate; +}; + +#endif /* __LEDS_RENESAS_TPU_H__ */ -- cgit v1.2.3 From 55036ba76b2d2fd53b5c00993fcec5ed56e83922 Mon Sep 17 00:00:00 2001 From: Andy Shevchenko Date: Mon, 31 Oct 2011 17:12:41 -0700 Subject: lib: rename pack_hex_byte() to hex_byte_pack() As suggested by Andrew Morton in [1] there is better to have most significant part first in the function name. [1] https://lkml.org/lkml/2011/9/20/22 There is no functional change. Signed-off-by: Andy Shevchenko Cc: Jesper Nilsson Cc: David Howells Cc: Koichi Yasutake Cc: Jason Wessel Cc: Mimi Zohar Cc: James Morris Cc: OGAWA Hirofumi Cc: "John W. Linville" Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds --- include/linux/kernel.h | 7 ++++++- lib/vsprintf.c | 14 +++++++------- 2 files changed, 13 insertions(+), 8 deletions(-) (limited to 'include/linux') diff --git a/include/linux/kernel.h b/include/linux/kernel.h index e40c950e1d62..4824f26a0dc2 100644 --- a/include/linux/kernel.h +++ b/include/linux/kernel.h @@ -372,13 +372,18 @@ extern const char hex_asc[]; #define hex_asc_lo(x) hex_asc[((x) & 0x0f)] #define hex_asc_hi(x) hex_asc[((x) & 0xf0) >> 4] -static inline char *pack_hex_byte(char *buf, u8 byte) +static inline char *hex_byte_pack(char *buf, u8 byte) { *buf++ = hex_asc_hi(byte); *buf++ = hex_asc_lo(byte); return buf; } +static inline char * __deprecated pack_hex_byte(char *buf, u8 byte) +{ + return hex_byte_pack(buf, byte); +} + extern int hex_to_bin(char ch); extern int __must_check hex2bin(u8 *dst, const char *src, size_t count); diff --git a/lib/vsprintf.c b/lib/vsprintf.c index c1a3927326eb..993599e66e5a 100644 --- a/lib/vsprintf.c +++ b/lib/vsprintf.c @@ -547,7 +547,7 @@ char *mac_address_string(char *buf, char *end, u8 *addr, } for (i = 0; i < 6; i++) { - p = pack_hex_byte(p, addr[i]); + p = hex_byte_pack(p, addr[i]); if (fmt[0] == 'M' && i != 5) *p++ = separator; } @@ -667,13 +667,13 @@ char *ip6_compressed_string(char *p, const char *addr) lo = word & 0xff; if (hi) { if (hi > 0x0f) - p = pack_hex_byte(p, hi); + p = hex_byte_pack(p, hi); else *p++ = hex_asc_lo(hi); - p = pack_hex_byte(p, lo); + p = hex_byte_pack(p, lo); } else if (lo > 0x0f) - p = pack_hex_byte(p, lo); + p = hex_byte_pack(p, lo); else *p++ = hex_asc_lo(lo); needcolon = true; @@ -695,8 +695,8 @@ char *ip6_string(char *p, const char *addr, const char *fmt) int i; for (i = 0; i < 8; i++) { - p = pack_hex_byte(p, *addr++); - p = pack_hex_byte(p, *addr++); + p = hex_byte_pack(p, *addr++); + p = hex_byte_pack(p, *addr++); if (fmt[0] == 'I' && i != 7) *p++ = ':'; } @@ -754,7 +754,7 @@ char *uuid_string(char *buf, char *end, const u8 *addr, } for (i = 0; i < 16; i++) { - p = pack_hex_byte(p, addr[index[i]]); + p = hex_byte_pack(p, addr[index[i]]); switch (i) { case 3: case 5: -- cgit v1.2.3 From fc23af34b00ef444eec088f744983b9ca6c7f5d1 Mon Sep 17 00:00:00 2001 From: Andrew Morton Date: Mon, 31 Oct 2011 17:13:08 -0700 Subject: llist-return-whether-list-is-empty-before-adding-in-llist_add-fix clarify comment Cc: Huang Ying Cc: Mathieu Desnoyers Cc: Peter Zijlstra Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds --- include/linux/llist.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'include/linux') diff --git a/include/linux/llist.h b/include/linux/llist.h index 7287734e08d1..801b44b07aac 100644 --- a/include/linux/llist.h +++ b/include/linux/llist.h @@ -148,7 +148,7 @@ static inline struct llist_node *llist_next(struct llist_node *node) * @new: new entry to be added * @head: the head for your lock-less list * - * Return whether list is empty before adding. + * Returns true if the list was empty prior to adding this entry. */ static inline bool llist_add(struct llist_node *new, struct llist_head *head) { -- cgit v1.2.3 From 67d0a0754455f89ef3946946159d8ec9e45ce33a Mon Sep 17 00:00:00 2001 From: Joe Perches Date: Mon, 31 Oct 2011 17:13:10 -0700 Subject: kernel.h/checkpatch: mark strict_strto and simple_strto as obsolete Mark obsolete/deprecated strict_strto and simple_strto functions and macros as obsolete. Update checkpatch to warn about their use. Signed-off-by: Joe Perches Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds --- include/linux/kernel.h | 4 ++++ scripts/checkpatch.pl | 6 +++--- 2 files changed, 7 insertions(+), 3 deletions(-) (limited to 'include/linux') diff --git a/include/linux/kernel.h b/include/linux/kernel.h index 4824f26a0dc2..4c0d3b2fd5fc 100644 --- a/include/linux/kernel.h +++ b/include/linux/kernel.h @@ -287,6 +287,8 @@ static inline int __must_check kstrtos32_from_user(const char __user *s, size_t return kstrtoint_from_user(s, count, base, res); } +/* Obsolete, do not use. Use kstrto instead */ + extern unsigned long simple_strtoul(const char *,char **,unsigned int); extern long simple_strtol(const char *,char **,unsigned int); extern unsigned long long simple_strtoull(const char *,char **,unsigned int); @@ -296,6 +298,8 @@ extern long long simple_strtoll(const char *,char **,unsigned int); #define strict_strtoull kstrtoull #define strict_strtoll kstrtoll +/* lib/printf utilities */ + extern __printf(2, 3) int sprintf(char *buf, const char * fmt, ...); extern __printf(2, 0) int vsprintf(char *buf, const char *, va_list); extern __printf(3, 4) diff --git a/scripts/checkpatch.pl b/scripts/checkpatch.pl index 0b3e35c9ef08..5ba679c8cde6 100755 --- a/scripts/checkpatch.pl +++ b/scripts/checkpatch.pl @@ -3151,10 +3151,10 @@ sub process { "consider using a completion\n" . $herecurr); } -# recommend kstrto* over simple_strto* - if ($line =~ /\bsimple_(strto.*?)\s*\(/) { +# recommend kstrto* over simple_strto* and strict_strto* + if ($line =~ /\b((simple|strict)_(strto(l|ll|ul|ull)))\s*\(/) { WARN("CONSIDER_KSTRTO", - "consider using kstrto* in preference to simple_$1\n" . $herecurr); + "$1 is obsolete, use k$3 instead\n" . $herecurr); } # check for __initcall(), use device_initcall() explicitly please if ($line =~ /^.\s*__initcall\s*\(/) { -- cgit v1.2.3