kernel/linux.git/include/linux/wait.h, branch linux-2.6.28.y

wait: prevent exclusive waiter starvation

2009-02-12T17:50:23+00:00

commit 777c6c5f1f6e757ae49ecca2ed72d6b1f523c007 upstream. With exclusive waiters, every process woken up through the wait queue must ensure that the next waiter down the line is woken when it has finished. Interruptible waiters don't do that when aborting due to a signal. And if an aborting waiter is concurrently woken up through the waitqueue, noone will ever wake up the next waiter. This has been observed with __wait_on_bit_lock() used by lock_page_killable(): the first contender on the queue was aborting when the actual lock holder woke it up concurrently. The aborted contender didn't acquire the lock and therefor never did an unlock followed by waking up the next waiter. Add abort_exclusive_wait() which removes the process' wait descriptor from the waitqueue, iff still queued, or wakes up the next waiter otherwise. It does so under the waitqueue lock. Racing with a wake up means the aborting process is either already woken (removed from the queue) and will wake up the next waiter, or it will remove itself from the queue and the concurrent wake up will apply to the next waiter after it. Use abort_exclusive_wait() in __wait_event_interruptible_exclusive() and __wait_on_bit_lock() when they were interrupted by other means than a wake up through the queue. [akpm@linux-foundation.org: coding-style fixes] Reported-by: Chris Mason Signed-off-by: Johannes Weiner Mentored-by: Oleg Nesterov Cc: Peter Zijlstra Cc: Matthew Wilcox Cc: Chuck Lever Cc: Nick Piggin Cc: Ingo Molnar Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds Signed-off-by: Greg Kroah-Hartman

wait: kill is_sync_wait()

2008-10-16T18:21:31+00:00

is_sync_wait() is used to distinguish between sync and async waits. Basically sync waits are the ones initialized with init_waitqueue_entry() and async ones with init_waitqueue_func_entry(). The sync/async distinction is used only in prepare_to_wait[_exclusive]() and its only function is to skip setting the current task state if the wait is async. This has a few problems. * No one uses it. None of func_entry users use prepare_to_wait() functions, so the code path never gets executed. * The distinction is bogus. Maybe back when func_entry is used only by aio but it's now also used by epoll and in future possibly by 9p and poll/select. * Taking @state as argument and ignoring it silenly depending on how @wait is initialized is just a bad error-prone API. * It prevents func_entry waits from using wait->private for no good reason. This patch kills is_sync_wait() and the associated code paths from prepare_to_wait[_exclusive](). As there was no user of these code paths, this patch doesn't cause any behavior difference. Signed-off-by: Tejun Heo Acked-by: Ingo Molnar Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds

include/linux: Remove all users of FASTCALL() macro

2008-02-14T00:21:18+00:00

FASTCALL() is always expanded to empty, remove it. [akpm@linux-foundation.org: coding-style fixes] Signed-off-by: Harvey Harrison Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds

lockdep: annotate epoll

2008-02-05T17:44:07+00:00

On Sat, 2008-01-05 at 13:35 -0800, Davide Libenzi wrote: > I remember I talked with Arjan about this time ago. Basically, since 1) > you can drop an epoll fd inside another epoll fd 2) callback-based wakeups > are used, you can see a wake_up() from inside another wake_up(), but they > will never refer to the same lock instance. > Think about: > > dfd = socket(...); > efd1 = epoll_create(); > efd2 = epoll_create(); > epoll_ctl(efd1, EPOLL_CTL_ADD, dfd, ...); > epoll_ctl(efd2, EPOLL_CTL_ADD, efd1, ...); > > When a packet arrives to the device underneath "dfd", the net code will > issue a wake_up() on its poll wake list. Epoll (efd1) has installed a > callback wakeup entry on that queue, and the wake_up() performed by the > "dfd" net code will end up in ep_poll_callback(). At this point epoll > (efd1) notices that it may have some event ready, so it needs to wake up > the waiters on its poll wait list (efd2). So it calls ep_poll_safewake() > that ends up in another wake_up(), after having checked about the > recursion constraints. That are, no more than EP_MAX_POLLWAKE_NESTS, to > avoid stack blasting. Never hit the same queue, to avoid loops like: > > epoll_ctl(efd2, EPOLL_CTL_ADD, efd1, ...); > epoll_ctl(efd3, EPOLL_CTL_ADD, efd2, ...); > epoll_ctl(efd4, EPOLL_CTL_ADD, efd3, ...); > epoll_ctl(efd1, EPOLL_CTL_ADD, efd4, ...); > > The code "if (tncur->wq == wq || ..." prevents re-entering the same > queue/lock. Since the epoll code is very careful to not nest same instance locks allow the recursion. Signed-off-by: Peter Zijlstra Tested-by: Stefan Richter Acked-by: Davide Libenzi Cc: Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds

Add wait_event_killable

2007-12-06T22:40:14+00:00

Signed-off-by: Matthew Wilcox

wait: Use TASK_NORMAL

2007-12-06T22:34:36+00:00

Also move wake_up_locked() to be with the related functions Signed-off-by: Matthew Wilcox

sched: clean up sleep_on() APIs

2007-07-09T16:52:01+00:00

clean up the sleep_on() APIs: - do not use fastcall - replace fragile macro magic with proper inline functions Signed-off-by: Ingo Molnar

[PATCH] lockdep: annotate DECLARE_WAIT_QUEUE_HEAD

2006-10-30T20:08:40+00:00

kernel: INFO: trying to register non-static key. kernel: the code is fine but needs lockdep annotation. kernel: turning off the locking correctness validator. kernel: [] show_trace_log_lvl+0x58/0x16a kernel: [] show_trace+0xd/0x10 kernel: [] dump_stack+0x19/0x1b kernel: [] __lock_acquire+0xf0/0x90d kernel: [] lock_acquire+0x4b/0x6b kernel: [] _spin_lock_irqsave+0x22/0x32 kernel: [] prepare_to_wait+0x17/0x4b kernel: [] lpfc_do_work+0xdd/0xcc2 [lpfc] kernel: [] kthread+0xc3/0xf2 kernel: [] kernel_thread_helper+0x5/0xb Another case of non-static lockdep keys; duplicate the paradigm set by DECLARE_COMPLETION_ONSTACK and introduce DECLARE_WAIT_QUEUE_HEAD_ONSTACK. Signed-off-by: Peter Zijlstra Cc: Greg KH Cc: Markus Lidel Acked-by: Ingo Molnar Cc: Arjan van de Ven Cc: James Bottomley Cc: Marcel Holtmann Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds

[PATCH] uninline init_waitqueue_head()

2006-07-10T20:24:25+00:00

allyesconfig vmlinux size delta: text data bss dec filename 20736884 6073834 3075176 29885894 vmlinux.before 20721009 6073966 3075176 29870151 vmlinux.after ~18 bytes per callsite, 15K of text size (~0.1%) saved. (as an added bonus this also removes a lockdep annotation.) Signed-off-by: Ingo Molnar Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds

[PATCH] lockdep: annotate waitqueues

2006-07-03T22:27:07+00:00

Create one lock class for all waitqueue locks in the kernel. Has no effect on non-lockdep kernels. Signed-off-by: Ingo Molnar Signed-off-by: Arjan van de Ven Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds