diff options
| author | Steven Rostedt <srostedt@redhat.com> | 2010-10-08 21:51:48 +0400 |
|---|---|---|
| committer | Steven Rostedt <rostedt@goodmis.org> | 2010-10-19 21:22:36 +0400 |
| commit | 747e94ae3d1b4c9bf5380e569f614eb9040b79e7 (patch) | |
| tree | 29404444fa5106e6d3361fc6c85b4463777ec3a3 | |
| parent | 51ea8a88aae425915b458bfe295f1f10a3d52781 (diff) | |
| download | linux-747e94ae3d1b4c9bf5380e569f614eb9040b79e7.tar.xz | |
ring-buffer: Make write slow path out of line
Gcc inlines the slow path of the ring buffer write which can
hurt performance. This patch simply forces the slow path function
rb_move_tail() to always be a function.
The ring_buffer_benchmark module with reader_disabled=1 shows that
this patch changes the time to record an event from 135 ns to
132 ns. (3 ns or 2.22% improvement)
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
| -rw-r--r-- | kernel/trace/ring_buffer.c | 7 |
1 files changed, 5 insertions, 2 deletions
diff --git a/kernel/trace/ring_buffer.c b/kernel/trace/ring_buffer.c index bca96377fd4e..0b88df849a59 100644 --- a/kernel/trace/ring_buffer.c +++ b/kernel/trace/ring_buffer.c @@ -1823,7 +1823,10 @@ rb_reset_tail(struct ring_buffer_per_cpu *cpu_buffer, local_sub(length, &tail_page->write); } -static struct ring_buffer_event * +/* + * This is the slow path, force gcc not to inline it. + */ +static noinline struct ring_buffer_event * rb_move_tail(struct ring_buffer_per_cpu *cpu_buffer, unsigned long length, unsigned long tail, struct buffer_page *tail_page, u64 *ts) @@ -1943,7 +1946,7 @@ __rb_reserve_next(struct ring_buffer_per_cpu *cpu_buffer, tail = write - length; /* See if we shot pass the end of this buffer page */ - if (write > BUF_PAGE_SIZE) + if (unlikely(write > BUF_PAGE_SIZE)) return rb_move_tail(cpu_buffer, length, tail, tail_page, ts); |