diff options
| author | Andrey Ignatov <rdna@fb.com> | 2021-10-06 03:18:38 +0300 |
|---|---|---|
| committer | Alexei Starovoitov <ast@kernel.org> | 2021-10-06 05:22:33 +0300 |
| commit | 0640c77c46cb84328d5e08aa85a781a60be8b02b (patch) | |
| tree | 0b3f943af8c3e17e5ab44d78eba68c56864487cf /arch/x86/net | |
| parent | 32a16f6bfe512a29fbe7e10ccaced118801a178f (diff) | |
| download | linux-0640c77c46cb84328d5e08aa85a781a60be8b02b.tar.xz | |
bpf: Avoid retpoline for bpf_for_each_map_elem
Similarly to 09772d92cd5a ("bpf: avoid retpoline for
lookup/update/delete calls on maps") and 84430d4232c3 ("bpf, verifier:
avoid retpoline for map push/pop/peek operation") avoid indirect call
while calling bpf_for_each_map_elem.
Before (a program fragment):
; if (rules_map) {
142: (15) if r4 == 0x0 goto pc+8
143: (bf) r3 = r10
; bpf_for_each_map_elem(rules_map, process_each_rule, &ctx, 0);
144: (07) r3 += -24
145: (bf) r1 = r4
146: (18) r2 = subprog[+5]
148: (b7) r4 = 0
149: (85) call bpf_for_each_map_elem#143680 <-- indirect call via
helper
After (same program fragment):
; if (rules_map) {
142: (15) if r4 == 0x0 goto pc+8
143: (bf) r3 = r10
; bpf_for_each_map_elem(rules_map, process_each_rule, &ctx, 0);
144: (07) r3 += -24
145: (bf) r1 = r4
146: (18) r2 = subprog[+5]
148: (b7) r4 = 0
149: (85) call bpf_for_each_array_elem#170336 <-- direct call
On a benchmark that calls bpf_for_each_map_elem() once and does many
other things (mostly checking fields in skb) with CONFIG_RETPOLINE=y it
makes program faster.
Before:
============================================================================
Benchmark.cpp time/iter iters/s
============================================================================
IngressMatchByRemoteEndpoint 80.78ns 12.38M
IngressMatchByRemoteIP 80.66ns 12.40M
IngressMatchByRemotePort 80.87ns 12.37M
After:
============================================================================
Benchmark.cpp time/iter iters/s
============================================================================
IngressMatchByRemoteEndpoint 73.49ns 13.61M
IngressMatchByRemoteIP 71.48ns 13.99M
IngressMatchByRemotePort 70.39ns 14.21M
Signed-off-by: Andrey Ignatov <rdna@fb.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://lore.kernel.org/bpf/20211006001838.75607-1-rdna@fb.com
Diffstat (limited to 'arch/x86/net')
0 files changed, 0 insertions, 0 deletions