kernel/linux.git/tools/perf/Documentation, branch v5.19

perf docs: Correct typo of event_sources

2022-06-03T19:14:51+00:00

The sysfs directory is called event_source. Before: $ ls -la /sys/bus/event_sources/devices/cpu/format/ ls: cannot access '/sys/bus/event_sources/devices/cpu/format/': No such file or directory $ After: $ ls -la /sys/bus/event_source/devices/cpu/format/ total 0 drwxr-xr-x. 2 root root 0 Jun 2 15:36 . drwxr-xr-x. 6 root root 0 Jun 2 15:35 .. -r--r--r--. 1 root root 4096 Jun 2 15:36 any -r--r--r--. 1 root root 4096 Jun 2 15:36 cmask -r--r--r--. 1 root root 4096 Jun 2 15:36 edge -r--r--r--. 1 root root 4096 Jun 2 15:36 event -r--r--r--. 1 root root 4096 Jun 2 15:36 frontend -r--r--r--. 1 root root 4096 Jun 2 15:36 inv -r--r--r--. 1 root root 4096 Jun 2 15:36 ldlat -r--r--r--. 1 root root 4096 Jun 2 15:36 offcore_rsp -r--r--r--. 1 root root 4096 Jun 2 15:36 pc -r--r--r--. 1 root root 4096 Jun 2 15:36 umask $ Reviewed-by: Sandipan Das Signed-off-by: Ian Rogers Tested-by: Arnaldo Carvalho de Melo Cc: Alexander Shishkin Cc: Alexey Bayduraev Cc: Alyssa Ross Cc: German Gomez Cc: Ingo Molnar Cc: Jin Yao Cc: Jiri Olsa Cc: Joshua Martinez Cc: Kan Liang Cc: Like Xu Cc: Mark Rutland Cc: Namhyung Kim Cc: Peter Zijlstra Cc: Zhengjun Xing Link: https://lore.kernel.org/r/20220603045744.2815559-1-irogers@google.com Reported-by: Kevin Nomura Signed-off-by: Arnaldo Carvalho de Melo

perf record: Enable off-cpu analysis with BPF

2022-05-26T15:36:57+00:00

Add --off-cpu option to enable the off-cpu profiling with BPF. It'd use a bpf_output event and rename it to "offcpu-time". Samples will be synthesized at the end of the record session using data from a BPF map which contains the aggregated off-cpu time at context switches. So it needs root privilege to get the off-cpu profiling. Each sample will have a separate user stacktrace so it will skip kernel threads. The sample ip will be set from the stacktrace and other sample data will be updated accordingly. Currently it only handles some basic sample types. The sample timestamp is set to a dummy value just not to bother with other events during the sorting. So it has a very big initial value and increase it on processing each samples. Good thing is that it can be used together with regular profiling like cpu cycles. If you don't want to that, you can use a dummy event to enable off-cpu profiling only. Example output: $ sudo perf record --off-cpu perf bench sched messaging -l 1000 $ sudo perf report --stdio --call-graph=no # Total Lost Samples: 0 # # Samples: 41K of event 'cycles' # Event count (approx.): 42137343851 ... # Samples: 1K of event 'offcpu-time' # Event count (approx.): 587990831640 # # Children Self Command Shared Object Symbol # ........ ........ ............... .................. ......................... # 81.66% 0.00% sched-messaging libc-2.33.so [.] __libc_start_main 81.66% 0.00% sched-messaging perf [.] cmd_bench 81.66% 0.00% sched-messaging perf [.] main 81.66% 0.00% sched-messaging perf [.] run_builtin 81.43% 0.00% sched-messaging perf [.] bench_sched_messaging 40.86% 40.86% sched-messaging libpthread-2.33.so [.] __read 37.66% 37.66% sched-messaging libpthread-2.33.so [.] __write 2.91% 2.91% sched-messaging libc-2.33.so [.] __poll ... As you can see it spent most of off-cpu time in read and write in bench_sched_messaging(). The --call-graph=no was added just to make the output concise here. It uses perf hooks facility to control BPF program during the record session rather than adding new BPF/off-cpu specific calls. Signed-off-by: Namhyung Kim Acked-by: Ian Rogers Tested-by: Arnaldo Carvalho de Melo Cc: Andi Kleen Cc: Blake Jones Cc: Hao Luo Cc: Ingo Molnar Cc: Jiri Olsa Cc: Milian Wolff Cc: Peter Zijlstra Cc: Song Liu Cc: bpf@vger.kernel.org Link: https://lore.kernel.org/r/20220518224725.742882-3-namhyung@kernel.org Signed-off-by: Arnaldo Carvalho de Melo

perf intel-pt: Add guest_code support

2022-05-23T13:19:24+00:00

A common case for KVM test programs is that the test program acts as the hypervisor, creating, running and destroying the virtual machine, and providing the guest object code from its own object code. In this case, the VM is not running an OS, but only the functions loaded into it by the hypervisor test program, and conveniently, loaded at the same virtual addresses. To support that, a new option "--guest-code" has been added in previous patches. In this patch, add support also to Intel PT. In particular, ensure guest_code thread is set up before attempting to walk object code or synthesize samples. Example: # perf record --kcore -e intel_pt/cyc/ -- tools/testing/selftests/kselftest_install/kvm/tsc_msrs_test [ perf record: Woken up 1 times to write data ] [ perf record: Captured and wrote 0.280 MB perf.data ] # perf script --guest-code --itrace=bep --ns -F-period,+addr,+flags [SNIP] tsc_msrs_test 18436 [007] 10897.962087733: branches: call ffffffffc13b2ff5 __vmx_vcpu_run+0x15 (vmlinux) => ffffffffc13b2f50 vmx_update_host_rsp+0x0 (vmlinux) tsc_msrs_test 18436 [007] 10897.962087733: branches: return ffffffffc13b2f5d vmx_update_host_rsp+0xd (vmlinux) => ffffffffc13b2ffa __vmx_vcpu_run+0x1a (vmlinux) tsc_msrs_test 18436 [007] 10897.962087733: branches: call ffffffffc13b303b __vmx_vcpu_run+0x5b (vmlinux) => ffffffffc13b2f80 vmx_vmenter+0x0 (vmlinux) tsc_msrs_test 18436 [007] 10897.962087836: branches: vmentry ffffffffc13b2f82 vmx_vmenter+0x2 (vmlinux) => 0 [unknown] ([unknown]) [guest/18436] 18436 [007] 10897.962087836: branches: vmentry 0 [unknown] ([unknown]) => 402c81 guest_code+0x131 (/home/ahunter/git/work/tools/testing/selftests/kselftest_install/kvm/tsc_msrs_test) [guest/18436] 18436 [007] 10897.962087836: branches: call 402c81 guest_code+0x131 (/home/ahunter/git/work/tools/testing/selftests/kselftest_install/kvm/tsc_msrs_test) => 40dba0 ucall+0x0 (/home/ahunter/git/work/tools/testing/selftests/kselftest_install/kvm/tsc_msrs_test) [guest/18436] 18436 [007] 10897.962088248: branches: vmexit 40dba0 ucall+0x0 (/home/ahunter/git/work/tools/testing/selftests/kselftest_install/kvm/tsc_msrs_test) => 0 [unknown] ([unknown]) tsc_msrs_test 18436 [007] 10897.962088248: branches: vmexit 0 [unknown] ([unknown]) => ffffffffc13b2fa0 vmx_vmexit+0x0 (vmlinux) tsc_msrs_test 18436 [007] 10897.962088248: branches: jmp ffffffffc13b2fa0 vmx_vmexit+0x0 (vmlinux) => ffffffffc13b2fd2 vmx_vmexit+0x32 (vmlinux) tsc_msrs_test 18436 [007] 10897.962088256: branches: return ffffffffc13b2fd2 vmx_vmexit+0x32 (vmlinux) => ffffffffc13b3040 __vmx_vcpu_run+0x60 (vmlinux) tsc_msrs_test 18436 [007] 10897.962088270: branches: return ffffffffc13b30b6 __vmx_vcpu_run+0xd6 (vmlinux) => ffffffffc13b2f2e vmx_vcpu_enter_exit+0x4e (vmlinux) [SNIP] tsc_msrs_test 18436 [007] 10897.962089321: branches: call ffffffffc13b2ff5 __vmx_vcpu_run+0x15 (vmlinux) => ffffffffc13b2f50 vmx_update_host_rsp+0x0 (vmlinux) tsc_msrs_test 18436 [007] 10897.962089321: branches: return ffffffffc13b2f5d vmx_update_host_rsp+0xd (vmlinux) => ffffffffc13b2ffa __vmx_vcpu_run+0x1a (vmlinux) tsc_msrs_test 18436 [007] 10897.962089321: branches: call ffffffffc13b303b __vmx_vcpu_run+0x5b (vmlinux) => ffffffffc13b2f80 vmx_vmenter+0x0 (vmlinux) tsc_msrs_test 18436 [007] 10897.962089424: branches: vmentry ffffffffc13b2f82 vmx_vmenter+0x2 (vmlinux) => 0 [unknown] ([unknown]) [guest/18436] 18436 [007] 10897.962089424: branches: vmentry 0 [unknown] ([unknown]) => 40dba0 ucall+0x0 (/home/ahunter/git/work/tools/testing/selftests/kselftest_install/kvm/tsc_msrs_test) [guest/18436] 18436 [007] 10897.962089701: branches: jmp 40dc1b ucall+0x7b (/home/ahunter/git/work/tools/testing/selftests/kselftest_install/kvm/tsc_msrs_test) => 40dc39 ucall+0x99 (/home/ahunter/git/work/tools/testing/selftests/kselftest_install/kvm/tsc_msrs_test) [guest/18436] 18436 [007] 10897.962089701: branches: jcc 40dc3c ucall+0x9c (/home/ahunter/git/work/tools/testing/selftests/kselftest_install/kvm/tsc_msrs_test) => 40dc20 ucall+0x80 (/home/ahunter/git/work/tools/testing/selftests/kselftest_install/kvm/tsc_msrs_test) [guest/18436] 18436 [007] 10897.962089701: branches: jcc 40dc3c ucall+0x9c (/home/ahunter/git/work/tools/testing/selftests/kselftest_install/kvm/tsc_msrs_test) => 40dc20 ucall+0x80 (/home/ahunter/git/work/tools/testing/selftests/kselftest_install/kvm/tsc_msrs_test) [guest/18436] 18436 [007] 10897.962089701: branches: jcc 40dc37 ucall+0x97 (/home/ahunter/git/work/tools/testing/selftests/kselftest_install/kvm/tsc_msrs_test) => 40dc50 ucall+0xb0 (/home/ahunter/git/work/tools/testing/selftests/kselftest_install/kvm/tsc_msrs_test) [guest/18436] 18436 [007] 10897.962089878: branches: vmexit 40dc55 ucall+0xb5 (/home/ahunter/git/work/tools/testing/selftests/kselftest_install/kvm/tsc_msrs_test) => 0 [unknown] ([unknown]) tsc_msrs_test 18436 [007] 10897.962089878: branches: vmexit 0 [unknown] ([unknown]) => ffffffffc13b2fa0 vmx_vmexit+0x0 (vmlinux) tsc_msrs_test 18436 [007] 10897.962089878: branches: jmp ffffffffc13b2fa0 vmx_vmexit+0x0 (vmlinux) => ffffffffc13b2fd2 vmx_vmexit+0x32 (vmlinux) tsc_msrs_test 18436 [007] 10897.962089887: branches: return ffffffffc13b2fd2 vmx_vmexit+0x32 (vmlinux) => ffffffffc13b3040 __vmx_vcpu_run+0x60 (vmlinux) tsc_msrs_test 18436 [007] 10897.962089901: branches: return ffffffffc13b30b6 __vmx_vcpu_run+0xd6 (vmlinux) => ffffffffc13b2f2e vmx_vcpu_enter_exit+0x4e (vmlinux) [SNIP] # perf kvm --guest-code --guest --host report -i perf.data --stdio | head -20 # To display the perf.data header info, please use --header/--header-only options. # # # Total Lost Samples: 0 # # Samples: 12 of event 'instructions' # Event count (approx.): 2274583 # # Children Self Command Shared Object Symbol # ........ ........ ............. .................... ........................................... # 54.70% 0.00% tsc_msrs_test [kernel.vmlinux] [k] entry_SYSCALL_64_after_hwframe | ---entry_SYSCALL_64_after_hwframe do_syscall_64 | |--29.44%--syscall_exit_to_user_mode | exit_to_user_mode_prepare | task_work_run | __fput For more information about Perf tools support for Intel® Processor Trace refer: https://perf.wiki.kernel.org/index.php/Perf_tools_support_for_Intel%C2%AE_Processor_Trace Signed-off-by: Adrian Hunter Cc: Andi Kleen Cc: Ian Rogers Cc: Jiri Olsa Cc: Leo Yan Cc: Namhyung Kim Cc: kvm@vger.kernel.org Link: https://lore.kernel.org/r/20220517131011.6117-7-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo

perf kvm report: Add guest_code support

2022-05-23T13:19:15+00:00

Add an option to indicate that guest code can be found in the hypervisor process. Signed-off-by: Adrian Hunter Cc: Andi Kleen Cc: Ian Rogers Cc: Jiri Olsa Cc: Leo Yan Cc: Namhyung Kim Cc: kvm@vger.kernel.org Link: https://lore.kernel.org/r/20220517131011.6117-6-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo

perf script: Add guest_code support

2022-05-23T13:19:04+00:00

Add an option to indicate that guest code can be found in the hypervisor process. Signed-off-by: Adrian Hunter Cc: Andi Kleen Cc: Ian Rogers Cc: Jiri Olsa Cc: Leo Yan Cc: Namhyung Kim Cc: kvm@vger.kernel.org Link: https://lore.kernel.org/r/20220517131011.6117-5-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo

perf lock: Add -t/--thread option for report

2022-05-23T12:49:35+00:00

The -t option is to show per-thread lock stat like below: $ perf lock report -t -F acquired,contended,avg_wait Name acquired contended avg wait (ns) perf 240569 9 5784 swapper 106610 19 543 :15789 17370 2 14538 ContainerMgr 8981 6 874 sleep 5275 1 11281 ContainerThread 4416 4 944 RootPressureThr 3215 5 1215 rcu_preempt 2954 0 0 ContainerMgr 2560 0 0 unnamed 1873 0 0 EventManager_De 1845 1 636 futex-default-S 1609 0 0 ... Committer notes: Add that option to the 'perf lock report' man page. Signed-off-by: Namhyung Kim Cc: Ian Rogers Cc: Ingo Molnar Cc: Jiri Olsa Cc: Peter Zijlstra Link: https://lore.kernel.org/r/20220521010811.932703-2-namhyung@kernel.org Signed-off-by: Arnaldo Carvalho de Melo

perf c2c: Update documentation for store metric 'N/A'

2022-05-23T12:36:47+00:00

The 'N/A' metric is added for store operations, update documentation to reflect changes in the report table. Signed-off-by: Leo Yan Acked-by: Jiri Olsa Cc: Adam Li Cc: Alexander Shishkin Cc: Ali Saidi Cc: Alyssa Ross Cc: German Gomez Cc: Ian Rogers Cc: Ingo Molnar Cc: James Clark Cc: Joe Mario Cc: Kajol Jain Cc: Kan Liang Cc: Li Huafei Cc: Like Xu Cc: Mark Rutland Cc: Namhyung Kim Cc: Peter Zijlstra Link: https://lore.kernel.org/r/20220518055729.1869566-4-leo.yan@linaro.org Signed-off-by: Arnaldo Carvalho de Melo

perf intel-pt: Add support for emulated ptwrite

2022-05-17T14:55:49+00:00

ptwrite is an Intel x86 instruction that writes arbitrary values into an Intel PT trace. It is not supported on all hardware, so provide an alternative that makes use of TNT packets to convey the payload data. TNT packets encode Taken/Not-taken conditional branch information, so taking branches based on the payload value will encode the value into the TNT packet. Refer to the changes to the documentation file perf-intel-pt.txt in this patch for an example. Signed-off-by: Adrian Hunter Cc: Jiri Olsa Link: https://lore.kernel.org/r/20220509152400.376613-2-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo

perf annotate: Add --percent-limit option

2022-05-10T17:37:55+00:00

Like in 'perf report' and 'perf top', Add this option to limit the number of functions it displays based on the overhead value in percent. This affects only stdio and stdio2 output modes. Without this, it shows very long disassembly lines for every function in the data file. If users don't want this behavior, they can set a value in percent to suppress that. Signed-off-by: Namhyung Kim Cc: Andi Kleen Cc: Ian Rogers Cc: Jiri Olsa Cc: Peter Zijlstra Link: http://lore.kernel.org/lkml/20220502232015.697243-1-namhyung@kernel.org Signed-off-by: Arnaldo Carvalho de Melo

perf intel-pt: Add link to the perf wiki's Intel PT page

2022-04-26T17:32:29+00:00

Add an EXAMPLE section and link to the perf wiki's Intel PT page. Signed-off-by: Adrian Hunter Cc: Jiri Olsa Link: http://lore.kernel.org/lkml/20220426133213.248475-1-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo