kernel/linux.git/tools/perf/util/scripting-engines/trace-event-python.c, branch v6.12.80

perf script python: Avoid buffer overflow in python PEBS register interface

2024-09-06T14:44:58+00:00

Running a script that processes PEBS records gives buffer overflows in valgrind. The problem is that the allocation of the register string doesn't include the terminating 0 byte. Fix this. I also replaced the very magic "28" with a more reasonable larger buffer that should fit all registers. There's no need to conserve memory here. ==2106591== Memcheck, a memory error detector ==2106591== Copyright (C) 2002-2022, and GNU GPL'd, by Julian Seward et al. ==2106591== Using Valgrind-3.22.0 and LibVEX; rerun with -h for copyright info ==2106591== Command: ../perf script -i tcall.data gcov.py tcall.gcov ==2106591== ==2106591== Invalid write of size 1 ==2106591== at 0x713354: regs_map (trace-event-python.c:748) ==2106591== by 0x7134EB: set_regs_in_dict (trace-event-python.c:784) ==2106591== by 0x713E58: get_perf_sample_dict (trace-event-python.c:940) ==2106591== by 0x716327: python_process_general_event (trace-event-python.c:1499) ==2106591== by 0x7164E1: python_process_event (trace-event-python.c:1531) ==2106591== by 0x44F9AF: process_sample_event (builtin-script.c:2549) ==2106591== by 0x6294DC: evlist__deliver_sample (session.c:1534) ==2106591== by 0x6296D0: machines__deliver_event (session.c:1573) ==2106591== by 0x629C39: perf_session__deliver_event (session.c:1655) ==2106591== by 0x625830: ordered_events__deliver_event (session.c:193) ==2106591== by 0x630B23: do_flush (ordered-events.c:245) ==2106591== by 0x630E7A: __ordered_events__flush (ordered-events.c:324) ==2106591== Address 0x7186fe0 is 0 bytes after a block of size 0 alloc'd ==2106591== at 0x484280F: malloc (vg_replace_malloc.c:442) ==2106591== by 0x7134AD: set_regs_in_dict (trace-event-python.c:780) ==2106591== by 0x713E58: get_perf_sample_dict (trace-event-python.c:940) ==2106591== by 0x716327: python_process_general_event (trace-event-python.c:1499) ==2106591== by 0x7164E1: python_process_event (trace-event-python.c:1531) ==2106591== by 0x44F9AF: process_sample_event (builtin-script.c:2549) ==2106591== by 0x6294DC: evlist__deliver_sample (session.c:1534) ==2106591== by 0x6296D0: machines__deliver_event (session.c:1573) ==2106591== by 0x629C39: perf_session__deliver_event (session.c:1655) ==2106591== by 0x625830: ordered_events__deliver_event (session.c:193) ==2106591== by 0x630B23: do_flush (ordered-events.c:245) ==2106591== by 0x630E7A: __ordered_events__flush (ordered-events.c:324) ==2106591== ==2106591== Invalid read of size 1 ==2106591== at 0x484B6C6: strlen (vg_replace_strmem.c:502) ==2106591== by 0x555D494: PyUnicode_FromString (unicodeobject.c:1899) ==2106591== by 0x7134F7: set_regs_in_dict (trace-event-python.c:786) ==2106591== by 0x713E58: get_perf_sample_dict (trace-event-python.c:940) ==2106591== by 0x716327: python_process_general_event (trace-event-python.c:1499) ==2106591== by 0x7164E1: python_process_event (trace-event-python.c:1531) ==2106591== by 0x44F9AF: process_sample_event (builtin-script.c:2549) ==2106591== by 0x6294DC: evlist__deliver_sample (session.c:1534) ==2106591== by 0x6296D0: machines__deliver_event (session.c:1573) ==2106591== by 0x629C39: perf_session__deliver_event (session.c:1655) ==2106591== by 0x625830: ordered_events__deliver_event (session.c:193) ==2106591== by 0x630B23: do_flush (ordered-events.c:245) ==2106591== Address 0x7186fe0 is 0 bytes after a block of size 0 alloc'd ==2106591== at 0x484280F: malloc (vg_replace_malloc.c:442) ==2106591== by 0x7134AD: set_regs_in_dict (trace-event-python.c:780) ==2106591== by 0x713E58: get_perf_sample_dict (trace-event-python.c:940) ==2106591== by 0x716327: python_process_general_event (trace-event-python.c:1499) ==2106591== by 0x7164E1: python_process_event (trace-event-python.c:1531) ==2106591== by 0x44F9AF: process_sample_event (builtin-script.c:2549) ==2106591== by 0x6294DC: evlist__deliver_sample (session.c:1534) ==2106591== by 0x6296D0: machines__deliver_event (session.c:1573) ==2106591== by 0x629C39: perf_session__deliver_event (session.c:1655) ==2106591== by 0x625830: ordered_events__deliver_event (session.c:193) ==2106591== by 0x630B23: do_flush (ordered-events.c:245) ==2106591== by 0x630E7A: __ordered_events__flush (ordered-events.c:324) ==2106591== ==2106591== Invalid write of size 1 ==2106591== at 0x713354: regs_map (trace-event-python.c:748) ==2106591== by 0x713539: set_regs_in_dict (trace-event-python.c:789) ==2106591== by 0x713E58: get_perf_sample_dict (trace-event-python.c:940) ==2106591== by 0x716327: python_process_general_event (trace-event-python.c:1499) ==2106591== by 0x7164E1: python_process_event (trace-event-python.c:1531) ==2106591== by 0x44F9AF: process_sample_event (builtin-script.c:2549) ==2106591== by 0x6294DC: evlist__deliver_sample (session.c:1534) ==2106591== by 0x6296D0: machines__deliver_event (session.c:1573) ==2106591== by 0x629C39: perf_session__deliver_event (session.c:1655) ==2106591== by 0x625830: ordered_events__deliver_event (session.c:193) ==2106591== by 0x630B23: do_flush (ordered-events.c:245) ==2106591== by 0x630E7A: __ordered_events__flush (ordered-events.c:324) ==2106591== Address 0x7186fe0 is 0 bytes after a block of size 0 alloc'd ==2106591== at 0x484280F: malloc (vg_replace_malloc.c:442) ==2106591== by 0x7134AD: set_regs_in_dict (trace-event-python.c:780) ==2106591== by 0x713E58: get_perf_sample_dict (trace-event-python.c:940) ==2106591== by 0x716327: python_process_general_event (trace-event-python.c:1499) ==2106591== by 0x7164E1: python_process_event (trace-event-python.c:1531) ==2106591== by 0x44F9AF: process_sample_event (builtin-script.c:2549) ==2106591== by 0x6294DC: evlist__deliver_sample (session.c:1534) ==2106591== by 0x6296D0: machines__deliver_event (session.c:1573) ==2106591== by 0x629C39: perf_session__deliver_event (session.c:1655) ==2106591== by 0x625830: ordered_events__deliver_event (session.c:193) ==2106591== by 0x630B23: do_flush (ordered-events.c:245) ==2106591== by 0x630E7A: __ordered_events__flush (ordered-events.c:324) ==2106591== ==2106591== Invalid read of size 1 ==2106591== at 0x484B6C6: strlen (vg_replace_strmem.c:502) ==2106591== by 0x555D494: PyUnicode_FromString (unicodeobject.c:1899) ==2106591== by 0x713545: set_regs_in_dict (trace-event-python.c:791) ==2106591== by 0x713E58: get_perf_sample_dict (trace-event-python.c:940) ==2106591== by 0x716327: python_process_general_event (trace-event-python.c:1499) ==2106591== by 0x7164E1: python_process_event (trace-event-python.c:1531) ==2106591== by 0x44F9AF: process_sample_event (builtin-script.c:2549) ==2106591== by 0x6294DC: evlist__deliver_sample (session.c:1534) ==2106591== by 0x6296D0: machines__deliver_event (session.c:1573) ==2106591== by 0x629C39: perf_session__deliver_event (session.c:1655) ==2106591== by 0x625830: ordered_events__deliver_event (session.c:193) ==2106591== by 0x630B23: do_flush (ordered-events.c:245) ==2106591== Address 0x7186fe0 is 0 bytes after a block of size 0 alloc'd ==2106591== at 0x484280F: malloc (vg_replace_malloc.c:442) ==2106591== by 0x7134AD: set_regs_in_dict (trace-event-python.c:780) ==2106591== by 0x713E58: get_perf_sample_dict (trace-event-python.c:940) ==2106591== by 0x716327: python_process_general_event (trace-event-python.c:1499) ==2106591== by 0x7164E1: python_process_event (trace-event-python.c:1531) ==2106591== by 0x44F9AF: process_sample_event (builtin-script.c:2549) ==2106591== by 0x6294DC: evlist__deliver_sample (session.c:1534) ==2106591== by 0x6296D0: machines__deliver_event (session.c:1573) ==2106591== by 0x629C39: perf_session__deliver_event (session.c:1655) ==2106591== by 0x625830: ordered_events__deliver_event (session.c:193) ==2106591== by 0x630B23: do_flush (ordered-events.c:245) ==2106591== by 0x630E7A: __ordered_events__flush (ordered-events.c:324) ==2106591== 73056 total, 29 ignored Signed-off-by: Andi Kleen Cc: Adrian Hunter Cc: Namhyung Kim Link: https://lore.kernel.org/r/20240905151058.2127122-2-ak@linux.intel.com Signed-off-by: Arnaldo Carvalho de Melo

perf script python: Add the 'ins_lat' field to event handler

2024-08-09T13:25:07+00:00

For example, when using the Alder Lake PMU memory load event, the instruction latency is stored in 'ins_lat', while the cache latency is stored in 'weight'. This patch reports the 'ins_lat' field for Python scripting. Committer testing: On a Rocket Lake Refresh Intel machine (14th gen): root@number:~# grep -m1 'model name' /proc/cpuinfo model name : Intel(R) Core(TM) i7-14700K root@number:~# perf mem record -a sleep 5 Memory events are enabled on a subset of CPUs: 16-27 [ perf record: Woken up 85 times to write data ] [ perf record: Captured and wrote 41.236 MB perf.data (191390 samples) ] root@number:~# perf evlist -v cpu_atom/mem-loads,ldlat=30/P: type: 10 (cpu_atom), size: 136, config: 0x5d0 (mem-loads), { sample_period, sample_freq }: 4000, sample_type: IP|TID|TIME|ADDR|CPU|PERIOD|IDENTIFIER|DATA_SRC|WEIGHT_STRUCT, read_format: ID|LOST, disabled: 1, inherit: 1, freq: 1, precise_ip: 3, sample_id_all: 1, { bp_addr, config1 }: 0x1f cpu_atom/mem-stores/P: type: 10 (cpu_atom), size: 136, config: 0x6d0 (mem-stores), { sample_period, sample_freq }: 4000, sample_type: IP|TID|TIME|ADDR|CPU|PERIOD|IDENTIFIER|DATA_SRC|WEIGHT_STRUCT, read_format: ID|LOST, disabled: 1, inherit: 1, freq: 1, precise_ip: 3, sample_id_all: 1 dummy:u: type: 1 (software), size: 136, config: 0x9 (PERF_COUNT_SW_DUMMY), { sample_period, sample_freq }: 1, sample_type: IP|TID|TIME|ADDR|CPU|IDENTIFIER|DATA_SRC|WEIGHT_STRUCT, read_format: ID|LOST, inherit: 1, exclude_kernel: 1, exclude_hv: 1, mmap: 1, comm: 1, task: 1, mmap_data: 1, sample_id_all: 1, exclude_guest: 1, mmap2: 1, comm_exec: 1, ksymbol: 1, bpf_event: 1 root@number:~# Now generate a python script to then dump the dictionary that now needs to have that 'ins_lat' field: root@number:~# perf script --gen python generated Python script: perf-script.py root@number:~# vim perf-script.py root@number:~# perf script -s perf-script.py | head -40 in trace_begin in trace_end root@number:~# vim perf-script.py Reviewed-by: Adrian Hunter Signed-off-by: Zixian Cai Cc: Alexander Shishkin Cc: Ben Gainey Cc: Ian Rogers Cc: Ingo Molnar Cc: Jiri Olsa Cc: Kan Liang Cc: Mark Rutland Cc: Namhyung Kim Cc: Paran Lee Cc: Peter Zijlstra Link: https://lore.kernel.org/r/20240809080137.3590148-1-fzczx123@gmail.com Signed-off-by: Arnaldo Carvalho de Melo

perf mem-info: Add reference count checking

2024-05-07T21:06:44+00:00

Add reference count checking and switch 'struct mem_info' usage to use accessor functions. Signed-off-by: Ian Rogers Cc: Adrian Hunter Cc: Alexander Shishkin Cc: Andi Kleen Cc: Athira Rajeev Cc: Ben Gainey Cc: Ingo Molnar Cc: James Clark Cc: Jiri Olsa Cc: K Prateek Nayak Cc: Kajol Jain Cc: Kan Liang Cc: Li Dong Cc: Mark Rutland Cc: Namhyung Kim Cc: Oliver Upton Cc: Paran Lee Cc: Peter Zijlstra Cc: Ravi Bangoria Cc: Sun Haiyong Cc: Tim Chen Cc: Yanteng Si Cc: Yicong Yang Link: https://lore.kernel.org/r/20240507183545.1236093-8-irogers@google.com Signed-off-by: Arnaldo Carvalho de Melo

perf mem-info: Move mem-info out of mem-events and symbol

2024-05-07T21:06:44+00:00

Move mem-info to its own header rather than having it split between mem-events and symbol. Signed-off-by: Ian Rogers Cc: Adrian Hunter Cc: Alexander Shishkin Cc: Andi Kleen Cc: Athira Rajeev Cc: Ben Gainey Cc: Ingo Molnar Cc: James Clark Cc: Jiri Olsa Cc: K Prateek Nayak Cc: Kajol Jain Cc: Kan Liang Cc: Li Dong Cc: Mark Rutland Cc: Namhyung Kim Cc: Oliver Upton Cc: Paran Lee Cc: Peter Zijlstra Cc: Ravi Bangoria Cc: Sun Haiyong Cc: Tim Chen Cc: Yanteng Si Cc: Yicong Yang Link: https://lore.kernel.org/r/20240507183545.1236093-7-irogers@google.com Signed-off-by: Arnaldo Carvalho de Melo

perf dso: Add reference count checking and accessor functions

2024-05-06T18:28:49+00:00

Add reference count checking to struct dso, this can help with implementing correct reference counting discipline. To avoid RC_CHK_ACCESS everywhere, add accessor functions for the variables in struct dso. The majority of the change is mechanical in nature and not easy to split up. Committer testing: 'perf test' up to this patch shows no regressions. But: util/symbol.c: In function ‘dso__load_bfd_symbols’: util/symbol.c:1683:9: error: too few arguments to function ‘dso__set_adjust_symbols’ 1683 | dso__set_adjust_symbols(dso); | ^~~~~~~~~~~~~~~~~~~~~~~ In file included from util/symbol.c:21: util/dso.h:268:20: note: declared here 268 | static inline void dso__set_adjust_symbols(struct dso *dso, bool val) | ^~~~~~~~~~~~~~~~~~~~~~~ make[6]: *** [/home/acme/git/perf-tools-next/tools/build/Makefile.build:106: /tmp/tmp.ZWHbQftdN6/util/symbol.o] Error 1 MKDIR /tmp/tmp.ZWHbQftdN6/tests/workloads/ make[6]: *** Waiting for unfinished jobs.... This was updated: - symbols__fixup_end(&dso->symbols, false); - symbols__fixup_duplicate(&dso->symbols); - dso->adjust_symbols = 1; + symbols__fixup_end(dso__symbols(dso), false); + symbols__fixup_duplicate(dso__symbols(dso)); + dso__set_adjust_symbols(dso); But not build tested with BUILD_NONDISTRO and libbfd devel files installed (binutils-devel on fedora). Add the missing argument: symbols__fixup_end(dso__symbols(dso), false); symbols__fixup_duplicate(dso__symbols(dso)); - dso__set_adjust_symbols(dso); + dso__set_adjust_symbols(dso, true); Signed-off-by: Ian Rogers Tested-by: Arnaldo Carvalho de Melo Cc: Adrian Hunter Cc: Ahelenia Ziemiańska Cc: Alexander Shishkin Cc: Andi Kleen Cc: Athira Rajeev Cc: Ben Gainey Cc: Changbin Du Cc: Chengen Du Cc: Colin Ian King Cc: Dima Kogan Cc: Ilkka Koskinen Cc: Ingo Molnar Cc: James Clark Cc: Jiri Olsa Cc: K Prateek Nayak Cc: Kan Liang Cc: Leo Yan Cc: Li Dong Cc: Mark Rutland Cc: Masami Hiramatsu Cc: Namhyung Kim Cc: Paran Lee Cc: Peter Zijlstra Cc: Song Liu Cc: Sun Haiyong Cc: Thomas Richter Cc: Tiezhu Yang Cc: Yanteng Si Cc: zhaimingbing Link: https://lore.kernel.org/r/20240504213803.218974-6-irogers@google.com Signed-off-by: Arnaldo Carvalho de Melo

perf cpumap: Use perf_cpu_map__for_each_cpu when possible

2024-03-21T13:41:28+00:00

Rather than manually iterating the CPU map, use perf_cpu_map__for_each_cpu(). When possible tidy local variables. Reviewed-by: James Clark Signed-off-by: Ian Rogers Acked-by: Namhyung Kim Cc: Adrian Hunter Cc: Alexander Shishkin Cc: Alexandre Ghiti Cc: Andrew Jones Cc: André Almeida Cc: Athira Rajeev Cc: Atish Patra Cc: Changbin Du Cc: Darren Hart Cc: Davidlohr Bueso Cc: Huacai Chen Cc: Ingo Molnar Cc: Jiri Olsa Cc: John Garry Cc: K Prateek Nayak Cc: Kajol Jain Cc: Kan Liang Cc: Leo Yan Cc: Mark Rutland Cc: Mike Leach Cc: Nick Desaulniers Cc: Paolo Bonzini Cc: Paran Lee Cc: Peter Zijlstra Cc: Ravi Bangoria Cc: Sandipan Das Cc: Sean Christopherson Cc: Steinar H. Gunderson Cc: Suzuki Poulouse Cc: Thomas Gleixner Cc: Will Deacon Cc: Yang Jihong Cc: Yang Li Cc: Yanteng Si Link: https://lore.kernel.org/r/20240202234057.2085863-9-irogers@google.com Signed-off-by: Arnaldo Carvalho de Melo

tools: perf: Expose sample ID / stream ID to python scripts

2024-02-03T02:05:40+00:00

perf script exposes the evsel_name to python scripts as part of the data passed to the sample or tracepoint handler function, and it passes the id and stream_id to the throttled/unthrottled handler functions. This makes matching throttle events and samples difficult. To make this possible, this change exposes the sample id and stream_id values to the script. Signed-off-by: Ben Gainey Reviewed-by: Adrian Hunter Cc: will@kernel.org Cc: linux-arm-kernel@lists.infradead.org Signed-off-by: Namhyung Kim Link: https://lore.kernel.org/r/20240123103137.1890779-2-ben.gainey@arm.com

perf script python: Fail check on dynamic allocation

2023-11-27T13:21:27+00:00

Add PyList_New() Fail check in get_field_numeric_entry() function and dynamic allocation checking for set_regs_in_dict(), python_start_script(). Reviewed-by: Adrian Hunter Reviewed-by: MichelleJin Signed-off-by: Paran Lee Cc: Alexander Shishkin Cc: Austin Kim Cc: Honggyu Kim Cc: Ian Rogers Cc: Ingo Molnar Cc: Jiri Olsa Cc: Li Dong Cc: Mark Rutland Cc: Namhyung Kim Cc: Peter Zijlstra Cc: Sean Christopherson Link: https://lore.kernel.org/r/20231120223218.9036-1-p4ranlee@gmail.com Signed-off-by: Arnaldo Carvalho de Melo

perf script: Initialize buffer for regs_map()

2023-06-23T05:10:51+00:00

The buffer is used to save register mapping in a sample. Normally perf samples don't have any register so the string should be empty. But it missed to initialize the buffer when the size is 0. And it's passed to PyUnicode_FromString() with a garbage data. So it returns NULL due to invalid input (instead of an empty unicode string object) which causes a segfault like below: Thread 2.1 "perf" received signal SIGSEGV, Segmentation fault. [Switching to Thread 0x7ffff7c83780 (LWP 193775)] 0x00007ffff6dbca2e in PyDict_SetItem () from /lib/x86_64-linux-gnu/libpython3.11.so.1.0 (gdb) bt #0 0x00007ffff6dbca2e in PyDict_SetItem () from /lib/x86_64-linux-gnu/libpython3.11.so.1.0 #1 0x00007ffff6dbf848 in PyDict_SetItemString () from /lib/x86_64-linux-gnu/libpython3.11.so.1.0 #2 0x000055555575824d in pydict_set_item_string_decref (val=0x0, key=0x5555557f96e3 "iregs", dict=0x7ffff5f7f780) at util/scripting-engines/trace-event-python.c:145 #3 set_regs_in_dict (evsel=0x555555efc370, sample=0x7fffffffb870, dict=0x7ffff5f7f780) at util/scripting-engines/trace-event-python.c:776 #4 get_perf_sample_dict (sample=sample@entry=0x7fffffffb870, evsel=evsel@entry=0x555555efc370, al=al@entry=0x7fffffffb2e0, addr_al=addr_al@entry=0x0, callchain=callchain@entry=0x7ffff63ef440) at util/scripting-engines/trace-event-python.c:923 #5 0x0000555555758ec1 in python_process_tracepoint (sample=0x7fffffffb870, evsel=0x555555efc370, al=0x7fffffffb2e0, addr_al=0x0) at util/scripting-engines/trace-event-python.c:1044 #6 0x00005555555c5db8 in process_sample_event (tool=, event=, sample=, evsel=0x555555efc370, machine=0x555555ef4d68) at builtin-script.c:2421 #7 0x00005555556b7793 in perf_session__deliver_event (session=0x555555ef4b60, event=0x7ffff62ff7d0, tool=0x7fffffffc150, file_offset=30672, file_path=0x555555efb8a0 "perf.data") at util/session.c:1639 #8 0x00005555556bc864 in do_flush (show_progress=true, oe=0x555555efb700) at util/ordered-events.c:245 #9 __ordered_events__flush (oe=oe@entry=0x555555efb700, how=how@entry=OE_FLUSH__FINAL, timestamp=timestamp@entry=0) at util/ordered-events.c:324 #10 0x00005555556bd06e in ordered_events__flush (oe=oe@entry=0x555555efb700, how=how@entry=OE_FLUSH__FINAL) at util/ordered-events.c:342 #11 0x00005555556b9d63 in __perf_session__process_events (session=0x555555ef4b60) at util/session.c:2465 #12 perf_session__process_events (session=0x555555ef4b60) at util/session.c:2627 #13 0x00005555555cb1d0 in __cmd_script (script=0x7fffffffc150) at builtin-script.c:2839 #14 cmd_script (argc=, argv=) at builtin-script.c:4365 #15 0x0000555555650811 in run_builtin (p=p@entry=0x555555ed8948 , argc=argc@entry=4, argv=argv@entry=0x7fffffffe240) at perf.c:323 #16 0x0000555555597eb3 in handle_internal_command (argv=0x7fffffffe240, argc=4) at perf.c:377 #17 run_argv (argv=, argcp=) at perf.c:421 #18 main (argc=4, argv=0x7fffffffe240) at perf.c:537 Fixes: 51cfe7a3e87e ("perf python: Avoid 2 leak sanitizer issues") Cc: Peter Zijlstra Cc: Adrian Hunter Cc: Arnaldo Carvalho de Melo Cc: Jiri Olsa Cc: Ingo Molnar Acked-by: Ian Rogers Signed-off-by: Namhyung Kim

tools: Fix incorrect calculation of object size by sizeof

2023-06-20T23:58:05+00:00

What we need to calculate is the size of the object, not the size of the pointer. Fixed: 51cfe7a3e87e ("perf python: Avoid 2 leak sanitizer issues") Signed-off-by: Li Dong Acked-by: Namhyung Kim Cc: Mark Rutland Cc: Ian Rogers Cc: Sean Christopherson Cc: Peter Zijlstra Cc: Adrian Hunter Cc: Arnaldo Carvalho de Melo Cc: Jiri Olsa Cc: Alexander Shishkin Cc: Ingo Molnar Cc: opensource.kernel@vivo.com Link: https://lore.kernel.org/r/20230619082036.410-1-lidong@vivo.com Signed-off-by: Namhyung Kim