diff options
Diffstat (limited to 'tools/perf/Documentation')
22 files changed, 830 insertions, 296 deletions
diff --git a/tools/perf/Documentation/callchain-overhead-calculation.txt b/tools/perf/Documentation/callchain-overhead-calculation.txt index 1a757927195e..e0202bf5bd1a 100644 --- a/tools/perf/Documentation/callchain-overhead-calculation.txt +++ b/tools/perf/Documentation/callchain-overhead-calculation.txt @@ -1,7 +1,8 @@ Overhead calculation -------------------- -The overhead can be shown in two columns as 'Children' and 'Self' when -perf collects callchains. The 'self' overhead is simply calculated by +The CPU overhead can be shown in two columns as 'Children' and 'Self' +when perf collects callchains (and corresponding 'Wall' columns for +wall-clock overhead). The 'self' overhead is simply calculated by adding all period values of the entry - usually a function (symbol). This is the value that perf shows traditionally and sum of all the 'self' overhead values should be 100%. diff --git a/tools/perf/Documentation/cpu-and-latency-overheads.txt b/tools/perf/Documentation/cpu-and-latency-overheads.txt new file mode 100644 index 000000000000..3b6d63705465 --- /dev/null +++ b/tools/perf/Documentation/cpu-and-latency-overheads.txt @@ -0,0 +1,85 @@ +CPU and latency overheads +------------------------- +There are two notions of time: wall-clock time and CPU time. +For a single-threaded program, or a program running on a single-core machine, +these notions are the same. However, for a multi-threaded/multi-process program +running on a multi-core machine, these notions are significantly different. +Each second of wall-clock time we have number-of-cores seconds of CPU time. +Perf can measure overhead for both of these times (shown in 'overhead' and +'latency' columns for CPU and wall-clock time correspondingly). + +Optimizing CPU overhead is useful to improve 'throughput', while optimizing +latency overhead is useful to improve 'latency'. It's important to understand +which one is useful in a concrete situation at hand. For example, the former +may be useful to improve max throughput of a CI build server that runs on 100% +CPU utilization, while the latter may be useful to improve user-perceived +latency of a single interactive program build. +These overheads may be significantly different in some cases. For example, +consider a program that executes function 'foo' for 9 seconds with 1 thread, +and then executes function 'bar' for 1 second with 128 threads (consumes +128 seconds of CPU time). The CPU overhead is: 'foo' - 6.6%, 'bar' - 93.4%. +While the latency overhead is: 'foo' - 90%, 'bar' - 10%. If we try to optimize +running time of the program looking at the (wrong in this case) CPU overhead, +we would concentrate on the function 'bar', but it can yield only 10% running +time improvement at best. + +By default, perf shows only CPU overhead. To show latency overhead, use +'perf record --latency' and 'perf report': + +----------------------------------- +Overhead Latency Command + 93.88% 25.79% cc1 + 1.90% 39.87% gzip + 0.99% 10.16% dpkg-deb + 0.57% 1.00% as + 0.40% 0.46% sh +----------------------------------- + +To sort by latency overhead, use 'perf report --latency': + +----------------------------------- +Latency Overhead Command + 39.87% 1.90% gzip + 25.79% 93.88% cc1 + 10.16% 0.99% dpkg-deb + 4.17% 0.29% git + 2.81% 0.11% objtool +----------------------------------- + +To get insight into the difference between the overheads, you may check +parallelization histogram with '--sort=latency,parallelism,comm,symbol --hierarchy' +flags. It shows fraction of (wall-clock) time the workload utilizes different +numbers of cores ('Parallelism' column). For example, in the following case +the workload utilizes only 1 core most of the time, but also has some +highly-parallel phases, which explains significant difference between +CPU and wall-clock overheads: + +----------------------------------- + Latency Overhead Parallelism / Command / Symbol ++ 56.98% 2.29% 1 ++ 16.94% 1.36% 2 ++ 4.00% 20.13% 125 ++ 3.66% 18.25% 124 ++ 3.48% 17.66% 126 ++ 3.26% 0.39% 3 ++ 2.61% 12.93% 123 +----------------------------------- + +By expanding corresponding lines, you may see what commands/functions run +at the given parallelism level: + +----------------------------------- + Latency Overhead Parallelism / Command / Symbol +- 56.98% 2.29% 1 + 32.80% 1.32% gzip + 4.46% 0.18% cc1 + 2.81% 0.11% objtool + 2.43% 0.10% dpkg-source + 2.22% 0.09% ld + 2.10% 0.08% dpkg-genchanges +----------------------------------- + +To see the normal function-level profile for particular parallelism levels +(number of threads actively running on CPUs), you may use '--parallelism' +filter. For example, to see the profile only for low parallelism phases +of a workload use '--latency --parallelism=1-2' flags. diff --git a/tools/perf/Documentation/intel-hybrid.txt b/tools/perf/Documentation/intel-hybrid.txt index e7a776ad25d7..0379903673a4 100644 --- a/tools/perf/Documentation/intel-hybrid.txt +++ b/tools/perf/Documentation/intel-hybrid.txt @@ -8,15 +8,15 @@ Part of events are available on core cpu, part of events are available on atom cpu and even part of events are available on both. Kernel exports two new cpu pmus via sysfs: -/sys/devices/cpu_core -/sys/devices/cpu_atom +/sys/bus/event_source/devices/cpu_core +/sys/bus/event_source/devices/cpu_atom The 'cpus' files are created under the directories. For example, -cat /sys/devices/cpu_core/cpus +cat /sys/bus/event_source/devices/cpu_core/cpus 0-15 -cat /sys/devices/cpu_atom/cpus +cat /sys/bus/event_source/devices/cpu_atom/cpus 16-23 It indicates cpu0-cpu15 are core cpus and cpu16-cpu23 are atom cpus. @@ -60,8 +60,8 @@ can't carry pmu information. So now this type is extended to be PMU aware type. The PMU type ID is stored at attr.config[63:32]. PMU type ID is retrieved from sysfs. -/sys/devices/cpu_atom/type -/sys/devices/cpu_core/type +/sys/bus/event_source/devices/cpu_atom/type +/sys/bus/event_source/devices/cpu_core/type The new attr.config layout for PERF_TYPE_HARDWARE: diff --git a/tools/perf/Documentation/perf-amd-ibs.txt b/tools/perf/Documentation/perf-amd-ibs.txt index 2fd31d9d7b71..548549935760 100644 --- a/tools/perf/Documentation/perf-amd-ibs.txt +++ b/tools/perf/Documentation/perf-amd-ibs.txt @@ -85,6 +85,15 @@ System-wide profile, uOps event, sampling period: 100000, L3MissOnly (Zen4 onwar # perf record -e ibs_op/cnt_ctl=1,l3missonly=1/ -c 100000 -a +System-wide profile, cycles event, sampling period: 100000, LdLat filtering (Zen5 +onward) + + # perf record -e ibs_op/ldlat=128/ -c 100000 -a + + Supported load latency threshold values are 128 to 2048 (both inclusive). + Latency value which is a multiple of 128 incurs a little less profiling + overhead compared to other values. + Per process(upstream v6.2 onward), uOps event, sampling period: 100000 # perf record -e ibs_op/cnt_ctl=1/ -c 100000 -p 1234 @@ -162,23 +171,48 @@ Below is a simple example of the perf mem tool. # perf mem report A normal perf mem report output will provide detailed memory access profile. -However, it can also be aggregated based on output fields. For example: - - # perf mem report -F mem,sample,snoop - Samples: 3M of event 'ibs_op//', Event count (approx.): 23524876 - Memory access Samples Snoop - N/A 1903343 N/A - L1 hit 1056754 N/A - L2 hit 75231 N/A - L3 hit 9496 HitM - L3 hit 2270 N/A - RAM hit 8710 N/A - Remote node, same socket RAM hit 3241 N/A - Remote core, same node Any cache hit 1572 HitM - Remote core, same node Any cache hit 514 N/A - Remote node, same socket Any cache hit 1216 HitM - Remote node, same socket Any cache hit 350 N/A - Uncached hit 18 N/A +New output fields will show related access info together. For example: + + # perf mem report -F overhead,cache,snoop,comm + ... + # Samples: 92K of event 'ibs_op//' + # Total weight : 531104 + # + # ---------- Cache ----------- --- Snoop ---- + # Overhead L1 L2 L1-buf Other HitM Other Command + # ........ ............................ .............. .......... + # + 76.07% 5.8% 35.7% 0.0% 34.6% 23.3% 52.8% cc1 + 5.79% 0.2% 0.0% 0.0% 5.6% 0.1% 5.7% make + 5.78% 0.1% 4.4% 0.0% 1.2% 0.5% 5.3% gcc + 5.33% 0.3% 3.9% 0.0% 1.1% 0.2% 5.2% as + 5.00% 0.1% 3.8% 0.0% 1.0% 0.3% 4.7% sh + 1.56% 0.1% 0.1% 0.0% 1.4% 0.6% 0.9% ld + 0.28% 0.1% 0.0% 0.0% 0.2% 0.1% 0.2% pkg-config + 0.09% 0.0% 0.0% 0.0% 0.1% 0.0% 0.1% git + 0.03% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% rm + ... + +Also, it can be aggregated based on various memory access info using the +sort keys. For example: + + # perf mem report -s mem,snoop + ... + # Samples: 92K of event 'ibs_op//' + # Total weight : 531104 + # Sort order : mem,snoop + # + # Overhead Samples Memory access Snoop + # ........ ............ ....................................... ............ + # + 47.99% 1509 L2 hit N/A + 25.08% 338 core, same node Any cache hit HitM + 10.24% 54374 N/A N/A + 6.77% 35938 L1 hit N/A + 6.39% 101 core, same node Any cache hit N/A + 3.50% 69 RAM hit N/A + 0.03% 158 LFB/MAB hit N/A + 0.00% 2 Uncached hit N/A Please refer to their man page for more detail. diff --git a/tools/perf/Documentation/perf-annotate.txt b/tools/perf/Documentation/perf-annotate.txt index 156c5f37b051..46090c5b42b4 100644 --- a/tools/perf/Documentation/perf-annotate.txt +++ b/tools/perf/Documentation/perf-annotate.txt @@ -168,6 +168,10 @@ include::itrace.txt[] --skip-empty:: Do not display empty (or dummy) events. +--code-with-type:: + Show data type info in code annotation (for memory instructions only). + Currently it only works with --stdio option. + SEE ALSO -------- diff --git a/tools/perf/Documentation/perf-arm-spe.txt b/tools/perf/Documentation/perf-arm-spe.txt index de2b0b479249..37afade4f1b2 100644 --- a/tools/perf/Documentation/perf-arm-spe.txt +++ b/tools/perf/Documentation/perf-arm-spe.txt @@ -150,6 +150,7 @@ arm_spe/load_filter=1,min_latency=10/' pct_enable=1 - collect physical timestamp instead of virtual timestamp (PMSCR.PCT) - requires privilege store_filter=1 - collect stores only (PMSFCR.ST) ts_enable=1 - enable timestamping with value of generic timer (PMSCR.TS) + discard=1 - enable SPE PMU events but don't collect sample data - see 'Discard mode' (PMBLIMITR.FM = DISCARD) +++*+++ Latency is the total latency from the point at which sampling started on that instruction, rather than only the execution latency. @@ -220,6 +221,31 @@ Common errors Increase sampling interval (see above) +PMU events +~~~~~~~~~~ + +SPE has events that can be counted on core PMUs. These are prefixed with +SAMPLE_, for example SAMPLE_POP, SAMPLE_FEED, SAMPLE_COLLISION and +SAMPLE_FEED_BR. + +These events will only count when an SPE event is running on the same core that +the PMU event is opened on, otherwise they read as 0. There are various ways to +ensure that the PMU event and SPE event are scheduled together depending on the +way the event is opened. For example opening both events as per-process events +on the same process, although it's not guaranteed that the PMU event is enabled +first when context switching. For that reason it may be better to open the PMU +event as a systemwide event and then open SPE on the process of interest. + +Discard mode +~~~~~~~~~~~~ + +SPE related (SAMPLE_* etc) core PMU events can be used without the overhead of +collecting sample data if discard mode is supported (optional from Armv8.6). +First run a system wide SPE session (or on the core of interest) using options +to minimize output. Then run perf stat: + + perf record -e arm_spe/discard/ -a -N -B --no-bpf-event -o - > /dev/null & + perf stat -e SAMPLE_FEED_LD SEE ALSO -------- diff --git a/tools/perf/Documentation/perf-c2c.txt b/tools/perf/Documentation/perf-c2c.txt index 856f0dfb8e5a..f4af2dd6ab31 100644 --- a/tools/perf/Documentation/perf-c2c.txt +++ b/tools/perf/Documentation/perf-c2c.txt @@ -54,8 +54,15 @@ RECORD OPTIONS -l:: --ldlat:: - Configure mem-loads latency. Supported on Intel and Arm64 processors - only. Ignored on other archs. + Configure mem-loads latency. Supported on Intel, Arm64 and some AMD + processors. Ignored on other archs. + + On supported AMD processors: + - /sys/bus/event_source/devices/ibs_op/caps/ldlat file contains '1'. + - Supported latency values are 128 to 2048 (both inclusive). + - Latency value which is a multiple of 128 incurs a little less profiling + overhead compared to other values. + - Load latency filtering is disabled by default. -k:: --all-kernel:: diff --git a/tools/perf/Documentation/perf-check.txt b/tools/perf/Documentation/perf-check.txt index 31741499e786..a764a4629220 100644 --- a/tools/perf/Documentation/perf-check.txt +++ b/tools/perf/Documentation/perf-check.txt @@ -51,7 +51,6 @@ feature:: dwarf_getlocations / HAVE_LIBDW_SUPPORT dwarf-unwind / HAVE_DWARF_UNWIND_SUPPORT auxtrace / HAVE_AUXTRACE_SUPPORT - libaudit / HAVE_LIBAUDIT_SUPPORT libbfd / HAVE_LIBBFD_SUPPORT libcapstone / HAVE_LIBCAPSTONE_SUPPORT libcrypto / HAVE_LIBCRYPTO_SUPPORT @@ -67,7 +66,6 @@ feature:: libunwind / HAVE_LIBUNWIND_SUPPORT lzma / HAVE_LZMA_SUPPORT numa_num_possible_cpus / HAVE_LIBNUMA_SUPPORT - syscall_table / HAVE_SYSCALL_TABLE_SUPPORT zlib / HAVE_ZLIB_SUPPORT zstd / HAVE_ZSTD_SUPPORT diff --git a/tools/perf/Documentation/perf-config.txt b/tools/perf/Documentation/perf-config.txt index 1f668d4724e3..c6f335659667 100644 --- a/tools/perf/Documentation/perf-config.txt +++ b/tools/perf/Documentation/perf-config.txt @@ -40,7 +40,7 @@ The '$HOME/.perfconfig' file is used to store a per-user configuration. The file '$(sysconfdir)/perfconfig' can be used to store a system-wide default configuration. -One an disable reading config files by setting the PERF_CONFIG environment +One can disable reading config files by setting the PERF_CONFIG environment variable to /dev/null, or provide an alternate config file by setting that variable. @@ -708,6 +708,10 @@ intel-pt.*:: the maximum is exceeded there will be a "Never-ending loop" error. The default is 100000. + intel-pt.all-switch-events:: + If the user has permission to do so, always record all context + switch events on all CPUs. + auxtrace.*:: auxtrace.dumpdir:: diff --git a/tools/perf/Documentation/perf-ftrace.txt b/tools/perf/Documentation/perf-ftrace.txt index eaec8253be68..b77f58c4d2fd 100644 --- a/tools/perf/Documentation/perf-ftrace.txt +++ b/tools/perf/Documentation/perf-ftrace.txt @@ -148,6 +148,17 @@ OPTIONS for 'perf ftrace latency' --use-nsec:: Use nano-second instead of micro-second as a base unit of the histogram. +--bucket-range=:: + Bucket range in ms or ns (according to -n/--use-nsec), default is log2() mode. + +--min-latency=:: + Minimum latency for the start of the first bucket, in ms or ns (according to + -n/--use-nsec). + +--max-latency=:: + Maximum latency for the start of the last bucket, in ms or ns (according to + -n/--use-nsec). The setting is ignored if the value results in more than + 22 buckets. OPTIONS for 'perf ftrace profile' --------------------------------- @@ -190,6 +201,14 @@ OPTIONS for 'perf ftrace profile' Sort the result by the given field. Available values are: total, avg, max, count, name. Default is 'total'. +--graph-opts:: + List of options allowed to set: + + - nosleep-time - Measure on-CPU time only for function_graph tracer. + - noirqs - Ignore functions that happen inside interrupt. + - thresh=<n> - Setup trace duration threshold in microseconds. + - depth=<n> - Set max depth for function graph tracer to follow. + SEE ALSO -------- diff --git a/tools/perf/Documentation/perf-intel-pt.txt b/tools/perf/Documentation/perf-intel-pt.txt index 59ab1ff9d75f..cc0f37f0fa5a 100644 --- a/tools/perf/Documentation/perf-intel-pt.txt +++ b/tools/perf/Documentation/perf-intel-pt.txt @@ -151,7 +151,7 @@ displayed as follows: There are two ways that instructions-per-cycle (IPC) can be calculated depending on the recording. -If the 'cyc' config term (see config terms section below) was used, then IPC +If the 'cyc' config term (see <<_config_terms,config terms>> section below) was used, then IPC and cycle events are calculated using the cycle count from CYC packets, otherwise MTC packets are used - refer to the 'mtc' config term. When MTC is used, however, the values are less accurate because the timing is less accurate. @@ -239,7 +239,7 @@ which is the same as -e intel_pt/tsc=1,noretcomp=0/ -Note there are now new config terms - see section 'config terms' further below. +Note there are other config terms - see section <<_config_terms,config terms>> further below. The config terms are listed in /sys/devices/intel_pt/format. They are bit fields within the config member of the struct perf_event_attr which is @@ -311,218 +311,271 @@ perf_event_attr is displayed if the -vv option is used e.g. config terms ~~~~~~~~~~~~ -The June 2015 version of Intel 64 and IA-32 Architectures Software Developer -Manuals, Chapter 36 Intel Processor Trace, defined new Intel PT features. -Some of the features are reflect in new config terms. All the config terms are -described below. - -tsc Always supported. Produces TSC timestamp packets to provide - timing information. In some cases it is possible to decode - without timing information, for example a per-thread context - that does not overlap executable memory maps. - - The default config selects tsc (i.e. tsc=1). - -noretcomp Always supported. Disables "return compression" so a TIP packet - is produced when a function returns. Causes more packets to be - produced but might make decoding more reliable. - - The default config does not select noretcomp (i.e. noretcomp=0). - -psb_period Allows the frequency of PSB packets to be specified. - - The PSB packet is a synchronization packet that provides a - starting point for decoding or recovery from errors. - - Support for psb_period is indicated by: - - /sys/bus/event_source/devices/intel_pt/caps/psb_cyc - - which contains "1" if the feature is supported and "0" - otherwise. - - Valid values are given by: - - /sys/bus/event_source/devices/intel_pt/caps/psb_periods - - which contains a hexadecimal value, the bits of which represent - valid values e.g. bit 2 set means value 2 is valid. - - The psb_period value is converted to the approximate number of - trace bytes between PSB packets as: - - 2 ^ (value + 11) - - e.g. value 3 means 16KiB bytes between PSBs - - If an invalid value is entered, the error message - will give a list of valid values e.g. - - $ perf record -e intel_pt/psb_period=15/u uname - Invalid psb_period for intel_pt. Valid values are: 0-5 - - If MTC packets are selected, the default config selects a value - of 3 (i.e. psb_period=3) or the nearest lower value that is - supported (0 is always supported). Otherwise the default is 0. - - If decoding is expected to be reliable and the buffer is large - then a large PSB period can be used. - - Because a TSC packet is produced with PSB, the PSB period can - also affect the granularity to timing information in the absence - of MTC or CYC. - -mtc Produces MTC timing packets. - - MTC packets provide finer grain timestamp information than TSC - packets. MTC packets record time using the hardware crystal - clock (CTC) which is related to TSC packets using a TMA packet. - - Support for this feature is indicated by: - - /sys/bus/event_source/devices/intel_pt/caps/mtc - - which contains "1" if the feature is supported and - "0" otherwise. - - The frequency of MTC packets can also be specified - see - mtc_period below. - -mtc_period Specifies how frequently MTC packets are produced - see mtc - above for how to determine if MTC packets are supported. - - Valid values are given by: - - /sys/bus/event_source/devices/intel_pt/caps/mtc_periods - - which contains a hexadecimal value, the bits of which represent - valid values e.g. bit 2 set means value 2 is valid. - - The mtc_period value is converted to the MTC frequency as: - - CTC-frequency / (2 ^ value) - - e.g. value 3 means one eighth of CTC-frequency - - Where CTC is the hardware crystal clock, the frequency of which - can be related to TSC via values provided in cpuid leaf 0x15. - - If an invalid value is entered, the error message - will give a list of valid values e.g. - - $ perf record -e intel_pt/mtc_period=15/u uname - Invalid mtc_period for intel_pt. Valid values are: 0,3,6,9 - - The default value is 3 or the nearest lower value - that is supported (0 is always supported). - -cyc Produces CYC timing packets. - - CYC packets provide even finer grain timestamp information than - MTC and TSC packets. A CYC packet contains the number of CPU - cycles since the last CYC packet. Unlike MTC and TSC packets, - CYC packets are only sent when another packet is also sent. - - Support for this feature is indicated by: - - /sys/bus/event_source/devices/intel_pt/caps/psb_cyc - - which contains "1" if the feature is supported and - "0" otherwise. - - The number of CYC packets produced can be reduced by specifying - a threshold - see cyc_thresh below. - -cyc_thresh Specifies how frequently CYC packets are produced - see cyc - above for how to determine if CYC packets are supported. - - Valid cyc_thresh values are given by: - - /sys/bus/event_source/devices/intel_pt/caps/cycle_thresholds - - which contains a hexadecimal value, the bits of which represent - valid values e.g. bit 2 set means value 2 is valid. - - The cyc_thresh value represents the minimum number of CPU cycles - that must have passed before a CYC packet can be sent. The - number of CPU cycles is: - - 2 ^ (value - 1) - - e.g. value 4 means 8 CPU cycles must pass before a CYC packet - can be sent. Note a CYC packet is still only sent when another - packet is sent, not at, e.g. every 8 CPU cycles. - - If an invalid value is entered, the error message - will give a list of valid values e.g. - - $ perf record -e intel_pt/cyc,cyc_thresh=15/u uname - Invalid cyc_thresh for intel_pt. Valid values are: 0-12 - - CYC packets are not requested by default. - -pt Specifies pass-through which enables the 'branch' config term. - - The default config selects 'pt' if it is available, so a user will - never need to specify this term. - -branch Enable branch tracing. Branch tracing is enabled by default so to - disable branch tracing use 'branch=0'. - - The default config selects 'branch' if it is available. - -ptw Enable PTWRITE packets which are produced when a ptwrite instruction - is executed. - - Support for this feature is indicated by: - - /sys/bus/event_source/devices/intel_pt/caps/ptwrite - - which contains "1" if the feature is supported and - "0" otherwise. - - As an alternative, refer to "Emulated PTWRITE" further below. - -fup_on_ptw Enable a FUP packet to follow the PTWRITE packet. The FUP packet - provides the address of the ptwrite instruction. In the absence of - fup_on_ptw, the decoder will use the address of the previous branch - if branch tracing is enabled, otherwise the address will be zero. - Note that fup_on_ptw will work even when branch tracing is disabled. - -pwr_evt Enable power events. The power events provide information about - changes to the CPU C-state. - - Support for this feature is indicated by: - - /sys/bus/event_source/devices/intel_pt/caps/power_event_trace - - which contains "1" if the feature is supported and - "0" otherwise. - -event Enable Event Trace. The events provide information about asynchronous - events. - - Support for this feature is indicated by: - - /sys/bus/event_source/devices/intel_pt/caps/event_trace - - which contains "1" if the feature is supported and - "0" otherwise. - -notnt Disable TNT packets. Without TNT packets, it is not possible to walk - executable code to reconstruct control flow, however FUP, TIP, TIP.PGE - and TIP.PGD packets still indicate asynchronous control flow, and (if - return compression is disabled - see noretcomp) return statements. - The advantage of eliminating TNT packets is reducing the size of the - trace and corresponding tracing overhead. - - Support for this feature is indicated by: - - /sys/bus/event_source/devices/intel_pt/caps/tnt_disable - - which contains "1" if the feature is supported and - "0" otherwise. - +Config terms are parameters specified with the -e intel_pt// event option, +for example: + + -e intel_pt/cyc/ + +which selects cycle accurate mode. Each config term can have a value which +defaults to 1, so the above is the same as: + + -e intel_pt/cyc=1/ + +Some terms are set by default, so must be set to 0 to turn them off. For +example, to turn off branch tracing: + + -e intel_pt/branch=0/ + +Multiple config terms are separated by commas, for example: + + -e intel_pt/cyc,mtc_period=9/ + +There are also common config terms, see linkperf:perf-record[1] documentation. + +Intel PT config terms are described below. + +*tsc*:: +Always supported. Produces TSC timestamp packets to provide +timing information. In some cases it is possible to decode +without timing information, for example a per-thread context +that does not overlap executable memory maps. ++ +The default config selects tsc (i.e. tsc=1). + +*noretcomp*:: +Always supported. Disables "return compression" so a TIP packet +is produced when a function returns. Causes more packets to be +produced but might make decoding more reliable. ++ +The default config does not select noretcomp (i.e. noretcomp=0). + +*psb_period*:: +Allows the frequency of PSB packets to be specified. ++ +The PSB packet is a synchronization packet that provides a +starting point for decoding or recovery from errors. ++ +Support for psb_period is indicated by: ++ + /sys/bus/event_source/devices/intel_pt/caps/psb_cyc ++ +which contains "1" if the feature is supported and "0" +otherwise. ++ +Valid values are given by: ++ + /sys/bus/event_source/devices/intel_pt/caps/psb_periods ++ +which contains a hexadecimal value, the bits of which represent +valid values e.g. bit 2 set means value 2 is valid. ++ +The psb_period value is converted to the approximate number of +trace bytes between PSB packets as: ++ + 2 ^ (value + 11) ++ +e.g. value 3 means 16KiB bytes between PSBs ++ +If an invalid value is entered, the error message +will give a list of valid values e.g. ++ + $ perf record -e intel_pt/psb_period=15/u uname + Invalid psb_period for intel_pt. Valid values are: 0-5 ++ +If MTC packets are selected, the default config selects a value +of 3 (i.e. psb_period=3) or the nearest lower value that is +supported (0 is always supported). Otherwise the default is 0. ++ +If decoding is expected to be reliable and the buffer is large +then a large PSB period can be used. ++ +Because a TSC packet is produced with PSB, the PSB period can +also affect the granularity to timing information in the absence +of MTC or CYC. + +*mtc*:: +Produces MTC timing packets. ++ +MTC packets provide finer grain timestamp information than TSC +packets. MTC packets record time using the hardware crystal +clock (CTC) which is related to TSC packets using a TMA packet. ++ +Support for this feature is indicated by: ++ + /sys/bus/event_source/devices/intel_pt/caps/mtc ++ +which contains "1" if the feature is supported and +"0" otherwise. ++ +The frequency of MTC packets can also be specified - see +mtc_period below. + +*mtc_period*:: +Specifies how frequently MTC packets are produced - see mtc +above for how to determine if MTC packets are supported. ++ +Valid values are given by: ++ + /sys/bus/event_source/devices/intel_pt/caps/mtc_periods ++ +which contains a hexadecimal value, the bits of which represent +valid values e.g. bit 2 set means value 2 is valid. ++ +The mtc_period value is converted to the MTC frequency as: + + CTC-frequency / (2 ^ value) ++ +e.g. value 3 means one eighth of CTC-frequency ++ +Where CTC is the hardware crystal clock, the frequency of which +can be related to TSC via values provided in cpuid leaf 0x15. ++ +If an invalid value is entered, the error message +will give a list of valid values e.g. ++ + $ perf record -e intel_pt/mtc_period=15/u uname + Invalid mtc_period for intel_pt. Valid values are: 0,3,6,9 ++ +The default value is 3 or the nearest lower value +that is supported (0 is always supported). + +*cyc*:: +Produces CYC timing packets. ++ +CYC packets provide even finer grain timestamp information than +MTC and TSC packets. A CYC packet contains the number of CPU +cycles since the last CYC packet. Unlike MTC and TSC packets, +CYC packets are only sent when another packet is also sent. ++ +Support for this feature is indicated by: ++ + /sys/bus/event_source/devices/intel_pt/caps/psb_cyc ++ +which contains "1" if the feature is supported and +"0" otherwise. ++ +The number of CYC packets produced can be reduced by specifying +a threshold - see cyc_thresh below. + +*cyc_thresh*:: +Specifies how frequently CYC packets are produced - see cyc +above for how to determine if CYC packets are supported. ++ +Valid cyc_thresh values are given by: ++ + /sys/bus/event_source/devices/intel_pt/caps/cycle_thresholds ++ +which contains a hexadecimal value, the bits of which represent +valid values e.g. bit 2 set means value 2 is valid. ++ +The cyc_thresh value represents the minimum number of CPU cycles +that must have passed before a CYC packet can be sent. The +number of CPU cycles is: ++ + 2 ^ (value - 1) ++ +e.g. value 4 means 8 CPU cycles must pass before a CYC packet +can be sent. Note a CYC packet is still only sent when another +packet is sent, not at, e.g. every 8 CPU cycles. ++ +If an invalid value is entered, the error message +will give a list of valid values e.g. ++ + $ perf record -e intel_pt/cyc,cyc_thresh=15/u uname + Invalid cyc_thresh for intel_pt. Valid values are: 0-12 ++ +CYC packets are not requested by default. + +*pt*:: +Specifies pass-through which enables the 'branch' config term. ++ +The default config selects 'pt' if it is available, so a user will +never need to specify this term. + +*branch*:: +Enable branch tracing. Branch tracing is enabled by default so to +disable branch tracing use 'branch=0'. ++ +The default config selects 'branch' if it is available. + +*ptw*:: +Enable PTWRITE packets which are produced when a ptwrite instruction +is executed. ++ +Support for this feature is indicated by: ++ + /sys/bus/event_source/devices/intel_pt/caps/ptwrite ++ +which contains "1" if the feature is supported and +"0" otherwise. ++ +As an alternative, refer to "Emulated PTWRITE" further below. + +*fup_on_ptw*:: +Enable a FUP packet to follow the PTWRITE packet. The FUP packet +provides the address of the ptwrite instruction. In the absence of +fup_on_ptw, the decoder will use the address of the previous branch +if branch tracing is enabled, otherwise the address will be zero. +Note that fup_on_ptw will work even when branch tracing is disabled. + +*pwr_evt*:: +Enable power events. The power events provide information about +changes to the CPU C-state. ++ +Support for this feature is indicated by: ++ + /sys/bus/event_source/devices/intel_pt/caps/power_event_trace ++ +which contains "1" if the feature is supported and +"0" otherwise. + +*event*:: +Enable Event Trace. The events provide information about asynchronous +events. ++ +Support for this feature is indicated by: ++ + /sys/bus/event_source/devices/intel_pt/caps/event_trace ++ +which contains "1" if the feature is supported and +"0" otherwise. + +*notnt*:: +Disable TNT packets. Without TNT packets, it is not possible to walk +executable code to reconstruct control flow, however FUP, TIP, TIP.PGE +and TIP.PGD packets still indicate asynchronous control flow, and (if +return compression is disabled - see noretcomp) return statements. +The advantage of eliminating TNT packets is reducing the size of the +trace and corresponding tracing overhead. ++ +Support for this feature is indicated by: ++ + /sys/bus/event_source/devices/intel_pt/caps/tnt_disable ++ +which contains "1" if the feature is supported and +"0" otherwise. + +*aux-action=start-paused*:: +Start tracing paused, refer to the section <<_pause_or_resume_tracing,Pause or Resume Tracing>> + + +config terms on other events +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Some Intel PT features work with other events, features such as AUX area sampling +and PEBS-via-PT. In those cases, the other events can have config terms below: + +*aux-sample-size*:: + Used to set the AUX area sample size, refer to the section + <<_aux_area_sampling_option,AUX area sampling option>> + +*aux-output*:: + Used to select PEBS-via-PT, refer to the + section <<_pebs_via_intel_pt,PEBS via Intel PT>> + +*aux-action*:: + Used to pause or resume tracing, refer to the section + <<_pause_or_resume_tracing,Pause or Resume Tracing>> AUX area sampling option ~~~~~~~~~~~~~~~~~~~~~~~~ @@ -596,7 +649,8 @@ The default snapshot size is the auxtrace mmap size. If neither auxtrace mmap s nor snapshot size is specified, then the default is 4MiB for privileged users (or if /proc/sys/kernel/perf_event_paranoid < 0), 128KiB for unprivileged users. If an unprivileged user does not specify mmap pages, the mmap pages will be -reduced as described in the 'new auxtrace mmap size option' section below. +reduced as described in the <<_new_auxtrace_mmap_size_option,new auxtrace mmap size option>> +section below. The snapshot size is displayed if the option -vv is used e.g. @@ -952,11 +1006,11 @@ transaction start, commit or abort. Note that "instructions", "cycles", "branches" and "transactions" events depend on code flow packets which can be disabled by using the config term -"branch=0". Refer to the config terms section above. +"branch=0". Refer to the <<_config_terms,config terms>> section above. "ptwrite" events record the payload of the ptwrite instruction and whether "fup_on_ptw" was used. "ptwrite" events depend on PTWRITE packets which are -recorded only if the "ptw" config term was used. Refer to the config terms +recorded only if the "ptw" config term was used. Refer to the <<_config_terms,config terms>> section above. perf script "synth" field displays "ptwrite" information like this: "ip: 0 payload: 0x123456789abcdef0" where "ip" is 1 if "fup_on_ptw" was used. @@ -964,7 +1018,7 @@ used. "Power" events correspond to power event packets and CBR (core-to-bus ratio) packets. While CBR packets are always recorded when tracing is enabled, power event packets are recorded only if the "pwr_evt" config term was used. Refer to -the config terms section above. The power events record information about +the <<_config_terms,config terms>> section above. The power events record information about C-state changes, whereas CBR is indicative of CPU frequency. perf script "event,synth" fields display information like this: @@ -1120,7 +1174,7 @@ What *will* be decoded with the (single) q option: - asynchronous branches such as interrupts - indirect branches - function return target address *if* the noretcomp config term (refer - config terms section) was used + <<_config_terms,config terms>> section) was used - start of (control-flow) tracing - end of (control-flow) tracing, if it is not out of context - power events, ptwrite, transaction start and abort @@ -1133,7 +1187,7 @@ Repeating the q option (double-q i.e. qq) results in even faster decoding and ev less detail. The decoder decodes only extended PSB (PSB+) packets, getting the instruction pointer if there is a FUP packet within PSB+ (i.e. between PSB and PSBEND). Note PSB packets occur regularly in the trace based on the psb_period -config term (refer config terms section). There will be a FUP packet if the +config term (refer <<_config_terms,config terms>> section). There will be a FUP packet if the PSB+ occurs while control flow is being traced. What will *not* be decoded with the qq option: @@ -1867,6 +1921,108 @@ For pipe mode, the order of events and timestamps can presumably be messed up. +Pause or Resume Tracing +----------------------- + +With newer Kernels, it is possible to use other selected events to pause +or resume Intel PT tracing. This is configured by using the "aux-action" +config term: + +"aux-action=pause" is used with events that are to pause Intel PT tracing. + +"aux-action=resume" is used with events that are to resume Intel PT tracing. + +"aux-action=start-paused" is used with the Intel PT event to start in a +paused state. + +For example, to trace only the uname system call (sys_newuname) when running the +command line utility uname: + + $ perf record --kcore -e intel_pt/aux-action=start-paused/k,syscalls:sys_enter_newuname/aux-action=resume/,syscalls:sys_exit_newuname/aux-action=pause/ uname + Linux + [ perf record: Woken up 1 times to write data ] + [ perf record: Captured and wrote 0.043 MB perf.data ] + $ perf script --call-trace + uname 30805 [000] 24001.058782799: name: 0x7ffc9c1865b0 + uname 30805 [000] 24001.058784424: psb offs: 0 + uname 30805 [000] 24001.058784424: cbr: 39 freq: 3904 MHz (139%) + uname 30805 [000] 24001.058784629: ([kernel.kallsyms]) debug_smp_processor_id + uname 30805 [000] 24001.058784629: ([kernel.kallsyms]) __x64_sys_newuname + uname 30805 [000] 24001.058784629: ([kernel.kallsyms]) down_read + uname 30805 [000] 24001.058784629: ([kernel.kallsyms]) __cond_resched + uname 30805 [000] 24001.058784629: ([kernel.kallsyms]) preempt_count_add + uname 30805 [000] 24001.058784629: ([kernel.kallsyms]) in_lock_functions + uname 30805 [000] 24001.058784629: ([kernel.kallsyms]) preempt_count_sub + uname 30805 [000] 24001.058784629: ([kernel.kallsyms]) up_read + uname 30805 [000] 24001.058784629: ([kernel.kallsyms]) preempt_count_add + uname 30805 [000] 24001.058784838: ([kernel.kallsyms]) in_lock_functions + uname 30805 [000] 24001.058784838: ([kernel.kallsyms]) preempt_count_sub + uname 30805 [000] 24001.058784838: ([kernel.kallsyms]) _copy_to_user + uname 30805 [000] 24001.058784838: ([kernel.kallsyms]) syscall_exit_to_user_mode + uname 30805 [000] 24001.058784838: ([kernel.kallsyms]) syscall_exit_work + uname 30805 [000] 24001.058784838: ([kernel.kallsyms]) perf_syscall_exit + uname 30805 [000] 24001.058784838: ([kernel.kallsyms]) debug_smp_processor_id + uname 30805 [000] 24001.058785046: ([kernel.kallsyms]) perf_trace_buf_alloc + uname 30805 [000] 24001.058785046: ([kernel.kallsyms]) perf_swevent_get_recursion_context + uname 30805 [000] 24001.058785046: ([kernel.kallsyms]) debug_smp_processor_id + uname 30805 [000] 24001.058785046: ([kernel.kallsyms]) debug_smp_processor_id + uname 30805 [000] 24001.058785046: ([kernel.kallsyms]) perf_tp_event + uname 30805 [000] 24001.058785046: ([kernel.kallsyms]) perf_trace_buf_update + uname 30805 [000] 24001.058785046: ([kernel.kallsyms]) tracing_gen_ctx_irq_test + uname 30805 [000] 24001.058785046: ([kernel.kallsyms]) perf_swevent_event + uname 30805 [000] 24001.058785046: ([kernel.kallsyms]) __perf_event_account_interrupt + uname 30805 [000] 24001.058785046: ([kernel.kallsyms]) __this_cpu_preempt_check + uname 30805 [000] 24001.058785046: ([kernel.kallsyms]) perf_event_output_forward + uname 30805 [000] 24001.058785046: ([kernel.kallsyms]) perf_event_aux_pause + uname 30805 [000] 24001.058785046: ([kernel.kallsyms]) ring_buffer_get + uname 30805 [000] 24001.058785046: ([kernel.kallsyms]) __rcu_read_lock + uname 30805 [000] 24001.058785046: ([kernel.kallsyms]) __rcu_read_unlock + uname 30805 [000] 24001.058785254: ([kernel.kallsyms]) pt_event_stop + uname 30805 [000] 24001.058785254: ([kernel.kallsyms]) debug_smp_processor_id + uname 30805 [000] 24001.058785254: ([kernel.kallsyms]) debug_smp_processor_id + uname 30805 [000] 24001.058785254: ([kernel.kallsyms]) native_write_msr + uname 30805 [000] 24001.058785463: ([kernel.kallsyms]) native_write_msr + uname 30805 [000] 24001.058785639: 0x0 + +The example above uses tracepoints, but any kind of sampled event can be used. + +For example: + + Tracing between arch_cpu_idle_enter() and arch_cpu_idle_exit() using breakpoint events: + + $ sudo cat /proc/kallsyms | sort | grep ' arch_cpu_idle_enter\| arch_cpu_idle_exit' + ffffffffb605bf60 T arch_cpu_idle_enter + ffffffffb614d8a0 W arch_cpu_idle_exit + $ sudo perf record --kcore -a -e intel_pt/aux-action=start-paused/k -e mem:0xffffffffb605bf60:x/aux-action=resume/ -e mem:0xffffffffb614d8a0:x/aux-action=pause/ -- sleep 1 + [ perf record: Woken up 1 times to write data ] + [ perf record: Captured and wrote 1.387 MB perf.data ] + + Tracing __alloc_pages() using kprobes: + + $ sudo perf probe --add '__alloc_pages order' + Added new event: probe:__alloc_pages (on __alloc_pages with order) + $ sudo perf probe --add __alloc_pages%return + Added new event: probe:__alloc_pages__return (on __alloc_pages%return) + $ sudo perf record --kcore -aR -e intel_pt/aux-action=start-paused/k -e probe:__alloc_pages/aux-action=resume/ -e probe:__alloc_pages__return/aux-action=pause/ -- sleep 1 + [ perf record: Woken up 1 times to write data ] + [ perf record: Captured and wrote 1.490 MB perf.data ] + + Tracing starting at main() using a uprobe event: + + $ sudo perf probe -x /usr/bin/uname main + Added new event: probe_uname:main (on main in /usr/bin/uname) + $ sudo perf record -e intel_pt/-aux-action=start-paused/u -e probe_uname:main/aux-action=resume/ -- uname + Linux + [ perf record: Woken up 1 times to write data ] + [ perf record: Captured and wrote 0.031 MB perf.data ] + + Tracing occasionally using cycles events with different periods: + + $ perf record --kcore -a -m,64M -e intel_pt/aux-action=start-paused/k -e cycles/aux-action=pause,period=1000000/Pk -e cycles/aux-action=resume,period=10500000/Pk -- firefox + [ perf record: Woken up 19 times to write data ] + [ perf record: Captured and wrote 16.561 MB perf.data ] + + EXAMPLE ------- diff --git a/tools/perf/Documentation/perf-list.txt b/tools/perf/Documentation/perf-list.txt index d0c65fad419a..ce0735021473 100644 --- a/tools/perf/Documentation/perf-list.txt +++ b/tools/perf/Documentation/perf-list.txt @@ -27,7 +27,7 @@ Don't print descriptions. -v:: --long-desc:: -Print longer event descriptions. +Print longer event descriptions and all similar PMUs with alphanumeric suffixes. --debug:: Enable debugging output. @@ -188,7 +188,7 @@ in the CPU vendor specific documentation. The available PMUs and their raw parameters can be listed with - ls /sys/devices/*/format + ls /sys/bus/event_source/devices/*/format For example the raw event "LSD.UOPS" core pmu event above could be specified as @@ -289,6 +289,15 @@ Sums up the event counts for all hardware threads in a core, e.g.: perf stat -e cpu/event=0,umask=0x3,percore=1/ +cpu: + +Specifies the CPU to open the event upon. The value may be repeated to +specify opening the event on multiple CPUs: + + + perf stat -e instructions/cpu=0,cpu=2/,cycles/cpu=1,cpu=2/ -a sleep 1 + perf stat -e data_read/cpu=0/,data_write/cpu=1/ -a sleep 1 + EVENT GROUPS ------------ diff --git a/tools/perf/Documentation/perf-lock.txt b/tools/perf/Documentation/perf-lock.txt index 57a940399de0..c17b3e318169 100644 --- a/tools/perf/Documentation/perf-lock.txt +++ b/tools/perf/Documentation/perf-lock.txt @@ -179,16 +179,17 @@ CONTENTION OPTIONS -o:: --lock-owner:: - Show lock contention stat by owners. Implies --threads and - requires --use-bpf. + Show lock contention stat by owners. This option can be combined with -t, + which shows owner's per thread lock stats, or -v, which shows owner's + stacktrace. Requires --use-bpf. -Y:: --type-filter=<value>:: Show lock contention only for given lock types (comma separated list). Available values are: semaphore, spinlock, rwlock, rwlock:R, rwlock:W, rwsem, rwsem:R, rwsem:W, - rtmutex, rwlock-rt, rwlock-rt:R, rwlock-rt:W, pcpu-sem, pcpu-sem:R, pcpu-sem:W, - mutex + rtmutex, rwlock-rt, rwlock-rt:R, rwlock-rt:W, percpu-rwmem, pcpu-sem, + pcpu-sem:R, pcpu-sem:W, mutex Note that RW-variant of locks have :R and :W suffix. Names without the suffix are shortcuts for the both variants. Ex) rwsem = rwsem:R + rwsem:W. @@ -215,6 +216,21 @@ CONTENTION OPTIONS --cgroup-filter=<value>:: Show lock contention only in the given cgroups (comma separated list). +-J:: +--inject-delay=<time@function>:: + Add delays to the given lock. It's added to the contention-end part so + that the (new) owner of the lock will be delayed. But by slowing down + the owner, the waiters will also be delayed as well. This is working + only with -b/--use-bpf. + + The 'time' is specified in nsec but it can have a unit suffix. Available + units are "ms", "us" and "ns". Currently it accepts up to 10ms of delays + for safety reasons. + + Note that it will busy-wait after it gets the lock. Delaying locks can + have significant consequences including potential kernel crashes. Please + use it at your own risk. + SEE ALSO -------- diff --git a/tools/perf/Documentation/perf-mem.txt b/tools/perf/Documentation/perf-mem.txt index 8a1bd9ff0f86..4d164836d094 100644 --- a/tools/perf/Documentation/perf-mem.txt +++ b/tools/perf/Documentation/perf-mem.txt @@ -28,6 +28,8 @@ and kernel support is required. See linkperf:perf-arm-spe[1] for a setup guide. Due to the statistical nature of SPE sampling, not every memory operation will be sampled. +On AMD this use IBS Op PMU to sample load-store operations. + COMMON OPTIONS -------------- -f:: @@ -67,8 +69,15 @@ RECORD OPTIONS Configure all used events to run in user space. --ldlat <n>:: - Specify desired latency for loads event. Supported on Intel and Arm64 - processors only. Ignored on other archs. + Specify desired latency for loads event. Supported on Intel, Arm64 and + some AMD processors. Ignored on other archs. + + On supported AMD processors: + - /sys/bus/event_source/devices/ibs_op/caps/ldlat file contains '1'. + - Supported latency values are 128 to 2048 (both inclusive). + - Latency value which is a multiple of 128 incurs a little less profiling + overhead compared to other values. + - Load latency filtering is disabled by default. REPORT OPTIONS -------------- @@ -110,6 +119,22 @@ REPORT OPTIONS And the default sort keys are changed to local_weight, mem, sym, dso, symbol_daddr, dso_daddr, snoop, tlb, locked, blocked, local_ins_lat. +-F:: +--fields=:: + Specify output field - multiple keys can be specified in CSV format. + Please see linkperf:perf-report[1] for details. + + In addition to the default fields, 'perf mem report' will provide the + following fields to break down sample periods. + + - op: operation in the sample instruction (load, store, prefetch, ...) + - cache: location in CPU cache (L1, L2, ...) where the sample hit + - mem: location in memory or other places the sample hit + - dtlb: location in Data TLB (L1, L2) where the sample hit + - snoop: snoop result for the sampled data access + + Please take a look at the OUTPUT FIELD SELECTION section for caveats. + -T:: --type-profile:: Show data-type profile result instead of code symbols. This requires @@ -128,6 +153,59 @@ REPORT OPTIONS In addition, for report all perf report options are valid, and for record all perf record options. +OVERHEAD CALCULATION +-------------------- +Unlike linkperf:perf-report[1], which calculates overhead from the actual +sample period, perf-mem overhead is calculated using sample weight. E.g. +there are two samples in perf.data file, both with the same sample period, +but one sample with weight 180 and the other with weight 20: + + $ perf script -F period,data_src,weight,ip,sym + 100000 629080842 |OP LOAD|LVL L3 hit|... 20 7e69b93ca524 strcmp + 100000 1a29081042 |OP LOAD|LVL RAM hit|... 180 ffffffff82429168 memcpy + + $ perf report -F overhead,symbol + 50% [.] strcmp + 50% [k] memcpy + + $ perf mem report -F overhead,symbol + 90% [k] memcpy + 10% [.] strcmp + +OUTPUT FIELD SELECTION +---------------------- +"perf mem report" adds a number of new output fields specific to data source +information in the sample. Some of them have the same name with the existing +sort keys ("mem" and "snoop"). So unlike other fields and sort keys, they'll +behave differently when it's used by -F/--fields or -s/--sort. + +Using those two as output fields will aggregate samples altogether and show +breakdown. + + $ perf mem report -F mem,snoop + ... + # ------ Memory ------- --- Snoop ---- + # RAM Uncach Other HitM Other + # ..................... .............. + # + 3.5% 0.0% 96.5% 25.1% 74.9% + +But using the same name for sort keys will aggregate samples for each type +separately. + + $ perf mem report -s mem,snoop + # Overhead Samples Memory access Snoop + # ........ ............ ....................................... ............ + # + 47.99% 1509 L2 hit N/A + 25.08% 338 core, same node Any cache hit HitM + 10.24% 54374 N/A N/A + 6.77% 35938 L1 hit N/A + 6.39% 101 core, same node Any cache hit N/A + 3.50% 69 RAM hit N/A + 0.03% 158 LFB/MAB hit N/A + 0.00% 2 Uncached hit N/A + SEE ALSO -------- linkperf:perf-record[1], linkperf:perf-report[1], linkperf:perf-arm-spe[1] diff --git a/tools/perf/Documentation/perf-record.txt b/tools/perf/Documentation/perf-record.txt index 242223240a08..612612fa2d80 100644 --- a/tools/perf/Documentation/perf-record.txt +++ b/tools/perf/Documentation/perf-record.txt @@ -68,6 +68,10 @@ OPTIONS like this: name=\'CPU_CLK_UNHALTED.THREAD:cmask=0x1\'. - 'aux-output': Generate AUX records instead of events. This requires that an AUX area event is also provided. + - 'aux-action': "pause" or "resume" to pause or resume an AUX + area event (the group leader) when this event occurs. + "start-paused" on an AUX area event itself, will + start in a paused state. - 'aux-sample-size': Set sample size for AUX area sampling. If the '--aux-sample' option has been used, set aux-sample-size=0 to disable AUX area sampling for the event. @@ -223,6 +227,10 @@ OPTIONS '--filter' exists, the new filter expression will be combined with them by '&&'. +--latency:: + Enable data collection for latency profiling. + Use perf report --latency for latency-centric profile. + -a:: --all-cpus:: System-wide collection from all CPUs (default if no target is specified). @@ -332,7 +340,7 @@ OPTIONS -d:: --data:: - Record the sample virtual addresses. + Record the sample virtual addresses. Implies --sample-mem-info. --phys-data:: Record the sample physical addresses. @@ -360,6 +368,11 @@ OPTIONS the sample_type member of the struct perf_event_attr argument to the perf_event_open system call. +--sample-mem-info:: + Record the sample data source information for memory operations. + It requires hardware supports and may work on specific events only. + Please consider using 'perf mem record' instead if you're not sure. + -n:: --no-samples:: Don't sample. @@ -829,6 +842,15 @@ filtered through the mask provided by -C option. only, as of now. So the applications built without the frame pointer might see bogus addresses. + off-cpu profiling consists two types of samples: direct samples, which + share the same behavior as regular samples, and the accumulated + samples, stored in BPF stack trace map, presented after all the regular + samples. + +--off-cpu-thresh:: + Once a task's off-cpu time reaches this threshold (in milliseconds), it + generates a direct off-cpu sample. The default is 500ms. + --setup-filter=<action>:: Prepare BPF filter to be used by regular users. The action should be either "pin" or "unpin". The filter can be used after it's pinned. diff --git a/tools/perf/Documentation/perf-report.txt b/tools/perf/Documentation/perf-report.txt index 87f864519406..acef3ff4178e 100644 --- a/tools/perf/Documentation/perf-report.txt +++ b/tools/perf/Documentation/perf-report.txt @@ -44,7 +44,7 @@ OPTIONS --comms=:: Only consider symbols in these comms. CSV that understands file://filename entries. This option will affect the percentage of - the overhead column. See --percentage for more info. + the overhead and latency columns. See --percentage for more info. --pid=:: Only show events for given process ID (comma separated list). @@ -54,12 +54,12 @@ OPTIONS --dsos=:: Only consider symbols in these dsos. CSV that understands file://filename entries. This option will affect the percentage of - the overhead column. See --percentage for more info. + the overhead and latency columns. See --percentage for more info. -S:: --symbols=:: Only consider these symbols. CSV that understands file://filename entries. This option will affect the percentage of - the overhead column. See --percentage for more info. + the overhead and latency columns. See --percentage for more info. --symbol-filter=:: Only show symbols that match (partially) with this filter. @@ -68,6 +68,21 @@ OPTIONS --hide-unresolved:: Only display entries resolved to a symbol. +--parallelism:: + Only consider these parallelism levels. Parallelism level is the number + of threads that actively run on CPUs at the time of sample. The flag + accepts single number, comma-separated list, and ranges (for example: + "1", "7,8", "1,64-128"). This is useful in understanding what a program + is doing during sequential/low-parallelism phases as compared to + high-parallelism phases. This option will affect the percentage of + the overhead and latency columns. See --percentage for more info. + Also see the `CPU and latency overheads' section for more details. + +--latency:: + Show latency-centric profile rather than the default + CPU-consumption-centric profile + (requires perf record --latency flag). + -s:: --sort=:: Sort histogram entries by given key(s) - multiple keys can be specified @@ -79,6 +94,7 @@ OPTIONS - comm: command (name) of the task which can be read via /proc/<pid>/comm - pid: command and tid of the task + - tgid: command and tgid of the task - dso: name of library or module executed at the time of sample - dso_size: size of library or module executed at the time of sample - symbol: name of function executed at the time of sample @@ -87,6 +103,7 @@ OPTIONS entries are displayed as "[other]". - cpu: cpu number the task ran at the time of sample - socket: processor socket number the task ran at the time of sample + - parallelism: number of running threads at the time of sample - srcline: filename and line number executed at the time of sample. The DWARF debugging info must be provided. - srcfile: file name of the source file of the samples. Requires dwarf @@ -97,12 +114,14 @@ OPTIONS - cgroup_id: ID derived from cgroup namespace device and inode numbers. - cgroup: cgroup pathname in the cgroupfs. - transaction: Transaction abort flags. - - overhead: Overhead percentage of sample - - overhead_sys: Overhead percentage of sample running in system mode - - overhead_us: Overhead percentage of sample running in user mode - - overhead_guest_sys: Overhead percentage of sample running in system mode + - overhead: CPU overhead percentage of sample. + - latency: latency (wall-clock) overhead percentage of sample. + See the `CPU and latency overheads' section for more details. + - overhead_sys: CPU overhead percentage of sample running in system mode + - overhead_us: CPU overhead percentage of sample running in user mode + - overhead_guest_sys: CPU overhead percentage of sample running in system mode on guest machine - - overhead_guest_us: Overhead percentage of sample running in user mode on + - overhead_guest_us: CPU overhead percentage of sample running in user mode on guest machine - sample: Number of sample - period: Raw number of event count of sample @@ -125,8 +144,8 @@ OPTIONS - weight2: Average value of event specific weight (2nd field of weight_struct). - weight3: Average value of event specific weight (3rd field of weight_struct). - By default, comm, dso and symbol keys are used. - (i.e. --sort comm,dso,symbol) + By default, overhead, comm, dso and symbol keys are used. + (i.e. --sort overhead,comm,dso,symbol). If --branch-stack option is used, following sort keys are also available: @@ -201,9 +220,9 @@ OPTIONS --fields=:: Specify output field - multiple keys can be specified in CSV format. Following fields are available: - overhead, overhead_sys, overhead_us, overhead_children, sample, period, - weight1, weight2, weight3, ins_lat, p_stage_cyc and retire_lat. The - last 3 names are alias for the corresponding weights. When the weight + overhead, latency, overhead_sys, overhead_us, overhead_children, sample, + period, weight1, weight2, weight3, ins_lat, p_stage_cyc and retire_lat. + The last 3 names are alias for the corresponding weights. When the weight fields are used, they will show the average value of the weight. Also it can contain any sort key(s). @@ -289,7 +308,7 @@ OPTIONS Accumulate callchain of children to parent entry so that then can show up in the output. The output will have a new "Children" column and will be sorted on the data. It requires callchains are recorded. - See the `overhead calculation' section for more details. Enabled by + See the `Overhead calculation' section for more details. Enabled by default, disable with --no-children. --max-stack:: @@ -442,9 +461,9 @@ OPTIONS --call-graph option for details. --percentage:: - Determine how to display the overhead percentage of filtered entries. - Filters can be applied by --comms, --dsos and/or --symbols options and - Zoom operations on the TUI (thread, dso, etc). + Determine how to display the CPU and latency overhead percentage + of filtered entries. Filters can be applied by --comms, --dsos, --symbols + and/or --parallelism options and Zoom operations on the TUI (thread, dso, etc). "relative" means it's relative to filtered entries only so that the sum of shown entries will be always 100%. "absolute" means it retains @@ -627,6 +646,8 @@ include::itrace.txt[] --skip-empty:: Do not print 0 results in the --stat output. +include::cpu-and-latency-overheads.txt[] + include::callchain-overhead-calculation.txt[] SEE ALSO diff --git a/tools/perf/Documentation/perf-script.txt b/tools/perf/Documentation/perf-script.txt index b72866ef270b..28bec7e78bc8 100644 --- a/tools/perf/Documentation/perf-script.txt +++ b/tools/perf/Documentation/perf-script.txt @@ -239,13 +239,22 @@ OPTIONS i.e., -F "" is not allowed. The brstack output includes branch related information with raw addresses using the - /v/v/v/v/cycles syntax in the following order: - FROM: branch source instruction - TO : branch target instruction - M/P/-: M=branch target mispredicted or branch direction was mispredicted, P=target predicted or direction predicted, -=not supported - X/- : X=branch inside a transactional region, -=not in transaction region or not supported - A/- : A=TSX abort entry, -=not aborted region or not supported - cycles + FROM/TO/EVENT/INTX/ABORT/CYCLES/TYPE/SPEC syntax in the following order: + FROM : branch source instruction + TO : branch target instruction + EVENT : M=branch target or direction was mispredicted + P=branch target or direction was predicted + N=branch not-taken + -=no event or not supported + INTX : X=branch inside a transactional region + -=branch not in transaction region or not supported + ABORT : A=TSX abort entry + -=not aborted region or not supported + CYCLES: the number of cycles that have elapsed since the last branch was recorded + TYPE : branch type: COND/UNCOND/IND/CALL/IND_CALL/RET etc. + -=not supported + SPEC : branch speculation info: SPEC_WRONG_PATH/NON_SPEC_CORRECT_PATH/SPEC_CORRECT_PATH + -=not supported The brstacksym is identical to brstack, except that the FROM and TO addresses are printed in a symbolic form if possible. diff --git a/tools/perf/Documentation/perf-stat.txt b/tools/perf/Documentation/perf-stat.txt index 2bc063672486..61d091670dee 100644 --- a/tools/perf/Documentation/perf-stat.txt +++ b/tools/perf/Documentation/perf-stat.txt @@ -506,6 +506,13 @@ this option is not set. The TPEBS hardware feature starts from Intel Granite Rapids microarchitecture. This option only exists in X86_64 and is meaningful on Intel platforms with TPEBS feature. +--tpebs-mode=[mean|min|max|last]:: +Set how retirement latency events have their sample times +combined. The default "mean" gives the average of retirement +latency. "min" or "max" give the smallest or largest retirment latency +times respectively. "last" uses the last retirment latency sample's +time. + --td-level:: Print the top-down statistics that equal the input level. It allows users to print the interested top-down metrics level instead of the diff --git a/tools/perf/Documentation/perf-test.txt b/tools/perf/Documentation/perf-test.txt index efcdec528a8f..32da0d1fa86a 100644 --- a/tools/perf/Documentation/perf-test.txt +++ b/tools/perf/Documentation/perf-test.txt @@ -28,18 +28,22 @@ OPTIONS Tests to skip (comma separated numeric list). -v:: +-vv:: +-vvv:: --verbose:: - Be more verbose. + With a single '-v', verbose level 1, only failing test output + is displayed. With '-vv' and higher all test output is shown. -S:: --sequential:: - Run tests one after the other, this is the default mode. - --p:: ---parallel:: - Run tests in parallel, speeds up the whole process but is not safe with - the current infrastructure, where some tests that compete for some resources, - for instance, 'perf probe' tests that add/remove probes or clean all probes, etc. + Run all tests one after the other. By default "exclusive" + tests are run sequentially, but other tests are run in + parallel to speed execution. + +-r:: +--runs-per-test:: + Run each test the given number of times, by default once. This + option can be useful to determine if a test is flaky. -F:: --dont-fork:: diff --git a/tools/perf/Documentation/perf-trace.txt b/tools/perf/Documentation/perf-trace.txt index 6e0cc50bbc13..c1fb6056a0d3 100644 --- a/tools/perf/Documentation/perf-trace.txt +++ b/tools/perf/Documentation/perf-trace.txt @@ -150,6 +150,11 @@ the thread executes on the designated CPUs. Default is to monitor all CPUs. To be used with -s or -S, to show stats for the errnos experienced by syscalls, using only this option will trigger --summary. +--summary-mode=mode:: + To be used with -s or -S, to select how to show summary. By default it'll + show the syscall summary by thread. Possible values are: thread, total, + cgroup. + --tool_stats:: Show tool stats such as number of times fd->pathname was discovered thru hooking the open syscall return + vfs_getname or via reading /proc/pid/fd, etc. @@ -241,6 +246,17 @@ the thread executes on the designated CPUs. Default is to monitor all CPUs. printing using the existing 'perf trace' syscall arg beautifiers to map integer arguments to strings (pid to comm, syscall id to syscall name, etc). +--force-btf:: + Use btf_dump to pretty print syscall argument data, instead of using hand-crafted pretty + printers. This option is intended for testing BTF integration in perf trace. btf_dump-based + pretty-printing serves as a fallback to hand-crafted pretty printers, as the latter can + better pretty-print integer flags and struct pointers. + +--bpf-summary:: + Collect system call statistics in BPF. This is only for live mode and + works well with -s/--summary option where no argument information is + required. + PAGEFAULTS ---------- diff --git a/tools/perf/Documentation/perf.data-file-format.txt b/tools/perf/Documentation/perf.data-file-format.txt index 010a4edcd384..cd95ba09f727 100644 --- a/tools/perf/Documentation/perf.data-file-format.txt +++ b/tools/perf/Documentation/perf.data-file-format.txt @@ -370,7 +370,7 @@ struct { u32 mmap_len; }; -Indicates that trace contains records of PERF_RECORD_COMPRESSED type +Indicates that trace contains records of PERF_RECORD_COMPRESSED2 type that have perf_events records in compressed form. HEADER_CPU_PMU_CAPS = 28, @@ -602,7 +602,14 @@ struct auxtrace_error_event { Describes a header feature. These are records used in pipe-mode that contain information that otherwise would be in perf.data file's header. - PERF_RECORD_COMPRESSED = 81, + PERF_RECORD_COMPRESSED = 81, /* deprecated */ + +The header is followed by compressed data frame that can be decompressed +into array of perf trace records. The size of the entire compressed event +record including the header is limited by the max value of header.size. + +It is deprecated and new files should use PERF_RECORD_COMPRESSED2 to gurantee +8-byte alignment. struct compressed_event { struct perf_event_header header; @@ -618,10 +625,17 @@ This is used, for instance, to 'perf inject' events after init and before regular events, those emitted by the kernel, to support combining guest and host records. + PERF_RECORD_COMPRESSED2 = 83, -The header is followed by compressed data frame that can be decompressed -into array of perf trace records. The size of the entire compressed event -record including the header is limited by the max value of header.size. +8-byte aligned version of `PERF_RECORD_COMPRESSED`. `header.size` indicates the +total record size, including padding for 8-byte alignment, and `data_size` +specifies the actual size of the compressed data. + +struct perf_record_compressed2 { + struct perf_event_header header; + __u64 data_size; + char data[]; +}; Event types diff --git a/tools/perf/Documentation/tips.txt b/tools/perf/Documentation/tips.txt index 67b326ba0040..3fee9b2a88ea 100644 --- a/tools/perf/Documentation/tips.txt +++ b/tools/perf/Documentation/tips.txt @@ -62,3 +62,7 @@ To show context switches in perf report sample context add --switch-events to pe To show time in nanoseconds in record/report add --ns To compare hot regions in two workloads use perf record -b -o file ... ; perf diff --stream file1 file2 To compare scalability of two workload samples use perf diff -c ratio file1 file2 +For latency profiling, try: perf record/report --latency +For parallelism histogram, try: perf report --hierarchy --sort latency,parallelism,comm,symbol +To analyze particular parallelism levels, try: perf report --latency --parallelism=32-64 +To see how parallelism changes over time, try: perf report -F time,latency,parallelism --time-quantum=1s |