summaryrefslogtreecommitdiff
path: root/tools/tracing/rtla/src/timerlat_hist.c
AgeCommit message (Collapse)AuthorFilesLines
2025-03-07rtla/timerlat_hist: Set OSNOISE_WORKLOAD for kernel threadsTomas Glozar1-6/+9
commit d8d866171a414ed88bd0d720864095fd75461134 upstream. When using rtla timerlat with userspace threads (-u or -U), rtla disables the OSNOISE_WORKLOAD option in /sys/kernel/tracing/osnoise/options. This option is not re-enabled in a subsequent run with kernel-space threads, leading to rtla collecting no results if the previous run exited abnormally: $ rtla timerlat hist -u ^\Quit (core dumped) $ rtla timerlat hist -k -d 1s Index over: count: min: avg: max: ALL: IRQ Thr Usr count: 0 0 0 min: - - - avg: - - - max: - - - The issue persists until OSNOISE_WORKLOAD is set manually by running: $ echo OSNOISE_WORKLOAD > /sys/kernel/tracing/osnoise/options Set OSNOISE_WORKLOAD when running rtla with kernel-space threads if available to fix the issue. Cc: stable@vger.kernel.org Cc: John Kacur <jkacur@redhat.com> Cc: Luis Goncalves <lgoncalv@redhat.com> Link: https://lore.kernel.org/20250107144823.239782-3-tglozar@redhat.com Fixes: ed774f7481fa ("rtla/timerlat_hist: Add timerlat user-space support") Signed-off-by: Tomas Glozar <tglozar@redhat.com> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org> [ params->kernel_workload does not exist in 6.6, use !params->user_hist ] Signed-off-by: Tomas Glozar <tglozar@redhat.com> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2025-03-07Revert "rtla/timerlat_hist: Set OSNOISE_WORKLOAD for kernel threads"Tomas Glozar1-9/+6
This reverts commit 83b74901bdc9b58739193b8ee6989254391b6ba7. The commit breaks rtla build, since params->kernel_workload is not present on 6.6-stable. Signed-off-by: Tomas Glozar <tglozar@redhat.com> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2025-02-21rtla/timerlat_hist: Abort event processing on second signalTomas Glozar1-0/+8
[ Upstream commit d6899e560366e10141189697502bc5521940c588 ] If either SIGINT is received twice, or after a SIGALRM (that is, after timerlat was supposed to stop), abort processing events currently left in the tracefs buffer and exit immediately. This allows the user to exit rtla without waiting for processing all events, should that take longer than wanted, at the cost of not processing all samples. Cc: John Kacur <jkacur@redhat.com> Cc: Luis Goncalves <lgoncalv@redhat.com> Cc: Gabriele Monaco <gmonaco@redhat.com> Link: https://lore.kernel.org/20250116144931.649593-5-tglozar@redhat.com Signed-off-by: Tomas Glozar <tglozar@redhat.com> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org> Signed-off-by: Sasha Levin <sashal@kernel.org>
2025-02-17rtla/timerlat_hist: Stop timerlat tracer on signalTomas Glozar1-1/+10
commit c73cab9dbed04d8f65ca69177b4b21ed3e09dfa7 upstream. Currently, when either SIGINT from the user or SIGALRM from the duration timer is caught by rtla-timerlat, stop_tracing is set to break out of the main loop. This is not sufficient for cases where the timerlat tracer is producing more data than rtla can consume, since in that case, rtla is looping indefinitely inside tracefs_iterate_raw_events, never reaches the check of stop_tracing and hangs. In addition to setting stop_tracing, also stop the timerlat tracer on received signal (SIGINT or SIGALRM). This will stop new samples so that the existing samples may be processed and tracefs_iterate_raw_events eventually exits. Cc: stable@vger.kernel.org Cc: John Kacur <jkacur@redhat.com> Cc: Luis Goncalves <lgoncalv@redhat.com> Cc: Gabriele Monaco <gmonaco@redhat.com> Link: https://lore.kernel.org/20250116144931.649593-3-tglozar@redhat.com Fixes: 1eeb6328e8b3 ("rtla/timerlat: Add timerlat hist mode") Signed-off-by: Tomas Glozar <tglozar@redhat.com> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2025-02-17rtla/timerlat_hist: Set OSNOISE_WORKLOAD for kernel threadsTomas Glozar1-6/+9
commit d8d866171a414ed88bd0d720864095fd75461134 upstream. When using rtla timerlat with userspace threads (-u or -U), rtla disables the OSNOISE_WORKLOAD option in /sys/kernel/tracing/osnoise/options. This option is not re-enabled in a subsequent run with kernel-space threads, leading to rtla collecting no results if the previous run exited abnormally: $ rtla timerlat hist -u ^\Quit (core dumped) $ rtla timerlat hist -k -d 1s Index over: count: min: avg: max: ALL: IRQ Thr Usr count: 0 0 0 min: - - - avg: - - - max: - - - The issue persists until OSNOISE_WORKLOAD is set manually by running: $ echo OSNOISE_WORKLOAD > /sys/kernel/tracing/osnoise/options Set OSNOISE_WORKLOAD when running rtla with kernel-space threads if available to fix the issue. Cc: stable@vger.kernel.org Cc: John Kacur <jkacur@redhat.com> Cc: Luis Goncalves <lgoncalv@redhat.com> Link: https://lore.kernel.org/20250107144823.239782-3-tglozar@redhat.com Fixes: ed774f7481fa ("rtla/timerlat_hist: Add timerlat user-space support") Signed-off-by: Tomas Glozar <tglozar@redhat.com> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2024-12-19rtla/timerlat: Make timerlat_hist_cpu->*_count unsigned long longTomas Glozar1-6/+6
commit 76b3102148135945b013797fac9b206273f0f777 upstream. Do the same fix as in previous commit also for timerlat-hist. Link: https://lore.kernel.org/20241011121015.2868751-2-tglozar@redhat.com Reported-by: Attila Fazekas <afazekas@redhat.com> Signed-off-by: Tomas Glozar <tglozar@redhat.com> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org> [ Drop hunk fixing printf in timerlat_print_stats_all since that is not in 6.6 ] Signed-off-by: Tomas Glozar <tglozar@redhat.com> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2024-06-16rtla/timerlat: Fix histogram report when a cpu count is 0John Kacur1-18/+42
commit 01b05fc0e5f3aec443a9a8ffa0022cbca2fd3608 upstream. On short runs it is possible to get no samples on a cpu, like this: # rtla timerlat hist -u -T50 Index IRQ-001 Thr-001 Usr-001 IRQ-002 Thr-002 Usr-002 2 1 0 0 0 0 0 33 0 1 0 0 0 0 36 0 0 1 0 0 0 49 0 0 0 1 0 0 52 0 0 0 0 1 0 over: 0 0 0 0 0 0 count: 1 1 1 1 1 0 min: 2 33 36 49 52 18446744073709551615 avg: 2 33 36 49 52 - max: 2 33 36 49 52 0 rtla timerlat hit stop tracing IRQ handler delay: (exit from idle) 48.21 us (91.09 %) IRQ latency: 49.11 us Timerlat IRQ duration: 2.17 us (4.09 %) Blocking thread: 1.01 us (1.90 %) swapper/2:0 1.01 us ------------------------------------------------------------------------ Thread latency: 52.93 us (100%) Max timerlat IRQ latency from idle: 49.11 us in cpu 2 Note, the value 18446744073709551615 is the same as ~0. Fix this by reporting no results for the min, avg and max if the count is 0. Link: https://lkml.kernel.org/r/20240510190318.44295-1-jkacur@redhat.com Cc: stable@vger.kernel.org Fixes: 1eeb6328e8b3 ("rtla/timerlat: Add timerlat hist mode") Suggested-by: Daniel Bristot de Oliveria <bristot@kernel.org> Signed-off-by: John Kacur <jkacur@redhat.com> Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2024-02-23tools/rtla: Fix uninitialized bucket/data->bucket_size warningDaniel Bristot de Oliveira1-2/+1
commit 64dc40f7523369912d7adb22c8cb655f71610505 upstream. When compiling rtla with clang, I am getting the following warnings: $ make HOSTCC=clang CC=clang LLVM_IAS=1 [..] clang -O -g -DVERSION=\"6.8.0-rc3\" -flto=auto -fexceptions -fstack-protector-strong -fasynchronous-unwind-tables -fstack-clash-protection -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS $(pkg-config --cflags libtracefs) -c -o src/osnoise_hist.o src/osnoise_hist.c src/osnoise_hist.c:138:6: warning: variable 'bucket' is used uninitialized whenever 'if' condition is false [-Wsometimes-uninitialized] 138 | if (data->bucket_size) | ^~~~~~~~~~~~~~~~~ src/osnoise_hist.c:149:6: note: uninitialized use occurs here 149 | if (bucket < entries) | ^~~~~~ src/osnoise_hist.c:138:2: note: remove the 'if' if its condition is always true 138 | if (data->bucket_size) | ^~~~~~~~~~~~~~~~~~~~~~ 139 | bucket = duration / data->bucket_size; src/osnoise_hist.c:132:12: note: initialize the variable 'bucket' to silence this warning 132 | int bucket; | ^ | = 0 1 warning generated. [...] clang -O -g -DVERSION=\"6.8.0-rc3\" -flto=auto -fexceptions -fstack-protector-strong -fasynchronous-unwind-tables -fstack-clash-protection -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS $(pkg-config --cflags libtracefs) -c -o src/timerlat_hist.o src/timerlat_hist.c src/timerlat_hist.c:181:6: warning: variable 'bucket' is used uninitialized whenever 'if' condition is false [-Wsometimes-uninitialized] 181 | if (data->bucket_size) | ^~~~~~~~~~~~~~~~~ src/timerlat_hist.c:204:6: note: uninitialized use occurs here 204 | if (bucket < entries) | ^~~~~~ src/timerlat_hist.c:181:2: note: remove the 'if' if its condition is always true 181 | if (data->bucket_size) | ^~~~~~~~~~~~~~~~~~~~~~ 182 | bucket = latency / data->bucket_size; src/timerlat_hist.c:175:12: note: initialize the variable 'bucket' to silence this warning 175 | int bucket; | ^ | = 0 1 warning generated. This is a legit warning, but data->bucket_size is always > 0 (see timerlat_hist_parse_args()), so the if is not necessary. Remove the unneeded if (data->bucket_size) to avoid the warning. Link: https://lkml.kernel.org/r/6e1b1665cd99042ae705b3e0fc410858c4c42346.1707217097.git.bristot@kernel.org Cc: stable@vger.kernel.org Cc: Masami Hiramatsu <mhiramat@kernel.org> Cc: Nathan Chancellor <nathan@kernel.org> Cc: Nick Desaulniers <ndesaulniers@google.com> Cc: Bill Wendling <morbo@google.com> Cc: Justin Stitt <justinstitt@google.com> Cc: Donald Zickus <dzickus@redhat.com> Fixes: 1eeb6328e8b3 ("rtla/timerlat: Add timerlat hist mode") Fixes: 829a6c0b5698 ("rtla/osnoise: Add the hist mode") Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2024-02-23tools/rtla: Exit with EXIT_SUCCESS when help is invokedJohn Kacur1-1/+5
commit b5f319360371087d52070d8f3fc7789e80ce69a6 upstream. Fix rtla so that the following commands exit with 0 when help is invoked rtla osnoise top -h rtla osnoise hist -h rtla timerlat top -h rtla timerlat hist -h Link: https://lore.kernel.org/linux-trace-devel/20240203001607.69703-1-jkacur@redhat.com Cc: stable@vger.kernel.org Fixes: 1eeb6328e8b3 ("rtla/timerlat: Add timerlat hist mode") Signed-off-by: John Kacur <jkacur@redhat.com> Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2023-06-13rtla/timerlat_hist: Add timerlat user-space supportDaniel Bristot de Oliveira1-11/+118
Add the support for running timerlat threads in user-space. In this mode, enabled with -u/--user-threads, timerlat dispatches user-space processes that will loop in the timerlat_fd, measuring the overhead for going to user-space and then returning to the kernel - in addition to the existing measurements. Here is one example of the tool's output with -u enabled: $ sudo timerlat hist -u -c 1-3 -d 600 # RTLA timerlat histogram # Time unit is microseconds (us) # Duration: 0 00:10:01 Index IRQ-001 Thr-001 Usr-001 IRQ-002 Thr-002 Usr-002 IRQ-003 Thr-003 Usr-003 0 477555 0 0 425287 0 0 474357 0 0 1 122385 7998 0 174616 1921 0 125412 3138 0 2 47 587376 492150 89 594717 447830 147 593463 454872 3 11 2549 101930 7 2682 145580 64 2530 138680 4 3 1954 2833 1 463 4917 11 548 4656 5 0 60 1037 0 138 1117 6 179 1130 6 0 26 1837 0 38 277 1 76 339 7 0 15 143 0 28 147 2 37 156 8 0 10 23 0 11 75 0 12 80 9 0 7 17 0 0 26 0 11 42 10 0 2 11 0 0 18 0 2 20 11 0 0 7 0 1 8 0 2 12 12 0 0 6 0 1 4 0 2 8 13 0 1 3 0 0 0 0 0 1 14 0 1 0 0 0 1 0 0 2 15 0 1 0 0 0 0 0 0 2 16 0 1 2 0 0 0 0 0 0 17 0 0 1 0 0 0 0 0 0 19 0 0 1 0 0 0 0 0 0 over: 0 0 0 0 0 0 0 0 0 count: 600001 600001 600001 600000 600000 600000 600000 600000 600000 min: 0 1 2 0 1 2 0 1 2 avg: 0 1 2 0 2 2 0 2 2 max: 4 16 19 4 12 14 7 12 15 The tuning setup like -p or -C work for the user-space threads as well. Link: https://lkml.kernel.org/r/b6a042d55003c4a67ff7dce28d96044b7044f00d.1686066600.git.bristot@kernel.org Cc: William White <chwhite@redhat.com> Cc: Jonathan Corbet <corbet@lwn.net> Tested-by: Juri Lelli <juri.lelli@redhat.com> Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
2023-06-13rtla: Start the tracers after creating all instancesDaniel Bristot de Oliveira1-5/+12
Group all start tracing after finishing creating all instances. The tracing instance starts first for the case of hitting a stop tracing while enabling other instances. The trace instance is the one with most valuable information. Link: https://lkml.kernel.org/r/67da7a703a56f75d7cd46568525145a65501a7e8.1686066600.git.bristot@kernel.org Cc: William White <chwhite@redhat.com> Cc: Jonathan Corbet <corbet@lwn.net> Tested-by: Juri Lelli <juri.lelli@redhat.com> Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
2023-06-13rtla/timerlat_hist: Add auto-analysis supportDaniel Bristot de Oliveira1-2/+49
Add auto-analysis to timerlat hist, including the --no-aa option to reduce overhead and --dump-task. --aa-only was not added as it is already on timerlat top. Link: https://lkml.kernel.org/r/c2693f47ee83e659a7723fed8035f5d2534f528e.1686066600.git.bristot@kernel.org Cc: William White <chwhite@redhat.com> Cc: Jonathan Corbet <corbet@lwn.net> Tested-by: Juri Lelli <juri.lelli@redhat.com> Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
2023-06-13rtla: Automatically move rtla to a house-keeping cpuDaniel Bristot de Oliveira1-0/+9
When the user sets -c <cpu-list> try to move rtla out of the <cpu-list>, even without an -H option. This is useful to avoid having rtla interfering with the workload. This works by removing <cpu-list> from rtla's current affinity. If rtla fails to move itself away it is not that of a problem as this is an automatic measure. Link: https://lkml.kernel.org/r/c54304d90c777310fb85a3e658d1449173759aab.1686066600.git.bristot@kernel.org Cc: William White <chwhite@redhat.com> Cc: Jonathan Corbet <corbet@lwn.net> Tested-by: Juri Lelli <juri.lelli@redhat.com> Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
2023-06-13rtla: Change monitored_cpus from char * to cpu_set_tDaniel Bristot de Oliveira1-9/+9
Use a cpumask instead of a char *, reducing memory footprint and code. No functional change, and in preparation for auto house-keeping. Link: https://lkml.kernel.org/r/54c46293261d13cb1042d0314486539eeb45fe5d.1686066600.git.bristot@kernel.org Cc: William White <chwhite@redhat.com> Cc: Jonathan Corbet <corbet@lwn.net> Tested-by: Juri Lelli <juri.lelli@redhat.com> Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
2023-06-13rtla: Add --house-keeping optionDaniel Bristot de Oliveira1-2/+25
To avoid having rtla interfering with the measurement threads, add an option for the user to set the CPUs in which rtla should run. For instance: # rtla timerlat top -H 0 -c 1-7 Will place rtla in the CPU 0, while running the measurement threads in the CPU 1-7. Link: https://lkml.kernel.org/r/6a6c78a579a96ba8b02ae67ee1e0ba2cb5e03c4a.1686066600.git.bristot@kernel.org Cc: William White <chwhite@redhat.com> Cc: Jonathan Corbet <corbet@lwn.net> Tested-by: Juri Lelli <juri.lelli@redhat.com> Suggested-by: Juri Lelli <juri.lelli@redhat.com> Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
2023-06-13rtla: Add -C cgroup supportDaniel Bristot de Oliveira1-3/+24
The -C option sets a cgroup to the tracer's threads. If the -C option is passed without arguments, the tracer's thread will inherit rtla's cgroup. Otherwise, the threads will be placed on the cgroup passed to the option. Link: https://lkml.kernel.org/r/cb051477331d292f17c08bf1d66f0e0384bbe5a5.1686066600.git.bristot@kernel.org Cc: William White <chwhite@redhat.com> Cc: Jonathan Corbet <corbet@lwn.net> Tested-by: Juri Lelli <juri.lelli@redhat.com> Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
2022-08-10rtla: Fix tracer nameAlexandre Vicenzi1-1/+1
The correct tracer name is timerlat and not timelat. Link: https://lore.kernel.org/linux-trace-devel/20220808180343.22262-1-alexandre.vicenzi@suse.com Signed-off-by: Alexandre Vicenzi <alexandre.vicenzi@suse.com> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
2022-05-26rtla: Avoid record NULL pointer dereferenceWan Jiabing1-5/+6
Fix the following null/deref_null.cocci errors: ./tools/tracing/rtla/src/osnoise_hist.c:870:31-36: ERROR: record is NULL but dereferenced. ./tools/tracing/rtla/src/osnoise_top.c:650:31-36: ERROR: record is NULL but dereferenced. ./tools/tracing/rtla/src/timerlat_hist.c:905:31-36: ERROR: record is NULL but dereferenced. ./tools/tracing/rtla/src/timerlat_top.c:700:31-36: ERROR: record is NULL but dereferenced. "record" is NULL before calling osnoise_init_trace_tool. Add a tag "out_free" to avoid dereferring a NULL pointer. Link: https://lkml.kernel.org/r/ae0e4500d383db0884eb2820286afe34ca303778.1651247710.git.bristot@kernel.org Link: https://lore.kernel.org/r/20220408151406.34823-1-wanjiabing@vivo.com/ Cc: kael_w@yeah.net Cc: Daniel Bristot de Oliveira <bristot@kernel.org> Fixes: 51d64c3a1819 ("rtla: Add -e/--event support") Acked-by: Daniel Bristot de Oliveira <bristot@kernel.org> Signed-off-by: Wan Jiabing <wanjiabing@vivo.com> Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
2022-03-15rtla: Tools main loop cleanupDaniel Bristot de Oliveira1-1/+1
I probably started using "do {} while();", but changed all but osnoise_top to "while(){};" leaving the ; behind. Cleanup the main loop code, making all tools use "while() {}" Changcheng Deng reported this problem, as reported by coccicheck: Fix the following coccicheck review: ./tools/tracing/rtla/src/timerlat_hist.c: 800: 2-3: Unneeded semicolon ./tools/tracing/rtla/src/osnoise_hist.c: 776: 2-3: Unneeded semicolon ./tools/tracing/rtla/src/timerlat_top.c: 596: 2-3: Unneeded semicolon Link: https://lkml.kernel.org/r/3c1642110aa87c396f5da4a037dabc72dbb9c601.1646247211.git.bristot@kernel.org Cc: Daniel Bristot de Oliveira <bristot@kernel.org> Cc: Clark Williams <williams@redhat.com> Cc: Juri Lelli <juri.lelli@redhat.com> Cc: Jonathan Corbet <corbet@lwn.net> Reported-by: Changcheng Deng <deng.changcheng@zte.com.cn> Reported-by: Zeal Robot <zealci@zte.com.cn> Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
2022-03-15rtla/timerlat: Add --dma-latency optionDaniel Bristot de Oliveira1-2/+26
Add the --dma-latency to set /dev/cpu_dma_latency to the specified value, this aims to avoid having exit from idle states latencies that could be influencing the analysis. Link: https://lkml.kernel.org/r/72ddb0d913459f13217086dadafad88a7c46dd28.1646247211.git.bristot@kernel.org Cc: Daniel Bristot de Oliveira <bristot@kernel.org> Cc: Clark Williams <williams@redhat.com> Cc: Juri Lelli <juri.lelli@redhat.com> Cc: Jonathan Corbet <corbet@lwn.net> Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
2022-03-15rtla: Check for trace off also in the trace instanceDaniel Bristot de Oliveira1-2/+2
With the addition of --trigger option, it is also possible to stop the trace from the -t tracing instance using the traceoff trigger. Make rtla tools to check if the trace is stopped also in the trace instance, stopping the execution of the tool. Link: https://lkml.kernel.org/r/59fc7c6f23dddd5c8b7ef1782cf3da51ea2ce0f5.1646247211.git.bristot@kernel.org Cc: Daniel Bristot de Oliveira <bristot@kernel.org> Cc: Clark Williams <williams@redhat.com> Cc: Juri Lelli <juri.lelli@redhat.com> Cc: Jonathan Corbet <corbet@lwn.net> Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
2022-03-15rtla: Add --filter supportDaniel Bristot de Oliveira1-3/+17
Add --filter option. This option enables a trace event filtering of the previous -e sys:event argument. This option is available for all current tools. Link: https://lkml.kernel.org/r/509d70b6348d3e5bcbf1f07ab725ce08d063149a.1646247211.git.bristot@kernel.org Cc: Daniel Bristot de Oliveira <bristot@kernel.org> Cc: Clark Williams <williams@redhat.com> Cc: Juri Lelli <juri.lelli@redhat.com> Cc: Jonathan Corbet <corbet@lwn.net> Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
2022-03-15rtla: Add --trigger supportDaniel Bristot de Oliveira1-3/+16
Add --trigger option. This option enables a trace event trigger to the previous -e sys:event argument, allowing some advanced tracing options. For instance, in a system with CPUs 2:23 isolated, it is possible to get a stack trace of thread wakeup targeting those CPUs while running osnoise with the following command line: # osnoise top -c 2-23 -a 50 -e sched:sched_wakeup --trigger="stacktrace if target_cpu >= 2" This option is available for all current tools. Link: https://lkml.kernel.org/r/07d2983d5f71261d4da89dbaf02efcad100ab8ee.1646247211.git.bristot@kernel.org Cc: Daniel Bristot de Oliveira <bristot@kernel.org> Cc: Clark Williams <williams@redhat.com> Cc: Juri Lelli <juri.lelli@redhat.com> Cc: Jonathan Corbet <corbet@lwn.net> Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
2022-03-15rtla: Add -e/--event supportDaniel Bristot de Oliveira1-4/+29
Add -e/--event option. This option enables an event in the trace (-t) session. The argument can be a specific event, e.g., -e sched:sched_switch, or all events of a system group, e.g., -e sched. Multiple -e are allowed. It is only active when -t or -a are set. This option is available for all current tools. Link: https://lkml.kernel.org/r/6a3b753be9b1e811953995f7f21a86918ad13390.1646247211.git.bristot@kernel.org Cc: Daniel Bristot de Oliveira <bristot@kernel.org> Cc: Clark Williams <williams@redhat.com> Cc: Juri Lelli <juri.lelli@redhat.com> Cc: Jonathan Corbet <corbet@lwn.net> Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
2022-03-15rtla/timerlat: Add the automatic trace optionDaniel Bristot de Oliveira1-4/+20
Add the -a/--auto <arg in us> option. This option sets some commonly used options while debugging the system. It aims to help users produce reports in the field, reducing the number of arguments passed to the tool in the first approach to a problem. It is equivalent to setting osnoise/stop_tracing_total_us and print_stack with the argument, and saving the trace to timerlat_trace.txt file if the trace is stopped automatically. Link: https://lkml.kernel.org/r/92438f7ef132c731f538cebdf77850300afe04a5.1646247211.git.bristot@kernel.org Cc: Daniel Bristot de Oliveira <bristot@kernel.org> Cc: Clark Williams <williams@redhat.com> Cc: Juri Lelli <juri.lelli@redhat.com> Cc: Jonathan Corbet <corbet@lwn.net> Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
2022-02-26rtla/hist: Make -E the short version of --entriesDaniel Bristot de Oliveira1-5/+5
Currently, --entries uses -e as the short version in the hist mode of timerlat and osnoise tools. But as -e is already used to enable events on trace sessions by other tools, thus let's keep it available for the same usage for all rtla tools. Make -E the short version of --entries for hist mode on all tools. Note: rtla was merged in this merge window, so rtla was not released yet. Link: https://lkml.kernel.org/r/5dbf0cbe7364d3a05e708926b41a097c59a02b1e.1645206561.git.bristot@kernel.org Cc: Jonathan Corbet <corbet@lwn.net> Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
2022-02-08rtla: Fix segmentation fault when failing to enable -tDaniel Bristot de Oliveira1-4/+3
rtla osnoise and timerlat are causing a segmentation fault when running with the --trace option on a kernel that does not support multiple instances. For example: [root@f34 rtla]# rtla osnoise top -t failed to enable the tracer osnoise Could not enable osnoiser tracer for tracing Failed to enable the trace instance Segmentation fault (core dumped) This error happens because the exit code of the tools is trying to destroy the trace instance that failed to be created. Make osnoise_destroy_tool() aware of possible NULL osnoise_tool *, and do not attempt to destroy it. This also simplifies the exit code. Link: https://lkml.kernel.org/r/5660a2b6bf66c2655842360f2d7f6b48db5dba23.1644327249.git.bristot@kernel.org Suggested-by: Steven Rostedt <rostedt@goodmis.org> Fixes: 1eceb2fc2ca5 ("rtla/osnoise: Add osnoise top mode") Fixes: 829a6c0b5698 ("rtla/osnoise: Add the hist mode") Fixes: a828cd18bc4a ("rtla: Add timerlat tool and timelart top mode") Fixes: 1eeb6328e8b3 ("rtla/timerlat: Add timerlat hist mode") Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
2022-01-14rtla/timerlat: Add timerlat hist modeDaniel Bristot de Oliveira1-0/+822
The rtla hist hist mode displays a histogram of each tracer event occurrence, both for IRQ and timer latencies. The tool also allows many configurations of the timerlat tracer and the collection of the tracer output. Here is one example of the rtla timerlat hist mode output: ---------- %< ---------- [root@alien ~]# rtla timerlat hist -c 0-3 -d 1M # RTLA timerlat histogram # Time unit is microseconds (us) # Duration: 0 00:01:00 Index IRQ-000 Thr-000 IRQ-001 Thr-001 IRQ-002 Thr-002 IRQ-003 Thr-003 0 58572 0 59373 0 58691 0 58895 0 1 1422 57021 628 57241 1310 56160 1102 56805 2 6 2931 0 2695 0 3567 4 3031 3 1 40 0 53 0 260 0 142 4 0 7 0 5 0 6 0 17 5 0 2 0 5 0 7 0 4 6 0 0 0 2 0 1 0 1 8 0 0 0 0 0 0 0 1 over: 0 0 0 0 0 0 0 0 count: 60001 60001 60001 60001 60001 60001 60001 60001 min: 0 1 0 1 0 1 0 1 avg: 0 1 0 1 0 1 0 1 max: 3 5 1 6 1 6 2 8 ---------- >% ---------- Running - rtla timerlat hist --help provides information about the available options. Link: https://lkml.kernel.org/r/7049ed3c46b7d6aceab18ffe7770003dfc4ddceb.1639158831.git.bristot@kernel.org Cc: Tao Zhou <tao.zhou@linux.dev> Cc: Ingo Molnar <mingo@redhat.com> Cc: Tom Zanussi <zanussi@kernel.org> Cc: Masami Hiramatsu <mhiramat@kernel.org> Cc: Juri Lelli <juri.lelli@redhat.com> Cc: Clark Williams <williams@redhat.com> Cc: John Kacur <jkacur@redhat.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: Sebastian Andrzej Siewior <bigeasy@linutronix.de> Cc: Daniel Bristot de Oliveira <bristot@kernel.org> Cc: linux-rt-users@vger.kernel.org Cc: linux-trace-devel@vger.kernel.org Cc: linux-kernel@vger.kernel.org Signed-off-by: Daniel Bristot de Oliveira <bristot@kernel.org> Signed-off-by: Steven Rostedt <rostedt@goodmis.org>