1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
|
.. SPDX-License-Identifier: GPL-2.0
=================
Boot-time tracing
=================
:Author: Masami Hiramatsu <mhiramat@kernel.org>
Overview
========
Boot-time tracing allows users to trace boot-time process including
device initialization with full features of ftrace including per-event
filter and actions, histograms, kprobe-events and synthetic-events,
and trace instances.
Since kernel command line is not enough to control these complex features,
this uses bootconfig file to describe tracing feature programming.
Options in the Boot Config
==========================
Here is the list of available options list for boot time tracing in
boot config file [1]_. All options are under "ftrace." or "kernel."
prefix. See kernel parameters for the options which starts
with "kernel." prefix [2]_.
.. [1] See :ref:`Documentation/admin-guide/bootconfig.rst <bootconfig>`
.. [2] See :ref:`Documentation/admin-guide/kernel-parameters.rst <kernelparameters>`
Ftrace Global Options
---------------------
Ftrace global options have "kernel." prefix in boot config, which means
these options are passed as a part of kernel legacy command line.
kernel.tp_printk
Output trace-event data on printk buffer too.
kernel.dump_on_oops [= MODE]
Dump ftrace on Oops. If MODE = 1 or omitted, dump trace buffer
on all CPUs. If MODE = 2, dump a buffer on a CPU which kicks Oops.
kernel.traceoff_on_warning
Stop tracing if WARN_ON() occurs.
kernel.fgraph_max_depth = MAX_DEPTH
Set MAX_DEPTH to maximum depth of fgraph tracer.
kernel.fgraph_filters = FILTER[, FILTER2...]
Add fgraph tracing function filters.
kernel.fgraph_notraces = FILTER[, FILTER2...]
Add fgraph non-tracing function filters.
Ftrace Per-instance Options
---------------------------
These options can be used for each instance including global ftrace node.
ftrace.[instance.INSTANCE.]options = OPT1[, OPT2[...]]
Enable given ftrace options.
ftrace.[instance.INSTANCE.]tracing_on = 0|1
Enable/Disable tracing on this instance when starting boot-time tracing.
(you can enable it by the "traceon" event trigger action)
ftrace.[instance.INSTANCE.]trace_clock = CLOCK
Set given CLOCK to ftrace's trace_clock.
ftrace.[instance.INSTANCE.]buffer_size = SIZE
Configure ftrace buffer size to SIZE. You can use "KB" or "MB"
for that SIZE.
ftrace.[instance.INSTANCE.]alloc_snapshot
Allocate snapshot buffer.
ftrace.[instance.INSTANCE.]cpumask = CPUMASK
Set CPUMASK as trace cpu-mask.
ftrace.[instance.INSTANCE.]events = EVENT[, EVENT2[...]]
Enable given events on boot. You can use a wild card in EVENT.
ftrace.[instance.INSTANCE.]tracer = TRACER
Set TRACER to current tracer on boot. (e.g. function)
ftrace.[instance.INSTANCE.]ftrace.filters
This will take an array of tracing function filter rules.
ftrace.[instance.INSTANCE.]ftrace.notraces
This will take an array of NON-tracing function filter rules.
Ftrace Per-Event Options
------------------------
These options are setting per-event options.
ftrace.[instance.INSTANCE.]event.GROUP.EVENT.enable
Enable GROUP:EVENT tracing.
ftrace.[instance.INSTANCE.]event.GROUP.enable
Enable all event tracing within GROUP.
ftrace.[instance.INSTANCE.]event.enable
Enable all event tracing.
ftrace.[instance.INSTANCE.]event.GROUP.EVENT.filter = FILTER
Set FILTER rule to the GROUP:EVENT.
ftrace.[instance.INSTANCE.]event.GROUP.EVENT.actions = ACTION[, ACTION2[...]]
Set ACTIONs to the GROUP:EVENT.
ftrace.[instance.INSTANCE.]event.kprobes.EVENT.probes = PROBE[, PROBE2[...]]
Defines new kprobe event based on PROBEs. It is able to define
multiple probes on one event, but those must have same type of
arguments. This option is available only for the event which
group name is "kprobes".
ftrace.[instance.INSTANCE.]event.synthetic.EVENT.fields = FIELD[, FIELD2[...]]
Defines new synthetic event with FIELDs. Each field should be
"type varname".
Note that kprobe and synthetic event definitions can be written under
instance node, but those are also visible from other instances. So please
take care for event name conflict.
When to Start
=============
All boot-time tracing options starting with ``ftrace`` will be enabled at the
end of core_initcall. This means you can trace the events from postcore_initcall.
Most of the subsystems and architecture dependent drivers will be initialized
after that (arch_initcall or subsys_initcall). Thus, you can trace those with
boot-time tracing.
If you want to trace events before core_initcall, you can use the options
starting with ``kernel``. Some of them will be enabled eariler than the initcall
processing (for example,. ``kernel.ftrace=function`` and ``kernel.trace_event``
will start before the initcall.)
Examples
========
For example, to add filter and actions for each event, define kprobe
events, and synthetic events with histogram, write a boot config like
below::
ftrace.event {
task.task_newtask {
filter = "pid < 128"
enable
}
kprobes.vfs_read {
probes = "vfs_read $arg1 $arg2"
filter = "common_pid < 200"
enable
}
synthetic.initcall_latency {
fields = "unsigned long func", "u64 lat"
actions = "hist:keys=func.sym,lat:vals=lat:sort=lat"
}
initcall.initcall_start {
actions = "hist:keys=func:ts0=common_timestamp.usecs"
}
initcall.initcall_finish {
actions = "hist:keys=func:lat=common_timestamp.usecs-$ts0:onmatch(initcall.initcall_start).initcall_latency(func,$lat)"
}
}
Also, boot-time tracing supports "instance" node, which allows us to run
several tracers for different purpose at once. For example, one tracer
is for tracing functions starting with "user\_", and others tracing
"kernel\_" functions, you can write boot config as below::
ftrace.instance {
foo {
tracer = "function"
ftrace.filters = "user_*"
}
bar {
tracer = "function"
ftrace.filters = "kernel_*"
}
}
The instance node also accepts event nodes so that each instance
can customize its event tracing.
With the trigger action and kprobes, you can trace function-graph while
a function is called. For example, this will trace all function calls in
the pci_proc_init()::
ftrace {
tracing_on = 0
tracer = function_graph
event.kprobes {
start_event {
probes = "pci_proc_init"
actions = "traceon"
}
end_event {
probes = "pci_proc_init%return"
actions = "traceoff"
}
}
}
This boot-time tracing also supports ftrace kernel parameters via boot
config.
For example, following kernel parameters::
trace_options=sym-addr trace_event=initcall:* tp_printk trace_buf_size=1M ftrace=function ftrace_filter="vfs*"
This can be written in boot config like below::
kernel {
trace_options = sym-addr
trace_event = "initcall:*"
tp_printk
trace_buf_size = 1M
ftrace = function
ftrace_filter = "vfs*"
}
Note that parameters start with "kernel" prefix instead of "ftrace".
|
