Merge branch 'turbostat' of git://git.kernel.org/pub/scm/linux/kernel/git/lenb/linux

Pull turbostat updates from Len Brown:
 "Only updating the turbostat tool here, no kernel changes"

* 'turbostat' of git://git.kernel.org/pub/scm/linux/kernel/git/lenb/linux:
  tools/power turbostat: version 2022.07.28
  tools/power turbostat: do not decode ACC for ICX and SPR
  tools/power turbostat: fix SPR PC6 limits
  tools/power turbostat: cleanup 'automatic_cstate_conversion_probe()'
  tools/power turbostat: separate SPR from ICX
  tools/power turbosstat: fix comment
  tools/power turbostat: Support RAPTORLAKE P
  tools/power turbostat: add support for ALDERLAKE_N
  tools/power turbostat: dump secondary Turbo-Ratio-Limit
  tools/power turbostat: simplify dump_turbo_ratio_limits()
  tools/power turbostat: dump CPUID.7.EDX.Hybrid
  tools/power turbostat: update turbostat.8
  tools/power turbostat: Show uncore frequency
  tools/power turbostat: Fix file pointer leak
  tools/power turbostat: replace strncmp with single character compare
  tools/power turbostat: print the kernel boot commandline
  tools/power turbostat: Introduce support for RaptorLake

2 files changed, 301 insertions, 139 deletions

diff --git a/tools/power/x86/turbostat/turbostat.8 b/tools/power/x86/turbostat/turbostat.8
index 1e7d3de55a94..c7b26a3603af 100644
--- a/tools/power/x86/turbostat/turbostat.8
+++ b/tools/power/x86/turbostat/turbostat.8
@@ -92,40 +92,66 @@ displays the statistics gathered since it was forked.
 .SH ROW DESCRIPTIONS
 The system configuration dump (if --quiet is not used) is followed by statistics.  The first row of the statistics labels the content of each column (below).  The second row of statistics is the system summary line.  The system summary line has a '-' in the columns for the Package, Core, and CPU.  The contents of the system summary line depends on the type of column.  Columns that count items (eg. IRQ) show the sum across all CPUs in the system.  Columns that show a percentage show the average across all CPUs in the system.  Columns that dump raw MSR values simply show 0 in the summary.  After the system summary row, each row describes a specific Package/Core/CPU.  Note that if the --cpu parameter is used to limit which specific CPUs are displayed, turbostat will still collect statistics for all CPUs in the system and will still show the system summary for all CPUs in the system.
 .SH COLUMN DESCRIPTIONS
-.nf
+.PP
 \fBusec\fP For each CPU, the number of microseconds elapsed during counter collection, including thread migration -- if any.  This counter is disabled by default, and is enabled with "--enable usec", or --debug.  On the summary row, usec refers to the total elapsed time to collect the counters on all cpus.
+.PP
 \fBTime_Of_Day_Seconds\fP For each CPU, the gettimeofday(2) value (seconds.subsec since Epoch) when the counters ending the measurement interval were collected.  This column is disabled by default, and can be enabled with "--enable Time_Of_Day_Seconds" or "--debug".  On the summary row, Time_Of_Day_Seconds refers to the timestamp following collection of counters on the last CPU.
+.PP
 \fBCore\fP processor core number.  Note that multiple CPUs per core indicate support for Intel(R) Hyper-Threading Technology (HT).
+.PP
 \fBCPU\fP Linux CPU (logical processor) number.  Yes, it is okay that on many systems the CPUs are not listed in numerical order -- for efficiency reasons, turbostat runs in topology order, so HT siblings appear together.
+.PP
 \fBPackage\fP processor package number -- not present on systems with a single processor package.
+.PP
 \fBAvg_MHz\fP number of cycles executed divided by time elapsed.  Note that this includes idle-time when 0 instructions are executed.
+.PP
 \fBBusy%\fP percent of the measurement interval that the CPU executes instructions, aka. % of time in "C0" state.
+.PP
 \fBBzy_MHz\fP average clock rate while the CPU was not idle (ie. in "c0" state).
+.PP
 \fBTSC_MHz\fP average MHz that the TSC ran during the entire interval.
+.PP
 \fBIRQ\fP The number of interrupts serviced by that CPU during the measurement interval.  The system total line is the sum of interrupts serviced across all CPUs.  turbostat parses /proc/interrupts to generate this summary.
+.PP
 \fBSMI\fP The number of System Management Interrupts  serviced CPU during the measurement interval.  While this counter is actually per-CPU, SMI are triggered on all processors, so the number should be the same for all CPUs.
+.PP
 \fBC1, C2, C3...\fP The number times Linux requested the C1, C2, C3 idle state during the measurement interval.  The system summary line shows the sum for all CPUs.  These are C-state names as exported in /sys/devices/system/cpu/cpu*/cpuidle/state*/name.  While their names are generic, their attributes are processor specific. They the system description section of output shows what MWAIT sub-states they are mapped to on each system.
+.PP
 \fBC1%, C2%, C3%\fP The residency percentage that Linux requested C1, C2, C3....  The system summary is the average of all CPUs in the system.  Note that these are software, reflecting what was requested.  The hardware counters reflect what was actually achieved.
+.PP
 \fBCPU%c1, CPU%c3, CPU%c6, CPU%c7\fP show the percentage residency in hardware core idle states.  These numbers are from hardware residency counters.
+.PP
 \fBCoreTmp\fP Degrees Celsius reported by the per-core Digital Thermal Sensor.
+.PP
 \fBPkgTmp\fP Degrees Celsius reported by the per-package Package Thermal Monitor.
+.PP
 \fBGFX%rc6\fP The percentage of time the GPU is in the "render C6" state, rc6, during the measurement interval. From /sys/class/drm/card0/power/rc6_residency_ms.
+.PP
 \fBGFXMHz\fP Instantaneous snapshot of what sysfs presents at the end of the measurement interval. From /sys/class/graphics/fb0/device/drm/card0/gt_cur_freq_mhz.
+.PP
 \fBPkg%pc2, Pkg%pc3, Pkg%pc6, Pkg%pc7\fP percentage residency in hardware package idle states.  These numbers are from hardware residency counters.
+.PP
 \fBPkgWatt\fP Watts consumed by the whole package.
+.PP
 \fBCorWatt\fP Watts consumed by the core part of the package.
+.PP
 \fBGFXWatt\fP Watts consumed by the Graphics part of the package -- available only on client processors.
+.PP
 \fBRAMWatt\fP Watts consumed by the DRAM DIMMS -- available only on server processors.
+.PP
 \fBPKG_%\fP percent of the interval that RAPL throttling was active on the Package.  Note that the system summary is the sum of the package throttling time, and thus may be higher than 100% on a multi-package system.  Note that the meaning of this field is model specific.  For example, some hardware increments this counter when RAPL responds to thermal limits, but does not increment this counter when RAPL responds to power limits.  Comparing PkgWatt and PkgTmp to system limits is necessary.
+.PP
 \fBRAM_%\fP percent of the interval that RAPL throttling was active on DRAM.
-.fi
+.PP
+\fBUncMHz\fP uncore MHz, instantaneous sample.
 .SH TOO MUCH INFORMATION EXAMPLE
 By default, turbostat dumps all possible information -- a system configuration header, followed by columns for all counters.
 This is ideal for remote debugging, use the "--out" option to save everything to a text file, and get that file to the expert helping you debug.
 .PP
 When you are not interested in all that information, and there are several ways to see only what you want.  First the "--quiet" option will skip the configuration information, and turbostat will show only the counter columns.  Second, you can reduce the columns with the "--hide" and "--show" options.  If you use the "--show" option, then turbostat will show only the columns you list.  If you use the "--hide" option, turbostat will show all columns, except the ones you list.
 .PP
-To find out what columns are available for --show and --hide, the "--list" option is available.  For convenience, the special strings "sysfs" can be used to refer to all of the sysfs C-state counters at once:
+To find out what columns are available for --show and --hide, the "--list" option is available.  Usually, the CATEGORY names above are used to refer to groups of counters.  Also, for convenience, the special string "sysfs" can be used to refer to all of the sysfs C-state counters at once:
+.PP
 .nf
 sudo ./turbostat --show sysfs --quiet sleep 10
 10.003837 sec
@@ -158,32 +184,29 @@ Without a command to fork, turbostat displays statistics ever 5 seconds.
 Periodic output goes to stdout, by default, unless --out is used to specify an output file.
 The 5-second interval can be changed with the "-i sec" option.
 .nf
-sudo ./turbostat --quiet --hide sysfs,IRQ,SMI,CoreTmp,PkgTmp,GFX%rc6,GFXMHz,PkgWatt,CorWatt,GFXWatt
-	Core	CPU	Avg_MHz	Busy%	Bzy_MHz	TSC_MHz	CPU%c1	CPU%c3	CPU%c6	CPU%c7
-	-	-	488	12.52	3900	3498	12.50	0.00	0.00	74.98
-	0	0	5	0.13	3900	3498	99.87	0.00	0.00	0.00
-	0	4	3897	99.99	3900	3498	0.01
-	1	1	0	0.00	3856	3498	0.01	0.00	0.00	99.98
-	1	5	0	0.00	3861	3498	0.01
-	2	2	1	0.02	3889	3498	0.03	0.00	0.00	99.95
-	2	6	0	0.00	3863	3498	0.05
-	3	3	0	0.01	3869	3498	0.02	0.00	0.00	99.97
-	3	7	0	0.00	3878	3498	0.03
-	Core	CPU	Avg_MHz	Busy%	Bzy_MHz	TSC_MHz	CPU%c1	CPU%c3	CPU%c6	CPU%c7
-	-	-	491	12.59	3900	3498	12.42	0.00	0.00	74.99
-	0	0	27	0.69	3900	3498	99.31	0.00	0.00	0.00
-	0	4	3898	99.99	3900	3498	0.01
-	1	1	0	0.00	3883	3498	0.01	0.00	0.00	99.99
-	1	5	0	0.00	3898	3498	0.01
-	2	2	0	0.01	3889	3498	0.02	0.00	0.00	99.98
-	2	6	0	0.00	3889	3498	0.02
-	3	3	0	0.00	3856	3498	0.01	0.00	0.00	99.99
-	3	7	0	0.00	3897	3498	0.01
+sudo turbostat --quiet --show CPU,frequency
+	Core	CPU	Avg_MHz	Busy%	Bzy_MHz	TSC_MHz	CPU%c7	UncMhz
+	-	-	524	12.48	4198	3096	74.53	3800
+	0	0	4	0.09	4081	3096	98.88	3800
+	0	4	1	0.02	4063	3096
+	1	1	2	0.06	4063	3096	99.60
+	1	5	2	0.05	4070	3096
+	2	2	4178	99.52	4199	3096	0.00
+	2	6	3	0.08	4159	3096
+	3	3	1	0.04	4046	3096	99.66
+	3	7	0	0.01	3989	3096
+	Core	CPU	Avg_MHz	Busy%	Bzy_MHz	TSC_MHz	CPU%c7	UncMhz
+	-	-	525	12.52	4198	3096	74.54	3800
+	0	0	4	0.10	4051	3096	99.49	3800
+	0	4	2	0.04	3993	3096
+	1	1	3	0.07	4054	3096	99.56
+	1	5	4	0.10	4018	3096
+	2	2	4178	99.51	4199	3096	0.00
+	2	6	4	0.09	4143	3096
+	3	3	2	0.06	4026	3096	99.10
+	3	7	7	0.17	4074	3096
 .fi
-This example also shows the use of the --hide option to skip columns that are not wanted.
-Note that cpu4 in this example is 99.99% busy, while the other CPUs are all under 1% busy.
-Notice that cpu4's HT sibling is cpu0, which is under 1% busy, but can get into CPU%c1 only,
-because its cpu4's activity on shared hardware keeps it from entering a deeper C-state.
+This example also shows the use of the --show option to show only the desired columns.
 
 .SH SYSTEM CONFIGURATION INFORMATION EXAMPLE
 
@@ -191,61 +214,86 @@ By default, turbostat always dumps system configuration information
 before taking measurements.  In the example above, "--quiet" is used
 to suppress that output.  Here is an example of the configuration information:
 .nf
-turbostat version 2017.02.15 - Len Brown <lenb@kernel.org>
-CPUID(0): GenuineIntel 13 CPUID levels; family:model:stepping 0x6:3c:3 (6:60:3)
-CPUID(1): SSE3 MONITOR - EIST TM2 TSC MSR ACPI-TM TM
-CPUID(6): APERF, TURBO, DTS, PTM, No-HWP, No-HWPnotify, No-HWPwindow, No-HWPepp, No-HWPpkg, EPB
-cpu4: MSR_IA32_MISC_ENABLE: 0x00850089 (TCC EIST No-MWAIT PREFETCH TURBO)
-CPUID(7): No-SGX
-cpu4: MSR_MISC_PWR_MGMT: 0x00400000 (ENable-EIST_Coordination DISable-EPB DISable-OOB)
-RAPL: 3121 sec. Joule Counter Range, at 84 Watts
-cpu4: MSR_PLATFORM_INFO: 0x80838f3012300
+turbostat version 2022.04.16 - Len Brown <lenb@kernel.org>
+Kernel command line: BOOT_IMAGE=/boot/vmlinuz-5.18.0-rc6-00001-ge6891250e3b5 ...
+CPUID(0): GenuineIntel 0x16 CPUID levels
+CPUID(1): family:model:stepping 0x6:9e:9 (6:158:9) microcode 0xea
+CPUID(0x80000000): max_extended_levels: 0x80000008
+CPUID(1): SSE3 MONITOR - EIST TM2 TSC MSR ACPI-TM HT TM
+CPUID(6): APERF, TURBO, DTS, PTM, HWP, HWPnotify, HWPwindow, HWPepp, No-HWPpkg, EPB
+cpu7: MSR_IA32_MISC_ENABLE: 0x00850089 (TCC EIST MWAIT PREFETCH TURBO)
+CPUID(7): SGX
+cpu7: MSR_IA32_FEATURE_CONTROL: 0x00000005 (Locked )
+CPUID(0x15): eax_crystal: 2 ebx_tsc: 258 ecx_crystal_hz: 0
+TSC: 3096 MHz (24000000 Hz * 258 / 2 / 1000000)
+CPUID(0x16): base_mhz: 3100 max_mhz: 4200 bus_mhz: 100
+cpu7: MSR_MISC_PWR_MGMT: 0x00401cc0 (ENable-EIST_Coordination DISable-EPB DISable-OOB)
+RAPL: 5825 sec. Joule Counter Range, at 45 Watts
+cpu7: MSR_PLATFORM_INFO: 0x80839f1011f00
 8 * 100.0 = 800.0 MHz max efficiency frequency
-35 * 100.0 = 3500.0 MHz base frequency
-cpu4: MSR_IA32_POWER_CTL: 0x0004005d (C1E auto-promotion: DISabled)
-cpu4: MSR_TURBO_RATIO_LIMIT: 0x25262727
-37 * 100.0 = 3700.0 MHz max turbo 4 active cores
-38 * 100.0 = 3800.0 MHz max turbo 3 active cores
-39 * 100.0 = 3900.0 MHz max turbo 2 active cores
-39 * 100.0 = 3900.0 MHz max turbo 1 active cores
-cpu4: MSR_CONFIG_TDP_NOMINAL: 0x00000023 (base_ratio=35)
-cpu4: MSR_CONFIG_TDP_LEVEL_1: 0x00000000 ()
-cpu4: MSR_CONFIG_TDP_LEVEL_2: 0x00000000 ()
-cpu4: MSR_CONFIG_TDP_CONTROL: 0x80000000 ( lock=1)
-cpu4: MSR_TURBO_ACTIVATION_RATIO: 0x00000000 (MAX_NON_TURBO_RATIO=0 lock=0)
-cpu4: MSR_PKG_CST_CONFIG_CONTROL: 0x1e000400 (UNdemote-C3, UNdemote-C1, demote-C3, demote-C1, UNlocked: pkg-cstate-limit=0: pc0)
-cpu4: POLL: CPUIDLE CORE POLL IDLE
-cpu4: C1: MWAIT 0x00
-cpu4: C1E: MWAIT 0x01
-cpu4: C3: MWAIT 0x10
-cpu4: C6: MWAIT 0x20
-cpu4: C7s: MWAIT 0x32
-cpu4: MSR_MISC_FEATURE_CONTROL: 0x00000000 (L2-Prefetch L2-Prefetch-pair L1-Prefetch L1-IP-Prefetch)
-cpu0: MSR_IA32_ENERGY_PERF_BIAS: 0x00000006 (balanced)
-cpu0: MSR_CORE_PERF_LIMIT_REASONS, 0x31200000 (Active: ) (Logged: Transitions, MultiCoreTurbo, Amps, Auto-HWP, )
-cpu0: MSR_GFX_PERF_LIMIT_REASONS, 0x00000000 (Active: ) (Logged: )
-cpu0: MSR_RING_PERF_LIMIT_REASONS, 0x0d000000 (Active: ) (Logged: Amps, PkgPwrL1, PkgPwrL2, )
+31 * 100.0 = 3100.0 MHz base frequency
+cpu7: MSR_IA32_POWER_CTL: 0x002c005d (C1E auto-promotion: DISabled)
+cpu7: MSR_TURBO_RATIO_LIMIT: 0x2728292a
+39 * 100.0 = 3900.0 MHz max turbo 4 active cores
+40 * 100.0 = 4000.0 MHz max turbo 3 active cores
+41 * 100.0 = 4100.0 MHz max turbo 2 active cores
+42 * 100.0 = 4200.0 MHz max turbo 1 active cores
+cpu7: MSR_CONFIG_TDP_NOMINAL: 0x0000001f (base_ratio=31)
+cpu7: MSR_CONFIG_TDP_LEVEL_1: 0x00000000 ()
+cpu7: MSR_CONFIG_TDP_LEVEL_2: 0x00000000 ()
+cpu7: MSR_CONFIG_TDP_CONTROL: 0x80000000 ( lock=1)
+cpu7: MSR_TURBO_ACTIVATION_RATIO: 0x00000000 (MAX_NON_TURBO_RATIO=0 lock=0)
+cpu7: MSR_PKG_CST_CONFIG_CONTROL: 0x1e008008 (UNdemote-C3, UNdemote-C1, demote-C3, demote-C1, locked, pkg-cstate-limit=8 (unlimited))
+Uncore Frequency pkg0 die0: 800 - 3900 MHz (800 - 3900 MHz)
+/dev/cpu_dma_latency: 2000000000 usec (default)
+current_driver: intel_idle
+current_governor: menu
+current_governor_ro: menu
+cpu7: POLL: CPUIDLE CORE POLL IDLE
+cpu7: C1: MWAIT 0x00
+cpu7: C1E: MWAIT 0x01
+cpu7: C3: MWAIT 0x10
+cpu7: C6: MWAIT 0x20
+cpu7: C7s: MWAIT 0x33
+cpu7: C8: MWAIT 0x40
+cpu7: C9: MWAIT 0x50
+cpu7: C10: MWAIT 0x60
+cpu7: cpufreq driver: intel_pstate
+cpu7: cpufreq governor: performance
+cpufreq intel_pstate no_turbo: 0
+cpu7: MSR_MISC_FEATURE_CONTROL: 0x00000000 (L2-Prefetch L2-Prefetch-pair L1-Prefetch L1-IP-Prefetch)
+cpu0: MSR_PM_ENABLE: 0x00000001 (HWP)
+cpu0: MSR_HWP_CAPABILITIES: 0x01101f53 (high 83 guar 31 eff 16 low 1)
+cpu0: MSR_HWP_REQUEST: 0x00005353 (min 83 max 83 des 0 epp 0x0 window 0x0 pkg 0x0)
+cpu0: MSR_HWP_INTERRUPT: 0x00000001 (EN_Guaranteed_Perf_Change, Dis_Excursion_Min)
+cpu0: MSR_HWP_STATUS: 0x00000004 (No-Guaranteed_Perf_Change, No-Excursion_Min)
+cpu0: EPB: 6 (balanced)
 cpu0: MSR_RAPL_POWER_UNIT: 0x000a0e03 (0.125000 Watts, 0.000061 Joules, 0.000977 sec.)
-cpu0: MSR_PKG_POWER_INFO: 0x000002a0 (84 W TDP, RAPL 0 - 0 W, 0.000000 sec.)
-cpu0: MSR_PKG_POWER_LIMIT: 0x428348001a82a0 (UNlocked)
-cpu0: PKG Limit #1: ENabled (84.000000 Watts, 8.000000 sec, clamp DISabled)
-cpu0: PKG Limit #2: ENabled (105.000000 Watts, 0.002441* sec, clamp DISabled)
+cpu0: MSR_PKG_POWER_INFO: 0x00000168 (45 W TDP, RAPL 0 - 0 W, 0.000000 sec.)
+cpu0: MSR_PKG_POWER_LIMIT: 0x42820800218208 (UNlocked)
+cpu0: PKG Limit #1: ENabled (65.000 Watts, 64.000000 sec, clamp ENabled)
+cpu0: PKG Limit #2: ENabled (65.000 Watts, 0.002441* sec, clamp DISabled)
+cpu0: MSR_VR_CURRENT_CONFIG: 0x00000000
+cpu0: PKG Limit #4: 0.000000 Watts (UNlocked)
+cpu0: MSR_DRAM_POWER_LIMIT: 0x5400de00000000 (UNlocked)
+cpu0: DRAM Limit: DISabled (0.000 Watts, 0.000977 sec, clamp DISabled)
 cpu0: MSR_PP0_POLICY: 0
 cpu0: MSR_PP0_POWER_LIMIT: 0x00000000 (UNlocked)
-cpu0: Cores Limit: DISabled (0.000000 Watts, 0.000977 sec, clamp DISabled)
+cpu0: Cores Limit: DISabled (0.000 Watts, 0.000977 sec, clamp DISabled)
 cpu0: MSR_PP1_POLICY: 0
 cpu0: MSR_PP1_POWER_LIMIT: 0x00000000 (UNlocked)
-cpu0: GFX Limit: DISabled (0.000000 Watts, 0.000977 sec, clamp DISabled)
-cpu0: MSR_IA32_TEMPERATURE_TARGET: 0x00641400 (100 C)
-cpu0: MSR_IA32_PACKAGE_THERM_STATUS: 0x884c0800 (24 C)
-cpu0: MSR_IA32_THERM_STATUS: 0x884c0000 (24 C +/- 1)
-cpu1: MSR_IA32_THERM_STATUS: 0x88510000 (19 C +/- 1)
-cpu2: MSR_IA32_THERM_STATUS: 0x884e0000 (22 C +/- 1)
-cpu3: MSR_IA32_THERM_STATUS: 0x88510000 (19 C +/- 1)
-cpu4: MSR_PKGC3_IRTL: 0x00008842 (valid, 67584 ns)
-cpu4: MSR_PKGC6_IRTL: 0x00008873 (valid, 117760 ns)
-cpu4: MSR_PKGC7_IRTL: 0x00008891 (valid, 148480 ns)
+cpu0: GFX Limit: DISabled (0.000 Watts, 0.000977 sec, clamp DISabled)
+cpu0: MSR_IA32_TEMPERATURE_TARGET: 0x00640000 (100 C) (100 default - 0 offset)
+cpu0: MSR_IA32_PACKAGE_THERM_STATUS: 0x88200800 (68 C)
+cpu0: MSR_IA32_PACKAGE_THERM_INTERRUPT: 0x00000003 (100 C, 100 C)
+cpu7: MSR_PKGC3_IRTL: 0x0000884e (valid, 79872 ns)
+cpu7: MSR_PKGC6_IRTL: 0x00008876 (valid, 120832 ns)
+cpu7: MSR_PKGC7_IRTL: 0x00008894 (valid, 151552 ns)
+cpu7: MSR_PKGC8_IRTL: 0x000088fa (valid, 256000 ns)
+cpu7: MSR_PKGC9_IRTL: 0x0000894c (valid, 339968 ns)
+cpu7: MSR_PKGC10_IRTL: 0x00008bf2 (valid, 1034240 ns)
 .fi
+.PP
 The \fBmax efficiency\fP frequency, a.k.a. Low Frequency Mode, is the frequency
 available at the minimum package voltage.  The \fBTSC frequency\fP is the base
 frequency of the processor -- this should match the brand string
diff --git a/tools/power/x86/turbostat/turbostat.c b/tools/power/x86/turbostat/turbostat.c
index ede31a4287a0..831dc32d45fa 100644
--- a/tools/power/x86/turbostat/turbostat.c
+++ b/tools/power/x86/turbostat/turbostat.c
@@ -126,6 +126,7 @@ struct msr_counter bic[] = {
 	{ 0x0, "GFXAMHz", "", 0, 0, 0, NULL, 0 },
 	{ 0x0, "IPC", "", 0, 0, 0, NULL, 0 },
 	{ 0x0, "CoreThr", "", 0, 0, 0, NULL, 0 },
+	{ 0x0, "UncMHz", "", 0, 0, 0, NULL, 0 },
 };
 
 #define MAX_BIC (sizeof(bic) / sizeof(struct msr_counter))
@@ -183,10 +184,11 @@ struct msr_counter bic[] = {
 #define	BIC_GFXACTMHz	(1ULL << 51)
 #define	BIC_IPC		(1ULL << 52)
 #define	BIC_CORE_THROT_CNT	(1ULL << 53)
+#define	BIC_UNCORE_MHZ		(1ULL << 54)
 
 #define BIC_TOPOLOGY (BIC_Package | BIC_Node | BIC_CoreCnt | BIC_PkgCnt | BIC_Core | BIC_CPU | BIC_Die )
 #define BIC_THERMAL_PWR ( BIC_CoreTmp | BIC_PkgTmp | BIC_PkgWatt | BIC_CorWatt | BIC_GFXWatt | BIC_RAMWatt | BIC_PKG__ | BIC_RAM__)
-#define BIC_FREQUENCY ( BIC_Avg_MHz | BIC_Busy | BIC_Bzy_MHz | BIC_TSC_MHz | BIC_GFXMHz | BIC_GFXACTMHz )
+#define BIC_FREQUENCY ( BIC_Avg_MHz | BIC_Busy | BIC_Bzy_MHz | BIC_TSC_MHz | BIC_GFXMHz | BIC_GFXACTMHz | BIC_UNCORE_MHZ)
 #define BIC_IDLE ( BIC_sysfs | BIC_CPU_c1 | BIC_CPU_c3 | BIC_CPU_c6 | BIC_CPU_c7 | BIC_GFX_rc6 | BIC_Pkgpc2 | BIC_Pkgpc3 | BIC_Pkgpc6 | BIC_Pkgpc7 | BIC_Pkgpc8 | BIC_Pkgpc9 | BIC_Pkgpc10 | BIC_CPU_LPI | BIC_SYS_LPI | BIC_Mod_c6 | BIC_Totl_c0 | BIC_Any_c0 | BIC_GFX_c0 | BIC_CPUGFX)
 #define BIC_OTHER ( BIC_IRQ | BIC_SMI | BIC_ThreadC | BIC_CoreTmp | BIC_IPC)
 
@@ -228,6 +230,7 @@ unsigned int do_slm_cstates;
 unsigned int use_c1_residency_msr;
 unsigned int has_aperf;
 unsigned int has_epb;
+unsigned int is_hybrid;
 unsigned int do_irtl_snb;
 unsigned int do_irtl_hsw;
 unsigned int units = 1000000;	/* MHz etc */
@@ -393,6 +396,7 @@ struct pkg_data {
 	unsigned long long rapl_pkg_perf_status;	/* MSR_PKG_PERF_STATUS */
 	unsigned long long rapl_dram_perf_status;	/* MSR_DRAM_PERF_STATUS */
 	unsigned int pkg_temp_c;
+	unsigned int uncore_mhz;
 	unsigned long long counter[MAX_ADDED_COUNTERS];
 } *package_even, *package_odd;
 
@@ -988,6 +992,9 @@ void print_header(char *delim)
 		if (DO_BIC(BIC_RAM__))
 			outp += sprintf(outp, "%sRAM_%%", (printed++ ? delim : ""));
 	}
+	if (DO_BIC(BIC_UNCORE_MHZ))
+		outp += sprintf(outp, "%sUncMHz", (printed++ ? delim : ""));
+
 	for (mp = sys.pp; mp; mp = mp->next) {
 		if (mp->format == FORMAT_RAW) {
 			if (mp->width == 64)
@@ -1370,6 +1377,9 @@ int format_counters(struct thread_data *t, struct core_data *c, struct pkg_data
 		outp +=
 		    sprintf(outp, fmt8, (printed++ ? delim : ""),
 			    100.0 * p->rapl_dram_perf_status * rapl_time_units / interval_float);
+	/* UncMHz */
+	if (DO_BIC(BIC_UNCORE_MHZ))
+		outp += sprintf(outp, "%s%d", (printed++ ? delim : ""), p->uncore_mhz);
 
 	for (i = 0, mp = sys.pp; mp; i++, mp = mp->next) {
 		if (mp->format == FORMAT_RAW) {
@@ -1471,6 +1481,7 @@ int delta_package(struct pkg_data *new, struct pkg_data *old)
 	else
 		old->gfx_rc6_ms = new->gfx_rc6_ms - old->gfx_rc6_ms;
 
+	old->uncore_mhz = new->uncore_mhz;
 	old->gfx_mhz = new->gfx_mhz;
 	old->gfx_act_mhz = new->gfx_act_mhz;
 
@@ -1689,6 +1700,7 @@ void clear_counters(struct thread_data *t, struct core_data *c, struct pkg_data
 	p->pkg_temp_c = 0;
 
 	p->gfx_rc6_ms = 0;
+	p->uncore_mhz = 0;
 	p->gfx_mhz = 0;
 	p->gfx_act_mhz = 0;
 	for (i = 0, mp = sys.tp; mp; i++, mp = mp->next)
@@ -1788,6 +1800,7 @@ int sum_counters(struct thread_data *t, struct core_data *c, struct pkg_data *p)
 	average.packages.energy_gfx += p->energy_gfx;
 
 	average.packages.gfx_rc6_ms = p->gfx_rc6_ms;
+	average.packages.uncore_mhz = p->uncore_mhz;
 	average.packages.gfx_mhz = p->gfx_mhz;
 	average.packages.gfx_act_mhz = p->gfx_act_mhz;
 
@@ -1948,6 +1961,16 @@ int get_mp(int cpu, struct msr_counter *mp, unsigned long long *counterp)
 	return 0;
 }
 
+unsigned long long get_uncore_mhz(int package, int die)
+{
+	char path[128];
+
+	sprintf(path, "/sys/devices/system/cpu/intel_uncore_frequency/package_0%d_die_0%d/current_freq_khz", package,
+		die);
+
+	return (snapshot_sysfs_counter(path) / 1000);
+}
+
 int get_epb(int cpu)
 {
 	char path[128 + PATH_BYTES];
@@ -2035,9 +2058,9 @@ int get_core_throt_cnt(int cpu, unsigned long long *cnt)
 	if (!fp)
 		return -1;
 	ret = fscanf(fp, "%lld", &tmp);
+	fclose(fp);
 	if (ret != 1)
 		return -1;
-	fclose(fp);
 	*cnt = tmp;
 
 	return 0;
@@ -2297,6 +2320,10 @@ retry:
 	if (DO_BIC(BIC_GFX_rc6))
 		p->gfx_rc6_ms = gfx_cur_rc6_ms;
 
+	/* n.b. assume die0 uncore frequency applies to whole package */
+	if (DO_BIC(BIC_UNCORE_MHZ))
+		p->uncore_mhz = get_uncore_mhz(p->package_id, 0);
+
 	if (DO_BIC(BIC_GFXMHz))
 		p->gfx_mhz = gfx_cur_mhz;
 
@@ -2494,6 +2521,7 @@ int has_turbo_ratio_group_limits(int family, int model)
 	case INTEL_FAM6_ATOM_GOLDMONT:
 	case INTEL_FAM6_SKYLAKE_X:
 	case INTEL_FAM6_ICELAKE_X:
+	case INTEL_FAM6_SAPPHIRERAPIDS_X:
 	case INTEL_FAM6_ATOM_GOLDMONT_D:
 	case INTEL_FAM6_ATOM_TREMONT_D:
 		return 1;
@@ -2502,13 +2530,14 @@ int has_turbo_ratio_group_limits(int family, int model)
 	}
 }
 
-static void dump_turbo_ratio_limits(int family, int model)
+static void dump_turbo_ratio_limits(int trl_msr_offset, int family, int model)
 {
 	unsigned long long msr, core_counts;
-	unsigned int ratio, group_size;
+	int shift;
 
-	get_msr(base_cpu, MSR_TURBO_RATIO_LIMIT, &msr);
-	fprintf(outf, "cpu%d: MSR_TURBO_RATIO_LIMIT: 0x%08llx\n", base_cpu, msr);
+	get_msr(base_cpu, trl_msr_offset, &msr);
+	fprintf(outf, "cpu%d: MSR_%sTURBO_RATIO_LIMIT: 0x%08llx\n",
+		base_cpu, trl_msr_offset == MSR_SECONDARY_TURBO_RATIO_LIMIT ? "SECONDARY" : "", msr);
 
 	if (has_turbo_ratio_group_limits(family, model)) {
 		get_msr(base_cpu, MSR_TURBO_RATIO_LIMIT1, &core_counts);
@@ -2517,53 +2546,16 @@ static void dump_turbo_ratio_limits(int family, int model)
 		core_counts = 0x0807060504030201;
 	}
 
-	ratio = (msr >> 56) & 0xFF;
-	group_size = (core_counts >> 56) & 0xFF;
-	if (ratio)
-		fprintf(outf, "%d * %.1f = %.1f MHz max turbo %d active cores\n",
-			ratio, bclk, ratio * bclk, group_size);
+	for (shift = 56; shift >= 0; shift -= 8) {
+		unsigned int ratio, group_size;
 
-	ratio = (msr >> 48) & 0xFF;
-	group_size = (core_counts >> 48) & 0xFF;
-	if (ratio)
-		fprintf(outf, "%d * %.1f = %.1f MHz max turbo %d active cores\n",
-			ratio, bclk, ratio * bclk, group_size);
-
-	ratio = (msr >> 40) & 0xFF;
-	group_size = (core_counts >> 40) & 0xFF;
-	if (ratio)
-		fprintf(outf, "%d * %.1f = %.1f MHz max turbo %d active cores\n",
-			ratio, bclk, ratio * bclk, group_size);
-
-	ratio = (msr >> 32) & 0xFF;
-	group_size = (core_counts >> 32) & 0xFF;
-	if (ratio)
-		fprintf(outf, "%d * %.1f = %.1f MHz max turbo %d active cores\n",
-			ratio, bclk, ratio * bclk, group_size);
-
-	ratio = (msr >> 24) & 0xFF;
-	group_size = (core_counts >> 24) & 0xFF;
-	if (ratio)
-		fprintf(outf, "%d * %.1f = %.1f MHz max turbo %d active cores\n",
-			ratio, bclk, ratio * bclk, group_size);
-
-	ratio = (msr >> 16) & 0xFF;
-	group_size = (core_counts >> 16) & 0xFF;
-	if (ratio)
-		fprintf(outf, "%d * %.1f = %.1f MHz max turbo %d active cores\n",
-			ratio, bclk, ratio * bclk, group_size);
-
-	ratio = (msr >> 8) & 0xFF;
-	group_size = (core_counts >> 8) & 0xFF;
-	if (ratio)
-		fprintf(outf, "%d * %.1f = %.1f MHz max turbo %d active cores\n",
-			ratio, bclk, ratio * bclk, group_size);
+		ratio = (msr >> shift) & 0xFF;
+		group_size = (core_counts >> shift) & 0xFF;
+		if (ratio)
+			fprintf(outf, "%d * %.1f = %.1f MHz max turbo %d active cores\n",
+				ratio, bclk, ratio * bclk, group_size);
+	}
 
-	ratio = (msr >> 0) & 0xFF;
-	group_size = (core_counts >> 0) & 0xFF;
-	if (ratio)
-		fprintf(outf, "%d * %.1f = %.1f MHz max turbo %d active cores\n",
-			ratio, bclk, ratio * bclk, group_size);
 	return;
 }
 
@@ -2976,7 +2968,7 @@ int get_thread_siblings(struct cpu_topology *thiscpu)
 				}
 			}
 		}
-	} while (!strncmp(&character, ",", 1));
+	} while (character == ',');
 	fclose(filep);
 
 	return CPU_COUNT_S(size, thiscpu->put_ids);
@@ -3742,6 +3734,7 @@ int probe_nhm_msrs(unsigned int family, unsigned int model)
 		has_misc_feature_control = 1;
 		break;
 	case INTEL_FAM6_SKYLAKE_X:	/* SKX */
+	case INTEL_FAM6_SAPPHIRERAPIDS_X:	/* SPR */
 		pkg_cstate_limits = skx_pkg_cstate_limits;
 		has_misc_feature_control = 1;
 		break;
@@ -3871,6 +3864,22 @@ int is_icx(unsigned int family, unsigned int model)
 	return 0;
 }
 
+int is_spr(unsigned int family, unsigned int model)
+{
+
+	if (!genuine_intel)
+		return 0;
+
+	if (family != 6)
+		return 0;
+
+	switch (model) {
+	case INTEL_FAM6_SAPPHIRERAPIDS_X:
+		return 1;
+	}
+	return 0;
+}
+
 int is_ehl(unsigned int family, unsigned int model)
 {
 	if (!genuine_intel)
@@ -3988,6 +3997,7 @@ int has_glm_turbo_ratio_limit(unsigned int family, unsigned int model)
 	case INTEL_FAM6_ATOM_GOLDMONT:
 	case INTEL_FAM6_SKYLAKE_X:
 	case INTEL_FAM6_ICELAKE_X:
+	case INTEL_FAM6_SAPPHIRERAPIDS_X:
 		return 1;
 	default:
 		return 0;
@@ -4015,7 +4025,7 @@ int has_config_tdp(unsigned int family, unsigned int model)
 	case INTEL_FAM6_CANNONLAKE_L:	/* CNL */
 	case INTEL_FAM6_SKYLAKE_X:	/* SKX */
 	case INTEL_FAM6_ICELAKE_X:	/* ICX */
-
+	case INTEL_FAM6_SAPPHIRERAPIDS_X:	/* SPR */
 	case INTEL_FAM6_XEON_PHI_KNL:	/* Knights Landing */
 		return 1;
 	default:
@@ -4083,8 +4093,12 @@ static void dump_cstate_pstate_config_info(unsigned int family, unsigned int mod
 	if (has_ivt_turbo_ratio_limit(family, model))
 		dump_ivt_turbo_ratio_limits();
 
-	if (has_turbo_ratio_limit(family, model))
-		dump_turbo_ratio_limits(family, model);
+	if (has_turbo_ratio_limit(family, model)) {
+		dump_turbo_ratio_limits(MSR_TURBO_RATIO_LIMIT, family, model);
+
+		if (is_hybrid)
+			dump_turbo_ratio_limits(MSR_SECONDARY_TURBO_RATIO_LIMIT, family, model);
+	}
 
 	if (has_atom_turbo_ratio_limit(family, model))
 		dump_atom_turbo_ratio_limits();
@@ -4098,6 +4112,24 @@ static void dump_cstate_pstate_config_info(unsigned int family, unsigned int mod
 	dump_nhm_cst_cfg();
 }
 
+static int read_sysfs_int(char *path)
+{
+	FILE *input;
+	int retval = -1;
+
+	input = fopen(path, "r");
+	if (input == NULL) {
+		if (debug)
+			fprintf(outf, "NSFOD %s\n", path);
+		return (-1);
+	}
+	if (fscanf(input, "%d", &retval) != 1)
+		err(1, "%s: failed to read int from file", path);
+	fclose(input);
+
+	return (retval);
+}
+
 static void dump_sysfs_file(char *path)
 {
 	FILE *input;
@@ -4116,6 +4148,48 @@ static void dump_sysfs_file(char *path)
 	fprintf(outf, "%s: %s", strrchr(path, '/') + 1, cpuidle_buf);
 }
 
+static void intel_uncore_frequency_probe(void)
+{
+	int i, j;
+	char path[128];
+
+	if (!genuine_intel)
+		return;
+
+	if (access("/sys/devices/system/cpu/intel_uncore_frequency/package_00_die_00", R_OK))
+		return;
+
+	if (!access("/sys/devices/system/cpu/intel_uncore_frequency/package_00_die_00/current_freq_khz", R_OK))
+		BIC_PRESENT(BIC_UNCORE_MHZ);
+
+	if (quiet)
+		return;
+
+	for (i = 0; i < topo.num_packages; ++i) {
+		for (j = 0; j < topo.num_die; ++j) {
+			int k, l;
+
+			sprintf(path, "/sys/devices/system/cpu/intel_uncore_frequency/package_0%d_die_0%d/min_freq_khz",
+				i, j);
+			k = read_sysfs_int(path);
+			sprintf(path, "/sys/devices/system/cpu/intel_uncore_frequency/package_0%d_die_0%d/max_freq_khz",
+				i, j);
+			l = read_sysfs_int(path);
+			fprintf(outf, "Uncore Frequency pkg%d die%d: %d - %d MHz ", i, j, k / 1000, l / 1000);
+
+			sprintf(path,
+				"/sys/devices/system/cpu/intel_uncore_frequency/package_0%d_die_0%d/initial_min_freq_khz",
+				i, j);
+			k = read_sysfs_int(path);
+			sprintf(path,
+				"/sys/devices/system/cpu/intel_uncore_frequency/package_0%d_die_0%d/initial_max_freq_khz",
+				i, j);
+			l = read_sysfs_int(path);
+			fprintf(outf, "(%d - %d MHz)\n", k / 1000, l / 1000);
+		}
+	}
+}
+
 static void dump_sysfs_cstate_config(void)
 {
 	char path[64];
@@ -4486,6 +4560,7 @@ static double rapl_dram_energy_units_probe(int model, double rapl_energy_units)
 	case INTEL_FAM6_SKYLAKE_X:	/* SKX */
 	case INTEL_FAM6_XEON_PHI_KNL:	/* KNL */
 	case INTEL_FAM6_ICELAKE_X:	/* ICX */
+	case INTEL_FAM6_SAPPHIRERAPIDS_X:	/* SPR */
 		return (rapl_dram_energy_units = 15.3 / 1000000);
 	default:
 		return (rapl_energy_units);
@@ -4575,6 +4650,7 @@ void rapl_probe_intel(unsigned int family, unsigned int model)
 	case INTEL_FAM6_BROADWELL_X:	/* BDX */
 	case INTEL_FAM6_SKYLAKE_X:	/* SKX */
 	case INTEL_FAM6_ICELAKE_X:	/* ICX */
+	case INTEL_FAM6_SAPPHIRERAPIDS_X:	/* SPR */
 	case INTEL_FAM6_XEON_PHI_KNL:	/* KNL */
 		do_rapl =
 		    RAPL_PKG | RAPL_DRAM | RAPL_DRAM_POWER_INFO | RAPL_DRAM_PERF_STATUS | RAPL_PKG_PERF_STATUS |
@@ -4740,13 +4816,19 @@ void perf_limit_reasons_probe(unsigned int family, unsigned int model)
 
 void automatic_cstate_conversion_probe(unsigned int family, unsigned int model)
 {
-	if (is_skx(family, model) || is_bdx(family, model) || is_icx(family, model))
+	if (family != 6)
+		return;
+
+	switch (model) {
+	case INTEL_FAM6_BROADWELL_X:
+	case INTEL_FAM6_SKYLAKE_X:
 		has_automatic_cstate_conversion = 1;
+	}
 }
 
 void prewake_cstate_probe(unsigned int family, unsigned int model)
 {
-	if (is_icx(family, model))
+	if (is_icx(family, model) || is_spr(family, model))
 		dis_cstate_prewake = 1;
 }
 
@@ -4975,6 +5057,7 @@ int has_snb_msrs(unsigned int family, unsigned int model)
 	case INTEL_FAM6_CANNONLAKE_L:	/* CNL */
 	case INTEL_FAM6_SKYLAKE_X:	/* SKX */
 	case INTEL_FAM6_ICELAKE_X:	/* ICX */
+	case INTEL_FAM6_SAPPHIRERAPIDS_X:	/* SPR */
 	case INTEL_FAM6_ATOM_GOLDMONT:	/* BXT */
 	case INTEL_FAM6_ATOM_GOLDMONT_PLUS:
 	case INTEL_FAM6_ATOM_GOLDMONT_D:	/* DNV */
@@ -5361,13 +5444,15 @@ unsigned int intel_model_duplicates(unsigned int model)
 	case INTEL_FAM6_LAKEFIELD:
 	case INTEL_FAM6_ALDERLAKE:
 	case INTEL_FAM6_ALDERLAKE_L:
+	case INTEL_FAM6_ALDERLAKE_N:
+	case INTEL_FAM6_RAPTORLAKE:
+	case INTEL_FAM6_RAPTORLAKE_P:
 		return INTEL_FAM6_CANNONLAKE_L;
 
 	case INTEL_FAM6_ATOM_TREMONT_L:
 		return INTEL_FAM6_ATOM_TREMONT;
 
 	case INTEL_FAM6_ICELAKE_D:
-	case INTEL_FAM6_SAPPHIRERAPIDS_X:
 		return INTEL_FAM6_ICELAKE_X;
 	}
 	return model;
@@ -5398,7 +5483,7 @@ void print_dev_latency(void)
 }
 
 /*
- * Linux-perf manages the the HW instructions-retired counter
+ * Linux-perf manages the HW instructions-retired counter
  * by enabling when requested, and hiding rollover
  */
 void linux_perf_init(void)
@@ -5543,7 +5628,10 @@ void process_cpuid()
 		__cpuid_count(0x7, 0, eax, ebx, ecx, edx);
 
 		has_sgx = ebx & (1 << 2);
-		fprintf(outf, "CPUID(7): %sSGX\n", has_sgx ? "" : "No-");
+
+		is_hybrid = edx & (1 << 15);
+
+		fprintf(outf, "CPUID(7): %sSGX %sHybrid\n", has_sgx ? "" : "No-", is_hybrid ? "" : "No-");
 
 		if (has_sgx)
 			decode_feature_control_msr();
@@ -5654,7 +5742,7 @@ void process_cpuid()
 		BIC_NOT_PRESENT(BIC_Pkgpc7);
 		use_c1_residency_msr = 1;
 	}
-	if (is_skx(family, model) || is_icx(family, model)) {
+	if (is_skx(family, model) || is_icx(family, model) || is_spr(family, model)) {
 		BIC_NOT_PRESENT(BIC_CPU_c3);
 		BIC_NOT_PRESENT(BIC_Pkgpc3);
 		BIC_NOT_PRESENT(BIC_CPU_c7);
@@ -5699,6 +5787,7 @@ void process_cpuid()
 
 	if (!quiet)
 		dump_cstate_pstate_config_info(family, model);
+	intel_uncore_frequency_probe();
 
 	if (!quiet)
 		print_dev_latency();
@@ -6128,7 +6217,30 @@ int get_and_dump_counters(void)
 
 void print_version()
 {
-	fprintf(outf, "turbostat version 2022.04.16 - Len Brown <lenb@kernel.org>\n");
+	fprintf(outf, "turbostat version 2022.07.28 - Len Brown <lenb@kernel.org>\n");
+}
+
+#define COMMAND_LINE_SIZE 2048
+
+void print_bootcmd(void)
+{
+	char bootcmd[COMMAND_LINE_SIZE];
+	FILE *fp;
+	int ret;
+
+	memset(bootcmd, 0, COMMAND_LINE_SIZE);
+	fp = fopen("/proc/cmdline", "r");
+	if (!fp)
+		return;
+
+	ret = fread(bootcmd, sizeof(char), COMMAND_LINE_SIZE - 1, fp);
+	if (ret) {
+		bootcmd[ret] = '\0';
+		/* the last character is already '\n' */
+		fprintf(outf, "Kernel command line: %s", bootcmd);
+	}
+
+	fclose(fp);
 }
 
 int add_counter(unsigned int msr_num, char *path, char *name,
@@ -6602,8 +6714,10 @@ int main(int argc, char **argv)
 	outf = stderr;
 	cmdline(argc, argv);
 
-	if (!quiet)
+	if (!quiet) {
 		print_version();
+		print_bootcmd();
+	}
 
 	probe_sysfs();