From 6e8ba494d87d05cba49224d2068dd313190adbd0 Mon Sep 17 00:00:00 2001 From: Thomas Weißschuh Date: Wed, 4 Dec 2024 20:37:45 +0100 Subject: kbuild/btf: Propagate CONFIG_WERROR to resolve_btfids MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Use CONFIG_WERROR to also fail on warnings emitted by resolve_btfids. Allow the CI bots to prevent the introduction of new warnings. Signed-off-by: Thomas Weißschuh Signed-off-by: Daniel Borkmann Acked-by: Jiri Olsa Link: https://lore.kernel.org/bpf/20241204-resolve_btfids-v3-2-e6a279a74cfd@weissschuh.net --- scripts/link-vmlinux.sh | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) (limited to 'scripts') diff --git a/scripts/link-vmlinux.sh b/scripts/link-vmlinux.sh index d853ddb3b28c..56a077d204cf 100755 --- a/scripts/link-vmlinux.sh +++ b/scripts/link-vmlinux.sh @@ -283,7 +283,11 @@ vmlinux_link vmlinux # fill in BTF IDs if is_enabled CONFIG_DEBUG_INFO_BTF; then info BTFIDS vmlinux - ${RESOLVE_BTFIDS} vmlinux + RESOLVE_BTFIDS_ARGS="" + if is_enabled CONFIG_WERROR; then + RESOLVE_BTFIDS_ARGS=" --fatal_warnings " + fi + ${RESOLVE_BTFIDS} ${RESOLVE_BTFIDS_ARGS} vmlinux fi mksysmap vmlinux System.map -- cgit v1.2.3 From 41d7ea30494cc0dde3e124a75ce0add93f988ba9 Mon Sep 17 00:00:00 2001 From: Vladimir Oltean Date: Tue, 10 Dec 2024 12:27:12 -0800 Subject: lib: packing: add pack_fields() and unpack_fields() This is new API which caters to the following requirements: - Pack or unpack a large number of fields to/from a buffer with a small code footprint. The current alternative is to open-code a large number of calls to pack() and unpack(), or to use packing() to reduce that number to half. But packing() is not const-correct. - Use unpacked numbers stored in variables smaller than u64. This reduces the rodata footprint of the stored field arrays. - Perform error checking at compile time, rather than runtime, and return void from the API functions. Because the C preprocessor can't generate variable length code (loops), this is a bit tricky to do with macros. To handle this, implement macros which sanity check the packed field definitions based on their size. Finally, a single macro with a chain of __builtin_choose_expr() is used to select the appropriate macros. We enforce the use of ascending or descending order to avoid O(N^2) scaling when checking for overlap. Note that the macros are written with care to ensure that the compilers can correctly evaluate the resulting code at compile time. In particular, care was taken with avoiding too many nested statement expressions. Nested statement expressions trip up some compilers, especially when passing down variables created in previous statement expressions. There are two key design choices intended to keep the overall macro code size small. First, the definition of each CHECK_PACKED_FIELDS_N macro is implemented recursively, by calling the N-1 macro. This avoids needing the code to repeat multiple times. Second, the CHECK_PACKED_FIELD macro enforces that the fields in the array are sorted in order. This allows checking for overlap only with neighboring fields, rather than the general overlap case where each field would need to be checked against other fields. The overlap checks use the first two fields to determine the order of the remaining fields, thus allowing either ascending or descending order. This enables drivers the flexibility to keep the fields ordered in which ever order most naturally fits their hardware design and its associated documentation. The CHECK_PACKED_FIELDS macro is directly called from within pack_fields and unpack_fields, ensuring that all drivers using the API receive the benefits of the compile-time checks. Users do not need to directly call any of the macros directly. The CHECK_PACKED_FIELDS and its helper macros CHECK_PACKED_FIELDS_(0..50) are generated using a simple C program in scripts/gen_packed_field_checks.c This program can be compiled on demand and executed to generate the macro code in include/linux/packing.h. This will aid in the event that a driver needs more than 50 fields. The generator can be updated with a new size, and used to update the packing.h header file. In practice, the ice driver will need to support 27 fields, and the sja1105 driver will need to support 0 fields. This on-demand generation avoids the need to modify Kbuild. We do not anticipate the maximum number of fields to grow very often. - Reduced rodata footprint for the storage of the packed field arrays. To that end, we have struct packed_field_u8 and packed_field_u16, which define the fields with the associated type. More can be added as needed (unlikely for now). On these types, the same generic pack_fields() and unpack_fields() API can be used, thanks to the new C11 _Generic() selection feature, which can call pack_fields_u8() or pack_fields_16(), depending on the type of the "fields" array - a simplistic form of polymorphism. It is evaluated at compile time which function will actually be called. Over time, packing() is expected to be completely replaced either with pack() or with pack_fields(). Signed-off-by: Vladimir Oltean Co-developed-by: Jacob Keller Signed-off-by: Jacob Keller Reviewed-by: Vladimir Oltean Link: https://patch.msgid.link/20241210-packing-pack-fields-and-ice-implementation-v10-3-ee56a47479ac@intel.com Signed-off-by: Jakub Kicinski --- MAINTAINERS | 1 + Makefile | 4 + include/linux/packing.h | 425 ++++++++++++++++++++++++++++++++++++++ lib/packing.c | 153 ++++++++++++++ lib/packing_test.c | 61 ++++++ scripts/.gitignore | 1 + scripts/Makefile | 2 +- scripts/gen_packed_field_checks.c | 37 ++++ 8 files changed, 683 insertions(+), 1 deletion(-) create mode 100644 scripts/gen_packed_field_checks.c (limited to 'scripts') diff --git a/MAINTAINERS b/MAINTAINERS index af35519be320..15cf366c0aec 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -17644,6 +17644,7 @@ F: Documentation/core-api/packing.rst F: include/linux/packing.h F: lib/packing.c F: lib/packing_test.c +F: scripts/gen_packed_field_checks.c PADATA PARALLEL EXECUTION MECHANISM M: Steffen Klassert diff --git a/Makefile b/Makefile index 93ab62cef244..9a9fd5504ae8 100644 --- a/Makefile +++ b/Makefile @@ -1367,6 +1367,10 @@ PHONY += scripts_unifdef scripts_unifdef: scripts_basic $(Q)$(MAKE) $(build)=scripts scripts/unifdef +PHONY += scripts_gen_packed_field_checks +scripts_gen_packed_field_checks: scripts_basic + $(Q)$(MAKE) $(build)=scripts scripts/gen_packed_field_checks + # --------------------------------------------------------------------------- # Install diff --git a/include/linux/packing.h b/include/linux/packing.h index 5d36dcd06f60..0589d70bbe04 100644 --- a/include/linux/packing.h +++ b/include/linux/packing.h @@ -8,6 +8,83 @@ #include #include +#define GEN_PACKED_FIELD_STRUCT(__type) \ + struct packed_field_ ## __type { \ + __type startbit; \ + __type endbit; \ + __type offset; \ + __type size; \ + } + +/* struct packed_field_u8. Use with bit offsets < 256, buffers < 32B and + * unpacked structures < 256B. + */ +GEN_PACKED_FIELD_STRUCT(u8); + +/* struct packed_field_u16. Use with bit offsets < 65536, buffers < 8KB and + * unpacked structures < 64KB. + */ +GEN_PACKED_FIELD_STRUCT(u16); + +#define PACKED_FIELD(start, end, struct_name, struct_field) \ +{ \ + (start), \ + (end), \ + offsetof(struct_name, struct_field), \ + sizeof_field(struct_name, struct_field), \ +} + +#define CHECK_PACKED_FIELD_OVERLAP(fields, index1, index2) ({ \ + typeof(&(fields)[0]) __f = (fields); \ + typeof(__f[0]) _f1 = __f[index1]; typeof(__f[0]) _f2 = __f[index2]; \ + const bool _ascending = __f[0].startbit < __f[1].startbit; \ + BUILD_BUG_ON_MSG(_ascending && _f1.startbit >= _f2.startbit, \ + __stringify(fields) " field " __stringify(index2) \ + " breaks ascending order"); \ + BUILD_BUG_ON_MSG(!_ascending && _f1.startbit <= _f2.startbit, \ + __stringify(fields) " field " __stringify(index2) \ + " breaks descending order"); \ + BUILD_BUG_ON_MSG(max(_f1.endbit, _f2.endbit) <= \ + min(_f1.startbit, _f2.startbit), \ + __stringify(fields) " field " __stringify(index2) \ + " overlaps with previous field"); \ +}) + +#define CHECK_PACKED_FIELD(fields, index) ({ \ + typeof(&(fields)[0]) _f = (fields); \ + typeof(_f[0]) __f = _f[index]; \ + BUILD_BUG_ON_MSG(__f.startbit < __f.endbit, \ + __stringify(fields) " field " __stringify(index) \ + " start bit must not be smaller than end bit"); \ + BUILD_BUG_ON_MSG(__f.size != 1 && __f.size != 2 && \ + __f.size != 4 && __f.size != 8, \ + __stringify(fields) " field " __stringify(index) \ + " has unsupported unpacked storage size"); \ + BUILD_BUG_ON_MSG(__f.startbit - __f.endbit >= BITS_PER_BYTE * __f.size, \ + __stringify(fields) " field " __stringify(index) \ + " exceeds unpacked storage size"); \ + __builtin_choose_expr(index != 0, \ + CHECK_PACKED_FIELD_OVERLAP(fields, index - 1, index), \ + 1); \ +}) + +/* Note that the packed fields may be either in ascending or descending order. + * Thus, we must check that both the first and last field wit within the + * packed buffer size. + */ +#define CHECK_PACKED_FIELDS_SIZE(fields, pbuflen) ({ \ + typeof(&(fields)[0]) _f = (fields); \ + typeof(pbuflen) _len = (pbuflen); \ + const size_t num_fields = ARRAY_SIZE(fields); \ + BUILD_BUG_ON_MSG(!__builtin_constant_p(_len), \ + __stringify(fields) " pbuflen " __stringify(pbuflen) \ + " must be a compile time constant"); \ + BUILD_BUG_ON_MSG(_f[0].startbit >= BITS_PER_BYTE * _len, \ + __stringify(fields) " first field exceeds packed buffer size"); \ + BUILD_BUG_ON_MSG(_f[num_fields - 1].startbit >= BITS_PER_BYTE * _len, \ + __stringify(fields) " last field exceeds packed buffer size"); \ +}) + #define QUIRK_MSB_ON_THE_RIGHT BIT(0) #define QUIRK_LITTLE_ENDIAN BIT(1) #define QUIRK_LSW32_IS_FIRST BIT(2) @@ -26,4 +103,352 @@ int pack(void *pbuf, u64 uval, size_t startbit, size_t endbit, size_t pbuflen, int unpack(const void *pbuf, u64 *uval, size_t startbit, size_t endbit, size_t pbuflen, u8 quirks); +void pack_fields_u8(void *pbuf, size_t pbuflen, const void *ustruct, + const struct packed_field_u8 *fields, size_t num_fields, + u8 quirks); + +void pack_fields_u16(void *pbuf, size_t pbuflen, const void *ustruct, + const struct packed_field_u16 *fields, size_t num_fields, + u8 quirks); + +void unpack_fields_u8(const void *pbuf, size_t pbuflen, void *ustruct, + const struct packed_field_u8 *fields, size_t num_fields, + u8 quirks); + +void unpack_fields_u16(const void *pbuf, size_t pbuflen, void *ustruct, + const struct packed_field_u16 *fields, size_t num_fields, + u8 quirks); + +/* Do not hand-edit the following packed field check macros! + * + * They are generated using scripts/gen_packed_field_checks.c, which may be + * built via "make scripts_gen_packed_field_checks". If larger macro sizes are + * needed in the future, please use this program to re-generate the macros and + * insert them here. + */ + +#define CHECK_PACKED_FIELDS_1(fields) \ + CHECK_PACKED_FIELD(fields, 0) + +#define CHECK_PACKED_FIELDS_2(fields) do { \ + CHECK_PACKED_FIELDS_1(fields); \ + CHECK_PACKED_FIELD(fields, 1); \ +} while (0) + +#define CHECK_PACKED_FIELDS_3(fields) do { \ + CHECK_PACKED_FIELDS_2(fields); \ + CHECK_PACKED_FIELD(fields, 2); \ +} while (0) + +#define CHECK_PACKED_FIELDS_4(fields) do { \ + CHECK_PACKED_FIELDS_3(fields); \ + CHECK_PACKED_FIELD(fields, 3); \ +} while (0) + +#define CHECK_PACKED_FIELDS_5(fields) do { \ + CHECK_PACKED_FIELDS_4(fields); \ + CHECK_PACKED_FIELD(fields, 4); \ +} while (0) + +#define CHECK_PACKED_FIELDS_6(fields) do { \ + CHECK_PACKED_FIELDS_5(fields); \ + CHECK_PACKED_FIELD(fields, 5); \ +} while (0) + +#define CHECK_PACKED_FIELDS_7(fields) do { \ + CHECK_PACKED_FIELDS_6(fields); \ + CHECK_PACKED_FIELD(fields, 6); \ +} while (0) + +#define CHECK_PACKED_FIELDS_8(fields) do { \ + CHECK_PACKED_FIELDS_7(fields); \ + CHECK_PACKED_FIELD(fields, 7); \ +} while (0) + +#define CHECK_PACKED_FIELDS_9(fields) do { \ + CHECK_PACKED_FIELDS_8(fields); \ + CHECK_PACKED_FIELD(fields, 8); \ +} while (0) + +#define CHECK_PACKED_FIELDS_10(fields) do { \ + CHECK_PACKED_FIELDS_9(fields); \ + CHECK_PACKED_FIELD(fields, 9); \ +} while (0) + +#define CHECK_PACKED_FIELDS_11(fields) do { \ + CHECK_PACKED_FIELDS_10(fields); \ + CHECK_PACKED_FIELD(fields, 10); \ +} while (0) + +#define CHECK_PACKED_FIELDS_12(fields) do { \ + CHECK_PACKED_FIELDS_11(fields); \ + CHECK_PACKED_FIELD(fields, 11); \ +} while (0) + +#define CHECK_PACKED_FIELDS_13(fields) do { \ + CHECK_PACKED_FIELDS_12(fields); \ + CHECK_PACKED_FIELD(fields, 12); \ +} while (0) + +#define CHECK_PACKED_FIELDS_14(fields) do { \ + CHECK_PACKED_FIELDS_13(fields); \ + CHECK_PACKED_FIELD(fields, 13); \ +} while (0) + +#define CHECK_PACKED_FIELDS_15(fields) do { \ + CHECK_PACKED_FIELDS_14(fields); \ + CHECK_PACKED_FIELD(fields, 14); \ +} while (0) + +#define CHECK_PACKED_FIELDS_16(fields) do { \ + CHECK_PACKED_FIELDS_15(fields); \ + CHECK_PACKED_FIELD(fields, 15); \ +} while (0) + +#define CHECK_PACKED_FIELDS_17(fields) do { \ + CHECK_PACKED_FIELDS_16(fields); \ + CHECK_PACKED_FIELD(fields, 16); \ +} while (0) + +#define CHECK_PACKED_FIELDS_18(fields) do { \ + CHECK_PACKED_FIELDS_17(fields); \ + CHECK_PACKED_FIELD(fields, 17); \ +} while (0) + +#define CHECK_PACKED_FIELDS_19(fields) do { \ + CHECK_PACKED_FIELDS_18(fields); \ + CHECK_PACKED_FIELD(fields, 18); \ +} while (0) + +#define CHECK_PACKED_FIELDS_20(fields) do { \ + CHECK_PACKED_FIELDS_19(fields); \ + CHECK_PACKED_FIELD(fields, 19); \ +} while (0) + +#define CHECK_PACKED_FIELDS_21(fields) do { \ + CHECK_PACKED_FIELDS_20(fields); \ + CHECK_PACKED_FIELD(fields, 20); \ +} while (0) + +#define CHECK_PACKED_FIELDS_22(fields) do { \ + CHECK_PACKED_FIELDS_21(fields); \ + CHECK_PACKED_FIELD(fields, 21); \ +} while (0) + +#define CHECK_PACKED_FIELDS_23(fields) do { \ + CHECK_PACKED_FIELDS_22(fields); \ + CHECK_PACKED_FIELD(fields, 22); \ +} while (0) + +#define CHECK_PACKED_FIELDS_24(fields) do { \ + CHECK_PACKED_FIELDS_23(fields); \ + CHECK_PACKED_FIELD(fields, 23); \ +} while (0) + +#define CHECK_PACKED_FIELDS_25(fields) do { \ + CHECK_PACKED_FIELDS_24(fields); \ + CHECK_PACKED_FIELD(fields, 24); \ +} while (0) + +#define CHECK_PACKED_FIELDS_26(fields) do { \ + CHECK_PACKED_FIELDS_25(fields); \ + CHECK_PACKED_FIELD(fields, 25); \ +} while (0) + +#define CHECK_PACKED_FIELDS_27(fields) do { \ + CHECK_PACKED_FIELDS_26(fields); \ + CHECK_PACKED_FIELD(fields, 26); \ +} while (0) + +#define CHECK_PACKED_FIELDS_28(fields) do { \ + CHECK_PACKED_FIELDS_27(fields); \ + CHECK_PACKED_FIELD(fields, 27); \ +} while (0) + +#define CHECK_PACKED_FIELDS_29(fields) do { \ + CHECK_PACKED_FIELDS_28(fields); \ + CHECK_PACKED_FIELD(fields, 28); \ +} while (0) + +#define CHECK_PACKED_FIELDS_30(fields) do { \ + CHECK_PACKED_FIELDS_29(fields); \ + CHECK_PACKED_FIELD(fields, 29); \ +} while (0) + +#define CHECK_PACKED_FIELDS_31(fields) do { \ + CHECK_PACKED_FIELDS_30(fields); \ + CHECK_PACKED_FIELD(fields, 30); \ +} while (0) + +#define CHECK_PACKED_FIELDS_32(fields) do { \ + CHECK_PACKED_FIELDS_31(fields); \ + CHECK_PACKED_FIELD(fields, 31); \ +} while (0) + +#define CHECK_PACKED_FIELDS_33(fields) do { \ + CHECK_PACKED_FIELDS_32(fields); \ + CHECK_PACKED_FIELD(fields, 32); \ +} while (0) + +#define CHECK_PACKED_FIELDS_34(fields) do { \ + CHECK_PACKED_FIELDS_33(fields); \ + CHECK_PACKED_FIELD(fields, 33); \ +} while (0) + +#define CHECK_PACKED_FIELDS_35(fields) do { \ + CHECK_PACKED_FIELDS_34(fields); \ + CHECK_PACKED_FIELD(fields, 34); \ +} while (0) + +#define CHECK_PACKED_FIELDS_36(fields) do { \ + CHECK_PACKED_FIELDS_35(fields); \ + CHECK_PACKED_FIELD(fields, 35); \ +} while (0) + +#define CHECK_PACKED_FIELDS_37(fields) do { \ + CHECK_PACKED_FIELDS_36(fields); \ + CHECK_PACKED_FIELD(fields, 36); \ +} while (0) + +#define CHECK_PACKED_FIELDS_38(fields) do { \ + CHECK_PACKED_FIELDS_37(fields); \ + CHECK_PACKED_FIELD(fields, 37); \ +} while (0) + +#define CHECK_PACKED_FIELDS_39(fields) do { \ + CHECK_PACKED_FIELDS_38(fields); \ + CHECK_PACKED_FIELD(fields, 38); \ +} while (0) + +#define CHECK_PACKED_FIELDS_40(fields) do { \ + CHECK_PACKED_FIELDS_39(fields); \ + CHECK_PACKED_FIELD(fields, 39); \ +} while (0) + +#define CHECK_PACKED_FIELDS_41(fields) do { \ + CHECK_PACKED_FIELDS_40(fields); \ + CHECK_PACKED_FIELD(fields, 40); \ +} while (0) + +#define CHECK_PACKED_FIELDS_42(fields) do { \ + CHECK_PACKED_FIELDS_41(fields); \ + CHECK_PACKED_FIELD(fields, 41); \ +} while (0) + +#define CHECK_PACKED_FIELDS_43(fields) do { \ + CHECK_PACKED_FIELDS_42(fields); \ + CHECK_PACKED_FIELD(fields, 42); \ +} while (0) + +#define CHECK_PACKED_FIELDS_44(fields) do { \ + CHECK_PACKED_FIELDS_43(fields); \ + CHECK_PACKED_FIELD(fields, 43); \ +} while (0) + +#define CHECK_PACKED_FIELDS_45(fields) do { \ + CHECK_PACKED_FIELDS_44(fields); \ + CHECK_PACKED_FIELD(fields, 44); \ +} while (0) + +#define CHECK_PACKED_FIELDS_46(fields) do { \ + CHECK_PACKED_FIELDS_45(fields); \ + CHECK_PACKED_FIELD(fields, 45); \ +} while (0) + +#define CHECK_PACKED_FIELDS_47(fields) do { \ + CHECK_PACKED_FIELDS_46(fields); \ + CHECK_PACKED_FIELD(fields, 46); \ +} while (0) + +#define CHECK_PACKED_FIELDS_48(fields) do { \ + CHECK_PACKED_FIELDS_47(fields); \ + CHECK_PACKED_FIELD(fields, 47); \ +} while (0) + +#define CHECK_PACKED_FIELDS_49(fields) do { \ + CHECK_PACKED_FIELDS_48(fields); \ + CHECK_PACKED_FIELD(fields, 48); \ +} while (0) + +#define CHECK_PACKED_FIELDS_50(fields) do { \ + CHECK_PACKED_FIELDS_49(fields); \ + CHECK_PACKED_FIELD(fields, 49); \ +} while (0) + +#define CHECK_PACKED_FIELDS(fields) \ + __builtin_choose_expr(ARRAY_SIZE(fields) == 1, ({ CHECK_PACKED_FIELDS_1(fields); }), \ + __builtin_choose_expr(ARRAY_SIZE(fields) == 2, ({ CHECK_PACKED_FIELDS_2(fields); }), \ + __builtin_choose_expr(ARRAY_SIZE(fields) == 3, ({ CHECK_PACKED_FIELDS_3(fields); }), \ + __builtin_choose_expr(ARRAY_SIZE(fields) == 4, ({ CHECK_PACKED_FIELDS_4(fields); }), \ + __builtin_choose_expr(ARRAY_SIZE(fields) == 5, ({ CHECK_PACKED_FIELDS_5(fields); }), \ + __builtin_choose_expr(ARRAY_SIZE(fields) == 6, ({ CHECK_PACKED_FIELDS_6(fields); }), \ + __builtin_choose_expr(ARRAY_SIZE(fields) == 7, ({ CHECK_PACKED_FIELDS_7(fields); }), \ + __builtin_choose_expr(ARRAY_SIZE(fields) == 8, ({ CHECK_PACKED_FIELDS_8(fields); }), \ + __builtin_choose_expr(ARRAY_SIZE(fields) == 9, ({ CHECK_PACKED_FIELDS_9(fields); }), \ + __builtin_choose_expr(ARRAY_SIZE(fields) == 10, ({ CHECK_PACKED_FIELDS_10(fields); }), \ + __builtin_choose_expr(ARRAY_SIZE(fields) == 11, ({ CHECK_PACKED_FIELDS_11(fields); }), \ + __builtin_choose_expr(ARRAY_SIZE(fields) == 12, ({ CHECK_PACKED_FIELDS_12(fields); }), \ + __builtin_choose_expr(ARRAY_SIZE(fields) == 13, ({ CHECK_PACKED_FIELDS_13(fields); }), \ + __builtin_choose_expr(ARRAY_SIZE(fields) == 14, ({ CHECK_PACKED_FIELDS_14(fields); }), \ + __builtin_choose_expr(ARRAY_SIZE(fields) == 15, ({ CHECK_PACKED_FIELDS_15(fields); }), \ + __builtin_choose_expr(ARRAY_SIZE(fields) == 16, ({ CHECK_PACKED_FIELDS_16(fields); }), \ + __builtin_choose_expr(ARRAY_SIZE(fields) == 17, ({ CHECK_PACKED_FIELDS_17(fields); }), \ + __builtin_choose_expr(ARRAY_SIZE(fields) == 18, ({ CHECK_PACKED_FIELDS_18(fields); }), \ + __builtin_choose_expr(ARRAY_SIZE(fields) == 19, ({ CHECK_PACKED_FIELDS_19(fields); }), \ + __builtin_choose_expr(ARRAY_SIZE(fields) == 20, ({ CHECK_PACKED_FIELDS_20(fields); }), \ + __builtin_choose_expr(ARRAY_SIZE(fields) == 21, ({ CHECK_PACKED_FIELDS_21(fields); }), \ + __builtin_choose_expr(ARRAY_SIZE(fields) == 22, ({ CHECK_PACKED_FIELDS_22(fields); }), \ + __builtin_choose_expr(ARRAY_SIZE(fields) == 23, ({ CHECK_PACKED_FIELDS_23(fields); }), \ + __builtin_choose_expr(ARRAY_SIZE(fields) == 24, ({ CHECK_PACKED_FIELDS_24(fields); }), \ + __builtin_choose_expr(ARRAY_SIZE(fields) == 25, ({ CHECK_PACKED_FIELDS_25(fields); }), \ + __builtin_choose_expr(ARRAY_SIZE(fields) == 26, ({ CHECK_PACKED_FIELDS_26(fields); }), \ + __builtin_choose_expr(ARRAY_SIZE(fields) == 27, ({ CHECK_PACKED_FIELDS_27(fields); }), \ + __builtin_choose_expr(ARRAY_SIZE(fields) == 28, ({ CHECK_PACKED_FIELDS_28(fields); }), \ + __builtin_choose_expr(ARRAY_SIZE(fields) == 29, ({ CHECK_PACKED_FIELDS_29(fields); }), \ + __builtin_choose_expr(ARRAY_SIZE(fields) == 30, ({ CHECK_PACKED_FIELDS_30(fields); }), \ + __builtin_choose_expr(ARRAY_SIZE(fields) == 31, ({ CHECK_PACKED_FIELDS_31(fields); }), \ + __builtin_choose_expr(ARRAY_SIZE(fields) == 32, ({ CHECK_PACKED_FIELDS_32(fields); }), \ + __builtin_choose_expr(ARRAY_SIZE(fields) == 33, ({ CHECK_PACKED_FIELDS_33(fields); }), \ + __builtin_choose_expr(ARRAY_SIZE(fields) == 34, ({ CHECK_PACKED_FIELDS_34(fields); }), \ + __builtin_choose_expr(ARRAY_SIZE(fields) == 35, ({ CHECK_PACKED_FIELDS_35(fields); }), \ + __builtin_choose_expr(ARRAY_SIZE(fields) == 36, ({ CHECK_PACKED_FIELDS_36(fields); }), \ + __builtin_choose_expr(ARRAY_SIZE(fields) == 37, ({ CHECK_PACKED_FIELDS_37(fields); }), \ + __builtin_choose_expr(ARRAY_SIZE(fields) == 38, ({ CHECK_PACKED_FIELDS_38(fields); }), \ + __builtin_choose_expr(ARRAY_SIZE(fields) == 39, ({ CHECK_PACKED_FIELDS_39(fields); }), \ + __builtin_choose_expr(ARRAY_SIZE(fields) == 40, ({ CHECK_PACKED_FIELDS_40(fields); }), \ + __builtin_choose_expr(ARRAY_SIZE(fields) == 41, ({ CHECK_PACKED_FIELDS_41(fields); }), \ + __builtin_choose_expr(ARRAY_SIZE(fields) == 42, ({ CHECK_PACKED_FIELDS_42(fields); }), \ + __builtin_choose_expr(ARRAY_SIZE(fields) == 43, ({ CHECK_PACKED_FIELDS_43(fields); }), \ + __builtin_choose_expr(ARRAY_SIZE(fields) == 44, ({ CHECK_PACKED_FIELDS_44(fields); }), \ + __builtin_choose_expr(ARRAY_SIZE(fields) == 45, ({ CHECK_PACKED_FIELDS_45(fields); }), \ + __builtin_choose_expr(ARRAY_SIZE(fields) == 46, ({ CHECK_PACKED_FIELDS_46(fields); }), \ + __builtin_choose_expr(ARRAY_SIZE(fields) == 47, ({ CHECK_PACKED_FIELDS_47(fields); }), \ + __builtin_choose_expr(ARRAY_SIZE(fields) == 48, ({ CHECK_PACKED_FIELDS_48(fields); }), \ + __builtin_choose_expr(ARRAY_SIZE(fields) == 49, ({ CHECK_PACKED_FIELDS_49(fields); }), \ + __builtin_choose_expr(ARRAY_SIZE(fields) == 50, ({ CHECK_PACKED_FIELDS_50(fields); }), \ + ({ BUILD_BUG_ON_MSG(1, "CHECK_PACKED_FIELDS() must be regenerated to support array sizes larger than 50."); }) \ +)))))))))))))))))))))))))))))))))))))))))))))))))) + +/* End of generated content */ + +#define pack_fields(pbuf, pbuflen, ustruct, fields, quirks) \ + ({ \ + CHECK_PACKED_FIELDS(fields); \ + CHECK_PACKED_FIELDS_SIZE((fields), (pbuflen)); \ + _Generic((fields), \ + const struct packed_field_u8 * : pack_fields_u8, \ + const struct packed_field_u16 * : pack_fields_u16 \ + )((pbuf), (pbuflen), (ustruct), (fields), ARRAY_SIZE(fields), (quirks)); \ + }) + +#define unpack_fields(pbuf, pbuflen, ustruct, fields, quirks) \ + ({ \ + CHECK_PACKED_FIELDS(fields); \ + CHECK_PACKED_FIELDS_SIZE((fields), (pbuflen)); \ + _Generic((fields), \ + const struct packed_field_u8 * : unpack_fields_u8, \ + const struct packed_field_u16 * : unpack_fields_u16 \ + )((pbuf), (pbuflen), (ustruct), (fields), ARRAY_SIZE(fields), (quirks)); \ + }) + #endif diff --git a/lib/packing.c b/lib/packing.c index 09a2d195b943..bb1643d9e64d 100644 --- a/lib/packing.c +++ b/lib/packing.c @@ -5,10 +5,37 @@ #include #include #include +#include #include #include #include +#define __pack_fields(pbuf, pbuflen, ustruct, fields, num_fields, quirks) \ + ({ \ + for (size_t i = 0; i < (num_fields); i++) { \ + typeof(&(fields)[0]) field = &(fields)[i]; \ + u64 uval; \ + \ + uval = ustruct_field_to_u64(ustruct, field->offset, field->size); \ + \ + __pack(pbuf, uval, field->startbit, field->endbit, \ + pbuflen, quirks); \ + } \ + }) + +#define __unpack_fields(pbuf, pbuflen, ustruct, fields, num_fields, quirks) \ + ({ \ + for (size_t i = 0; i < (num_fields); i++) { \ + typeof(&(fields)[0]) field = &fields[i]; \ + u64 uval; \ + \ + __unpack(pbuf, &uval, field->startbit, field->endbit, \ + pbuflen, quirks); \ + \ + u64_to_ustruct_field(ustruct, field->offset, field->size, uval); \ + } \ + }) + /** * calculate_box_addr - Determine physical location of byte in buffer * @box: Index of byte within buffer seen as a logical big-endian big number @@ -322,4 +349,130 @@ int packing(void *pbuf, u64 *uval, int startbit, int endbit, size_t pbuflen, } EXPORT_SYMBOL(packing); +static u64 ustruct_field_to_u64(const void *ustruct, size_t field_offset, + size_t field_size) +{ + switch (field_size) { + case 1: + return *((u8 *)(ustruct + field_offset)); + case 2: + return *((u16 *)(ustruct + field_offset)); + case 4: + return *((u32 *)(ustruct + field_offset)); + default: + return *((u64 *)(ustruct + field_offset)); + } +} + +static void u64_to_ustruct_field(void *ustruct, size_t field_offset, + size_t field_size, u64 uval) +{ + switch (field_size) { + case 1: + *((u8 *)(ustruct + field_offset)) = uval; + break; + case 2: + *((u16 *)(ustruct + field_offset)) = uval; + break; + case 4: + *((u32 *)(ustruct + field_offset)) = uval; + break; + default: + *((u64 *)(ustruct + field_offset)) = uval; + break; + } +} + +/** + * pack_fields_u8 - Pack array of fields + * + * @pbuf: Pointer to a buffer holding the packed value. + * @pbuflen: The length in bytes of the packed buffer pointed to by @pbuf. + * @ustruct: Pointer to CPU-readable structure holding the unpacked value. + * It is expected (but not checked) that this has the same data type + * as all struct packed_field_u8 definitions. + * @fields: Array of packed_field_u8 field definition. They must not overlap. + * @num_fields: Length of @fields array. + * @quirks: A bit mask of QUIRK_LITTLE_ENDIAN, QUIRK_LSW32_IS_FIRST and + * QUIRK_MSB_ON_THE_RIGHT. + * + * Use the pack_fields() macro instead of calling this directly. + */ +void pack_fields_u8(void *pbuf, size_t pbuflen, const void *ustruct, + const struct packed_field_u8 *fields, size_t num_fields, + u8 quirks) +{ + __pack_fields(pbuf, pbuflen, ustruct, fields, num_fields, quirks); +} +EXPORT_SYMBOL(pack_fields_u8); + +/** + * pack_fields_u16 - Pack array of fields + * + * @pbuf: Pointer to a buffer holding the packed value. + * @pbuflen: The length in bytes of the packed buffer pointed to by @pbuf. + * @ustruct: Pointer to CPU-readable structure holding the unpacked value. + * It is expected (but not checked) that this has the same data type + * as all struct packed_field_u16 definitions. + * @fields: Array of packed_field_u16 field definitions. They must not overlap. + * @num_fields: Length of @fields array. + * @quirks: A bit mask of QUIRK_LITTLE_ENDIAN, QUIRK_LSW32_IS_FIRST and + * QUIRK_MSB_ON_THE_RIGHT. + * + * Use the pack_fields() macro instead of calling this directly. + */ +void pack_fields_u16(void *pbuf, size_t pbuflen, const void *ustruct, + const struct packed_field_u16 *fields, size_t num_fields, + u8 quirks) +{ + __pack_fields(pbuf, pbuflen, ustruct, fields, num_fields, quirks); +} +EXPORT_SYMBOL(pack_fields_u16); + +/** + * unpack_fields_u8 - Unpack array of fields + * + * @pbuf: Pointer to a buffer holding the packed value. + * @pbuflen: The length in bytes of the packed buffer pointed to by @pbuf. + * @ustruct: Pointer to CPU-readable structure holding the unpacked value. + * It is expected (but not checked) that this has the same data type + * as all struct packed_field_u8 definitions. + * @fields: Array of packed_field_u8 field definitions. They must not overlap. + * @num_fields: Length of @fields array. + * @quirks: A bit mask of QUIRK_LITTLE_ENDIAN, QUIRK_LSW32_IS_FIRST and + * QUIRK_MSB_ON_THE_RIGHT. + * + * Use the unpack_fields() macro instead of calling this directly. + */ +void unpack_fields_u8(const void *pbuf, size_t pbuflen, void *ustruct, + const struct packed_field_u8 *fields, size_t num_fields, + u8 quirks) +{ + __unpack_fields(pbuf, pbuflen, ustruct, fields, num_fields, quirks); +} +EXPORT_SYMBOL(unpack_fields_u8); + +/** + * unpack_fields_u16 - Unpack array of fields + * + * @pbuf: Pointer to a buffer holding the packed value. + * @pbuflen: The length in bytes of the packed buffer pointed to by @pbuf. + * @ustruct: Pointer to CPU-readable structure holding the unpacked value. + * It is expected (but not checked) that this has the same data type + * as all struct packed_field_u16 definitions. + * @fields: Array of packed_field_u16 field definitions. They must not overlap. + * @num_fields: Length of @fields array. + * @quirks: A bit mask of QUIRK_LITTLE_ENDIAN, QUIRK_LSW32_IS_FIRST and + * QUIRK_MSB_ON_THE_RIGHT. + * + * Use the unpack_fields() macro instead of calling this directly. + */ +void unpack_fields_u16(const void *pbuf, size_t pbuflen, void *ustruct, + const struct packed_field_u16 *fields, size_t num_fields, + u8 quirks) +{ + __unpack_fields(pbuf, pbuflen, ustruct, fields, num_fields, quirks); +} +EXPORT_SYMBOL(unpack_fields_u16); + MODULE_DESCRIPTION("Generic bitfield packing and unpacking"); diff --git a/lib/packing_test.c b/lib/packing_test.c index b38ea43c03fd..ce3b83d33b04 100644 --- a/lib/packing_test.c +++ b/lib/packing_test.c @@ -396,9 +396,70 @@ static void packing_test_unpack(struct kunit *test) KUNIT_EXPECT_EQ(test, uval, params->uval); } +#define PACKED_BUF_SIZE 8 + +typedef struct __packed { u8 buf[PACKED_BUF_SIZE]; } packed_buf_t; + +struct test_data { + u32 field3; + u16 field2; + u16 field4; + u16 field6; + u8 field1; + u8 field5; +}; + +static const struct packed_field_u8 test_fields[] = { + PACKED_FIELD(63, 61, struct test_data, field1), + PACKED_FIELD(60, 52, struct test_data, field2), + PACKED_FIELD(51, 28, struct test_data, field3), + PACKED_FIELD(27, 14, struct test_data, field4), + PACKED_FIELD(13, 9, struct test_data, field5), + PACKED_FIELD(8, 0, struct test_data, field6), +}; + +static void packing_test_pack_fields(struct kunit *test) +{ + const struct test_data data = { + .field1 = 0x2, + .field2 = 0x100, + .field3 = 0xF00050, + .field4 = 0x7D3, + .field5 = 0x9, + .field6 = 0x10B, + }; + packed_buf_t expect = { + .buf = { 0x50, 0x0F, 0x00, 0x05, 0x01, 0xF4, 0xD3, 0x0B }, + }; + packed_buf_t buf = {}; + + pack_fields(&buf, sizeof(buf), &data, test_fields, 0); + + KUNIT_EXPECT_MEMEQ(test, &expect, &buf, sizeof(buf)); +} + +static void packing_test_unpack_fields(struct kunit *test) +{ + const packed_buf_t buf = { + .buf = { 0x17, 0x28, 0x10, 0x19, 0x3D, 0xA9, 0x07, 0x9C }, + }; + struct test_data data = {}; + + unpack_fields(&buf, sizeof(buf), &data, test_fields, 0); + + KUNIT_EXPECT_EQ(test, 0, data.field1); + KUNIT_EXPECT_EQ(test, 0x172, data.field2); + KUNIT_EXPECT_EQ(test, 0x810193, data.field3); + KUNIT_EXPECT_EQ(test, 0x36A4, data.field4); + KUNIT_EXPECT_EQ(test, 0x3, data.field5); + KUNIT_EXPECT_EQ(test, 0x19C, data.field6); +} + static struct kunit_case packing_test_cases[] = { KUNIT_CASE_PARAM(packing_test_pack, packing_gen_params), KUNIT_CASE_PARAM(packing_test_unpack, packing_gen_params), + KUNIT_CASE(packing_test_pack_fields), + KUNIT_CASE(packing_test_unpack_fields), {}, }; diff --git a/scripts/.gitignore b/scripts/.gitignore index 3dbb8bb2457b..c2ef68848da5 100644 --- a/scripts/.gitignore +++ b/scripts/.gitignore @@ -1,5 +1,6 @@ # SPDX-License-Identifier: GPL-2.0-only /asn1_compiler +/gen_packed_field_checks /generate_rust_target /insert-sys-cert /kallsyms diff --git a/scripts/Makefile b/scripts/Makefile index 6bcda4b9d054..546e8175e1c4 100644 --- a/scripts/Makefile +++ b/scripts/Makefile @@ -47,7 +47,7 @@ HOSTCFLAGS_sorttable.o += -DMCOUNT_SORT_ENABLED endif # The following programs are only built on demand -hostprogs += unifdef +hostprogs += unifdef gen_packed_field_checks # The module linker script is preprocessed on demand targets += module.lds diff --git a/scripts/gen_packed_field_checks.c b/scripts/gen_packed_field_checks.c new file mode 100644 index 000000000000..60042b7616ee --- /dev/null +++ b/scripts/gen_packed_field_checks.c @@ -0,0 +1,37 @@ +// SPDX-License-Identifier: GPL-2.0 +// Copyright (c) 2024, Intel Corporation +#include +#include + +#define MAX_PACKED_FIELD_SIZE 50 + +int main(int argc, char **argv) +{ + /* The first macro doesn't need a 'do {} while(0)' loop */ + printf("#define CHECK_PACKED_FIELDS_1(fields) \\\n"); + printf("\tCHECK_PACKED_FIELD(fields, 0)\n\n"); + + /* Remaining macros require a do/while loop, and are implemented + * recursively by calling the previous iteration's macro. + */ + for (int i = 2; i <= MAX_PACKED_FIELD_SIZE; i++) { + printf("#define CHECK_PACKED_FIELDS_%d(fields) do { \\\n", i); + printf("\tCHECK_PACKED_FIELDS_%d(fields); \\\n", i - 1); + printf("\tCHECK_PACKED_FIELD(fields, %d); \\\n", i - 1); + printf("} while (0)\n\n"); + } + + printf("#define CHECK_PACKED_FIELDS(fields) \\\n"); + + for (int i = 1; i <= MAX_PACKED_FIELD_SIZE; i++) + printf("\t__builtin_choose_expr(ARRAY_SIZE(fields) == %d, ({ CHECK_PACKED_FIELDS_%d(fields); }), \\\n", + i, i); + + printf("\t({ BUILD_BUG_ON_MSG(1, \"CHECK_PACKED_FIELDS() must be regenerated to support array sizes larger than %d.\"); }) \\\n", + MAX_PACKED_FIELD_SIZE); + + for (int i = 1; i <= MAX_PACKED_FIELD_SIZE; i++) + printf(")"); + + printf("\n"); +} -- cgit v1.2.3 From 20d00cfae627f048560c46ba5849011a34515103 Mon Sep 17 00:00:00 2001 From: Przemek Kitszel Date: Mon, 16 Dec 2024 15:15:30 +0100 Subject: checkpatch: don't complain on _Generic() use Improve CamelCase recognition logic to avoid reporting on _Generic() use. Other C keywords, such as _Bool, are intentionally omitted, as those should be rather avoided in new source code. Reviewed-by: Wojciech Drewek Reviewed-by: Simon Horman Signed-off-by: Mateusz Polchlopek Acked-by: Joe Perches Signed-off-by: Przemek Kitszel Signed-off-by: Tony Nguyen --- scripts/checkpatch.pl | 2 ++ 1 file changed, 2 insertions(+) (limited to 'scripts') diff --git a/scripts/checkpatch.pl b/scripts/checkpatch.pl index 9eed3683ad76..a2066a6c9dd8 100755 --- a/scripts/checkpatch.pl +++ b/scripts/checkpatch.pl @@ -5843,6 +5843,8 @@ sub process { #CamelCase if ($var !~ /^$Constant$/ && $var =~ /[A-Z][a-z]|[a-z][A-Z]/ && +#Ignore C keywords + $var !~ /^_Generic$/ && #Ignore some autogenerated defines and enum values $var !~ /^(?:[A-Z]+_){1,5}[A-Z]{1,3}[a-z]/ && #Ignore Page variants -- cgit v1.2.3 From 8541bf0239b8509ecc1192b2e26768a36fd9c944 Mon Sep 17 00:00:00 2001 From: Abhishek Pandit-Subedi Date: Fri, 13 Dec 2024 15:35:42 -0800 Subject: usb: typec: Only use SVID for matching altmodes Mode in struct typec_altmode is used to indicate the index of the altmode on a port, partner or plug. It is used in enter mode VDMs but doesn't make much sense for matching against altmode drivers or for matching partner to port altmodes. Signed-off-by: Abhishek Pandit-Subedi Reviewed-by: Heikki Krogerus Reviewed-by: Benson Leung Link: https://lore.kernel.org/r/20241213153543.v5.1.Ie0d37646f18461234777d88b4c3e21faed92ed4f@changeid Signed-off-by: Greg Kroah-Hartman --- drivers/usb/typec/altmodes/displayport.c | 2 +- drivers/usb/typec/altmodes/nvidia.c | 2 +- drivers/usb/typec/bus.c | 6 ++---- drivers/usb/typec/class.c | 4 ++-- scripts/mod/devicetable-offsets.c | 1 - scripts/mod/file2alias.c | 9 ++------- 6 files changed, 8 insertions(+), 16 deletions(-) (limited to 'scripts') diff --git a/drivers/usb/typec/altmodes/displayport.c b/drivers/usb/typec/altmodes/displayport.c index 2f03190a9873..3245e03d59e6 100644 --- a/drivers/usb/typec/altmodes/displayport.c +++ b/drivers/usb/typec/altmodes/displayport.c @@ -791,7 +791,7 @@ void dp_altmode_remove(struct typec_altmode *alt) EXPORT_SYMBOL_GPL(dp_altmode_remove); static const struct typec_device_id dp_typec_id[] = { - { USB_TYPEC_DP_SID, USB_TYPEC_DP_MODE }, + { USB_TYPEC_DP_SID }, { }, }; MODULE_DEVICE_TABLE(typec, dp_typec_id); diff --git a/drivers/usb/typec/altmodes/nvidia.c b/drivers/usb/typec/altmodes/nvidia.c index fe70b36f078f..2b77d931e494 100644 --- a/drivers/usb/typec/altmodes/nvidia.c +++ b/drivers/usb/typec/altmodes/nvidia.c @@ -24,7 +24,7 @@ static void nvidia_altmode_remove(struct typec_altmode *alt) } static const struct typec_device_id nvidia_typec_id[] = { - { USB_TYPEC_NVIDIA_VLINK_SID, TYPEC_ANY_MODE }, + { USB_TYPEC_NVIDIA_VLINK_SID }, { }, }; MODULE_DEVICE_TABLE(typec, nvidia_typec_id); diff --git a/drivers/usb/typec/bus.c b/drivers/usb/typec/bus.c index aa879253d3b8..ae90688d23e4 100644 --- a/drivers/usb/typec/bus.c +++ b/drivers/usb/typec/bus.c @@ -454,8 +454,7 @@ static int typec_match(struct device *dev, const struct device_driver *driver) const struct typec_device_id *id; for (id = drv->id_table; id->svid; id++) - if (id->svid == altmode->svid && - (id->mode == TYPEC_ANY_MODE || id->mode == altmode->mode)) + if (id->svid == altmode->svid) return 1; return 0; } @@ -470,8 +469,7 @@ static int typec_uevent(const struct device *dev, struct kobj_uevent_env *env) if (add_uevent_var(env, "MODE=%u", altmode->mode)) return -ENOMEM; - return add_uevent_var(env, "MODALIAS=typec:id%04Xm%02X", - altmode->svid, altmode->mode); + return add_uevent_var(env, "MODALIAS=typec:id%04X", altmode->svid); } static int typec_altmode_create_links(struct altmode *alt) diff --git a/drivers/usb/typec/class.c b/drivers/usb/typec/class.c index 4b3047e055a3..febe453b96be 100644 --- a/drivers/usb/typec/class.c +++ b/drivers/usb/typec/class.c @@ -237,13 +237,13 @@ static int altmode_match(struct device *dev, void *data) if (!is_typec_altmode(dev)) return 0; - return ((adev->svid == id->svid) && (adev->mode == id->mode)); + return (adev->svid == id->svid); } static void typec_altmode_set_partner(struct altmode *altmode) { struct typec_altmode *adev = &altmode->adev; - struct typec_device_id id = { adev->svid, adev->mode, }; + struct typec_device_id id = { adev->svid }; struct typec_port *port = typec_altmode2port(adev); struct altmode *partner; struct device *dev; diff --git a/scripts/mod/devicetable-offsets.c b/scripts/mod/devicetable-offsets.c index 9c7b404defbd..d3d00e85edf7 100644 --- a/scripts/mod/devicetable-offsets.c +++ b/scripts/mod/devicetable-offsets.c @@ -237,7 +237,6 @@ int main(void) DEVID(typec_device_id); DEVID_FIELD(typec_device_id, svid); - DEVID_FIELD(typec_device_id, mode); DEVID(tee_client_device_id); DEVID_FIELD(tee_client_device_id, uuid); diff --git a/scripts/mod/file2alias.c b/scripts/mod/file2alias.c index 5b5745f00eb3..7049c31062c6 100644 --- a/scripts/mod/file2alias.c +++ b/scripts/mod/file2alias.c @@ -1221,17 +1221,12 @@ static void do_tbsvc_entry(struct module *mod, void *symval) module_alias_printf(mod, true, "tbsvc:%s", alias); } -/* Looks like: typec:idNmN */ +/* Looks like: typec:idN */ static void do_typec_entry(struct module *mod, void *symval) { - char alias[256] = {}; - DEF_FIELD(symval, typec_device_id, svid); - DEF_FIELD(symval, typec_device_id, mode); - - ADD(alias, "m", mode != TYPEC_ANY_MODE, mode); - module_alias_printf(mod, false, "typec:id%04X%s", svid, alias); + module_alias_printf(mod, false, "typec:id%04X", svid); } /* Looks like: tee:uuid */ -- cgit v1.2.3 From da3ecf00ffc7b169b9a31a1218bbb52aa19d8df7 Mon Sep 17 00:00:00 2001 From: Vegard Nossum Date: Sat, 21 Dec 2024 23:22:14 +0100 Subject: scripts/kernel-doc: fix identifier parsing regex John wrote: > kernel-doc gets confused by code like the following: > > /** > * define HOMA_MIN_DEFAULT_PORT - The 16-bit port space is divided into > * two nonoverlapping regions. Ports 1-32767 are reserved exclusively > * for well-defined server ports. The remaining ports are used for client > * ports; these are allocated automatically by Homa. Port 0 is reserved. > */ > #define HOMA_MIN_DEFAULT_PORT 0x8000 > > It seems to use the last "-" on the line (the one in "16-bit") rather > than the first one, so it produces the following false error message: > > homa.h:50: warning: expecting prototype for HOMA_MIN_DEFAULT_PORT - > The 16(). Prototype was for HOMA_MIN_DEFAULT_PORT() instead > > There are similar problems if there is a ":" later on the line. The problem is the regex for the identifier, which is a greedy /.*/ that matches everything up to the last - or : (i.e. $decl_end). Fix it by tightening up this regex and not matching those characters as part of the identifier. Link: https://lore.kernel.org/all/CAGXJAmzfRzE=A94NT5ETtj3bZc-=2oLg-9E5Kjh4o_-iuw1q8g@mail.gmail.com/ Reported-by: John Ousterhout Reviewed-by: Randy Dunlap Tested-by: Randy Dunlap Signed-off-by: Vegard Nossum Signed-off-by: Jonathan Corbet Link: https://lore.kernel.org/r/20241221222214.1969823-1-vegard.nossum@oracle.com --- scripts/kernel-doc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'scripts') diff --git a/scripts/kernel-doc b/scripts/kernel-doc index 4ee843d3600e..e57c5e989a0a 100755 --- a/scripts/kernel-doc +++ b/scripts/kernel-doc @@ -2085,7 +2085,7 @@ sub process_name($$) { # Look for foo() or static void foo() - description; or misspelt # identifier elsif (/^$decl_start$fn_type?(\w+)\s*$parenthesis?\s*$decl_end?$/ || - /^$decl_start$fn_type?(\w+.*)$parenthesis?\s*$decl_end$/) { + /^$decl_start$fn_type?(\w+[^-:]*)$parenthesis?\s*$decl_end$/) { $identifier = $1; $decl_type = 'function'; $identifier =~ s/^define\s+//; -- cgit v1.2.3 From 6356f18f09dc0781650c4f128ea48745fa48c415 Mon Sep 17 00:00:00 2001 From: Geert Uytterhoeven Date: Thu, 5 Dec 2024 19:16:34 +0100 Subject: Align git commit ID abbreviation guidelines and checks The guidelines for git commit ID abbreviation are inconsistent: some places state to use 12 characters exactly, while other places recommend 12 characters or more. The same issue is present in the checkpatch.pl script. E.g. Documentation/dev-tools/checkpatch.rst says: **GIT_COMMIT_ID** The proper way to reference a commit id is: commit <12+ chars of sha1> ("") However, scripts/checkpatch.pl has two different checks: one warning check accepting 12 characters exactly: # Check Fixes: styles is correct Please use correct Fixes: style 'Fixes: <12 chars of sha1> (\"<title line>\")' and a second error check accepting 12-40 characters: # Check for git id commit length and improperly formed commit descriptions # A correctly formed commit description is: # commit <SHA-1 hash length 12+ chars> ("Complete commit subject") Please use git commit description style 'commit <12+ chars of sha1> Hence patches containing commit IDs with more than 12 characters are flagged by checkpatch, and sometimes rejected by maintainers or reviewers. This is becoming more important with the growth of the repository, as git may decide to use more characters in case of local conflicts. Fix this by settling on at least 12 characters, in both the documentation and in the checkpatch.pl script. Fixes: bd17e036b495bebb ("checkpatch: warn for non-standard fixes tag style") Signed-off-by: Geert Uytterhoeven <geert+renesas@glider.be> Link: https://lore.kernel.org/r/1c244040bf6ce304656e31036e5178b4b9dfb719.1733421037.git.geert+renesas@glider.be Signed-off-by: Jonathan Corbet <corbet@lwn.net> --- Documentation/process/maintainer-tip.rst | 2 +- Documentation/process/submitting-patches.rst | 8 ++++---- scripts/checkpatch.pl | 4 ++-- 3 files changed, 7 insertions(+), 7 deletions(-) (limited to 'scripts') diff --git a/Documentation/process/maintainer-tip.rst b/Documentation/process/maintainer-tip.rst index e374b67b3277..41d5855700cd 100644 --- a/Documentation/process/maintainer-tip.rst +++ b/Documentation/process/maintainer-tip.rst @@ -270,7 +270,7 @@ Ordering of commit tags To have a uniform view of the commit tags, the tip maintainers use the following tag ordering scheme: - - Fixes: 12char-SHA1 ("sub/sys: Original subject line") + - Fixes: 12+char-SHA1 ("sub/sys: Original subject line") A Fixes tag should be added even for changes which do not need to be backported to stable kernels, i.e. when addressing a recently introduced diff --git a/Documentation/process/submitting-patches.rst b/Documentation/process/submitting-patches.rst index 1474c84b3ac5..816bdb037e02 100644 --- a/Documentation/process/submitting-patches.rst +++ b/Documentation/process/submitting-patches.rst @@ -143,10 +143,10 @@ also track such tags and take certain actions. Private bug trackers and invalid URLs are forbidden. If your patch fixes a bug in a specific commit, e.g. you found an issue using -``git bisect``, please use the 'Fixes:' tag with the first 12 characters of -the SHA-1 ID, and the one line summary. Do not split the tag across multiple -lines, tags are exempt from the "wrap at 75 columns" rule in order to simplify -parsing scripts. For example:: +``git bisect``, please use the 'Fixes:' tag with at least the first 12 +characters of the SHA-1 ID, and the one line summary. Do not split the tag +across multiple lines, tags are exempt from the "wrap at 75 columns" rule in +order to simplify parsing scripts. For example:: Fixes: 54a4f0239f2e ("KVM: MMU: make kvm_mmu_zap_page() return the number of pages it actually freed") diff --git a/scripts/checkpatch.pl b/scripts/checkpatch.pl index 9eed3683ad76..f7087dda9ac9 100755 --- a/scripts/checkpatch.pl +++ b/scripts/checkpatch.pl @@ -3230,7 +3230,7 @@ sub process { my $tag_case = not ($tag eq "Fixes:"); my $tag_space = not ($line =~ /^fixes:? [0-9a-f]{5,40} ($balanced_parens)/i); - my $id_length = not ($orig_commit =~ /^[0-9a-f]{12}$/i); + my $id_length = not ($orig_commit =~ /^[0-9a-f]{12,40}$/i); my $id_case = not ($orig_commit !~ /[A-F]/); my $id = "0123456789ab"; @@ -3240,7 +3240,7 @@ sub process { if ($ctitle ne $title || $tag_case || $tag_space || $id_length || $id_case || !$title_has_quotes) { if (WARN("BAD_FIXES_TAG", - "Please use correct Fixes: style 'Fixes: <12 chars of sha1> (\"<title line>\")' - ie: 'Fixes: $cid (\"$ctitle\")'\n" . $herecurr) && + "Please use correct Fixes: style 'Fixes: <12+ chars of sha1> (\"<title line>\")' - ie: 'Fixes: $cid (\"$ctitle\")'\n" . $herecurr) && $fix) { $fixed[$fixlinenr] = "Fixes: $cid (\"$ctitle\")"; } -- cgit v1.2.3 From 28b24394c6e9a3166fcb4480cba054562526657c Mon Sep 17 00:00:00 2001 From: Steven Rostedt <rostedt@goodmis.org> Date: Sun, 5 Jan 2025 11:22:12 -0500 Subject: scripts/sorttable: Remove unused macro defines The code of sorttable.h was copied from the recordmcount.h which defined a bunch of Elf MACROs so that they could be used between 32bit and 64bit functions. But there's several MACROs that sorttable.h does not use but was copied over. Remove them to clean up the code. Cc: bpf <bpf@vger.kernel.org> Cc: Masami Hiramatsu <mhiramat@kernel.org> Cc: Mark Rutland <mark.rutland@arm.com> Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Masahiro Yamada <masahiroy@kernel.org> Cc: Nathan Chancellor <nathan@kernel.org> Cc: Nicolas Schier <nicolas@fjasle.eu> Cc: Zheng Yejian <zhengyejian1@huawei.com> Cc: Martin Kelly <martin.kelly@crowdstrike.com> Cc: Christophe Leroy <christophe.leroy@csgroup.eu> Cc: Josh Poimboeuf <jpoimboe@redhat.com> Link: https://lore.kernel.org/20250105162344.128870118@goodmis.org Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org> --- scripts/sorttable.h | 27 --------------------------- 1 file changed, 27 deletions(-) (limited to 'scripts') diff --git a/scripts/sorttable.h b/scripts/sorttable.h index a7c5445baf00..14d0c4d843e8 100644 --- a/scripts/sorttable.h +++ b/scripts/sorttable.h @@ -27,19 +27,10 @@ #undef Elf_Ehdr #undef Elf_Shdr #undef Elf_Rel -#undef Elf_Rela #undef Elf_Sym -#undef ELF_R_SYM -#undef Elf_r_sym -#undef ELF_R_INFO -#undef Elf_r_info -#undef ELF_ST_BIND #undef ELF_ST_TYPE -#undef fn_ELF_R_SYM -#undef fn_ELF_R_INFO #undef uint_t #undef _r -#undef _w #ifdef SORTTABLE_64 # define extable_ent_size 16 @@ -52,19 +43,10 @@ # define Elf_Ehdr Elf64_Ehdr # define Elf_Shdr Elf64_Shdr # define Elf_Rel Elf64_Rel -# define Elf_Rela Elf64_Rela # define Elf_Sym Elf64_Sym -# define ELF_R_SYM ELF64_R_SYM -# define Elf_r_sym Elf64_r_sym -# define ELF_R_INFO ELF64_R_INFO -# define Elf_r_info Elf64_r_info -# define ELF_ST_BIND ELF64_ST_BIND # define ELF_ST_TYPE ELF64_ST_TYPE -# define fn_ELF_R_SYM fn_ELF64_R_SYM -# define fn_ELF_R_INFO fn_ELF64_R_INFO # define uint_t uint64_t # define _r r8 -# define _w w8 #else # define extable_ent_size 8 # define compare_extable compare_extable_32 @@ -76,19 +58,10 @@ # define Elf_Ehdr Elf32_Ehdr # define Elf_Shdr Elf32_Shdr # define Elf_Rel Elf32_Rel -# define Elf_Rela Elf32_Rela # define Elf_Sym Elf32_Sym -# define ELF_R_SYM ELF32_R_SYM -# define Elf_r_sym Elf32_r_sym -# define ELF_R_INFO ELF32_R_INFO -# define Elf_r_info Elf32_r_info -# define ELF_ST_BIND ELF32_ST_BIND # define ELF_ST_TYPE ELF32_ST_TYPE -# define fn_ELF_R_SYM fn_ELF32_R_SYM -# define fn_ELF_R_INFO fn_ELF32_R_INFO # define uint_t uint32_t # define _r r -# define _w w #endif #if defined(SORTTABLE_64) && defined(UNWINDER_ORC_ENABLED) -- cgit v1.2.3 From 4f48a28b37d594dab38092514a42ae9f4b781553 Mon Sep 17 00:00:00 2001 From: Steven Rostedt <rostedt@goodmis.org> Date: Sun, 5 Jan 2025 11:22:13 -0500 Subject: scripts/sorttable: Remove unused write functions The code of sorttable.h was copied from the recordmcount.h which defined various write functions for different sizes (2, 4, 8 byte lengths). But sorttable only uses the 4 byte writes. Remove the extra versions as they are not used. Cc: bpf <bpf@vger.kernel.org> Cc: Masami Hiramatsu <mhiramat@kernel.org> Cc: Mark Rutland <mark.rutland@arm.com> Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Masahiro Yamada <masahiroy@kernel.org> Cc: Nathan Chancellor <nathan@kernel.org> Cc: Nicolas Schier <nicolas@fjasle.eu> Cc: Zheng Yejian <zhengyejian1@huawei.com> Cc: Martin Kelly <martin.kelly@crowdstrike.com> Cc: Christophe Leroy <christophe.leroy@csgroup.eu> Cc: Josh Poimboeuf <jpoimboe@redhat.com> Link: https://lore.kernel.org/20250105162344.314385504@goodmis.org Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org> --- scripts/sorttable.c | 26 -------------------------- 1 file changed, 26 deletions(-) (limited to 'scripts') diff --git a/scripts/sorttable.c b/scripts/sorttable.c index 83cdb843d92f..4dcdbf7a5e26 100644 --- a/scripts/sorttable.c +++ b/scripts/sorttable.c @@ -68,8 +68,6 @@ static uint32_t (*r)(const uint32_t *); static uint16_t (*r2)(const uint16_t *); static uint64_t (*r8)(const uint64_t *); static void (*w)(uint32_t, uint32_t *); -static void (*w2)(uint16_t, uint16_t *); -static void (*w8)(uint64_t, uint64_t *); typedef void (*table_sort_t)(char *, int); /* @@ -146,31 +144,11 @@ static void wbe(uint32_t val, uint32_t *x) put_unaligned_be32(val, x); } -static void w2be(uint16_t val, uint16_t *x) -{ - put_unaligned_be16(val, x); -} - -static void w8be(uint64_t val, uint64_t *x) -{ - put_unaligned_be64(val, x); -} - static void wle(uint32_t val, uint32_t *x) { put_unaligned_le32(val, x); } -static void w2le(uint16_t val, uint16_t *x) -{ - put_unaligned_le16(val, x); -} - -static void w8le(uint64_t val, uint64_t *x) -{ - put_unaligned_le64(val, x); -} - /* * Move reserved section indices SHN_LORESERVE..SHN_HIRESERVE out of * the way to -256..-1, to avoid conflicting with real section @@ -277,16 +255,12 @@ static int do_file(char const *const fname, void *addr) r2 = r2le; r8 = r8le; w = wle; - w2 = w2le; - w8 = w8le; break; case ELFDATA2MSB: r = rbe; r2 = r2be; r8 = r8be; w = wbe; - w2 = w2be; - w8 = w8be; break; default: fprintf(stderr, "unrecognized ELF data encoding %d: %s\n", -- cgit v1.2.3 From 6f2c2f93a190467cebd6ebd03feb49514fead5ca Mon Sep 17 00:00:00 2001 From: Steven Rostedt <rostedt@goodmis.org> Date: Sun, 5 Jan 2025 11:22:14 -0500 Subject: scripts/sorttable: Remove unneeded Elf_Rel The code had references to initialize the Elf_Rel relocation tables, but it was never used. Remove it. Cc: bpf <bpf@vger.kernel.org> Cc: Masami Hiramatsu <mhiramat@kernel.org> Cc: Mark Rutland <mark.rutland@arm.com> Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Masahiro Yamada <masahiroy@kernel.org> Cc: Nathan Chancellor <nathan@kernel.org> Cc: Nicolas Schier <nicolas@fjasle.eu> Cc: Zheng Yejian <zhengyejian1@huawei.com> Cc: Martin Kelly <martin.kelly@crowdstrike.com> Cc: Christophe Leroy <christophe.leroy@csgroup.eu> Cc: Josh Poimboeuf <jpoimboe@redhat.com> Link: https://lore.kernel.org/20250105162344.515342233@goodmis.org Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org> --- scripts/sorttable.h | 23 ++--------------------- 1 file changed, 2 insertions(+), 21 deletions(-) (limited to 'scripts') diff --git a/scripts/sorttable.h b/scripts/sorttable.h index 14d0c4d843e8..18d07fdb2716 100644 --- a/scripts/sorttable.h +++ b/scripts/sorttable.h @@ -26,7 +26,6 @@ #undef Elf_Addr #undef Elf_Ehdr #undef Elf_Shdr -#undef Elf_Rel #undef Elf_Sym #undef ELF_ST_TYPE #undef uint_t @@ -42,7 +41,6 @@ # define Elf_Addr Elf64_Addr # define Elf_Ehdr Elf64_Ehdr # define Elf_Shdr Elf64_Shdr -# define Elf_Rel Elf64_Rel # define Elf_Sym Elf64_Sym # define ELF_ST_TYPE ELF64_ST_TYPE # define uint_t uint64_t @@ -57,7 +55,6 @@ # define Elf_Addr Elf32_Addr # define Elf_Ehdr Elf32_Ehdr # define Elf_Shdr Elf32_Shdr -# define Elf_Rel Elf32_Rel # define Elf_Sym Elf32_Sym # define ELF_ST_TYPE ELF32_ST_TYPE # define uint_t uint32_t @@ -248,14 +245,10 @@ static int do_sort(Elf_Ehdr *ehdr, Elf32_Word *symtab_shndx = NULL; Elf_Sym *sort_needed_sym = NULL; Elf_Shdr *sort_needed_sec; - Elf_Rel *relocs = NULL; - int relocs_size = 0; uint32_t *sort_needed_loc; const char *secstrings; const char *strtab; char *extab_image; - int extab_index = 0; - int i; int idx; unsigned int shnum; unsigned int shstrndx; @@ -279,23 +272,15 @@ static int do_sort(Elf_Ehdr *ehdr, if (shnum == SHN_UNDEF) shnum = _r(&shdr[0].sh_size); - for (i = 0, s = shdr; s < shdr + shnum; i++, s++) { + for (s = shdr; s < shdr + shnum; s++) { idx = r(&s->sh_name); - if (!strcmp(secstrings + idx, "__ex_table")) { + if (!strcmp(secstrings + idx, "__ex_table")) extab_sec = s; - extab_index = i; - } if (!strcmp(secstrings + idx, ".symtab")) symtab_sec = s; if (!strcmp(secstrings + idx, ".strtab")) strtab_sec = s; - if ((r(&s->sh_type) == SHT_REL || - r(&s->sh_type) == SHT_RELA) && - r(&s->sh_info) == extab_index) { - relocs = (void *)ehdr + _r(&s->sh_offset); - relocs_size = _r(&s->sh_size); - } if (r(&s->sh_type) == SHT_SYMTAB_SHNDX) symtab_shndx = (Elf32_Word *)((const char *)ehdr + _r(&s->sh_offset)); @@ -397,10 +382,6 @@ static int do_sort(Elf_Ehdr *ehdr, extable_ent_size, compare_extable); } - /* If there were relocations, we no longer need them. */ - if (relocs) - memset(relocs, 0, relocs_size); - /* find the flag main_extable_sort_needed */ for (sym = (void *)ehdr + _r(&symtab_sec->sh_offset); sym < sym + _r(&symtab_sec->sh_size) / sizeof(Elf_Sym); -- cgit v1.2.3 From 66990c003306c240d570b3ba274ec4f68cf18c91 Mon Sep 17 00:00:00 2001 From: Steven Rostedt <rostedt@goodmis.org> Date: Sun, 5 Jan 2025 11:22:15 -0500 Subject: scripts/sorttable: Have the ORC code use the _r() functions to read The ORC code reads the section information directly from the file. This currently works because the default read function is for 64bit little endian machines. But if for some reason that ever changes, this will break. Instead of having a surprise breakage, use the _r() functions that will read the values from the file properly. Cc: bpf <bpf@vger.kernel.org> Cc: Masami Hiramatsu <mhiramat@kernel.org> Cc: Mark Rutland <mark.rutland@arm.com> Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Masahiro Yamada <masahiroy@kernel.org> Cc: Nathan Chancellor <nathan@kernel.org> Cc: Nicolas Schier <nicolas@fjasle.eu> Cc: Zheng Yejian <zhengyejian1@huawei.com> Cc: Martin Kelly <martin.kelly@crowdstrike.com> Cc: Christophe Leroy <christophe.leroy@csgroup.eu> Cc: Josh Poimboeuf <jpoimboe@redhat.com> Link: https://lore.kernel.org/20250105162344.721480386@goodmis.org Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org> --- scripts/sorttable.h | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) (limited to 'scripts') diff --git a/scripts/sorttable.h b/scripts/sorttable.h index 18d07fdb2716..58f7ab5f5644 100644 --- a/scripts/sorttable.h +++ b/scripts/sorttable.h @@ -299,14 +299,14 @@ static int do_sort(Elf_Ehdr *ehdr, #if defined(SORTTABLE_64) && defined(UNWINDER_ORC_ENABLED) /* locate the ORC unwind tables */ if (!strcmp(secstrings + idx, ".orc_unwind_ip")) { - orc_ip_size = s->sh_size; + orc_ip_size = _r(&s->sh_size); g_orc_ip_table = (int *)((void *)ehdr + - s->sh_offset); + _r(&s->sh_offset)); } if (!strcmp(secstrings + idx, ".orc_unwind")) { - orc_size = s->sh_size; + orc_size = _r(&s->sh_size); g_orc_table = (struct orc_entry *)((void *)ehdr + - s->sh_offset); + _r(&s->sh_offset)); } #endif } /* for loop */ -- cgit v1.2.3 From 7ffc0d0819f438779ed592e2e2e3576f43ce14f0 Mon Sep 17 00:00:00 2001 From: Steven Rostedt <rostedt@goodmis.org> Date: Sun, 5 Jan 2025 11:22:16 -0500 Subject: scripts/sorttable: Make compare_extable() into two functions Instead of having the compare_extable() part of the sorttable.h header where it get's defined twice, since it is a very simple function, just define it twice in sorttable.c, and then it can use the proper read functions for the word size and endianess and the Elf_Addr macro can be removed from sorttable.h. Also add a micro optimization. Instead of: if (a < b) return -1; if (a > b) return 1; return 0; That can be shorten to: if (a < b) return -1; return a > b; Cc: bpf <bpf@vger.kernel.org> Cc: Masami Hiramatsu <mhiramat@kernel.org> Cc: Mark Rutland <mark.rutland@arm.com> Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Masahiro Yamada <masahiroy@kernel.org> Cc: Nathan Chancellor <nathan@kernel.org> Cc: Nicolas Schier <nicolas@fjasle.eu> Cc: Zheng Yejian <zhengyejian1@huawei.com> Cc: Martin Kelly <martin.kelly@crowdstrike.com> Cc: Christophe Leroy <christophe.leroy@csgroup.eu> Cc: Josh Poimboeuf <jpoimboe@redhat.com> Link: https://lore.kernel.org/20250105162344.945299671@goodmis.org Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org> --- scripts/sorttable.c | 20 ++++++++++++++++++++ scripts/sorttable.h | 14 -------------- 2 files changed, 20 insertions(+), 14 deletions(-) (limited to 'scripts') diff --git a/scripts/sorttable.c b/scripts/sorttable.c index 4dcdbf7a5e26..3e2c17e91485 100644 --- a/scripts/sorttable.c +++ b/scripts/sorttable.c @@ -173,6 +173,26 @@ static inline unsigned int get_secindex(unsigned int shndx, return r(&symtab_shndx_start[sym_offs]); } +static int compare_extable_32(const void *a, const void *b) +{ + Elf32_Addr av = r(a); + Elf32_Addr bv = r(b); + + if (av < bv) + return -1; + return av > bv; +} + +static int compare_extable_64(const void *a, const void *b) +{ + Elf64_Addr av = r8(a); + Elf64_Addr bv = r8(b); + + if (av < bv) + return -1; + return av > bv; +} + /* 32 bit and 64 bit are very similar */ #include "sorttable.h" #define SORTTABLE_64 diff --git a/scripts/sorttable.h b/scripts/sorttable.h index 58f7ab5f5644..36655ff16b39 100644 --- a/scripts/sorttable.h +++ b/scripts/sorttable.h @@ -23,7 +23,6 @@ #undef sort_mcount_loc #undef elf_mcount_loc #undef do_sort -#undef Elf_Addr #undef Elf_Ehdr #undef Elf_Shdr #undef Elf_Sym @@ -38,7 +37,6 @@ # define sort_mcount_loc sort_mcount_loc_64 # define elf_mcount_loc elf_mcount_loc_64 # define do_sort do_sort_64 -# define Elf_Addr Elf64_Addr # define Elf_Ehdr Elf64_Ehdr # define Elf_Shdr Elf64_Shdr # define Elf_Sym Elf64_Sym @@ -52,7 +50,6 @@ # define sort_mcount_loc sort_mcount_loc_32 # define elf_mcount_loc elf_mcount_loc_32 # define do_sort do_sort_32 -# define Elf_Addr Elf32_Addr # define Elf_Ehdr Elf32_Ehdr # define Elf_Shdr Elf32_Shdr # define Elf_Sym Elf32_Sym @@ -160,17 +157,6 @@ static void *sort_orctable(void *arg) } #endif -static int compare_extable(const void *a, const void *b) -{ - Elf_Addr av = _r(a); - Elf_Addr bv = _r(b); - - if (av < bv) - return -1; - if (av > bv) - return 1; - return 0; -} #ifdef MCOUNT_SORT_ENABLED pthread_t mcount_sort_thread; -- cgit v1.2.3 From 157fb5b3cfd2cb5950314f926a76e567fc1921c5 Mon Sep 17 00:00:00 2001 From: Steven Rostedt <rostedt@goodmis.org> Date: Sun, 5 Jan 2025 11:22:17 -0500 Subject: scripts/sorttable: Convert Elf_Ehdr to union In order to remove the double #include of sorttable.h for 64 and 32 bit to create duplicate functions for both, replace the Elf_Ehdr macro with a union that defines both Elf64_Ehdr and Elf32_Ehdr, with field e64 for the 64bit version, and e32 for the 32bit version. Then a macro etype can be used instead to get to the proper value. This will eventually be replaced with just single functions that can handle both 32bit and 64bit ELF parsing. Cc: bpf <bpf@vger.kernel.org> Cc: Masami Hiramatsu <mhiramat@kernel.org> Cc: Mark Rutland <mark.rutland@arm.com> Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Masahiro Yamada <masahiroy@kernel.org> Cc: Nathan Chancellor <nathan@kernel.org> Cc: Nicolas Schier <nicolas@fjasle.eu> Cc: Zheng Yejian <zhengyejian1@huawei.com> Cc: Martin Kelly <martin.kelly@crowdstrike.com> Cc: Christophe Leroy <christophe.leroy@csgroup.eu> Cc: Josh Poimboeuf <jpoimboe@redhat.com> Link: https://lore.kernel.org/20250105162345.148224465@goodmis.org Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org> --- scripts/sorttable.c | 36 ++++++++++++++++++++---------------- scripts/sorttable.h | 12 ++++++------ 2 files changed, 26 insertions(+), 22 deletions(-) (limited to 'scripts') diff --git a/scripts/sorttable.c b/scripts/sorttable.c index 3e2c17e91485..67cbbfc8214d 100644 --- a/scripts/sorttable.c +++ b/scripts/sorttable.c @@ -64,6 +64,11 @@ #define EM_LOONGARCH 258 #endif +typedef union { + Elf32_Ehdr e32; + Elf64_Ehdr e64; +} Elf_Ehdr; + static uint32_t (*r)(const uint32_t *); static uint16_t (*r2)(const uint16_t *); static uint64_t (*r8)(const uint64_t *); @@ -266,10 +271,10 @@ static void sort_relative_table_with_data(char *extab_image, int image_size) static int do_file(char const *const fname, void *addr) { int rc = -1; - Elf32_Ehdr *ehdr = addr; + Elf_Ehdr *ehdr = addr; table_sort_t custom_sort = NULL; - switch (ehdr->e_ident[EI_DATA]) { + switch (ehdr->e32.e_ident[EI_DATA]) { case ELFDATA2LSB: r = rle; r2 = r2le; @@ -284,18 +289,18 @@ static int do_file(char const *const fname, void *addr) break; default: fprintf(stderr, "unrecognized ELF data encoding %d: %s\n", - ehdr->e_ident[EI_DATA], fname); + ehdr->e32.e_ident[EI_DATA], fname); return -1; } - if (memcmp(ELFMAG, ehdr->e_ident, SELFMAG) != 0 || - (r2(&ehdr->e_type) != ET_EXEC && r2(&ehdr->e_type) != ET_DYN) || - ehdr->e_ident[EI_VERSION] != EV_CURRENT) { + if (memcmp(ELFMAG, ehdr->e32.e_ident, SELFMAG) != 0 || + (r2(&ehdr->e32.e_type) != ET_EXEC && r2(&ehdr->e32.e_type) != ET_DYN) || + ehdr->e32.e_ident[EI_VERSION] != EV_CURRENT) { fprintf(stderr, "unrecognized ET_EXEC/ET_DYN file %s\n", fname); return -1; } - switch (r2(&ehdr->e_machine)) { + switch (r2(&ehdr->e32.e_machine)) { case EM_386: case EM_AARCH64: case EM_LOONGARCH: @@ -318,14 +323,14 @@ static int do_file(char const *const fname, void *addr) break; default: fprintf(stderr, "unrecognized e_machine %d %s\n", - r2(&ehdr->e_machine), fname); + r2(&ehdr->e32.e_machine), fname); return -1; } - switch (ehdr->e_ident[EI_CLASS]) { + switch (ehdr->e32.e_ident[EI_CLASS]) { case ELFCLASS32: - if (r2(&ehdr->e_ehsize) != sizeof(Elf32_Ehdr) || - r2(&ehdr->e_shentsize) != sizeof(Elf32_Shdr)) { + if (r2(&ehdr->e32.e_ehsize) != sizeof(Elf32_Ehdr) || + r2(&ehdr->e32.e_shentsize) != sizeof(Elf32_Shdr)) { fprintf(stderr, "unrecognized ET_EXEC/ET_DYN file: %s\n", fname); break; @@ -334,20 +339,19 @@ static int do_file(char const *const fname, void *addr) break; case ELFCLASS64: { - Elf64_Ehdr *const ghdr = (Elf64_Ehdr *)ehdr; - if (r2(&ghdr->e_ehsize) != sizeof(Elf64_Ehdr) || - r2(&ghdr->e_shentsize) != sizeof(Elf64_Shdr)) { + if (r2(&ehdr->e64.e_ehsize) != sizeof(Elf64_Ehdr) || + r2(&ehdr->e64.e_shentsize) != sizeof(Elf64_Shdr)) { fprintf(stderr, "unrecognized ET_EXEC/ET_DYN file: %s\n", fname); break; } - rc = do_sort_64(ghdr, fname, custom_sort); + rc = do_sort_64(ehdr, fname, custom_sort); } break; default: fprintf(stderr, "unrecognized ELF class %d %s\n", - ehdr->e_ident[EI_CLASS], fname); + ehdr->e32.e_ident[EI_CLASS], fname); break; } diff --git a/scripts/sorttable.h b/scripts/sorttable.h index 36655ff16b39..be8b529498fb 100644 --- a/scripts/sorttable.h +++ b/scripts/sorttable.h @@ -23,12 +23,12 @@ #undef sort_mcount_loc #undef elf_mcount_loc #undef do_sort -#undef Elf_Ehdr #undef Elf_Shdr #undef Elf_Sym #undef ELF_ST_TYPE #undef uint_t #undef _r +#undef etype #ifdef SORTTABLE_64 # define extable_ent_size 16 @@ -37,12 +37,12 @@ # define sort_mcount_loc sort_mcount_loc_64 # define elf_mcount_loc elf_mcount_loc_64 # define do_sort do_sort_64 -# define Elf_Ehdr Elf64_Ehdr # define Elf_Shdr Elf64_Shdr # define Elf_Sym Elf64_Sym # define ELF_ST_TYPE ELF64_ST_TYPE # define uint_t uint64_t # define _r r8 +# define etype e64 #else # define extable_ent_size 8 # define compare_extable compare_extable_32 @@ -50,12 +50,12 @@ # define sort_mcount_loc sort_mcount_loc_32 # define elf_mcount_loc elf_mcount_loc_32 # define do_sort do_sort_32 -# define Elf_Ehdr Elf32_Ehdr # define Elf_Shdr Elf32_Shdr # define Elf_Sym Elf32_Sym # define ELF_ST_TYPE ELF32_ST_TYPE # define uint_t uint32_t # define _r r +# define etype e32 #endif #if defined(SORTTABLE_64) && defined(UNWINDER_ORC_ENABLED) @@ -222,7 +222,7 @@ static int do_sort(Elf_Ehdr *ehdr, table_sort_t custom_sort) { int rc = -1; - Elf_Shdr *s, *shdr = (Elf_Shdr *)((char *)ehdr + _r(&ehdr->e_shoff)); + Elf_Shdr *s, *shdr = (Elf_Shdr *)((char *)ehdr + _r(&ehdr->etype.e_shoff)); Elf_Shdr *strtab_sec = NULL; Elf_Shdr *symtab_sec = NULL; Elf_Shdr *extab_sec = NULL; @@ -249,12 +249,12 @@ static int do_sort(Elf_Ehdr *ehdr, unsigned int orc_num_entries = 0; #endif - shstrndx = r2(&ehdr->e_shstrndx); + shstrndx = r2(&ehdr->etype.e_shstrndx); if (shstrndx == SHN_XINDEX) shstrndx = r(&shdr[0].sh_link); secstrings = (const char *)ehdr + _r(&shdr[shstrndx].sh_offset); - shnum = r2(&ehdr->e_shnum); + shnum = r2(&ehdr->etype.e_shnum); if (shnum == SHN_UNDEF) shnum = _r(&shdr[0].sh_size); -- cgit v1.2.3 From 545f6cf8f4c9a268e0bab2637f1d279679befdbf Mon Sep 17 00:00:00 2001 From: Steven Rostedt <rostedt@goodmis.org> Date: Sun, 5 Jan 2025 11:22:18 -0500 Subject: scripts/sorttable: Replace Elf_Shdr Macro with a union In order to remove the double #include of sorttable.h for 64 and 32 bit to create duplicate functions for both, replace the Elf_Shdr macro with a union that defines both Elf64_Shdr and Elf32_Shdr, with field e64 for the 64bit version, and e32 for the 32bit version. It can then use the macro etype to get the proper value. This will eventually be replaced with just single functions that can handle both 32bit and 64bit ELF parsing. Cc: bpf <bpf@vger.kernel.org> Cc: Masami Hiramatsu <mhiramat@kernel.org> Cc: Mark Rutland <mark.rutland@arm.com> Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Masahiro Yamada <masahiroy@kernel.org> Cc: Nathan Chancellor <nathan@kernel.org> Cc: Nicolas Schier <nicolas@fjasle.eu> Cc: Zheng Yejian <zhengyejian1@huawei.com> Cc: Martin Kelly <martin.kelly@crowdstrike.com> Cc: Christophe Leroy <christophe.leroy@csgroup.eu> Cc: Josh Poimboeuf <jpoimboe@redhat.com> Link: https://lore.kernel.org/20250105162345.339462681@goodmis.org Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org> --- scripts/sorttable.c | 10 ++++++++ scripts/sorttable.h | 74 +++++++++++++++++++++++++++++------------------------ 2 files changed, 51 insertions(+), 33 deletions(-) (limited to 'scripts') diff --git a/scripts/sorttable.c b/scripts/sorttable.c index 67cbbfc8214d..94497b8ab04c 100644 --- a/scripts/sorttable.c +++ b/scripts/sorttable.c @@ -69,6 +69,11 @@ typedef union { Elf64_Ehdr e64; } Elf_Ehdr; +typedef union { + Elf32_Shdr e32; + Elf64_Shdr e64; +} Elf_Shdr; + static uint32_t (*r)(const uint32_t *); static uint16_t (*r2)(const uint16_t *); static uint64_t (*r8)(const uint64_t *); @@ -198,6 +203,11 @@ static int compare_extable_64(const void *a, const void *b) return av > bv; } +static inline void *get_index(void *start, int entsize, int index) +{ + return start + (entsize * index); +} + /* 32 bit and 64 bit are very similar */ #include "sorttable.h" #define SORTTABLE_64 diff --git a/scripts/sorttable.h b/scripts/sorttable.h index be8b529498fb..3daf37bb6b9a 100644 --- a/scripts/sorttable.h +++ b/scripts/sorttable.h @@ -23,7 +23,6 @@ #undef sort_mcount_loc #undef elf_mcount_loc #undef do_sort -#undef Elf_Shdr #undef Elf_Sym #undef ELF_ST_TYPE #undef uint_t @@ -37,7 +36,6 @@ # define sort_mcount_loc sort_mcount_loc_64 # define elf_mcount_loc elf_mcount_loc_64 # define do_sort do_sort_64 -# define Elf_Shdr Elf64_Shdr # define Elf_Sym Elf64_Sym # define ELF_ST_TYPE ELF64_ST_TYPE # define uint_t uint64_t @@ -50,7 +48,6 @@ # define sort_mcount_loc sort_mcount_loc_32 # define elf_mcount_loc elf_mcount_loc_32 # define do_sort do_sort_32 -# define Elf_Shdr Elf32_Shdr # define Elf_Sym Elf32_Sym # define ELF_ST_TYPE ELF32_ST_TYPE # define uint_t uint32_t @@ -171,8 +168,8 @@ struct elf_mcount_loc { static void *sort_mcount_loc(void *arg) { struct elf_mcount_loc *emloc = (struct elf_mcount_loc *)arg; - uint_t offset = emloc->start_mcount_loc - _r(&(emloc->init_data_sec)->sh_addr) - + _r(&(emloc->init_data_sec)->sh_offset); + uint_t offset = emloc->start_mcount_loc - _r(&(emloc->init_data_sec)->etype.sh_addr) + + _r(&(emloc->init_data_sec)->etype.sh_offset); uint_t count = emloc->stop_mcount_loc - emloc->start_mcount_loc; unsigned char *start_loc = (void *)emloc->ehdr + offset; @@ -222,10 +219,11 @@ static int do_sort(Elf_Ehdr *ehdr, table_sort_t custom_sort) { int rc = -1; - Elf_Shdr *s, *shdr = (Elf_Shdr *)((char *)ehdr + _r(&ehdr->etype.e_shoff)); + Elf_Shdr *shdr_start; Elf_Shdr *strtab_sec = NULL; Elf_Shdr *symtab_sec = NULL; Elf_Shdr *extab_sec = NULL; + Elf_Shdr *string_sec; Elf_Sym *sym; const Elf_Sym *symtab; Elf32_Word *symtab_shndx = NULL; @@ -235,7 +233,10 @@ static int do_sort(Elf_Ehdr *ehdr, const char *secstrings; const char *strtab; char *extab_image; + int sort_need_index; + int shentsize; int idx; + int i; unsigned int shnum; unsigned int shstrndx; #ifdef MCOUNT_SORT_ENABLED @@ -249,34 +250,40 @@ static int do_sort(Elf_Ehdr *ehdr, unsigned int orc_num_entries = 0; #endif + shdr_start = (Elf_Shdr *)((char *)ehdr + _r(&ehdr->etype.e_shoff)); + shentsize = r2(&ehdr->etype.e_shentsize); + shstrndx = r2(&ehdr->etype.e_shstrndx); if (shstrndx == SHN_XINDEX) - shstrndx = r(&shdr[0].sh_link); - secstrings = (const char *)ehdr + _r(&shdr[shstrndx].sh_offset); + shstrndx = r(&shdr_start->etype.sh_link); + string_sec = get_index(shdr_start, shentsize, shstrndx); + secstrings = (const char *)ehdr + _r(&string_sec->etype.sh_offset); shnum = r2(&ehdr->etype.e_shnum); if (shnum == SHN_UNDEF) - shnum = _r(&shdr[0].sh_size); + shnum = _r(&shdr_start->etype.sh_size); + + for (i = 0; i < shnum; i++) { + Elf_Shdr *shdr = get_index(shdr_start, shentsize, i); - for (s = shdr; s < shdr + shnum; s++) { - idx = r(&s->sh_name); + idx = r(&shdr->etype.sh_name); if (!strcmp(secstrings + idx, "__ex_table")) - extab_sec = s; + extab_sec = shdr; if (!strcmp(secstrings + idx, ".symtab")) - symtab_sec = s; + symtab_sec = shdr; if (!strcmp(secstrings + idx, ".strtab")) - strtab_sec = s; + strtab_sec = shdr; - if (r(&s->sh_type) == SHT_SYMTAB_SHNDX) + if (r(&shdr->etype.sh_type) == SHT_SYMTAB_SHNDX) symtab_shndx = (Elf32_Word *)((const char *)ehdr + - _r(&s->sh_offset)); + _r(&shdr->etype.sh_offset)); #ifdef MCOUNT_SORT_ENABLED /* locate the .init.data section in vmlinux */ if (!strcmp(secstrings + idx, ".init.data")) { get_mcount_loc(&_start_mcount_loc, &_stop_mcount_loc); mstruct.ehdr = ehdr; - mstruct.init_data_sec = s; + mstruct.init_data_sec = shdr; mstruct.start_mcount_loc = _start_mcount_loc; mstruct.stop_mcount_loc = _stop_mcount_loc; } @@ -285,14 +292,14 @@ static int do_sort(Elf_Ehdr *ehdr, #if defined(SORTTABLE_64) && defined(UNWINDER_ORC_ENABLED) /* locate the ORC unwind tables */ if (!strcmp(secstrings + idx, ".orc_unwind_ip")) { - orc_ip_size = _r(&s->sh_size); + orc_ip_size = _r(&shdr->etype.sh_size); g_orc_ip_table = (int *)((void *)ehdr + - _r(&s->sh_offset)); + _r(&shdr->etype.sh_offset)); } if (!strcmp(secstrings + idx, ".orc_unwind")) { - orc_size = _r(&s->sh_size); + orc_size = _r(&shdr->etype.sh_size); g_orc_table = (struct orc_entry *)((void *)ehdr + - _r(&s->sh_offset)); + _r(&shdr->etype.sh_offset)); } #endif } /* for loop */ @@ -355,22 +362,22 @@ static int do_sort(Elf_Ehdr *ehdr, goto out; } - extab_image = (void *)ehdr + _r(&extab_sec->sh_offset); - strtab = (const char *)ehdr + _r(&strtab_sec->sh_offset); + extab_image = (void *)ehdr + _r(&extab_sec->etype.sh_offset); + strtab = (const char *)ehdr + _r(&strtab_sec->etype.sh_offset); symtab = (const Elf_Sym *)((const char *)ehdr + - _r(&symtab_sec->sh_offset)); + _r(&symtab_sec->etype.sh_offset)); if (custom_sort) { - custom_sort(extab_image, _r(&extab_sec->sh_size)); + custom_sort(extab_image, _r(&extab_sec->etype.sh_size)); } else { - int num_entries = _r(&extab_sec->sh_size) / extable_ent_size; + int num_entries = _r(&extab_sec->etype.sh_size) / extable_ent_size; qsort(extab_image, num_entries, extable_ent_size, compare_extable); } /* find the flag main_extable_sort_needed */ - for (sym = (void *)ehdr + _r(&symtab_sec->sh_offset); - sym < sym + _r(&symtab_sec->sh_size) / sizeof(Elf_Sym); + for (sym = (void *)ehdr + _r(&symtab_sec->etype.sh_offset); + sym < sym + _r(&symtab_sec->etype.sh_size) / sizeof(Elf_Sym); sym++) { if (ELF_ST_TYPE(sym->st_info) != STT_OBJECT) continue; @@ -388,13 +395,14 @@ static int do_sort(Elf_Ehdr *ehdr, goto out; } - sort_needed_sec = &shdr[get_secindex(r2(&sym->st_shndx), - sort_needed_sym - symtab, - symtab_shndx)]; + sort_need_index = get_secindex(r2(&sym->st_shndx), + sort_needed_sym - symtab, + symtab_shndx); + sort_needed_sec = get_index(shdr_start, shentsize, sort_need_index); sort_needed_loc = (void *)ehdr + - _r(&sort_needed_sec->sh_offset) + + _r(&sort_needed_sec->etype.sh_offset) + _r(&sort_needed_sym->st_value) - - _r(&sort_needed_sec->sh_addr); + _r(&sort_needed_sec->etype.sh_addr); /* extable has been sorted, clear the flag */ w(0, sort_needed_loc); -- cgit v1.2.3 From 200d015e73b4da69bcd8212a7c58695452b12bad Mon Sep 17 00:00:00 2001 From: Steven Rostedt <rostedt@goodmis.org> Date: Sun, 5 Jan 2025 11:22:19 -0500 Subject: scripts/sorttable: Convert Elf_Sym MACRO over to a union In order to remove the double #include of sorttable.h for 64 and 32 bit to create duplicate functions for both, replace the Elf_Sym macro with a union that defines both Elf64_Sym and Elf32_Sym, with field e64 for the 64bit version, and e32 for the 32bit version. It can then use the macro etype to get the proper value. This will eventually be replaced with just single functions that can handle both 32bit and 64bit ELF parsing. Cc: bpf <bpf@vger.kernel.org> Cc: Masami Hiramatsu <mhiramat@kernel.org> Cc: Mark Rutland <mark.rutland@arm.com> Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Masahiro Yamada <masahiroy@kernel.org> Cc: Nathan Chancellor <nathan@kernel.org> Cc: Nicolas Schier <nicolas@fjasle.eu> Cc: Zheng Yejian <zhengyejian1@huawei.com> Cc: Martin Kelly <martin.kelly@crowdstrike.com> Cc: Christophe Leroy <christophe.leroy@csgroup.eu> Cc: Josh Poimboeuf <jpoimboe@redhat.com> Link: https://lore.kernel.org/20250105162345.528626969@goodmis.org Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org> --- scripts/sorttable.c | 5 +++++ scripts/sorttable.h | 25 ++++++++++++++----------- 2 files changed, 19 insertions(+), 11 deletions(-) (limited to 'scripts') diff --git a/scripts/sorttable.c b/scripts/sorttable.c index 94497b8ab04c..57792cf2aa89 100644 --- a/scripts/sorttable.c +++ b/scripts/sorttable.c @@ -74,6 +74,11 @@ typedef union { Elf64_Shdr e64; } Elf_Shdr; +typedef union { + Elf32_Sym e32; + Elf64_Sym e64; +} Elf_Sym; + static uint32_t (*r)(const uint32_t *); static uint16_t (*r2)(const uint16_t *); static uint64_t (*r8)(const uint64_t *); diff --git a/scripts/sorttable.h b/scripts/sorttable.h index 3daf37bb6b9a..cd4429c8a9f4 100644 --- a/scripts/sorttable.h +++ b/scripts/sorttable.h @@ -23,7 +23,6 @@ #undef sort_mcount_loc #undef elf_mcount_loc #undef do_sort -#undef Elf_Sym #undef ELF_ST_TYPE #undef uint_t #undef _r @@ -36,7 +35,6 @@ # define sort_mcount_loc sort_mcount_loc_64 # define elf_mcount_loc elf_mcount_loc_64 # define do_sort do_sort_64 -# define Elf_Sym Elf64_Sym # define ELF_ST_TYPE ELF64_ST_TYPE # define uint_t uint64_t # define _r r8 @@ -48,7 +46,6 @@ # define sort_mcount_loc sort_mcount_loc_32 # define elf_mcount_loc elf_mcount_loc_32 # define do_sort do_sort_32 -# define Elf_Sym Elf32_Sym # define ELF_ST_TYPE ELF32_ST_TYPE # define uint_t uint32_t # define _r r @@ -230,10 +227,13 @@ static int do_sort(Elf_Ehdr *ehdr, Elf_Sym *sort_needed_sym = NULL; Elf_Shdr *sort_needed_sec; uint32_t *sort_needed_loc; + void *sym_start; + void *sym_end; const char *secstrings; const char *strtab; char *extab_image; int sort_need_index; + int symentsize; int shentsize; int idx; int i; @@ -376,12 +376,15 @@ static int do_sort(Elf_Ehdr *ehdr, } /* find the flag main_extable_sort_needed */ - for (sym = (void *)ehdr + _r(&symtab_sec->etype.sh_offset); - sym < sym + _r(&symtab_sec->etype.sh_size) / sizeof(Elf_Sym); - sym++) { - if (ELF_ST_TYPE(sym->st_info) != STT_OBJECT) + sym_start = (void *)ehdr + _r(&symtab_sec->etype.sh_offset); + sym_end = sym_start + _r(&symtab_sec->etype.sh_size); + symentsize = _r(&symtab_sec->etype.sh_entsize); + + for (sym = sym_start; (void *)sym + symentsize < sym_end; + sym = (void *)sym + symentsize) { + if (ELF_ST_TYPE(sym->etype.st_info) != STT_OBJECT) continue; - if (!strcmp(strtab + r(&sym->st_name), + if (!strcmp(strtab + r(&sym->etype.st_name), "main_extable_sort_needed")) { sort_needed_sym = sym; break; @@ -395,13 +398,13 @@ static int do_sort(Elf_Ehdr *ehdr, goto out; } - sort_need_index = get_secindex(r2(&sym->st_shndx), - sort_needed_sym - symtab, + sort_need_index = get_secindex(r2(&sym->etype.st_shndx), + ((void *)sort_needed_sym - (void *)symtab) / symentsize, symtab_shndx); sort_needed_sec = get_index(shdr_start, shentsize, sort_need_index); sort_needed_loc = (void *)ehdr + _r(&sort_needed_sec->etype.sh_offset) + - _r(&sort_needed_sym->st_value) - + _r(&sort_needed_sym->etype.st_value) - _r(&sort_needed_sec->etype.sh_addr); /* extable has been sorted, clear the flag */ -- cgit v1.2.3 From 1dfb59a228dde59ad7d99b2fa2104e90004995c7 Mon Sep 17 00:00:00 2001 From: Steven Rostedt <rostedt@goodmis.org> Date: Sun, 5 Jan 2025 11:22:20 -0500 Subject: scripts/sorttable: Add helper functions for Elf_Ehdr In order to remove the double #include of sorttable.h for 64 and 32 bit to create duplicate functions, add helper functions for Elf_Ehdr. This will create a function pointer for each helper that will get assigned to the appropriate function to handle either the 64bit or 32bit version. This also moves the _r()/r() wrappers for the Elf_Ehdr references that handle endian and size differences between the different architectures, into the helper function and out of the open code which is more error prone. Cc: bpf <bpf@vger.kernel.org> Cc: Masami Hiramatsu <mhiramat@kernel.org> Cc: Mark Rutland <mark.rutland@arm.com> Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Masahiro Yamada <masahiroy@kernel.org> Cc: Nathan Chancellor <nathan@kernel.org> Cc: Nicolas Schier <nicolas@fjasle.eu> Cc: Zheng Yejian <zhengyejian1@huawei.com> Cc: Martin Kelly <martin.kelly@crowdstrike.com> Cc: Christophe Leroy <christophe.leroy@csgroup.eu> Cc: Josh Poimboeuf <jpoimboe@redhat.com> Link: https://lore.kernel.org/20250105162345.736369526@goodmis.org Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org> --- scripts/sorttable.c | 25 +++++++++++++++++++++++++ scripts/sorttable.h | 20 ++++++++++++++++---- 2 files changed, 41 insertions(+), 4 deletions(-) (limited to 'scripts') diff --git a/scripts/sorttable.c b/scripts/sorttable.c index 57792cf2aa89..5dfa734eff09 100644 --- a/scripts/sorttable.c +++ b/scripts/sorttable.c @@ -85,6 +85,31 @@ static uint64_t (*r8)(const uint64_t *); static void (*w)(uint32_t, uint32_t *); typedef void (*table_sort_t)(char *, int); +static uint64_t ehdr64_shoff(Elf_Ehdr *ehdr) +{ + return r8(&ehdr->e64.e_shoff); +} + +static uint64_t ehdr32_shoff(Elf_Ehdr *ehdr) +{ + return r(&ehdr->e32.e_shoff); +} + +#define EHDR_HALF(fn_name) \ +static uint16_t ehdr64_##fn_name(Elf_Ehdr *ehdr) \ +{ \ + return r2(&ehdr->e64.e_##fn_name); \ +} \ + \ +static uint16_t ehdr32_##fn_name(Elf_Ehdr *ehdr) \ +{ \ + return r2(&ehdr->e32.e_##fn_name); \ +} + +EHDR_HALF(shentsize) +EHDR_HALF(shstrndx) +EHDR_HALF(shnum) + /* * Get the whole file as a programming convenience in order to avoid * malloc+lseek+read+free of many pieces. If successful, then mmap diff --git a/scripts/sorttable.h b/scripts/sorttable.h index cd4429c8a9f4..97278c973bc9 100644 --- a/scripts/sorttable.h +++ b/scripts/sorttable.h @@ -27,6 +27,10 @@ #undef uint_t #undef _r #undef etype +#undef ehdr_shoff +#undef ehdr_shentsize +#undef ehdr_shstrndx +#undef ehdr_shnum #ifdef SORTTABLE_64 # define extable_ent_size 16 @@ -39,6 +43,10 @@ # define uint_t uint64_t # define _r r8 # define etype e64 +# define ehdr_shoff ehdr64_shoff +# define ehdr_shentsize ehdr64_shentsize +# define ehdr_shstrndx ehdr64_shstrndx +# define ehdr_shnum ehdr64_shnum #else # define extable_ent_size 8 # define compare_extable compare_extable_32 @@ -50,6 +58,10 @@ # define uint_t uint32_t # define _r r # define etype e32 +# define ehdr_shoff ehdr32_shoff +# define ehdr_shentsize ehdr32_shentsize +# define ehdr_shstrndx ehdr32_shstrndx +# define ehdr_shnum ehdr32_shnum #endif #if defined(SORTTABLE_64) && defined(UNWINDER_ORC_ENABLED) @@ -250,16 +262,16 @@ static int do_sort(Elf_Ehdr *ehdr, unsigned int orc_num_entries = 0; #endif - shdr_start = (Elf_Shdr *)((char *)ehdr + _r(&ehdr->etype.e_shoff)); - shentsize = r2(&ehdr->etype.e_shentsize); + shdr_start = (Elf_Shdr *)((char *)ehdr + ehdr_shoff(ehdr)); + shentsize = ehdr_shentsize(ehdr); - shstrndx = r2(&ehdr->etype.e_shstrndx); + shstrndx = ehdr_shstrndx(ehdr); if (shstrndx == SHN_XINDEX) shstrndx = r(&shdr_start->etype.sh_link); string_sec = get_index(shdr_start, shentsize, shstrndx); secstrings = (const char *)ehdr + _r(&string_sec->etype.sh_offset); - shnum = r2(&ehdr->etype.e_shnum); + shnum = ehdr_shnum(ehdr); if (shnum == SHN_UNDEF) shnum = _r(&shdr_start->etype.sh_size); -- cgit v1.2.3 From 67afb7f504400e5b4e5ff895459fbb3eb63d4450 Mon Sep 17 00:00:00 2001 From: Steven Rostedt <rostedt@goodmis.org> Date: Sun, 5 Jan 2025 11:22:21 -0500 Subject: scripts/sorttable: Add helper functions for Elf_Shdr In order to remove the double #include of sorttable.h for 64 and 32 bit to create duplicate functions, add helper functions for Elf_Shdr. This will create a function pointer for each helper that will get assigned to the appropriate function to handle either the 64bit or 32bit version. This also moves the _r()/r() wrappers for the Elf_Shdr references that handle endian and size differences between the different architectures, into the helper function and out of the open code which is more error prone. Cc: bpf <bpf@vger.kernel.org> Cc: Masami Hiramatsu <mhiramat@kernel.org> Cc: Mark Rutland <mark.rutland@arm.com> Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Masahiro Yamada <masahiroy@kernel.org> Cc: Nathan Chancellor <nathan@kernel.org> Cc: Nicolas Schier <nicolas@fjasle.eu> Cc: Zheng Yejian <zhengyejian1@huawei.com> Cc: Martin Kelly <martin.kelly@crowdstrike.com> Cc: Christophe Leroy <christophe.leroy@csgroup.eu> Cc: Josh Poimboeuf <jpoimboe@redhat.com> Link: https://lore.kernel.org/20250105162345.940924221@goodmis.org Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org> --- scripts/sorttable.c | 42 ++++++++++++++++++++++++++++++++++ scripts/sorttable.h | 66 ++++++++++++++++++++++++++++++++++------------------- 2 files changed, 85 insertions(+), 23 deletions(-) (limited to 'scripts') diff --git a/scripts/sorttable.c b/scripts/sorttable.c index 5dfa734eff09..b2b96ff261d6 100644 --- a/scripts/sorttable.c +++ b/scripts/sorttable.c @@ -110,6 +110,48 @@ EHDR_HALF(shentsize) EHDR_HALF(shstrndx) EHDR_HALF(shnum) +#define SHDR_WORD(fn_name) \ +static uint32_t shdr64_##fn_name(Elf_Shdr *shdr) \ +{ \ + return r(&shdr->e64.sh_##fn_name); \ +} \ + \ +static uint32_t shdr32_##fn_name(Elf_Shdr *shdr) \ +{ \ + return r(&shdr->e32.sh_##fn_name); \ +} + +#define SHDR_ADDR(fn_name) \ +static uint64_t shdr64_##fn_name(Elf_Shdr *shdr) \ +{ \ + return r8(&shdr->e64.sh_##fn_name); \ +} \ + \ +static uint64_t shdr32_##fn_name(Elf_Shdr *shdr) \ +{ \ + return r(&shdr->e32.sh_##fn_name); \ +} + +#define SHDR_WORD(fn_name) \ +static uint32_t shdr64_##fn_name(Elf_Shdr *shdr) \ +{ \ + return r(&shdr->e64.sh_##fn_name); \ +} \ + \ +static uint32_t shdr32_##fn_name(Elf_Shdr *shdr) \ +{ \ + return r(&shdr->e32.sh_##fn_name); \ +} + +SHDR_ADDR(addr) +SHDR_ADDR(offset) +SHDR_ADDR(size) +SHDR_ADDR(entsize) + +SHDR_WORD(link) +SHDR_WORD(name) +SHDR_WORD(type) + /* * Get the whole file as a programming convenience in order to avoid * malloc+lseek+read+free of many pieces. If successful, then mmap diff --git a/scripts/sorttable.h b/scripts/sorttable.h index 97278c973bc9..af3a5f0209a3 100644 --- a/scripts/sorttable.h +++ b/scripts/sorttable.h @@ -31,6 +31,13 @@ #undef ehdr_shentsize #undef ehdr_shstrndx #undef ehdr_shnum +#undef shdr_addr +#undef shdr_offset +#undef shdr_link +#undef shdr_size +#undef shdr_name +#undef shdr_type +#undef shdr_entsize #ifdef SORTTABLE_64 # define extable_ent_size 16 @@ -47,6 +54,13 @@ # define ehdr_shentsize ehdr64_shentsize # define ehdr_shstrndx ehdr64_shstrndx # define ehdr_shnum ehdr64_shnum +# define shdr_addr shdr64_addr +# define shdr_offset shdr64_offset +# define shdr_link shdr64_link +# define shdr_size shdr64_size +# define shdr_name shdr64_name +# define shdr_type shdr64_type +# define shdr_entsize shdr64_entsize #else # define extable_ent_size 8 # define compare_extable compare_extable_32 @@ -62,6 +76,13 @@ # define ehdr_shentsize ehdr32_shentsize # define ehdr_shstrndx ehdr32_shstrndx # define ehdr_shnum ehdr32_shnum +# define shdr_addr shdr32_addr +# define shdr_offset shdr32_offset +# define shdr_link shdr32_link +# define shdr_size shdr32_size +# define shdr_name shdr32_name +# define shdr_type shdr32_type +# define shdr_entsize shdr32_entsize #endif #if defined(SORTTABLE_64) && defined(UNWINDER_ORC_ENABLED) @@ -177,8 +198,8 @@ struct elf_mcount_loc { static void *sort_mcount_loc(void *arg) { struct elf_mcount_loc *emloc = (struct elf_mcount_loc *)arg; - uint_t offset = emloc->start_mcount_loc - _r(&(emloc->init_data_sec)->etype.sh_addr) - + _r(&(emloc->init_data_sec)->etype.sh_offset); + uint_t offset = emloc->start_mcount_loc - shdr_addr(emloc->init_data_sec) + + shdr_offset(emloc->init_data_sec); uint_t count = emloc->stop_mcount_loc - emloc->start_mcount_loc; unsigned char *start_loc = (void *)emloc->ehdr + offset; @@ -267,18 +288,18 @@ static int do_sort(Elf_Ehdr *ehdr, shstrndx = ehdr_shstrndx(ehdr); if (shstrndx == SHN_XINDEX) - shstrndx = r(&shdr_start->etype.sh_link); + shstrndx = shdr_link(shdr_start); string_sec = get_index(shdr_start, shentsize, shstrndx); - secstrings = (const char *)ehdr + _r(&string_sec->etype.sh_offset); + secstrings = (const char *)ehdr + shdr_offset(string_sec); shnum = ehdr_shnum(ehdr); if (shnum == SHN_UNDEF) - shnum = _r(&shdr_start->etype.sh_size); + shnum = shdr_size(shdr_start); for (i = 0; i < shnum; i++) { Elf_Shdr *shdr = get_index(shdr_start, shentsize, i); - idx = r(&shdr->etype.sh_name); + idx = shdr_name(shdr); if (!strcmp(secstrings + idx, "__ex_table")) extab_sec = shdr; if (!strcmp(secstrings + idx, ".symtab")) @@ -286,9 +307,9 @@ static int do_sort(Elf_Ehdr *ehdr, if (!strcmp(secstrings + idx, ".strtab")) strtab_sec = shdr; - if (r(&shdr->etype.sh_type) == SHT_SYMTAB_SHNDX) + if (shdr_type(shdr) == SHT_SYMTAB_SHNDX) symtab_shndx = (Elf32_Word *)((const char *)ehdr + - _r(&shdr->etype.sh_offset)); + shdr_offset(shdr)); #ifdef MCOUNT_SORT_ENABLED /* locate the .init.data section in vmlinux */ @@ -304,14 +325,14 @@ static int do_sort(Elf_Ehdr *ehdr, #if defined(SORTTABLE_64) && defined(UNWINDER_ORC_ENABLED) /* locate the ORC unwind tables */ if (!strcmp(secstrings + idx, ".orc_unwind_ip")) { - orc_ip_size = _r(&shdr->etype.sh_size); + orc_ip_size = shdr_size(shdr); g_orc_ip_table = (int *)((void *)ehdr + - _r(&shdr->etype.sh_offset)); + shdr_offset(shdr)); } if (!strcmp(secstrings + idx, ".orc_unwind")) { - orc_size = _r(&shdr->etype.sh_size); + orc_size = shdr_size(shdr); g_orc_table = (struct orc_entry *)((void *)ehdr + - _r(&shdr->etype.sh_offset)); + shdr_offset(shdr)); } #endif } /* for loop */ @@ -374,23 +395,22 @@ static int do_sort(Elf_Ehdr *ehdr, goto out; } - extab_image = (void *)ehdr + _r(&extab_sec->etype.sh_offset); - strtab = (const char *)ehdr + _r(&strtab_sec->etype.sh_offset); - symtab = (const Elf_Sym *)((const char *)ehdr + - _r(&symtab_sec->etype.sh_offset)); + extab_image = (void *)ehdr + shdr_offset(extab_sec); + strtab = (const char *)ehdr + shdr_offset(strtab_sec); + symtab = (const Elf_Sym *)((const char *)ehdr + shdr_offset(symtab_sec)); if (custom_sort) { - custom_sort(extab_image, _r(&extab_sec->etype.sh_size)); + custom_sort(extab_image, shdr_size(extab_sec)); } else { - int num_entries = _r(&extab_sec->etype.sh_size) / extable_ent_size; + int num_entries = shdr_size(extab_sec) / extable_ent_size; qsort(extab_image, num_entries, extable_ent_size, compare_extable); } /* find the flag main_extable_sort_needed */ - sym_start = (void *)ehdr + _r(&symtab_sec->etype.sh_offset); - sym_end = sym_start + _r(&symtab_sec->etype.sh_size); - symentsize = _r(&symtab_sec->etype.sh_entsize); + sym_start = (void *)ehdr + shdr_offset(symtab_sec); + sym_end = sym_start + shdr_size(symtab_sec); + symentsize = shdr_entsize(symtab_sec); for (sym = sym_start; (void *)sym + symentsize < sym_end; sym = (void *)sym + symentsize) { @@ -415,9 +435,9 @@ static int do_sort(Elf_Ehdr *ehdr, symtab_shndx); sort_needed_sec = get_index(shdr_start, shentsize, sort_need_index); sort_needed_loc = (void *)ehdr + - _r(&sort_needed_sec->etype.sh_offset) + + shdr_offset(sort_needed_sec) + _r(&sort_needed_sym->etype.st_value) - - _r(&sort_needed_sec->etype.sh_addr); + shdr_addr(sort_needed_sec); /* extable has been sorted, clear the flag */ w(0, sort_needed_loc); -- cgit v1.2.3 From 17bed33ac12f011f4695059960e1b1d6457229a7 Mon Sep 17 00:00:00 2001 From: Steven Rostedt <rostedt@goodmis.org> Date: Sun, 5 Jan 2025 11:22:22 -0500 Subject: scripts/sorttable: Add helper functions for Elf_Sym In order to remove the double #include of sorttable.h for 64 and 32 bit to create duplicate functions, add helper functions for Elf_Sym. This will create a function pointer for each helper that will get assigned to the appropriate function to handle either the 64bit or 32bit version. This also removes the last references of etype and _r() macros from the sorttable.h file as their references are now just defined in the appropriate architecture version of the helper functions. All read functions now exist in the helper functions which makes it easier to maintain, as the helper functions define the necessary architecture sizes. Cc: bpf <bpf@vger.kernel.org> Cc: Masami Hiramatsu <mhiramat@kernel.org> Cc: Mark Rutland <mark.rutland@arm.com> Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Masahiro Yamada <masahiroy@kernel.org> Cc: Nathan Chancellor <nathan@kernel.org> Cc: Nicolas Schier <nicolas@fjasle.eu> Cc: Zheng Yejian <zhengyejian1@huawei.com> Cc: Martin Kelly <martin.kelly@crowdstrike.com> Cc: Christophe Leroy <christophe.leroy@csgroup.eu> Cc: Josh Poimboeuf <jpoimboe@redhat.com> Link: https://lore.kernel.org/20250105162346.185740651@goodmis.org Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org> --- scripts/sorttable.c | 47 +++++++++++++++++++++++++++++++++++++++++++++++ scripts/sorttable.h | 30 ++++++++++++++++-------------- 2 files changed, 63 insertions(+), 14 deletions(-) (limited to 'scripts') diff --git a/scripts/sorttable.c b/scripts/sorttable.c index b2b96ff261d6..20615de18276 100644 --- a/scripts/sorttable.c +++ b/scripts/sorttable.c @@ -152,6 +152,53 @@ SHDR_WORD(link) SHDR_WORD(name) SHDR_WORD(type) +#define SYM_ADDR(fn_name) \ +static uint64_t sym64_##fn_name(Elf_Sym *sym) \ +{ \ + return r8(&sym->e64.st_##fn_name); \ +} \ + \ +static uint64_t sym32_##fn_name(Elf_Sym *sym) \ +{ \ + return r(&sym->e32.st_##fn_name); \ +} + +#define SYM_WORD(fn_name) \ +static uint32_t sym64_##fn_name(Elf_Sym *sym) \ +{ \ + return r(&sym->e64.st_##fn_name); \ +} \ + \ +static uint32_t sym32_##fn_name(Elf_Sym *sym) \ +{ \ + return r(&sym->e32.st_##fn_name); \ +} + +#define SYM_HALF(fn_name) \ +static uint16_t sym64_##fn_name(Elf_Sym *sym) \ +{ \ + return r2(&sym->e64.st_##fn_name); \ +} \ + \ +static uint16_t sym32_##fn_name(Elf_Sym *sym) \ +{ \ + return r2(&sym->e32.st_##fn_name); \ +} + +static uint8_t sym64_type(Elf_Sym *sym) +{ + return ELF64_ST_TYPE(sym->e64.st_info); +} + +static uint8_t sym32_type(Elf_Sym *sym) +{ + return ELF32_ST_TYPE(sym->e32.st_info); +} + +SYM_ADDR(value) +SYM_WORD(name) +SYM_HALF(shndx) + /* * Get the whole file as a programming convenience in order to avoid * malloc+lseek+read+free of many pieces. If successful, then mmap diff --git a/scripts/sorttable.h b/scripts/sorttable.h index af3a5f0209a3..ef7e5161db31 100644 --- a/scripts/sorttable.h +++ b/scripts/sorttable.h @@ -23,10 +23,7 @@ #undef sort_mcount_loc #undef elf_mcount_loc #undef do_sort -#undef ELF_ST_TYPE #undef uint_t -#undef _r -#undef etype #undef ehdr_shoff #undef ehdr_shentsize #undef ehdr_shstrndx @@ -38,6 +35,10 @@ #undef shdr_name #undef shdr_type #undef shdr_entsize +#undef sym_type +#undef sym_name +#undef sym_value +#undef sym_shndx #ifdef SORTTABLE_64 # define extable_ent_size 16 @@ -46,10 +47,7 @@ # define sort_mcount_loc sort_mcount_loc_64 # define elf_mcount_loc elf_mcount_loc_64 # define do_sort do_sort_64 -# define ELF_ST_TYPE ELF64_ST_TYPE # define uint_t uint64_t -# define _r r8 -# define etype e64 # define ehdr_shoff ehdr64_shoff # define ehdr_shentsize ehdr64_shentsize # define ehdr_shstrndx ehdr64_shstrndx @@ -61,6 +59,10 @@ # define shdr_name shdr64_name # define shdr_type shdr64_type # define shdr_entsize shdr64_entsize +# define sym_type sym64_type +# define sym_name sym64_name +# define sym_value sym64_value +# define sym_shndx sym64_shndx #else # define extable_ent_size 8 # define compare_extable compare_extable_32 @@ -68,10 +70,7 @@ # define sort_mcount_loc sort_mcount_loc_32 # define elf_mcount_loc elf_mcount_loc_32 # define do_sort do_sort_32 -# define ELF_ST_TYPE ELF32_ST_TYPE # define uint_t uint32_t -# define _r r -# define etype e32 # define ehdr_shoff ehdr32_shoff # define ehdr_shentsize ehdr32_shentsize # define ehdr_shstrndx ehdr32_shstrndx @@ -83,6 +82,10 @@ # define shdr_name shdr32_name # define shdr_type shdr32_type # define shdr_entsize shdr32_entsize +# define sym_type sym32_type +# define sym_name sym32_name +# define sym_value sym32_value +# define sym_shndx sym32_shndx #endif #if defined(SORTTABLE_64) && defined(UNWINDER_ORC_ENABLED) @@ -414,9 +417,9 @@ static int do_sort(Elf_Ehdr *ehdr, for (sym = sym_start; (void *)sym + symentsize < sym_end; sym = (void *)sym + symentsize) { - if (ELF_ST_TYPE(sym->etype.st_info) != STT_OBJECT) + if (sym_type(sym) != STT_OBJECT) continue; - if (!strcmp(strtab + r(&sym->etype.st_name), + if (!strcmp(strtab + sym_name(sym), "main_extable_sort_needed")) { sort_needed_sym = sym; break; @@ -430,14 +433,13 @@ static int do_sort(Elf_Ehdr *ehdr, goto out; } - sort_need_index = get_secindex(r2(&sym->etype.st_shndx), + sort_need_index = get_secindex(sym_shndx(sym), ((void *)sort_needed_sym - (void *)symtab) / symentsize, symtab_shndx); sort_needed_sec = get_index(shdr_start, shentsize, sort_need_index); sort_needed_loc = (void *)ehdr + shdr_offset(sort_needed_sec) + - _r(&sort_needed_sym->etype.st_value) - - shdr_addr(sort_needed_sec); + sym_value(sort_needed_sym) - shdr_addr(sort_needed_sec); /* extable has been sorted, clear the flag */ w(0, sort_needed_loc); -- cgit v1.2.3 From 1b649e6ab8dc9188d82c64069493afe66ca0edad Mon Sep 17 00:00:00 2001 From: Steven Rostedt <rostedt@goodmis.org> Date: Sun, 5 Jan 2025 11:22:23 -0500 Subject: scripts/sorttable: Use uint64_t for mcount sorting The mcount sorting defines uint_t to uint64_t on 64bit architectures and uint32_t on 32bit architectures. It can work with just using uint64_t as that will hold the values of both, and they are not used to point into the ELF file. sizeof(uint_t) is used for defining the size of the mcount_loc section. Instead of using a type, define long_size and use that instead. This will allow the header code to be moved into the C file as generic functions and not need to include sorttable.h twice, once for 64bit and once for 32bit. Cc: bpf <bpf@vger.kernel.org> Cc: Masami Hiramatsu <mhiramat@kernel.org> Cc: Mark Rutland <mark.rutland@arm.com> Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Masahiro Yamada <masahiroy@kernel.org> Cc: Nathan Chancellor <nathan@kernel.org> Cc: Nicolas Schier <nicolas@fjasle.eu> Cc: Zheng Yejian <zhengyejian1@huawei.com> Cc: Martin Kelly <martin.kelly@crowdstrike.com> Cc: Christophe Leroy <christophe.leroy@csgroup.eu> Cc: Josh Poimboeuf <jpoimboe@redhat.com> Link: https://lore.kernel.org/20250105162346.373528925@goodmis.org Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org> --- scripts/sorttable.h | 22 +++++++++++----------- 1 file changed, 11 insertions(+), 11 deletions(-) (limited to 'scripts') diff --git a/scripts/sorttable.h b/scripts/sorttable.h index ef7e5161db31..17a8541a10d6 100644 --- a/scripts/sorttable.h +++ b/scripts/sorttable.h @@ -23,7 +23,6 @@ #undef sort_mcount_loc #undef elf_mcount_loc #undef do_sort -#undef uint_t #undef ehdr_shoff #undef ehdr_shentsize #undef ehdr_shstrndx @@ -39,6 +38,7 @@ #undef sym_name #undef sym_value #undef sym_shndx +#undef long_size #ifdef SORTTABLE_64 # define extable_ent_size 16 @@ -47,7 +47,6 @@ # define sort_mcount_loc sort_mcount_loc_64 # define elf_mcount_loc elf_mcount_loc_64 # define do_sort do_sort_64 -# define uint_t uint64_t # define ehdr_shoff ehdr64_shoff # define ehdr_shentsize ehdr64_shentsize # define ehdr_shstrndx ehdr64_shstrndx @@ -63,6 +62,7 @@ # define sym_name sym64_name # define sym_value sym64_value # define sym_shndx sym64_shndx +# define long_size 8 #else # define extable_ent_size 8 # define compare_extable compare_extable_32 @@ -70,7 +70,6 @@ # define sort_mcount_loc sort_mcount_loc_32 # define elf_mcount_loc elf_mcount_loc_32 # define do_sort do_sort_32 -# define uint_t uint32_t # define ehdr_shoff ehdr32_shoff # define ehdr_shentsize ehdr32_shentsize # define ehdr_shstrndx ehdr32_shstrndx @@ -86,6 +85,7 @@ # define sym_name sym32_name # define sym_value sym32_value # define sym_shndx sym32_shndx +# define long_size 4 #endif #if defined(SORTTABLE_64) && defined(UNWINDER_ORC_ENABLED) @@ -193,25 +193,25 @@ pthread_t mcount_sort_thread; struct elf_mcount_loc { Elf_Ehdr *ehdr; Elf_Shdr *init_data_sec; - uint_t start_mcount_loc; - uint_t stop_mcount_loc; + uint64_t start_mcount_loc; + uint64_t stop_mcount_loc; }; /* Sort the addresses stored between __start_mcount_loc to __stop_mcount_loc in vmlinux */ static void *sort_mcount_loc(void *arg) { struct elf_mcount_loc *emloc = (struct elf_mcount_loc *)arg; - uint_t offset = emloc->start_mcount_loc - shdr_addr(emloc->init_data_sec) + uint64_t offset = emloc->start_mcount_loc - shdr_addr(emloc->init_data_sec) + shdr_offset(emloc->init_data_sec); - uint_t count = emloc->stop_mcount_loc - emloc->start_mcount_loc; + uint64_t count = emloc->stop_mcount_loc - emloc->start_mcount_loc; unsigned char *start_loc = (void *)emloc->ehdr + offset; - qsort(start_loc, count/sizeof(uint_t), sizeof(uint_t), compare_extable); + qsort(start_loc, count/long_size, long_size, compare_extable); return NULL; } /* Get the address of __start_mcount_loc and __stop_mcount_loc in System.map */ -static void get_mcount_loc(uint_t *_start, uint_t *_stop) +static void get_mcount_loc(uint64_t *_start, uint64_t *_stop) { FILE *file_start, *file_stop; char start_buff[20]; @@ -277,8 +277,8 @@ static int do_sort(Elf_Ehdr *ehdr, unsigned int shstrndx; #ifdef MCOUNT_SORT_ENABLED struct elf_mcount_loc mstruct = {0}; - uint_t _start_mcount_loc = 0; - uint_t _stop_mcount_loc = 0; + uint64_t _start_mcount_loc = 0; + uint64_t _stop_mcount_loc = 0; #endif #if defined(SORTTABLE_64) && defined(UNWINDER_ORC_ENABLED) unsigned int orc_ip_size = 0; -- cgit v1.2.3 From 58d87678a0f46c6120904b4326aaf5ebf4454c69 Mon Sep 17 00:00:00 2001 From: Steven Rostedt <rostedt@goodmis.org> Date: Tue, 7 Jan 2025 22:32:17 -0500 Subject: scripts/sorttable: Move code from sorttable.h into sorttable.c Instead of having the main code live in a header file and included twice with MACROs that define the Elf structures for 64 bit or 32 bit, move the code in the C file now that the Elf structures are defined in a union that has both. All accesses to the Elf structure fields are done through helper function pointers. If the file being parsed if for a 64 bit architecture, all the helper functions point to the 64 bit versions to retrieve the Elf fields. The same is true if the architecture is 32 bit, where the function pointers will point to the 32 bit helper functions. Note, when the value of a field can be either 32 bit or 64 bit, a 64 bit is always returned, as it works for the 32 bit code as well. This makes the code easier to read and maintain, and it now all exists in sorttable.c and sorttable.h may be removed. Cc: bpf <bpf@vger.kernel.org> Cc: Masami Hiramatsu <mhiramat@kernel.org> Cc: Mark Rutland <mark.rutland@arm.com> Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Masahiro Yamada <masahiroy@kernel.org> Cc: Nathan Chancellor <nathan@kernel.org> Cc: Nicolas Schier <nicolas@fjasle.eu> Cc: Zheng Yejian <zhengyejian1@huawei.com> Cc: Martin Kelly <martin.kelly@crowdstrike.com> Cc: Christophe Leroy <christophe.leroy@csgroup.eu> Cc: Josh Poimboeuf <jpoimboe@redhat.com> Cc: Stephen Rothwell <sfr@canb.auug.org.au> Link: https://lore.kernel.org/20250107223217.6f7f96a5@gandalf.local.home Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org> --- scripts/sorttable.c | 473 ++++++++++++++++++++++++++++++++++++++++++++++++-- scripts/sorttable.h | 485 ---------------------------------------------------- 2 files changed, 460 insertions(+), 498 deletions(-) delete mode 100644 scripts/sorttable.h (limited to 'scripts') diff --git a/scripts/sorttable.c b/scripts/sorttable.c index 20615de18276..ff9b60fc0dd8 100644 --- a/scripts/sorttable.c +++ b/scripts/sorttable.c @@ -327,10 +327,423 @@ static inline void *get_index(void *start, int entsize, int index) return start + (entsize * index); } -/* 32 bit and 64 bit are very similar */ -#include "sorttable.h" -#define SORTTABLE_64 -#include "sorttable.h" + +static int (*compare_extable)(const void *a, const void *b); +static uint64_t (*ehdr_shoff)(Elf_Ehdr *ehdr); +static uint16_t (*ehdr_shstrndx)(Elf_Ehdr *ehdr); +static uint16_t (*ehdr_shentsize)(Elf_Ehdr *ehdr); +static uint16_t (*ehdr_shnum)(Elf_Ehdr *ehdr); +static uint64_t (*shdr_addr)(Elf_Shdr *shdr); +static uint64_t (*shdr_offset)(Elf_Shdr *shdr); +static uint64_t (*shdr_size)(Elf_Shdr *shdr); +static uint64_t (*shdr_entsize)(Elf_Shdr *shdr); +static uint32_t (*shdr_link)(Elf_Shdr *shdr); +static uint32_t (*shdr_name)(Elf_Shdr *shdr); +static uint32_t (*shdr_type)(Elf_Shdr *shdr); +static uint8_t (*sym_type)(Elf_Sym *sym); +static uint32_t (*sym_name)(Elf_Sym *sym); +static uint64_t (*sym_value)(Elf_Sym *sym); +static uint16_t (*sym_shndx)(Elf_Sym *sym); + +static int extable_ent_size; +static int long_size; + + +#ifdef UNWINDER_ORC_ENABLED +/* ORC unwinder only support X86_64 */ +#include <asm/orc_types.h> + +#define ERRSTR_MAXSZ 256 + +static char g_err[ERRSTR_MAXSZ]; +static int *g_orc_ip_table; +static struct orc_entry *g_orc_table; + +static pthread_t orc_sort_thread; + +static inline unsigned long orc_ip(const int *ip) +{ + return (unsigned long)ip + *ip; +} + +static int orc_sort_cmp(const void *_a, const void *_b) +{ + struct orc_entry *orc_a, *orc_b; + const int *a = g_orc_ip_table + *(int *)_a; + const int *b = g_orc_ip_table + *(int *)_b; + unsigned long a_val = orc_ip(a); + unsigned long b_val = orc_ip(b); + + if (a_val > b_val) + return 1; + if (a_val < b_val) + return -1; + + /* + * The "weak" section terminator entries need to always be on the left + * to ensure the lookup code skips them in favor of real entries. + * These terminator entries exist to handle any gaps created by + * whitelisted .o files which didn't get objtool generation. + */ + orc_a = g_orc_table + (a - g_orc_ip_table); + orc_b = g_orc_table + (b - g_orc_ip_table); + if (orc_a->type == ORC_TYPE_UNDEFINED && orc_b->type == ORC_TYPE_UNDEFINED) + return 0; + return orc_a->type == ORC_TYPE_UNDEFINED ? -1 : 1; +} + +static void *sort_orctable(void *arg) +{ + int i; + int *idxs = NULL; + int *tmp_orc_ip_table = NULL; + struct orc_entry *tmp_orc_table = NULL; + unsigned int *orc_ip_size = (unsigned int *)arg; + unsigned int num_entries = *orc_ip_size / sizeof(int); + unsigned int orc_size = num_entries * sizeof(struct orc_entry); + + idxs = (int *)malloc(*orc_ip_size); + if (!idxs) { + snprintf(g_err, ERRSTR_MAXSZ, "malloc idxs: %s", + strerror(errno)); + pthread_exit(g_err); + } + + tmp_orc_ip_table = (int *)malloc(*orc_ip_size); + if (!tmp_orc_ip_table) { + snprintf(g_err, ERRSTR_MAXSZ, "malloc tmp_orc_ip_table: %s", + strerror(errno)); + pthread_exit(g_err); + } + + tmp_orc_table = (struct orc_entry *)malloc(orc_size); + if (!tmp_orc_table) { + snprintf(g_err, ERRSTR_MAXSZ, "malloc tmp_orc_table: %s", + strerror(errno)); + pthread_exit(g_err); + } + + /* initialize indices array, convert ip_table to absolute address */ + for (i = 0; i < num_entries; i++) { + idxs[i] = i; + tmp_orc_ip_table[i] = g_orc_ip_table[i] + i * sizeof(int); + } + memcpy(tmp_orc_table, g_orc_table, orc_size); + + qsort(idxs, num_entries, sizeof(int), orc_sort_cmp); + + for (i = 0; i < num_entries; i++) { + if (idxs[i] == i) + continue; + + /* convert back to relative address */ + g_orc_ip_table[i] = tmp_orc_ip_table[idxs[i]] - i * sizeof(int); + g_orc_table[i] = tmp_orc_table[idxs[i]]; + } + + free(idxs); + free(tmp_orc_ip_table); + free(tmp_orc_table); + pthread_exit(NULL); +} +#endif + +#ifdef MCOUNT_SORT_ENABLED +static pthread_t mcount_sort_thread; + +struct elf_mcount_loc { + Elf_Ehdr *ehdr; + Elf_Shdr *init_data_sec; + uint64_t start_mcount_loc; + uint64_t stop_mcount_loc; +}; + +/* Sort the addresses stored between __start_mcount_loc to __stop_mcount_loc in vmlinux */ +static void *sort_mcount_loc(void *arg) +{ + struct elf_mcount_loc *emloc = (struct elf_mcount_loc *)arg; + uint64_t offset = emloc->start_mcount_loc - shdr_addr(emloc->init_data_sec) + + shdr_offset(emloc->init_data_sec); + uint64_t count = emloc->stop_mcount_loc - emloc->start_mcount_loc; + unsigned char *start_loc = (void *)emloc->ehdr + offset; + + qsort(start_loc, count/long_size, long_size, compare_extable); + return NULL; +} + +/* Get the address of __start_mcount_loc and __stop_mcount_loc in System.map */ +static void get_mcount_loc(uint64_t *_start, uint64_t *_stop) +{ + FILE *file_start, *file_stop; + char start_buff[20]; + char stop_buff[20]; + int len = 0; + + file_start = popen(" grep start_mcount System.map | awk '{print $1}' ", "r"); + if (!file_start) { + fprintf(stderr, "get start_mcount_loc error!"); + return; + } + + file_stop = popen(" grep stop_mcount System.map | awk '{print $1}' ", "r"); + if (!file_stop) { + fprintf(stderr, "get stop_mcount_loc error!"); + pclose(file_start); + return; + } + + while (fgets(start_buff, sizeof(start_buff), file_start) != NULL) { + len = strlen(start_buff); + start_buff[len - 1] = '\0'; + } + *_start = strtoul(start_buff, NULL, 16); + + while (fgets(stop_buff, sizeof(stop_buff), file_stop) != NULL) { + len = strlen(stop_buff); + stop_buff[len - 1] = '\0'; + } + *_stop = strtoul(stop_buff, NULL, 16); + + pclose(file_start); + pclose(file_stop); +} +#endif +static int do_sort(Elf_Ehdr *ehdr, + char const *const fname, + table_sort_t custom_sort) +{ + int rc = -1; + Elf_Shdr *shdr_start; + Elf_Shdr *strtab_sec = NULL; + Elf_Shdr *symtab_sec = NULL; + Elf_Shdr *extab_sec = NULL; + Elf_Shdr *string_sec; + Elf_Sym *sym; + const Elf_Sym *symtab; + Elf32_Word *symtab_shndx = NULL; + Elf_Sym *sort_needed_sym = NULL; + Elf_Shdr *sort_needed_sec; + uint32_t *sort_needed_loc; + void *sym_start; + void *sym_end; + const char *secstrings; + const char *strtab; + char *extab_image; + int sort_need_index; + int symentsize; + int shentsize; + int idx; + int i; + unsigned int shnum; + unsigned int shstrndx; +#ifdef MCOUNT_SORT_ENABLED + struct elf_mcount_loc mstruct = {0}; + uint64_t _start_mcount_loc = 0; + uint64_t _stop_mcount_loc = 0; +#endif +#ifdef UNWINDER_ORC_ENABLED + unsigned int orc_ip_size = 0; + unsigned int orc_size = 0; + unsigned int orc_num_entries = 0; +#endif + + shdr_start = (Elf_Shdr *)((char *)ehdr + ehdr_shoff(ehdr)); + shentsize = ehdr_shentsize(ehdr); + + shstrndx = ehdr_shstrndx(ehdr); + if (shstrndx == SHN_XINDEX) + shstrndx = shdr_link(shdr_start); + string_sec = get_index(shdr_start, shentsize, shstrndx); + secstrings = (const char *)ehdr + shdr_offset(string_sec); + + shnum = ehdr_shnum(ehdr); + if (shnum == SHN_UNDEF) + shnum = shdr_size(shdr_start); + + for (i = 0; i < shnum; i++) { + Elf_Shdr *shdr = get_index(shdr_start, shentsize, i); + + idx = shdr_name(shdr); + if (!strcmp(secstrings + idx, "__ex_table")) + extab_sec = shdr; + if (!strcmp(secstrings + idx, ".symtab")) + symtab_sec = shdr; + if (!strcmp(secstrings + idx, ".strtab")) + strtab_sec = shdr; + + if (shdr_type(shdr) == SHT_SYMTAB_SHNDX) + symtab_shndx = (Elf32_Word *)((const char *)ehdr + + shdr_offset(shdr)); + +#ifdef MCOUNT_SORT_ENABLED + /* locate the .init.data section in vmlinux */ + if (!strcmp(secstrings + idx, ".init.data")) { + get_mcount_loc(&_start_mcount_loc, &_stop_mcount_loc); + mstruct.ehdr = ehdr; + mstruct.init_data_sec = shdr; + mstruct.start_mcount_loc = _start_mcount_loc; + mstruct.stop_mcount_loc = _stop_mcount_loc; + } +#endif + +#ifdef UNWINDER_ORC_ENABLED + /* locate the ORC unwind tables */ + if (!strcmp(secstrings + idx, ".orc_unwind_ip")) { + orc_ip_size = shdr_size(shdr); + g_orc_ip_table = (int *)((void *)ehdr + + shdr_offset(shdr)); + } + if (!strcmp(secstrings + idx, ".orc_unwind")) { + orc_size = shdr_size(shdr); + g_orc_table = (struct orc_entry *)((void *)ehdr + + shdr_offset(shdr)); + } +#endif + } /* for loop */ + +#ifdef UNWINDER_ORC_ENABLED + if (!g_orc_ip_table || !g_orc_table) { + fprintf(stderr, + "incomplete ORC unwind tables in file: %s\n", fname); + goto out; + } + + orc_num_entries = orc_ip_size / sizeof(int); + if (orc_ip_size % sizeof(int) != 0 || + orc_size % sizeof(struct orc_entry) != 0 || + orc_num_entries != orc_size / sizeof(struct orc_entry)) { + fprintf(stderr, + "inconsistent ORC unwind table entries in file: %s\n", + fname); + goto out; + } + + /* create thread to sort ORC unwind tables concurrently */ + if (pthread_create(&orc_sort_thread, NULL, + sort_orctable, &orc_ip_size)) { + fprintf(stderr, + "pthread_create orc_sort_thread failed '%s': %s\n", + strerror(errno), fname); + goto out; + } +#endif + +#ifdef MCOUNT_SORT_ENABLED + if (!mstruct.init_data_sec || !_start_mcount_loc || !_stop_mcount_loc) { + fprintf(stderr, + "incomplete mcount's sort in file: %s\n", + fname); + goto out; + } + + /* create thread to sort mcount_loc concurrently */ + if (pthread_create(&mcount_sort_thread, NULL, &sort_mcount_loc, &mstruct)) { + fprintf(stderr, + "pthread_create mcount_sort_thread failed '%s': %s\n", + strerror(errno), fname); + goto out; + } +#endif + if (!extab_sec) { + fprintf(stderr, "no __ex_table in file: %s\n", fname); + goto out; + } + + if (!symtab_sec) { + fprintf(stderr, "no .symtab in file: %s\n", fname); + goto out; + } + + if (!strtab_sec) { + fprintf(stderr, "no .strtab in file: %s\n", fname); + goto out; + } + + extab_image = (void *)ehdr + shdr_offset(extab_sec); + strtab = (const char *)ehdr + shdr_offset(strtab_sec); + symtab = (const Elf_Sym *)((const char *)ehdr + shdr_offset(symtab_sec)); + + if (custom_sort) { + custom_sort(extab_image, shdr_size(extab_sec)); + } else { + int num_entries = shdr_size(extab_sec) / extable_ent_size; + qsort(extab_image, num_entries, + extable_ent_size, compare_extable); + } + + /* find the flag main_extable_sort_needed */ + sym_start = (void *)ehdr + shdr_offset(symtab_sec); + sym_end = sym_start + shdr_size(symtab_sec); + symentsize = shdr_entsize(symtab_sec); + + for (sym = sym_start; (void *)sym + symentsize < sym_end; + sym = (void *)sym + symentsize) { + if (sym_type(sym) != STT_OBJECT) + continue; + if (!strcmp(strtab + sym_name(sym), + "main_extable_sort_needed")) { + sort_needed_sym = sym; + break; + } + } + + if (!sort_needed_sym) { + fprintf(stderr, + "no main_extable_sort_needed symbol in file: %s\n", + fname); + goto out; + } + + sort_need_index = get_secindex(sym_shndx(sym), + ((void *)sort_needed_sym - (void *)symtab) / symentsize, + symtab_shndx); + sort_needed_sec = get_index(shdr_start, shentsize, sort_need_index); + sort_needed_loc = (void *)ehdr + + shdr_offset(sort_needed_sec) + + sym_value(sort_needed_sym) - shdr_addr(sort_needed_sec); + + /* extable has been sorted, clear the flag */ + w(0, sort_needed_loc); + rc = 0; + +out: +#ifdef UNWINDER_ORC_ENABLED + if (orc_sort_thread) { + void *retval = NULL; + /* wait for ORC tables sort done */ + rc = pthread_join(orc_sort_thread, &retval); + if (rc) { + fprintf(stderr, + "pthread_join failed '%s': %s\n", + strerror(errno), fname); + } else if (retval) { + rc = -1; + fprintf(stderr, + "failed to sort ORC tables '%s': %s\n", + (char *)retval, fname); + } + } +#endif + +#ifdef MCOUNT_SORT_ENABLED + if (mcount_sort_thread) { + void *retval = NULL; + /* wait for mcount sort done */ + rc = pthread_join(mcount_sort_thread, &retval); + if (rc) { + fprintf(stderr, + "pthread_join failed '%s': %s\n", + strerror(errno), fname); + } else if (retval) { + rc = -1; + fprintf(stderr, + "failed to sort mcount '%s': %s\n", + (char *)retval, fname); + } + } +#endif + return rc; +} static int compare_relative_table(const void *a, const void *b) { @@ -399,7 +812,6 @@ static void sort_relative_table_with_data(char *extab_image, int image_size) static int do_file(char const *const fname, void *addr) { - int rc = -1; Elf_Ehdr *ehdr = addr; table_sort_t custom_sort = NULL; @@ -462,29 +874,64 @@ static int do_file(char const *const fname, void *addr) r2(&ehdr->e32.e_shentsize) != sizeof(Elf32_Shdr)) { fprintf(stderr, "unrecognized ET_EXEC/ET_DYN file: %s\n", fname); - break; + return -1; } - rc = do_sort_32(ehdr, fname, custom_sort); + + compare_extable = compare_extable_32; + ehdr_shoff = ehdr32_shoff; + ehdr_shentsize = ehdr32_shentsize; + ehdr_shstrndx = ehdr32_shstrndx; + ehdr_shnum = ehdr32_shnum; + shdr_addr = shdr32_addr; + shdr_offset = shdr32_offset; + shdr_link = shdr32_link; + shdr_size = shdr32_size; + shdr_name = shdr32_name; + shdr_type = shdr32_type; + shdr_entsize = shdr32_entsize; + sym_type = sym32_type; + sym_name = sym32_name; + sym_value = sym32_value; + sym_shndx = sym32_shndx; + long_size = 4; + extable_ent_size = 8; break; case ELFCLASS64: - { if (r2(&ehdr->e64.e_ehsize) != sizeof(Elf64_Ehdr) || r2(&ehdr->e64.e_shentsize) != sizeof(Elf64_Shdr)) { fprintf(stderr, "unrecognized ET_EXEC/ET_DYN file: %s\n", fname); - break; - } - rc = do_sort_64(ehdr, fname, custom_sort); + return -1; } + + compare_extable = compare_extable_64; + ehdr_shoff = ehdr64_shoff; + ehdr_shentsize = ehdr64_shentsize; + ehdr_shstrndx = ehdr64_shstrndx; + ehdr_shnum = ehdr64_shnum; + shdr_addr = shdr64_addr; + shdr_offset = shdr64_offset; + shdr_link = shdr64_link; + shdr_size = shdr64_size; + shdr_name = shdr64_name; + shdr_type = shdr64_type; + shdr_entsize = shdr64_entsize; + sym_type = sym64_type; + sym_name = sym64_name; + sym_value = sym64_value; + sym_shndx = sym64_shndx; + long_size = 8; + extable_ent_size = 16; + break; default: fprintf(stderr, "unrecognized ELF class %d %s\n", ehdr->e32.e_ident[EI_CLASS], fname); - break; + return -1; } - return rc; + return do_sort(ehdr, fname, custom_sort); } int main(int argc, char *argv[]) diff --git a/scripts/sorttable.h b/scripts/sorttable.h deleted file mode 100644 index 17a8541a10d6..000000000000 --- a/scripts/sorttable.h +++ /dev/null @@ -1,485 +0,0 @@ -/* SPDX-License-Identifier: GPL-2.0-only */ -/* - * sorttable.h - * - * Added ORC unwind tables sort support and other updates: - * Copyright (C) 1999-2019 Alibaba Group Holding Limited. by: - * Shile Zhang <shile.zhang@linux.alibaba.com> - * - * Copyright 2011 - 2012 Cavium, Inc. - * - * Some of code was taken out of arch/x86/kernel/unwind_orc.c, written by: - * Copyright (C) 2017 Josh Poimboeuf <jpoimboe@redhat.com> - * - * Some of this code was taken out of recordmcount.h written by: - * - * Copyright 2009 John F. Reiser <jreiser@BitWagon.com>. All rights reserved. - * Copyright 2010 Steven Rostedt <srostedt@redhat.com>, Red Hat Inc. - */ - -#undef extable_ent_size -#undef compare_extable -#undef get_mcount_loc -#undef sort_mcount_loc -#undef elf_mcount_loc -#undef do_sort -#undef ehdr_shoff -#undef ehdr_shentsize -#undef ehdr_shstrndx -#undef ehdr_shnum -#undef shdr_addr -#undef shdr_offset -#undef shdr_link -#undef shdr_size -#undef shdr_name -#undef shdr_type -#undef shdr_entsize -#undef sym_type -#undef sym_name -#undef sym_value -#undef sym_shndx -#undef long_size - -#ifdef SORTTABLE_64 -# define extable_ent_size 16 -# define compare_extable compare_extable_64 -# define get_mcount_loc get_mcount_loc_64 -# define sort_mcount_loc sort_mcount_loc_64 -# define elf_mcount_loc elf_mcount_loc_64 -# define do_sort do_sort_64 -# define ehdr_shoff ehdr64_shoff -# define ehdr_shentsize ehdr64_shentsize -# define ehdr_shstrndx ehdr64_shstrndx -# define ehdr_shnum ehdr64_shnum -# define shdr_addr shdr64_addr -# define shdr_offset shdr64_offset -# define shdr_link shdr64_link -# define shdr_size shdr64_size -# define shdr_name shdr64_name -# define shdr_type shdr64_type -# define shdr_entsize shdr64_entsize -# define sym_type sym64_type -# define sym_name sym64_name -# define sym_value sym64_value -# define sym_shndx sym64_shndx -# define long_size 8 -#else -# define extable_ent_size 8 -# define compare_extable compare_extable_32 -# define get_mcount_loc get_mcount_loc_32 -# define sort_mcount_loc sort_mcount_loc_32 -# define elf_mcount_loc elf_mcount_loc_32 -# define do_sort do_sort_32 -# define ehdr_shoff ehdr32_shoff -# define ehdr_shentsize ehdr32_shentsize -# define ehdr_shstrndx ehdr32_shstrndx -# define ehdr_shnum ehdr32_shnum -# define shdr_addr shdr32_addr -# define shdr_offset shdr32_offset -# define shdr_link shdr32_link -# define shdr_size shdr32_size -# define shdr_name shdr32_name -# define shdr_type shdr32_type -# define shdr_entsize shdr32_entsize -# define sym_type sym32_type -# define sym_name sym32_name -# define sym_value sym32_value -# define sym_shndx sym32_shndx -# define long_size 4 -#endif - -#if defined(SORTTABLE_64) && defined(UNWINDER_ORC_ENABLED) -/* ORC unwinder only support X86_64 */ -#include <asm/orc_types.h> - -#define ERRSTR_MAXSZ 256 - -char g_err[ERRSTR_MAXSZ]; -int *g_orc_ip_table; -struct orc_entry *g_orc_table; - -pthread_t orc_sort_thread; - -static inline unsigned long orc_ip(const int *ip) -{ - return (unsigned long)ip + *ip; -} - -static int orc_sort_cmp(const void *_a, const void *_b) -{ - struct orc_entry *orc_a, *orc_b; - const int *a = g_orc_ip_table + *(int *)_a; - const int *b = g_orc_ip_table + *(int *)_b; - unsigned long a_val = orc_ip(a); - unsigned long b_val = orc_ip(b); - - if (a_val > b_val) - return 1; - if (a_val < b_val) - return -1; - - /* - * The "weak" section terminator entries need to always be on the left - * to ensure the lookup code skips them in favor of real entries. - * These terminator entries exist to handle any gaps created by - * whitelisted .o files which didn't get objtool generation. - */ - orc_a = g_orc_table + (a - g_orc_ip_table); - orc_b = g_orc_table + (b - g_orc_ip_table); - if (orc_a->type == ORC_TYPE_UNDEFINED && orc_b->type == ORC_TYPE_UNDEFINED) - return 0; - return orc_a->type == ORC_TYPE_UNDEFINED ? -1 : 1; -} - -static void *sort_orctable(void *arg) -{ - int i; - int *idxs = NULL; - int *tmp_orc_ip_table = NULL; - struct orc_entry *tmp_orc_table = NULL; - unsigned int *orc_ip_size = (unsigned int *)arg; - unsigned int num_entries = *orc_ip_size / sizeof(int); - unsigned int orc_size = num_entries * sizeof(struct orc_entry); - - idxs = (int *)malloc(*orc_ip_size); - if (!idxs) { - snprintf(g_err, ERRSTR_MAXSZ, "malloc idxs: %s", - strerror(errno)); - pthread_exit(g_err); - } - - tmp_orc_ip_table = (int *)malloc(*orc_ip_size); - if (!tmp_orc_ip_table) { - snprintf(g_err, ERRSTR_MAXSZ, "malloc tmp_orc_ip_table: %s", - strerror(errno)); - pthread_exit(g_err); - } - - tmp_orc_table = (struct orc_entry *)malloc(orc_size); - if (!tmp_orc_table) { - snprintf(g_err, ERRSTR_MAXSZ, "malloc tmp_orc_table: %s", - strerror(errno)); - pthread_exit(g_err); - } - - /* initialize indices array, convert ip_table to absolute address */ - for (i = 0; i < num_entries; i++) { - idxs[i] = i; - tmp_orc_ip_table[i] = g_orc_ip_table[i] + i * sizeof(int); - } - memcpy(tmp_orc_table, g_orc_table, orc_size); - - qsort(idxs, num_entries, sizeof(int), orc_sort_cmp); - - for (i = 0; i < num_entries; i++) { - if (idxs[i] == i) - continue; - - /* convert back to relative address */ - g_orc_ip_table[i] = tmp_orc_ip_table[idxs[i]] - i * sizeof(int); - g_orc_table[i] = tmp_orc_table[idxs[i]]; - } - - free(idxs); - free(tmp_orc_ip_table); - free(tmp_orc_table); - pthread_exit(NULL); -} -#endif - -#ifdef MCOUNT_SORT_ENABLED -pthread_t mcount_sort_thread; - -struct elf_mcount_loc { - Elf_Ehdr *ehdr; - Elf_Shdr *init_data_sec; - uint64_t start_mcount_loc; - uint64_t stop_mcount_loc; -}; - -/* Sort the addresses stored between __start_mcount_loc to __stop_mcount_loc in vmlinux */ -static void *sort_mcount_loc(void *arg) -{ - struct elf_mcount_loc *emloc = (struct elf_mcount_loc *)arg; - uint64_t offset = emloc->start_mcount_loc - shdr_addr(emloc->init_data_sec) - + shdr_offset(emloc->init_data_sec); - uint64_t count = emloc->stop_mcount_loc - emloc->start_mcount_loc; - unsigned char *start_loc = (void *)emloc->ehdr + offset; - - qsort(start_loc, count/long_size, long_size, compare_extable); - return NULL; -} - -/* Get the address of __start_mcount_loc and __stop_mcount_loc in System.map */ -static void get_mcount_loc(uint64_t *_start, uint64_t *_stop) -{ - FILE *file_start, *file_stop; - char start_buff[20]; - char stop_buff[20]; - int len = 0; - - file_start = popen(" grep start_mcount System.map | awk '{print $1}' ", "r"); - if (!file_start) { - fprintf(stderr, "get start_mcount_loc error!"); - return; - } - - file_stop = popen(" grep stop_mcount System.map | awk '{print $1}' ", "r"); - if (!file_stop) { - fprintf(stderr, "get stop_mcount_loc error!"); - pclose(file_start); - return; - } - - while (fgets(start_buff, sizeof(start_buff), file_start) != NULL) { - len = strlen(start_buff); - start_buff[len - 1] = '\0'; - } - *_start = strtoul(start_buff, NULL, 16); - - while (fgets(stop_buff, sizeof(stop_buff), file_stop) != NULL) { - len = strlen(stop_buff); - stop_buff[len - 1] = '\0'; - } - *_stop = strtoul(stop_buff, NULL, 16); - - pclose(file_start); - pclose(file_stop); -} -#endif -static int do_sort(Elf_Ehdr *ehdr, - char const *const fname, - table_sort_t custom_sort) -{ - int rc = -1; - Elf_Shdr *shdr_start; - Elf_Shdr *strtab_sec = NULL; - Elf_Shdr *symtab_sec = NULL; - Elf_Shdr *extab_sec = NULL; - Elf_Shdr *string_sec; - Elf_Sym *sym; - const Elf_Sym *symtab; - Elf32_Word *symtab_shndx = NULL; - Elf_Sym *sort_needed_sym = NULL; - Elf_Shdr *sort_needed_sec; - uint32_t *sort_needed_loc; - void *sym_start; - void *sym_end; - const char *secstrings; - const char *strtab; - char *extab_image; - int sort_need_index; - int symentsize; - int shentsize; - int idx; - int i; - unsigned int shnum; - unsigned int shstrndx; -#ifdef MCOUNT_SORT_ENABLED - struct elf_mcount_loc mstruct = {0}; - uint64_t _start_mcount_loc = 0; - uint64_t _stop_mcount_loc = 0; -#endif -#if defined(SORTTABLE_64) && defined(UNWINDER_ORC_ENABLED) - unsigned int orc_ip_size = 0; - unsigned int orc_size = 0; - unsigned int orc_num_entries = 0; -#endif - - shdr_start = (Elf_Shdr *)((char *)ehdr + ehdr_shoff(ehdr)); - shentsize = ehdr_shentsize(ehdr); - - shstrndx = ehdr_shstrndx(ehdr); - if (shstrndx == SHN_XINDEX) - shstrndx = shdr_link(shdr_start); - string_sec = get_index(shdr_start, shentsize, shstrndx); - secstrings = (const char *)ehdr + shdr_offset(string_sec); - - shnum = ehdr_shnum(ehdr); - if (shnum == SHN_UNDEF) - shnum = shdr_size(shdr_start); - - for (i = 0; i < shnum; i++) { - Elf_Shdr *shdr = get_index(shdr_start, shentsize, i); - - idx = shdr_name(shdr); - if (!strcmp(secstrings + idx, "__ex_table")) - extab_sec = shdr; - if (!strcmp(secstrings + idx, ".symtab")) - symtab_sec = shdr; - if (!strcmp(secstrings + idx, ".strtab")) - strtab_sec = shdr; - - if (shdr_type(shdr) == SHT_SYMTAB_SHNDX) - symtab_shndx = (Elf32_Word *)((const char *)ehdr + - shdr_offset(shdr)); - -#ifdef MCOUNT_SORT_ENABLED - /* locate the .init.data section in vmlinux */ - if (!strcmp(secstrings + idx, ".init.data")) { - get_mcount_loc(&_start_mcount_loc, &_stop_mcount_loc); - mstruct.ehdr = ehdr; - mstruct.init_data_sec = shdr; - mstruct.start_mcount_loc = _start_mcount_loc; - mstruct.stop_mcount_loc = _stop_mcount_loc; - } -#endif - -#if defined(SORTTABLE_64) && defined(UNWINDER_ORC_ENABLED) - /* locate the ORC unwind tables */ - if (!strcmp(secstrings + idx, ".orc_unwind_ip")) { - orc_ip_size = shdr_size(shdr); - g_orc_ip_table = (int *)((void *)ehdr + - shdr_offset(shdr)); - } - if (!strcmp(secstrings + idx, ".orc_unwind")) { - orc_size = shdr_size(shdr); - g_orc_table = (struct orc_entry *)((void *)ehdr + - shdr_offset(shdr)); - } -#endif - } /* for loop */ - -#if defined(SORTTABLE_64) && defined(UNWINDER_ORC_ENABLED) - if (!g_orc_ip_table || !g_orc_table) { - fprintf(stderr, - "incomplete ORC unwind tables in file: %s\n", fname); - goto out; - } - - orc_num_entries = orc_ip_size / sizeof(int); - if (orc_ip_size % sizeof(int) != 0 || - orc_size % sizeof(struct orc_entry) != 0 || - orc_num_entries != orc_size / sizeof(struct orc_entry)) { - fprintf(stderr, - "inconsistent ORC unwind table entries in file: %s\n", - fname); - goto out; - } - - /* create thread to sort ORC unwind tables concurrently */ - if (pthread_create(&orc_sort_thread, NULL, - sort_orctable, &orc_ip_size)) { - fprintf(stderr, - "pthread_create orc_sort_thread failed '%s': %s\n", - strerror(errno), fname); - goto out; - } -#endif - -#ifdef MCOUNT_SORT_ENABLED - if (!mstruct.init_data_sec || !_start_mcount_loc || !_stop_mcount_loc) { - fprintf(stderr, - "incomplete mcount's sort in file: %s\n", - fname); - goto out; - } - - /* create thread to sort mcount_loc concurrently */ - if (pthread_create(&mcount_sort_thread, NULL, &sort_mcount_loc, &mstruct)) { - fprintf(stderr, - "pthread_create mcount_sort_thread failed '%s': %s\n", - strerror(errno), fname); - goto out; - } -#endif - if (!extab_sec) { - fprintf(stderr, "no __ex_table in file: %s\n", fname); - goto out; - } - - if (!symtab_sec) { - fprintf(stderr, "no .symtab in file: %s\n", fname); - goto out; - } - - if (!strtab_sec) { - fprintf(stderr, "no .strtab in file: %s\n", fname); - goto out; - } - - extab_image = (void *)ehdr + shdr_offset(extab_sec); - strtab = (const char *)ehdr + shdr_offset(strtab_sec); - symtab = (const Elf_Sym *)((const char *)ehdr + shdr_offset(symtab_sec)); - - if (custom_sort) { - custom_sort(extab_image, shdr_size(extab_sec)); - } else { - int num_entries = shdr_size(extab_sec) / extable_ent_size; - qsort(extab_image, num_entries, - extable_ent_size, compare_extable); - } - - /* find the flag main_extable_sort_needed */ - sym_start = (void *)ehdr + shdr_offset(symtab_sec); - sym_end = sym_start + shdr_size(symtab_sec); - symentsize = shdr_entsize(symtab_sec); - - for (sym = sym_start; (void *)sym + symentsize < sym_end; - sym = (void *)sym + symentsize) { - if (sym_type(sym) != STT_OBJECT) - continue; - if (!strcmp(strtab + sym_name(sym), - "main_extable_sort_needed")) { - sort_needed_sym = sym; - break; - } - } - - if (!sort_needed_sym) { - fprintf(stderr, - "no main_extable_sort_needed symbol in file: %s\n", - fname); - goto out; - } - - sort_need_index = get_secindex(sym_shndx(sym), - ((void *)sort_needed_sym - (void *)symtab) / symentsize, - symtab_shndx); - sort_needed_sec = get_index(shdr_start, shentsize, sort_need_index); - sort_needed_loc = (void *)ehdr + - shdr_offset(sort_needed_sec) + - sym_value(sort_needed_sym) - shdr_addr(sort_needed_sec); - - /* extable has been sorted, clear the flag */ - w(0, sort_needed_loc); - rc = 0; - -out: -#if defined(SORTTABLE_64) && defined(UNWINDER_ORC_ENABLED) - if (orc_sort_thread) { - void *retval = NULL; - /* wait for ORC tables sort done */ - rc = pthread_join(orc_sort_thread, &retval); - if (rc) { - fprintf(stderr, - "pthread_join failed '%s': %s\n", - strerror(errno), fname); - } else if (retval) { - rc = -1; - fprintf(stderr, - "failed to sort ORC tables '%s': %s\n", - (char *)retval, fname); - } - } -#endif - -#ifdef MCOUNT_SORT_ENABLED - if (mcount_sort_thread) { - void *retval = NULL; - /* wait for mcount sort done */ - rc = pthread_join(mcount_sort_thread, &retval); - if (rc) { - fprintf(stderr, - "pthread_join failed '%s': %s\n", - strerror(errno), fname); - } else if (retval) { - rc = -1; - fprintf(stderr, - "failed to sort mcount '%s': %s\n", - (char *)retval, fname); - } - } -#endif - return rc; -} -- cgit v1.2.3 From 4acda8edefa1ce66d3de845f1c12745721cd14c3 Mon Sep 17 00:00:00 2001 From: Steven Rostedt <rostedt@goodmis.org> Date: Sun, 5 Jan 2025 11:22:25 -0500 Subject: scripts/sorttable: Get start/stop_mcount_loc from ELF file directly The get_mcount_loc() does a cheesy trick to find the start_mcount_loc and stop_mcount_loc values. That trick is: file_start = popen(" grep start_mcount System.map | awk '{print $1}' ", "r"); and file_stop = popen(" grep stop_mcount System.map | awk '{print $1}' ", "r"); Those values are stored in the Elf symbol table. Use that to capture those values. Using the symbol table is more efficient and more robust. The above could fail if another variable had "start_mcount" or "stop_mcount" as part of its name. Cc: bpf <bpf@vger.kernel.org> Cc: Masami Hiramatsu <mhiramat@kernel.org> Cc: Mark Rutland <mark.rutland@arm.com> Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Masahiro Yamada <masahiroy@kernel.org> Cc: Nathan Chancellor <nathan@kernel.org> Cc: Nicolas Schier <nicolas@fjasle.eu> Cc: Zheng Yejian <zhengyejian1@huawei.com> Cc: Martin Kelly <martin.kelly@crowdstrike.com> Cc: Christophe Leroy <christophe.leroy@csgroup.eu> Cc: Josh Poimboeuf <jpoimboe@redhat.com> Link: https://lore.kernel.org/20250105162346.817157047@goodmis.org Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org> --- scripts/sorttable.c | 95 +++++++++++++++++++++++++---------------------------- 1 file changed, 45 insertions(+), 50 deletions(-) (limited to 'scripts') diff --git a/scripts/sorttable.c b/scripts/sorttable.c index ff9b60fc0dd8..656c1e9b5ad9 100644 --- a/scripts/sorttable.c +++ b/scripts/sorttable.c @@ -472,42 +472,41 @@ static void *sort_mcount_loc(void *arg) } /* Get the address of __start_mcount_loc and __stop_mcount_loc in System.map */ -static void get_mcount_loc(uint64_t *_start, uint64_t *_stop) +static void get_mcount_loc(struct elf_mcount_loc *emloc, Elf_Shdr *symtab_sec, + const char *strtab) { - FILE *file_start, *file_stop; - char start_buff[20]; - char stop_buff[20]; - int len = 0; + Elf_Sym *sym, *end_sym; + int symentsize = shdr_entsize(symtab_sec); + int found = 0; + + sym = (void *)emloc->ehdr + shdr_offset(symtab_sec); + end_sym = (void *)sym + shdr_size(symtab_sec); + + while (sym < end_sym) { + if (!strcmp(strtab + sym_name(sym), "__start_mcount_loc")) { + emloc->start_mcount_loc = sym_value(sym); + if (++found == 2) + break; + } else if (!strcmp(strtab + sym_name(sym), "__stop_mcount_loc")) { + emloc->stop_mcount_loc = sym_value(sym); + if (++found == 2) + break; + } + sym = (void *)sym + symentsize; + } - file_start = popen(" grep start_mcount System.map | awk '{print $1}' ", "r"); - if (!file_start) { + if (!emloc->start_mcount_loc) { fprintf(stderr, "get start_mcount_loc error!"); return; } - file_stop = popen(" grep stop_mcount System.map | awk '{print $1}' ", "r"); - if (!file_stop) { + if (!emloc->stop_mcount_loc) { fprintf(stderr, "get stop_mcount_loc error!"); - pclose(file_start); return; } - - while (fgets(start_buff, sizeof(start_buff), file_start) != NULL) { - len = strlen(start_buff); - start_buff[len - 1] = '\0'; - } - *_start = strtoul(start_buff, NULL, 16); - - while (fgets(stop_buff, sizeof(stop_buff), file_stop) != NULL) { - len = strlen(stop_buff); - stop_buff[len - 1] = '\0'; - } - *_stop = strtoul(stop_buff, NULL, 16); - - pclose(file_start); - pclose(file_stop); } #endif + static int do_sort(Elf_Ehdr *ehdr, char const *const fname, table_sort_t custom_sort) @@ -538,8 +537,6 @@ static int do_sort(Elf_Ehdr *ehdr, unsigned int shstrndx; #ifdef MCOUNT_SORT_ENABLED struct elf_mcount_loc mstruct = {0}; - uint64_t _start_mcount_loc = 0; - uint64_t _stop_mcount_loc = 0; #endif #ifdef UNWINDER_ORC_ENABLED unsigned int orc_ip_size = 0; @@ -577,13 +574,8 @@ static int do_sort(Elf_Ehdr *ehdr, #ifdef MCOUNT_SORT_ENABLED /* locate the .init.data section in vmlinux */ - if (!strcmp(secstrings + idx, ".init.data")) { - get_mcount_loc(&_start_mcount_loc, &_stop_mcount_loc); - mstruct.ehdr = ehdr; + if (!strcmp(secstrings + idx, ".init.data")) mstruct.init_data_sec = shdr; - mstruct.start_mcount_loc = _start_mcount_loc; - mstruct.stop_mcount_loc = _stop_mcount_loc; - } #endif #ifdef UNWINDER_ORC_ENABLED @@ -627,23 +619,6 @@ static int do_sort(Elf_Ehdr *ehdr, goto out; } #endif - -#ifdef MCOUNT_SORT_ENABLED - if (!mstruct.init_data_sec || !_start_mcount_loc || !_stop_mcount_loc) { - fprintf(stderr, - "incomplete mcount's sort in file: %s\n", - fname); - goto out; - } - - /* create thread to sort mcount_loc concurrently */ - if (pthread_create(&mcount_sort_thread, NULL, &sort_mcount_loc, &mstruct)) { - fprintf(stderr, - "pthread_create mcount_sort_thread failed '%s': %s\n", - strerror(errno), fname); - goto out; - } -#endif if (!extab_sec) { fprintf(stderr, "no __ex_table in file: %s\n", fname); goto out; @@ -663,6 +638,26 @@ static int do_sort(Elf_Ehdr *ehdr, strtab = (const char *)ehdr + shdr_offset(strtab_sec); symtab = (const Elf_Sym *)((const char *)ehdr + shdr_offset(symtab_sec)); +#ifdef MCOUNT_SORT_ENABLED + mstruct.ehdr = ehdr; + get_mcount_loc(&mstruct, symtab_sec, strtab); + + if (!mstruct.init_data_sec || !mstruct.start_mcount_loc || !mstruct.stop_mcount_loc) { + fprintf(stderr, + "incomplete mcount's sort in file: %s\n", + fname); + goto out; + } + + /* create thread to sort mcount_loc concurrently */ + if (pthread_create(&mcount_sort_thread, NULL, &sort_mcount_loc, &mstruct)) { + fprintf(stderr, + "pthread_create mcount_sort_thread failed '%s': %s\n", + strerror(errno), fname); + goto out; + } +#endif + if (custom_sort) { custom_sort(extab_image, shdr_size(extab_sec)); } else { -- cgit v1.2.3 From def35da76073fcf43c2fae4942ebe55b60dc84a6 Mon Sep 17 00:00:00 2001 From: Costa Shulyupin <costa.shul@redhat.com> Date: Mon, 9 Dec 2024 10:29:57 +0200 Subject: scripts/tags.sh: Tag timer definitions For timer definitions like DEFINE_TIMER(mytimer, mytimer_handler); ctags generates tags `DEFINE_TIMER` and skips `mytimer` because it doesn't expand the DEFINE_TIMER macro. Configure ctags to generate tag for `mytimer` ans skip the `DEFINE_TIMER` tag in such cases. Signed-off-by: Costa Shulyupin <costa.shul@redhat.com> Link: https://lore.kernel.org/r/20241209083004.911013-2-costa.shul@redhat.com Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> --- scripts/tags.sh | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) (limited to 'scripts') diff --git a/scripts/tags.sh b/scripts/tags.sh index b21236377998..7102f14fc775 100755 --- a/scripts/tags.sh +++ b/scripts/tags.sh @@ -188,6 +188,7 @@ regex_c=( '/^PCI_OP_WRITE([[:space:]]*\(\w*\).*[1-4])/pci_bus_write_config_\1/' '/\<DEFINE_\(RT_MUTEX\|MUTEX\|SEMAPHORE\|SPINLOCK\)([[:space:]]*\([[:alnum:]_]*\)/\2/v/' '/\<DEFINE_\(RAW_SPINLOCK\|RWLOCK\|SEQLOCK\)([[:space:]]*\([[:alnum:]_]*\)/\2/v/' + '/\<DEFINE_TIMER(\([^,)]*\),/\1/' '/\<DECLARE_\(RWSEM\|COMPLETION\)([[:space:]]*\([[:alnum:]_]\+\)/\2/v/' '/\<DECLARE_BITMAP([[:space:]]*\([[:alnum:]_]\+\)/\1/v/' '/\(^\|\s\)\(\|L\|H\)LIST_HEAD([[:space:]]*\([[:alnum:]_]*\)/\3/v/' @@ -260,7 +261,7 @@ exuberant() # identifiers to ignore by ctags local ign=( ACPI_EXPORT_SYMBOL - DEFINE_{TRACE,MUTEX} + DEFINE_{TRACE,MUTEX,TIMER} EXPORT_SYMBOL EXPORT_SYMBOL_GPL EXPORT_TRACEPOINT_SYMBOL EXPORT_TRACEPOINT_SYMBOL_GPL ____cacheline_aligned ____cacheline_aligned_in_smp -- cgit v1.2.3 From 634d34e856ca0123f96613ffaf1852a1d3b46880 Mon Sep 17 00:00:00 2001 From: Thomas Gleixner <tglx@linutronix.de> Date: Fri, 29 Nov 2024 11:37:45 +0100 Subject: scripts/spdxcheck: Parse j2 comments correctly j2 files use '#}' as comment closure, which trips up the SPDX parser: tools/.../definition.j2: 1:36 Invalid token: #} Handle those comments correctly by removing the closure. Signed-off-by: Thomas Gleixner <tglx@linutronix.de> Link: https://lore.kernel.org/r/878qt2xr46.ffs@tglx Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> --- scripts/spdxcheck.py | 3 +++ 1 file changed, 3 insertions(+) (limited to 'scripts') diff --git a/scripts/spdxcheck.py b/scripts/spdxcheck.py index 8b8fb115fc81..fe57c9aecf37 100755 --- a/scripts/spdxcheck.py +++ b/scripts/spdxcheck.py @@ -217,6 +217,9 @@ class id_parser(object): # Special case for SH magic boot code files if line.startswith('LIST \"'): expr = expr.rstrip('\"').strip() + # Remove j2 comment closure + if line.startswith('{#'): + expr = expr.rstrip('#}').strip() self.parse(expr) self.spdx_valid += 1 # -- cgit v1.2.3 From 154916f4b59dc2228caad5fc98e9e6a7e1a84e19 Mon Sep 17 00:00:00 2001 From: Lukas Bulwahn <lukas.bulwahn@redhat.com> Date: Wed, 8 Jan 2025 13:52:07 +0100 Subject: scripts/spdxcheck: Handle license identifiers in Jinja comments Commit 4b132aacb076 ("tools: Add xdrgen") adds a tool, which uses Jinja template files, i.e., files with the j2 file extension, for its lightweight code generation. These template files for this tool have proper headers with the SPDX License information, which are included as Jinja comments by enclosing the text with '{#' and '#}'. Sofar, the spdxcheck script does not support to properly parse this license information in Jinja comments and it reports back with 'Invalid token: #}'. Parse Jinja comments properly by stripping the known Jinja comment suffix. Signed-off-by: Lukas Bulwahn <lukas.bulwahn@redhat.com> Link: https://lore.kernel.org/r/20250108125207.57486-1-lukas.bulwahn@redhat.com Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> --- scripts/spdxcheck.py | 3 +++ 1 file changed, 3 insertions(+) (limited to 'scripts') diff --git a/scripts/spdxcheck.py b/scripts/spdxcheck.py index fe57c9aecf37..8d608f61bf37 100755 --- a/scripts/spdxcheck.py +++ b/scripts/spdxcheck.py @@ -214,6 +214,9 @@ class id_parser(object): # Remove trailing xml comment closure if line.strip().endswith('-->'): expr = expr.rstrip('-->').strip() + # Remove trailing Jinja2 comment closure + if line.strip().endswith('#}'): + expr = expr.rstrip('#}').strip() # Special case for SH magic boot code files if line.startswith('LIST \"'): expr = expr.rstrip('\"').strip() -- cgit v1.2.3 From c23d1f7e15d11d6ae6c70824e04ba9ed0299de0a Mon Sep 17 00:00:00 2001 From: Miguel Ojeda <ojeda@kernel.org> Date: Mon, 9 Dec 2024 22:25:44 +0100 Subject: rust: document `bindgen` 0.71.0 regression `bindgen` 0.71.0 regressed [1] on the "`--version` requires header" issue which appeared in 0.69.0 first [2] and was fixed in 0.69.1. It has been fixed again in 0.71.1 [3]. Thus document it so that, when we upgrade the minimum past 0.69.0 in the future, we do not forget that we cannot remove the workaround until we arrive at 0.71.1 at least. Link: https://github.com/rust-lang/rust-bindgen/issues/3039 [1] Link: https://github.com/rust-lang/rust-bindgen/issues/2677 [2] Link: https://github.com/rust-lang/rust-bindgen/blob/main/CHANGELOG.md#v0711-2024-12-09 [3] Reviewed-by: Fiona Behrens <me@kloenk.dev> Reviewed-by: Alice Ryhl <aliceryhl@google.com> Link: https://lore.kernel.org/r/20241209212544.1977065-1-ojeda@kernel.org Signed-off-by: Miguel Ojeda <ojeda@kernel.org> --- init/Kconfig | 6 ++++-- scripts/rust_is_available.sh | 6 ++++-- 2 files changed, 8 insertions(+), 4 deletions(-) (limited to 'scripts') diff --git a/init/Kconfig b/init/Kconfig index a20e6efd3f0f..e8d2b5128f87 100644 --- a/init/Kconfig +++ b/init/Kconfig @@ -1989,8 +1989,10 @@ config BINDGEN_VERSION_TEXT string depends on RUST # The dummy parameter `workaround-for-0.69.0` is required to support 0.69.0 - # (https://github.com/rust-lang/rust-bindgen/pull/2678). It can be removed when - # the minimum version is upgraded past that (0.69.1 already fixed the issue). + # (https://github.com/rust-lang/rust-bindgen/pull/2678) and 0.71.0 + # (https://github.com/rust-lang/rust-bindgen/pull/3040). It can be removed + # when the minimum version is upgraded past the latter (0.69.1 and 0.71.1 + # both fixed the issue). default "$(shell,$(BINDGEN) --version workaround-for-0.69.0 2>/dev/null)" # diff --git a/scripts/rust_is_available.sh b/scripts/rust_is_available.sh index 93c0ef7fb3fb..d2323de0692c 100755 --- a/scripts/rust_is_available.sh +++ b/scripts/rust_is_available.sh @@ -123,8 +123,10 @@ fi # Non-stable and distributions' versions may have a version suffix, e.g. `-dev`. # # The dummy parameter `workaround-for-0.69.0` is required to support 0.69.0 -# (https://github.com/rust-lang/rust-bindgen/pull/2678). It can be removed when -# the minimum version is upgraded past that (0.69.1 already fixed the issue). +# (https://github.com/rust-lang/rust-bindgen/pull/2678) and 0.71.0 +# (https://github.com/rust-lang/rust-bindgen/pull/3040). It can be removed when +# the minimum version is upgraded past the latter (0.69.1 and 0.71.1 both fixed +# the issue). rust_bindings_generator_output=$( \ LC_ALL=C "$BINDGEN" --version workaround-for-0.69.0 2>/dev/null ) || rust_bindings_generator_code=$? -- cgit v1.2.3 From 0730422bced5e8325fb6806d9a80bb10673588e6 Mon Sep 17 00:00:00 2001 From: Tamir Duberstein <tamird@gmail.com> Date: Mon, 16 Dec 2024 10:54:22 -0500 Subject: rust: use host dylib naming convention to support macOS Because the `macros` crate exposes procedural macros, it must be compiled as a dynamic library (so it can be loaded by the compiler at compile-time). Before this change the resulting artifact was always named `libmacros.so`, which works on hosts where this matches the naming convention for dynamic libraries. However the proper name on macOS would be `libmacros.dylib`. This turns out to matter even when the dependency is passed with a path (`--extern macros=path/to/libmacros.so` rather than `--extern macros`) because rustc uses the file name to infer the type of the library (see link). This is because there's no way to specify both the path to and the type of the external library via CLI flags. The compiler could speculatively parse the file to determine its type, but it does not do so today. This means that libraries that match neither rustc's naming convention for static libraries nor the platform's naming convention for dynamic libraries are *rejected*. The only solution I've found is to follow the host platform's naming convention. This patch does that by querying the compiler to determine the appropriate name for the artifact. This allows the kernel to build with CONFIG_RUST=y on macOS. Link: https://github.com/rust-lang/rust/blob/d829780/compiler/rustc_metadata/src/locator.rs#L728-L752 Tested-by: Daniel Gomez <da.gomez@samsung.com> Co-developed-by: Fiona Behrens <me@kloenk.dev> Signed-off-by: Fiona Behrens <me@kloenk.dev> Signed-off-by: Tamir Duberstein <tamird@gmail.com> Tested-by: Andreas Hindborg <a.hindborg@kernel.org> Link: https://lore.kernel.org/r/20241216-b4-dylib-host-macos-v7-1-cfc507681447@gmail.com [ Added `MAKEFLAGS=`s to avoid jobserver warnings. Removed space. Reworded title. - Miguel ] Signed-off-by: Miguel Ojeda <ojeda@kernel.org> --- .gitignore | 1 + Makefile | 2 +- rust/Makefile | 22 ++++++++++++---------- scripts/generate_rust_analyzer.py | 15 +++++++++++---- 4 files changed, 25 insertions(+), 15 deletions(-) (limited to 'scripts') diff --git a/.gitignore b/.gitignore index 6839cf84acda..5937c74d3dc1 100644 --- a/.gitignore +++ b/.gitignore @@ -22,6 +22,7 @@ *.dtb.S *.dtbo.S *.dwo +*.dylib *.elf *.gcno *.gcda diff --git a/Makefile b/Makefile index e5b8a8832c0c..80d9943d9744 100644 --- a/Makefile +++ b/Makefile @@ -1571,7 +1571,7 @@ MRPROPER_FILES += include/config include/generated \ certs/x509.genkey \ vmlinux-gdb.py \ rpmbuild \ - rust/libmacros.so + rust/libmacros.so rust/libmacros.dylib # clean - Delete most, but leave enough to build external modules # diff --git a/rust/Makefile b/rust/Makefile index 161e7cf67c97..5c8f35f5ada2 100644 --- a/rust/Makefile +++ b/rust/Makefile @@ -11,9 +11,6 @@ always-$(CONFIG_RUST) += exports_core_generated.h obj-$(CONFIG_RUST) += helpers/helpers.o CFLAGS_REMOVE_helpers/helpers.o = -Wmissing-prototypes -Wmissing-declarations -always-$(CONFIG_RUST) += libmacros.so -no-clean-files += libmacros.so - always-$(CONFIG_RUST) += bindings/bindings_generated.rs bindings/bindings_helpers_generated.rs obj-$(CONFIG_RUST) += bindings.o kernel.o always-$(CONFIG_RUST) += exports_helpers_generated.h \ @@ -38,9 +35,14 @@ obj-$(CONFIG_RUST_KERNEL_DOCTESTS) += doctests_kernel_generated_kunit.o always-$(subst y,$(CONFIG_RUST),$(CONFIG_JUMP_LABEL)) += kernel/generated_arch_static_branch_asm.rs -# Avoids running `$(RUSTC)` for the sysroot when it may not be available. +# Avoids running `$(RUSTC)` when it may not be available. ifdef CONFIG_RUST +libmacros_name := $(shell MAKEFLAGS= $(RUSTC) --print file-names --crate-name macros --crate-type proc-macro - </dev/null) +libmacros_extension := $(patsubst libmacros.%,%,$(libmacros_name)) + +always-$(CONFIG_RUST) += $(libmacros_name) + # `$(rust_flags)` is passed in case the user added `--sysroot`. rustc_sysroot := $(shell MAKEFLAGS= $(RUSTC) $(rust_flags) --print sysroot) rustc_host_target := $(shell $(RUSTC) --version --verbose | grep -F 'host: ' | cut -d' ' -f2) @@ -109,10 +111,10 @@ rustdoc-ffi: $(src)/ffi.rs rustdoc-core FORCE +$(call if_changed,rustdoc) rustdoc-kernel: private rustc_target_flags = --extern ffi \ - --extern build_error --extern macros=$(objtree)/$(obj)/libmacros.so \ + --extern build_error --extern macros \ --extern bindings --extern uapi rustdoc-kernel: $(src)/kernel/lib.rs rustdoc-core rustdoc-ffi rustdoc-macros \ - rustdoc-compiler_builtins $(obj)/libmacros.so \ + rustdoc-compiler_builtins $(obj)/$(libmacros_name) \ $(obj)/bindings.o FORCE +$(call if_changed,rustdoc) @@ -362,10 +364,10 @@ quiet_cmd_rustc_procmacro = $(RUSTC_OR_CLIPPY_QUIET) P $@ -Clink-args='$(call escsq,$(KBUILD_HOSTLDFLAGS))' \ --emit=dep-info=$(depfile) --emit=link=$@ --extern proc_macro \ --crate-type proc-macro \ - --crate-name $(patsubst lib%.so,%,$(notdir $@)) $< + --crate-name $(patsubst lib%.$(libmacros_extension),%,$(notdir $@)) $< # Procedural macros can only be used with the `rustc` that compiled it. -$(obj)/libmacros.so: $(src)/macros/lib.rs FORCE +$(obj)/$(libmacros_name): $(src)/macros/lib.rs FORCE +$(call if_changed_dep,rustc_procmacro) quiet_cmd_rustc_library = $(if $(skip_clippy),RUSTC,$(RUSTC_OR_CLIPPY_QUIET)) L $@ @@ -382,7 +384,7 @@ quiet_cmd_rustc_library = $(if $(skip_clippy),RUSTC,$(RUSTC_OR_CLIPPY_QUIET)) L $(cmd_objtool) rust-analyzer: - $(Q)$(srctree)/scripts/generate_rust_analyzer.py \ + $(Q)MAKEFLAGS= $(srctree)/scripts/generate_rust_analyzer.py \ --cfgs='core=$(core-cfgs)' \ $(realpath $(srctree)) $(realpath $(objtree)) \ $(rustc_sysroot) $(RUST_LIB_SRC) $(if $(KBUILD_EXTMOD),$(srcroot)) \ @@ -443,7 +445,7 @@ $(obj)/uapi.o: $(src)/uapi/lib.rs \ $(obj)/kernel.o: private rustc_target_flags = --extern ffi \ --extern build_error --extern macros --extern bindings --extern uapi $(obj)/kernel.o: $(src)/kernel/lib.rs $(obj)/build_error.o \ - $(obj)/libmacros.so $(obj)/bindings.o $(obj)/uapi.o FORCE + $(obj)/$(libmacros_name) $(obj)/bindings.o $(obj)/uapi.o FORCE +$(call if_changed_rule,rustc_library) ifdef CONFIG_JUMP_LABEL diff --git a/scripts/generate_rust_analyzer.py b/scripts/generate_rust_analyzer.py index 09e1d166d8d2..aa8ea1a4dbe5 100755 --- a/scripts/generate_rust_analyzer.py +++ b/scripts/generate_rust_analyzer.py @@ -8,6 +8,7 @@ import json import logging import os import pathlib +import subprocess import sys def args_crates_cfgs(cfgs): @@ -35,8 +36,7 @@ def generate_crates(srctree, objtree, sysroot_src, external_src, cfgs): crates_cfgs = args_crates_cfgs(cfgs) def append_crate(display_name, root_module, deps, cfg=[], is_workspace_member=True, is_proc_macro=False): - crates_indexes[display_name] = len(crates) - crates.append({ + crate = { "display_name": display_name, "root_module": str(root_module), "is_workspace_member": is_workspace_member, @@ -47,7 +47,15 @@ def generate_crates(srctree, objtree, sysroot_src, external_src, cfgs): "env": { "RUST_MODFILE": "This is only for rust-analyzer" } - }) + } + if is_proc_macro: + proc_macro_dylib_name = subprocess.check_output( + [os.environ["RUSTC"], "--print", "file-names", "--crate-name", display_name, "--crate-type", "proc-macro", "-"], + stdin=subprocess.DEVNULL, + ).decode('utf-8').strip() + crate["proc_macro_dylib_path"] = f"{objtree}/rust/{proc_macro_dylib_name}" + crates_indexes[display_name] = len(crates) + crates.append(crate) # First, the ones in `rust/` since they are a bit special. append_crate( @@ -70,7 +78,6 @@ def generate_crates(srctree, objtree, sysroot_src, external_src, cfgs): [], is_proc_macro=True, ) - crates[-1]["proc_macro_dylib_path"] = f"{objtree}/rust/libmacros.so" append_crate( "build_error", -- cgit v1.2.3 From 1e5f6771c247b28135307058d2cfe3b0153733dc Mon Sep 17 00:00:00 2001 From: Steven Rostedt <rostedt@goodmis.org> Date: Fri, 10 Jan 2025 07:54:59 -0500 Subject: scripts/sorttable: Use a structure of function pointers for elf helpers Instead of having a series of function pointers that gets assigned to the Elf64 or Elf32 versions, put them all into a single structure and use that. Add the helper function that chooses the structure into the macros that build the different versions of the elf functions. Link: https://lore.kernel.org/all/CAHk-=wiafEyX7UgOeZgvd6fvuByE5WXUPh9599kwOc_d-pdeug@mail.gmail.com/ Cc: bpf <bpf@vger.kernel.org> Cc: Masami Hiramatsu <mhiramat@kernel.org> Cc: Mark Rutland <mark.rutland@arm.com> Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Masahiro Yamada <masahiroy@kernel.org> Cc: Nathan Chancellor <nathan@kernel.org> Cc: Nicolas Schier <nicolas@fjasle.eu> Cc: Zheng Yejian <zhengyejian1@huawei.com> Cc: Martin Kelly <martin.kelly@crowdstrike.com> Cc: Christophe Leroy <christophe.leroy@csgroup.eu> Cc: Josh Poimboeuf <jpoimboe@redhat.com> Cc: Stephen Rothwell <sfr@canb.auug.org.au> Link: https://lore.kernel.org/20250110075459.13d4b94c@gandalf.local.home Suggested-by: Linus Torvalds <torvalds@linux-foundation.org> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org> --- scripts/sorttable.c | 175 +++++++++++++++++++++++++++++++++++----------------- 1 file changed, 118 insertions(+), 57 deletions(-) (limited to 'scripts') diff --git a/scripts/sorttable.c b/scripts/sorttable.c index 656c1e9b5ad9..9f41575afd7a 100644 --- a/scripts/sorttable.c +++ b/scripts/sorttable.c @@ -85,6 +85,25 @@ static uint64_t (*r8)(const uint64_t *); static void (*w)(uint32_t, uint32_t *); typedef void (*table_sort_t)(char *, int); +static struct elf_funcs { + int (*compare_extable)(const void *a, const void *b); + uint64_t (*ehdr_shoff)(Elf_Ehdr *ehdr); + uint16_t (*ehdr_shstrndx)(Elf_Ehdr *ehdr); + uint16_t (*ehdr_shentsize)(Elf_Ehdr *ehdr); + uint16_t (*ehdr_shnum)(Elf_Ehdr *ehdr); + uint64_t (*shdr_addr)(Elf_Shdr *shdr); + uint64_t (*shdr_offset)(Elf_Shdr *shdr); + uint64_t (*shdr_size)(Elf_Shdr *shdr); + uint64_t (*shdr_entsize)(Elf_Shdr *shdr); + uint32_t (*shdr_link)(Elf_Shdr *shdr); + uint32_t (*shdr_name)(Elf_Shdr *shdr); + uint32_t (*shdr_type)(Elf_Shdr *shdr); + uint8_t (*sym_type)(Elf_Sym *sym); + uint32_t (*sym_name)(Elf_Sym *sym); + uint64_t (*sym_value)(Elf_Sym *sym); + uint16_t (*sym_shndx)(Elf_Sym *sym); +} e; + static uint64_t ehdr64_shoff(Elf_Ehdr *ehdr) { return r8(&ehdr->e64.e_shoff); @@ -95,6 +114,11 @@ static uint64_t ehdr32_shoff(Elf_Ehdr *ehdr) return r(&ehdr->e32.e_shoff); } +static uint64_t ehdr_shoff(Elf_Ehdr *ehdr) +{ + return e.ehdr_shoff(ehdr); +} + #define EHDR_HALF(fn_name) \ static uint16_t ehdr64_##fn_name(Elf_Ehdr *ehdr) \ { \ @@ -104,6 +128,11 @@ static uint16_t ehdr64_##fn_name(Elf_Ehdr *ehdr) \ static uint16_t ehdr32_##fn_name(Elf_Ehdr *ehdr) \ { \ return r2(&ehdr->e32.e_##fn_name); \ +} \ + \ +static uint16_t ehdr_##fn_name(Elf_Ehdr *ehdr) \ +{ \ + return e.ehdr_##fn_name(ehdr); \ } EHDR_HALF(shentsize) @@ -119,6 +148,11 @@ static uint32_t shdr64_##fn_name(Elf_Shdr *shdr) \ static uint32_t shdr32_##fn_name(Elf_Shdr *shdr) \ { \ return r(&shdr->e32.sh_##fn_name); \ +} \ + \ +static uint32_t shdr_##fn_name(Elf_Shdr *shdr) \ +{ \ + return e.shdr_##fn_name(shdr); \ } #define SHDR_ADDR(fn_name) \ @@ -130,6 +164,11 @@ static uint64_t shdr64_##fn_name(Elf_Shdr *shdr) \ static uint64_t shdr32_##fn_name(Elf_Shdr *shdr) \ { \ return r(&shdr->e32.sh_##fn_name); \ +} \ + \ +static uint64_t shdr_##fn_name(Elf_Shdr *shdr) \ +{ \ + return e.shdr_##fn_name(shdr); \ } #define SHDR_WORD(fn_name) \ @@ -141,6 +180,10 @@ static uint32_t shdr64_##fn_name(Elf_Shdr *shdr) \ static uint32_t shdr32_##fn_name(Elf_Shdr *shdr) \ { \ return r(&shdr->e32.sh_##fn_name); \ +} \ +static uint32_t shdr_##fn_name(Elf_Shdr *shdr) \ +{ \ + return e.shdr_##fn_name(shdr); \ } SHDR_ADDR(addr) @@ -161,6 +204,11 @@ static uint64_t sym64_##fn_name(Elf_Sym *sym) \ static uint64_t sym32_##fn_name(Elf_Sym *sym) \ { \ return r(&sym->e32.st_##fn_name); \ +} \ + \ +static uint64_t sym_##fn_name(Elf_Sym *sym) \ +{ \ + return e.sym_##fn_name(sym); \ } #define SYM_WORD(fn_name) \ @@ -172,6 +220,11 @@ static uint32_t sym64_##fn_name(Elf_Sym *sym) \ static uint32_t sym32_##fn_name(Elf_Sym *sym) \ { \ return r(&sym->e32.st_##fn_name); \ +} \ + \ +static uint32_t sym_##fn_name(Elf_Sym *sym) \ +{ \ + return e.sym_##fn_name(sym); \ } #define SYM_HALF(fn_name) \ @@ -183,6 +236,11 @@ static uint16_t sym64_##fn_name(Elf_Sym *sym) \ static uint16_t sym32_##fn_name(Elf_Sym *sym) \ { \ return r2(&sym->e32.st_##fn_name); \ +} \ + \ +static uint16_t sym_##fn_name(Elf_Sym *sym) \ +{ \ + return e.sym_##fn_name(sym); \ } static uint8_t sym64_type(Elf_Sym *sym) @@ -195,6 +253,11 @@ static uint8_t sym32_type(Elf_Sym *sym) return ELF32_ST_TYPE(sym->e32.st_info); } +static uint8_t sym_type(Elf_Sym *sym) +{ + return e.sym_type(sym); +} + SYM_ADDR(value) SYM_WORD(name) SYM_HALF(shndx) @@ -322,29 +385,16 @@ static int compare_extable_64(const void *a, const void *b) return av > bv; } +static int compare_extable(const void *a, const void *b) +{ + return e.compare_extable(a, b); +} + static inline void *get_index(void *start, int entsize, int index) { return start + (entsize * index); } - -static int (*compare_extable)(const void *a, const void *b); -static uint64_t (*ehdr_shoff)(Elf_Ehdr *ehdr); -static uint16_t (*ehdr_shstrndx)(Elf_Ehdr *ehdr); -static uint16_t (*ehdr_shentsize)(Elf_Ehdr *ehdr); -static uint16_t (*ehdr_shnum)(Elf_Ehdr *ehdr); -static uint64_t (*shdr_addr)(Elf_Shdr *shdr); -static uint64_t (*shdr_offset)(Elf_Shdr *shdr); -static uint64_t (*shdr_size)(Elf_Shdr *shdr); -static uint64_t (*shdr_entsize)(Elf_Shdr *shdr); -static uint32_t (*shdr_link)(Elf_Shdr *shdr); -static uint32_t (*shdr_name)(Elf_Shdr *shdr); -static uint32_t (*shdr_type)(Elf_Shdr *shdr); -static uint8_t (*sym_type)(Elf_Sym *sym); -static uint32_t (*sym_name)(Elf_Sym *sym); -static uint64_t (*sym_value)(Elf_Sym *sym); -static uint16_t (*sym_shndx)(Elf_Sym *sym); - static int extable_ent_size; static int long_size; @@ -864,7 +914,30 @@ static int do_file(char const *const fname, void *addr) } switch (ehdr->e32.e_ident[EI_CLASS]) { - case ELFCLASS32: + case ELFCLASS32: { + struct elf_funcs efuncs = { + .compare_extable = compare_extable_32, + .ehdr_shoff = ehdr32_shoff, + .ehdr_shentsize = ehdr32_shentsize, + .ehdr_shstrndx = ehdr32_shstrndx, + .ehdr_shnum = ehdr32_shnum, + .shdr_addr = shdr32_addr, + .shdr_offset = shdr32_offset, + .shdr_link = shdr32_link, + .shdr_size = shdr32_size, + .shdr_name = shdr32_name, + .shdr_type = shdr32_type, + .shdr_entsize = shdr32_entsize, + .sym_type = sym32_type, + .sym_name = sym32_name, + .sym_value = sym32_value, + .sym_shndx = sym32_shndx, + }; + + e = efuncs; + long_size = 4; + extable_ent_size = 8; + if (r2(&ehdr->e32.e_ehsize) != sizeof(Elf32_Ehdr) || r2(&ehdr->e32.e_shentsize) != sizeof(Elf32_Shdr)) { fprintf(stderr, @@ -872,26 +945,32 @@ static int do_file(char const *const fname, void *addr) return -1; } - compare_extable = compare_extable_32; - ehdr_shoff = ehdr32_shoff; - ehdr_shentsize = ehdr32_shentsize; - ehdr_shstrndx = ehdr32_shstrndx; - ehdr_shnum = ehdr32_shnum; - shdr_addr = shdr32_addr; - shdr_offset = shdr32_offset; - shdr_link = shdr32_link; - shdr_size = shdr32_size; - shdr_name = shdr32_name; - shdr_type = shdr32_type; - shdr_entsize = shdr32_entsize; - sym_type = sym32_type; - sym_name = sym32_name; - sym_value = sym32_value; - sym_shndx = sym32_shndx; - long_size = 4; - extable_ent_size = 8; + } break; - case ELFCLASS64: + case ELFCLASS64: { + struct elf_funcs efuncs = { + .compare_extable = compare_extable_64, + .ehdr_shoff = ehdr64_shoff, + .ehdr_shentsize = ehdr64_shentsize, + .ehdr_shstrndx = ehdr64_shstrndx, + .ehdr_shnum = ehdr64_shnum, + .shdr_addr = shdr64_addr, + .shdr_offset = shdr64_offset, + .shdr_link = shdr64_link, + .shdr_size = shdr64_size, + .shdr_name = shdr64_name, + .shdr_type = shdr64_type, + .shdr_entsize = shdr64_entsize, + .sym_type = sym64_type, + .sym_name = sym64_name, + .sym_value = sym64_value, + .sym_shndx = sym64_shndx, + }; + + e = efuncs; + long_size = 8; + extable_ent_size = 16; + if (r2(&ehdr->e64.e_ehsize) != sizeof(Elf64_Ehdr) || r2(&ehdr->e64.e_shentsize) != sizeof(Elf64_Shdr)) { fprintf(stderr, @@ -900,25 +979,7 @@ static int do_file(char const *const fname, void *addr) return -1; } - compare_extable = compare_extable_64; - ehdr_shoff = ehdr64_shoff; - ehdr_shentsize = ehdr64_shentsize; - ehdr_shstrndx = ehdr64_shstrndx; - ehdr_shnum = ehdr64_shnum; - shdr_addr = shdr64_addr; - shdr_offset = shdr64_offset; - shdr_link = shdr64_link; - shdr_size = shdr64_size; - shdr_name = shdr64_name; - shdr_type = shdr64_type; - shdr_entsize = shdr64_entsize; - sym_type = sym64_type; - sym_name = sym64_name; - sym_value = sym64_value; - sym_shndx = sym64_shndx; - long_size = 8; - extable_ent_size = 16; - + } break; default: fprintf(stderr, "unrecognized ELF class %d %s\n", -- cgit v1.2.3 From d9ecb92b4fbbbb0a9993017da8d044541ca35886 Mon Sep 17 00:00:00 2001 From: Masahiro Yamada <masahiroy@kernel.org> Date: Tue, 3 Dec 2024 22:59:58 +0900 Subject: kbuild: deb-pkg: do not include empty hook directories The linux-image package currently includes empty hook directories (/etc/kernel/{pre,post}{inst,rm}.d/ by default). These directories were perhaps intended as a fail-safe in case no hook scripts exist there. However, they are really unnecessary because the run-parts command is already guarded by the following check: test -d ${debhookdir}/${script}.d && run-parts ... The only difference is that the run-parts command either runs for empty directories (resulting in a no-op) or is skipped entirely. The maintainer scripts will succeed without these dummy directories. The linux-image packages from the Debian kernel do not contain /etc/kernel/*.d/, either. Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> --- scripts/package/builddeb | 2 -- 1 file changed, 2 deletions(-) (limited to 'scripts') diff --git a/scripts/package/builddeb b/scripts/package/builddeb index ad7aba0f268e..85fe8f56bb9b 100755 --- a/scripts/package/builddeb +++ b/scripts/package/builddeb @@ -76,8 +76,6 @@ install_maint_scripts () { # so do we; recent versions of dracut and initramfs-tools will obey this. debhookdir=${KDEB_HOOKDIR:-/etc/kernel} for script in postinst postrm preinst prerm; do - mkdir -p "${pdir}${debhookdir}/${script}.d" - mkdir -p "${pdir}/DEBIAN" cat <<-EOF > "${pdir}/DEBIAN/${script}" #!/bin/sh -- cgit v1.2.3 From ac2c30f98f28a6606af89ce44bff77af5d558fe8 Mon Sep 17 00:00:00 2001 From: Johannes Schauer Marin Rodrigues <josch@mister-muffin.de> Date: Tue, 3 Dec 2024 17:17:35 +0100 Subject: kbuild: deb-pkg: allow hooks also in /usr/share/kernel By passing an additional directory to run-parts, allow Debian and its derivatives to ship maintainer scripts in /usr while at the same time allowing the local admin to override or disable them by placing hooks of the same name in /etc. This adds support for the mechanism described in the UAPI Configuration Files Specification for kernel hooks. The same idea is also used by udev, systemd or modprobe for their config files. https://uapi-group.org/specifications/specs/configuration_files_specification/ This functionality relies on run-parts 5.21 or later. It is the responsibility of packages installing hooks into /usr/share/kernel to also declare a Depends: debianutils (>= 5.21). KDEB_HOOKDIR can be used to change the list of directories that is searched. By default, /etc/kernel and /usr/share/kernel are hook directories. Since the list of directories in KDEB_HOOKDIR is separated by spaces, the paths must not contain the space character themselves. Signed-off-by: Johannes Schauer Marin Rodrigues <josch@mister-muffin.de> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> --- scripts/package/builddeb | 22 ++++++++++++++++------ 1 file changed, 16 insertions(+), 6 deletions(-) (limited to 'scripts') diff --git a/scripts/package/builddeb b/scripts/package/builddeb index 85fe8f56bb9b..3627ca227e5a 100755 --- a/scripts/package/builddeb +++ b/scripts/package/builddeb @@ -5,10 +5,12 @@ # # Simple script to generate a deb package for a Linux kernel. All the # complexity of what to do with a kernel after it is installed or removed -# is left to other scripts and packages: they can install scripts in the -# /etc/kernel/{pre,post}{inst,rm}.d/ directories (or an alternative location -# specified in KDEB_HOOKDIR) that will be called on package install and -# removal. +# is left to other scripts and packages. Scripts can be placed into the +# preinst, postinst, prerm and postrm directories in /etc/kernel or +# /usr/share/kernel. A different list of search directories can be given +# via KDEB_HOOKDIR. Scripts in directories earlier in the list will +# override scripts of the same name in later directories. The script will +# be called on package installation and removal. set -eu @@ -74,7 +76,7 @@ install_maint_scripts () { # kernel packages, as well as kernel packages built using make-kpkg. # make-kpkg sets $INITRD to indicate whether an initramfs is wanted, and # so do we; recent versions of dracut and initramfs-tools will obey this. - debhookdir=${KDEB_HOOKDIR:-/etc/kernel} + debhookdir=${KDEB_HOOKDIR:-/etc/kernel /usr/share/kernel} for script in postinst postrm preinst prerm; do mkdir -p "${pdir}/DEBIAN" cat <<-EOF > "${pdir}/DEBIAN/${script}" @@ -88,7 +90,15 @@ install_maint_scripts () { # Tell initramfs builder whether it's wanted export INITRD=$(if_enabled_echo CONFIG_BLK_DEV_INITRD Yes No) - test -d ${debhookdir}/${script}.d && run-parts --arg="${KERNELRELEASE}" --arg="/${installed_image_path}" ${debhookdir}/${script}.d + # run-parts will error out if one of its directory arguments does not + # exist, so filter the list of hook directories accordingly. + hookdirs= + for dir in ${debhookdir}; do + test -d "\$dir/${script}.d" || continue + hookdirs="\$hookdirs \$dir/${script}.d" + done + hookdirs="\${hookdirs# }" + test -n "\$hookdirs" && run-parts --arg="${KERNELRELEASE}" --arg="/${installed_image_path}" \$hookdirs exit 0 EOF chmod 755 "${pdir}/DEBIAN/${script}" -- cgit v1.2.3 From 5f73e7d0386d970a7d0e9de5a58d53114de85033 Mon Sep 17 00:00:00 2001 From: Masahiro Yamada <masahiroy@kernel.org> Date: Tue, 10 Dec 2024 19:06:17 +0900 Subject: kbuild: refactor cross-compiling linux-headers package Since commit 13b25489b6f8 ("kbuild: change working directory to external module directory with M="), when cross-building host programs for the linux-headers package, the "Entering directory" and "Leaving directory" messages appear multiple times, and each object path shown is relative to the working directory. This makes it difficult to track which objects are being rebuilt. In hindsight, using the external module build (M=) was not a good idea. This commit simplifies the script by leveraging the run-command target, resulting in a cleaner build log again. [Before] $ make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- bindeb-pkg [ snip ] Rebuilding host programs with aarch64-linux-gnu-gcc... make[5]: Entering directory '/home/masahiro/linux' make[6]: Entering directory '/home/masahiro/linux/debian/linux-headers-6.13.0-rc1+/usr/src/linux-headers-6.13.0-rc1+' HOSTCC scripts/kallsyms HOSTCC scripts/sorttable HOSTCC scripts/asn1_compiler make[6]: Leaving directory '/home/masahiro/linux/debian/linux-headers-6.13.0-rc1+/usr/src/linux-headers-6.13.0-rc1+' make[5]: Leaving directory '/home/masahiro/linux' make[5]: Entering directory '/home/masahiro/linux' make[6]: Entering directory '/home/masahiro/linux/debian/linux-headers-6.13.0-rc1+/usr/src/linux-headers-6.13.0-rc1+' HOSTCC scripts/basic/fixdep HOSTCC scripts/mod/modpost.o HOSTCC scripts/mod/file2alias.o HOSTCC scripts/mod/sumversion.o HOSTCC scripts/mod/symsearch.o HOSTLD scripts/mod/modpost make[6]: Leaving directory '/home/masahiro/linux/debian/linux-headers-6.13.0-rc1+/usr/src/linux-headers-6.13.0-rc1+' make[5]: Leaving directory '/home/masahiro/linux' [After] $ make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- bindeb-pkg [ snip ] HOSTCC debian/linux-headers-6.13.0-rc1+/usr/src/linux-headers-6.13.0-rc1+/scripts/basic/fixdep HOSTCC debian/linux-headers-6.13.0-rc1+/usr/src/linux-headers-6.13.0-rc1+/scripts/kallsyms HOSTCC debian/linux-headers-6.13.0-rc1+/usr/src/linux-headers-6.13.0-rc1+/scripts/sorttable HOSTCC debian/linux-headers-6.13.0-rc1+/usr/src/linux-headers-6.13.0-rc1+/scripts/asn1_compiler HOSTCC debian/linux-headers-6.13.0-rc1+/usr/src/linux-headers-6.13.0-rc1+/scripts/mod/modpost.o HOSTCC debian/linux-headers-6.13.0-rc1+/usr/src/linux-headers-6.13.0-rc1+/scripts/mod/file2alias.o HOSTCC debian/linux-headers-6.13.0-rc1+/usr/src/linux-headers-6.13.0-rc1+/scripts/mod/sumversion.o HOSTCC debian/linux-headers-6.13.0-rc1+/usr/src/linux-headers-6.13.0-rc1+/scripts/mod/symsearch.o HOSTLD debian/linux-headers-6.13.0-rc1+/usr/src/linux-headers-6.13.0-rc1+/scripts/mod/modpost Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> --- scripts/package/install-extmod-build | 33 +++++++++------------------------ 1 file changed, 9 insertions(+), 24 deletions(-) (limited to 'scripts') diff --git a/scripts/package/install-extmod-build b/scripts/package/install-extmod-build index d3c5b104c063..bb6e23c1174e 100755 --- a/scripts/package/install-extmod-build +++ b/scripts/package/install-extmod-build @@ -49,17 +49,10 @@ mkdir -p "${destdir}" # This caters to host programs that participate in Kbuild. objtool and # resolve_btfids are out of scope. if [ "${CC}" != "${HOSTCC}" ]; then - echo "Rebuilding host programs with ${CC}..." - - # This leverages external module building. - # - Clear sub_make_done to allow the top-level Makefile to redo sub-make. - # - Filter out --no-print-directory to print "Entering directory" logs - # when Make changes the working directory. - unset sub_make_done - MAKEFLAGS=$(echo "${MAKEFLAGS}" | sed s/--no-print-directory//) - - cat <<-'EOF' > "${destdir}/Kbuild" - subdir-y := scripts + cat "${destdir}/scripts/Makefile" - <<-'EOF' > "${destdir}/scripts/Kbuild" + subdir-y += basic + hostprogs-always-y += mod/modpost + mod/modpost-objs := $(addprefix mod/, modpost.o file2alias.o sumversion.o symsearch.o) EOF # HOSTCXX is not overridden. The C++ compiler is used to build: @@ -67,20 +60,12 @@ if [ "${CC}" != "${HOSTCC}" ]; then # - GCC plugins, which will not work on the installed system even after # being rebuilt. # - # Use the single-target build to avoid the modpost invocation, which - # would overwrite Module.symvers. - "${MAKE}" HOSTCC="${CC}" KBUILD_OUTPUT=. KBUILD_EXTMOD="${destdir}" scripts/ - - cat <<-'EOF' > "${destdir}/scripts/Kbuild" - subdir-y := basic - hostprogs-always-y := mod/modpost - mod/modpost-objs := $(addprefix mod/, modpost.o file2alias.o sumversion.o symsearch.o) - EOF - - # Run once again to rebuild scripts/basic/ and scripts/mod/modpost. - "${MAKE}" HOSTCC="${CC}" KBUILD_OUTPUT=. KBUILD_EXTMOD="${destdir}" scripts/ + # Clear VPATH and srcroot because the source files reside in the output + # directory. + # shellcheck disable=SC2016 # $(MAKE), $(CC), and $(build) will be expanded by Make + "${MAKE}" run-command KBUILD_RUN_COMMAND='+$(MAKE) HOSTCC=$(CC) VPATH= srcroot=. $(build)='"${destdir}"/scripts - rm -f "${destdir}/Kbuild" "${destdir}/scripts/Kbuild" + rm -f "${destdir}/scripts/Kbuild" fi find "${destdir}" \( -name '.*.cmd' -o -name '*.o' \) -delete -- cgit v1.2.3 From 1f937a4bcb0472015818f30f4d3c5546d3f09933 Mon Sep 17 00:00:00 2001 From: Masahiro Yamada <masahiroy@kernel.org> Date: Tue, 10 Dec 2024 19:24:41 +0900 Subject: kbuild: suppress stdout from merge_config for silent builds merge_config does not respect the Make's -s (--silent) option. Let's sink the stdout from merge_config for silent builds. This commit does not cater to the direct invocation of merge_config.sh (e.g. arch/mips/Makefile). Reported-by: Leon Romanovsky <leon@kernel.org> Closes: https://lore.kernel.org/all/e534ce33b0e1060eb85ece8429810f087b034c88.1733234008.git.leonro@nvidia.com/ Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Tested-by: Leon Romanovsky <leon@kernel.org> Reviewed-by: Nathan Chancellor <nathan@kernel.org> --- scripts/Makefile.defconf | 13 +++++++------ scripts/kconfig/Makefile | 4 +++- 2 files changed, 10 insertions(+), 7 deletions(-) (limited to 'scripts') diff --git a/scripts/Makefile.defconf b/scripts/Makefile.defconf index 226ea3df3b4b..a44307f08e9d 100644 --- a/scripts/Makefile.defconf +++ b/scripts/Makefile.defconf @@ -1,6 +1,11 @@ # SPDX-License-Identifier: GPL-2.0 # Configuration heplers +cmd_merge_fragments = \ + $(srctree)/scripts/kconfig/merge_config.sh \ + $4 -m -O $(objtree) $(srctree)/arch/$(SRCARCH)/configs/$2 \ + $(foreach config,$3,$(srctree)/arch/$(SRCARCH)/configs/$(config).config) + # Creates 'merged defconfigs' # --------------------------------------------------------------------------- # Usage: @@ -8,9 +13,7 @@ # # Input config fragments without '.config' suffix define merge_into_defconfig - $(Q)$(CONFIG_SHELL) $(srctree)/scripts/kconfig/merge_config.sh \ - -m -O $(objtree) $(srctree)/arch/$(SRCARCH)/configs/$(1) \ - $(foreach config,$(2),$(srctree)/arch/$(SRCARCH)/configs/$(config).config) + $(call cmd,merge_fragments,$1,$2) +$(Q)$(MAKE) -f $(srctree)/Makefile olddefconfig endef @@ -22,8 +25,6 @@ endef # # Input config fragments without '.config' suffix define merge_into_defconfig_override - $(Q)$(CONFIG_SHELL) $(srctree)/scripts/kconfig/merge_config.sh \ - -Q -m -O $(objtree) $(srctree)/arch/$(SRCARCH)/configs/$(1) \ - $(foreach config,$(2),$(srctree)/arch/$(SRCARCH)/configs/$(config).config) + $(call cmd,merge_fragments,$1,$2,-Q) +$(Q)$(MAKE) -f $(srctree)/Makefile olddefconfig endef diff --git a/scripts/kconfig/Makefile b/scripts/kconfig/Makefile index a0a0be38cbdc..fb50bd4f4103 100644 --- a/scripts/kconfig/Makefile +++ b/scripts/kconfig/Makefile @@ -105,9 +105,11 @@ configfiles = $(wildcard $(srctree)/kernel/configs/$(1) $(srctree)/arch/$(SRCARC all-config-fragments = $(call configfiles,*.config) config-fragments = $(call configfiles,$@) +cmd_merge_fragments = $(srctree)/scripts/kconfig/merge_config.sh -m $(KCONFIG_CONFIG) $(config-fragments) + %.config: $(obj)/conf $(if $(config-fragments),, $(error $@ fragment does not exists on this architecture)) - $(Q)$(CONFIG_SHELL) $(srctree)/scripts/kconfig/merge_config.sh -m $(KCONFIG_CONFIG) $(config-fragments) + $(call cmd,merge_fragments) $(Q)$(MAKE) -f $(srctree)/Makefile olddefconfig PHONY += tinyconfig -- cgit v1.2.3 From 82a1978d0fdc28e561bc4d98ea155dd322f33c19 Mon Sep 17 00:00:00 2001 From: Masahiro Yamada <masahiroy@kernel.org> Date: Wed, 18 Dec 2024 19:37:08 +0900 Subject: kheaders: use 'tar' instead of 'cpio' for copying files The 'cpio' command is used solely for copying header files to the temporary directory. However, there is no strong reason to use 'cpio' for this purpose. For example, scripts/package/install-extmod-build uses the 'tar' command to copy files. This commit replaces the use of 'cpio' with 'tar' because 'tar' is already used in this script to generate kheaders_data.tar.xz anyway. Performance-wide, there is no significant difference between 'cpio' and 'tar'. [Before] $ rm -fr kheaders; mkdir kheaders $ time sh -c ' for f in include arch/x86/include do find "$f" -name "*.h" done | cpio --quiet -pd kheaders ' real 0m0.148s user 0m0.021s sys 0m0.140s [After] $ rm -fr kheaders; mkdir kheaders $ time sh -c ' for f in include arch/x86/include do find "$f" -name "*.h" done | tar -c -f - -T - | tar -xf - -C kheaders ' real 0m0.098s user 0m0.024s sys 0m0.131s Revert commit 69ef0920bdd3 ("Docs: Add cpio requirement to changes.rst") because 'cpio' is not used anywhere else during the kernel build. Please note that the built-in initramfs is created by the in-tree tool, usr/gen_init_cpio, so it does not rely on the external 'cpio' command at all. Remove 'cpio' from the package build dependencies as well. Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> --- Documentation/process/changes.rst | 6 ------ kernel/gen_kheaders.sh | 13 ++----------- scripts/package/PKGBUILD | 1 - scripts/package/mkdebian | 2 +- 4 files changed, 3 insertions(+), 19 deletions(-) (limited to 'scripts') diff --git a/Documentation/process/changes.rst b/Documentation/process/changes.rst index 82b5e378eebf..a0beca805362 100644 --- a/Documentation/process/changes.rst +++ b/Documentation/process/changes.rst @@ -59,7 +59,6 @@ iptables 1.4.2 iptables -V openssl & libcrypto 1.0.0 openssl version bc 1.06.95 bc --version Sphinx\ [#f1]_ 2.4.4 sphinx-build --version -cpio any cpio --version GNU tar 1.28 tar --version gtags (optional) 6.6.5 gtags --version mkimage (optional) 2017.01 mkimage --version @@ -536,11 +535,6 @@ mcelog - <https://www.mcelog.org/> -cpio ----- - -- <https://www.gnu.org/software/cpio/> - Networking ********** diff --git a/kernel/gen_kheaders.sh b/kernel/gen_kheaders.sh index ddfd1177567f..55f493d83b8f 100755 --- a/kernel/gen_kheaders.sh +++ b/kernel/gen_kheaders.sh @@ -14,13 +14,6 @@ include/ arch/$SRCARCH/include/ " -if ! command -v cpio >/dev/null; then - echo >&2 "***" - echo >&2 "*** 'cpio' could not be found." - echo >&2 "***" - exit 1 -fi - # Support incremental builds by skipping archive generation # if timestamps of files being archived are not changed. @@ -73,15 +66,13 @@ if [ "$building_out_of_srctree" ]; then cd $srctree for f in $dir_list do find "$f" -name "*.h"; - done | cpio --quiet -pd "${tmpdir}" + done | tar -c -f - -T - | tar -xf - -C "${tmpdir}" ) fi -# The second CPIO can complain if files already exist which can happen with out -# of tree builds having stale headers in srctree. Just silence CPIO for now. for f in $dir_list; do find "$f" -name "*.h"; -done | cpio --quiet -pdu "${tmpdir}" >/dev/null 2>&1 +done | tar -c -f - -T - | tar -xf - -C "${tmpdir}" # Always exclude include/generated/utsversion.h # Otherwise, the contents of the tarball may vary depending on the build steps. diff --git a/scripts/package/PKGBUILD b/scripts/package/PKGBUILD index dca706617adc..0cf3a55b05e1 100644 --- a/scripts/package/PKGBUILD +++ b/scripts/package/PKGBUILD @@ -22,7 +22,6 @@ license=(GPL-2.0-only) makedepends=( bc bison - cpio flex gettext kmod diff --git a/scripts/package/mkdebian b/scripts/package/mkdebian index b038a1380b8a..b6dd98ca860b 100755 --- a/scripts/package/mkdebian +++ b/scripts/package/mkdebian @@ -205,7 +205,7 @@ Priority: optional Maintainer: $maintainer Rules-Requires-Root: no Build-Depends: debhelper-compat (= 12) -Build-Depends-Arch: bc, bison, cpio, flex, +Build-Depends-Arch: bc, bison, flex, gcc-${host_gnu} <!pkg.${sourcename}.nokernelheaders>, kmod, libelf-dev:native, libssl-dev:native, libssl-dev <!pkg.${sourcename}.nokernelheaders>, -- cgit v1.2.3 From ad2091dee019a68145610081a75fae3b90f0c44d Mon Sep 17 00:00:00 2001 From: Rolf Eike Beer <eb@emlix.com> Date: Thu, 19 Dec 2024 08:20:34 +0100 Subject: kconfig: qconf: use preferred form of QString API A QString constructed from a character literal of length 0, i.e. "", is not "null" for historical reasons. This does not matter here so use the preferred method isEmpty() instead. Also directly construct empty QString objects instead of passing in an empty character literal that has to be parsed into an empty object first. Signed-off-by: Rolf Eike Beer <eb@emlix.com> Link: https://doc.qt.io/qt-6/qstring.html#distinction-between-null-and-empty-strings Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> --- scripts/kconfig/qconf.cc | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) (limited to 'scripts') diff --git a/scripts/kconfig/qconf.cc b/scripts/kconfig/qconf.cc index 6c92ef1e16ef..eaa465b0ccf9 100644 --- a/scripts/kconfig/qconf.cc +++ b/scripts/kconfig/qconf.cc @@ -1464,8 +1464,8 @@ void ConfigMainWindow::loadConfig(void) { QString str; - str = QFileDialog::getOpenFileName(this, "", configname); - if (str.isNull()) + str = QFileDialog::getOpenFileName(this, QString(), configname); + if (str.isEmpty()) return; if (conf_read(str.toLocal8Bit().constData())) @@ -1491,8 +1491,8 @@ void ConfigMainWindow::saveConfigAs(void) { QString str; - str = QFileDialog::getSaveFileName(this, "", configname); - if (str.isNull()) + str = QFileDialog::getSaveFileName(this, QString(), configname); + if (str.isEmpty()) return; if (conf_write(str.toLocal8Bit().constData())) { -- cgit v1.2.3 From 5963913bb57f15f198361bc7f1389c756b98f25f Mon Sep 17 00:00:00 2001 From: Masahiro Yamada <masahiroy@kernel.org> Date: Sun, 29 Dec 2024 00:45:29 +0900 Subject: modpost: zero-pad CRC values in modversion_info array I do not think the '#' flag is useful here because adding the explicit '0x' is clearer. Add the '0' flag to zero-pad the CRC values. This change gives better alignment in the generated *.mod.c files. There is no impact to the compiled modules. [Before] $ grep -A5 modversion_info fs/efivarfs/efivarfs.mod.c static const struct modversion_info ____versions[] __used __section("__versions") = { { 0x907d14d, "blocking_notifier_chain_register" }, { 0x53d3b64, "simple_inode_init_ts" }, { 0x65487097, "__x86_indirect_thunk_rax" }, { 0x122c3a7e, "_printk" }, [After] $ grep -A5 modversion_info fs/efivarfs/efivarfs.mod.c static const struct modversion_info ____versions[] __used __section("__versions") = { { 0x0907d14d, "blocking_notifier_chain_register" }, { 0x053d3b64, "simple_inode_init_ts" }, { 0x65487097, "__x86_indirect_thunk_rax" }, { 0x122c3a7e, "_printk" }, Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> --- scripts/mod/modpost.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'scripts') diff --git a/scripts/mod/modpost.c b/scripts/mod/modpost.c index 7ea59dc4926b..dc907014108b 100644 --- a/scripts/mod/modpost.c +++ b/scripts/mod/modpost.c @@ -1832,7 +1832,7 @@ static void add_versions(struct buffer *b, struct module *mod) s->name, mod->name); break; } - buf_printf(b, "\t{ %#8x, \"%s\" },\n", + buf_printf(b, "\t{ 0x%08x, \"%s\" },\n", s->crc, s->name); } -- cgit v1.2.3 From 45c9c4101d3d2fdfa00852274bbebba65fcc3cf2 Mon Sep 17 00:00:00 2001 From: Masahiro Yamada <masahiroy@kernel.org> Date: Fri, 3 Jan 2025 16:30:38 +0900 Subject: genksyms: fix memory leak when the same symbol is added from source When a symbol that is already registered is added again, __add_symbol() returns without freeing the symbol definition, making it unreachable. The following test cases demonstrate different memory leak points. [Test Case 1] Forward declaration with exactly the same definition $ cat foo.c #include <linux/export.h> void foo(void); void foo(void) {} EXPORT_SYMBOL(foo); [Test Case 2] Forward declaration with a different definition (e.g. attribute) $ cat foo.c #include <linux/export.h> void foo(void); __attribute__((__section__(".ref.text"))) void foo(void) {} EXPORT_SYMBOL(foo); [Test Case 3] Preserving an overridden symbol (compile with KBUILD_PRESERVE=1) $ cat foo.c #include <linux/export.h> void foo(void); void foo(void) { } EXPORT_SYMBOL(foo); $ cat foo.symref override foo void foo ( int ) The memory leaks in Test Case 1 and 2 have existed since the introduction of genksyms into the kernel tree. [1] The memory leak in Test Case 3 was introduced by commit 5dae9a550a74 ("genksyms: allow to ignore symbol checksum changes"). When multiple init_declarators are reduced to an init_declarator_list, the decl_spec must be duplicated. Otherwise, the following Test Case 4 would result in a double-free bug. [Test Case 4] $ cat foo.c #include <linux/export.h> extern int foo, bar; int foo, bar; EXPORT_SYMBOL(foo); In this case, 'foo' and 'bar' share the same decl_spec, 'int'. It must be unshared before being passed to add_symbol(). [1]: https://git.kernel.org/pub/scm/linux/kernel/git/history/history.git/commit/?id=46bd1da672d66ccd8a639d3c1f8a166048cca608 Fixes: 5dae9a550a74 ("genksyms: allow to ignore symbol checksum changes") Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> --- scripts/genksyms/genksyms.c | 3 +++ scripts/genksyms/parse.y | 14 ++++++++++++-- 2 files changed, 15 insertions(+), 2 deletions(-) (limited to 'scripts') diff --git a/scripts/genksyms/genksyms.c b/scripts/genksyms/genksyms.c index 07f9b8cfb233..8ca46f807b57 100644 --- a/scripts/genksyms/genksyms.c +++ b/scripts/genksyms/genksyms.c @@ -239,6 +239,7 @@ static struct symbol *__add_symbol(const char *name, enum symbol_type type, "unchanged\n"); } sym->is_declared = 1; + free_list(defn, NULL); return sym; } else if (!sym->is_declared) { if (sym->is_override && flag_preserve) { @@ -247,6 +248,7 @@ static struct symbol *__add_symbol(const char *name, enum symbol_type type, print_type_name(type, name); fprintf(stderr, " modversion change\n"); sym->is_declared = 1; + free_list(defn, NULL); return sym; } else { status = is_unknown_symbol(sym) ? @@ -254,6 +256,7 @@ static struct symbol *__add_symbol(const char *name, enum symbol_type type, } } else { error_with_pos("redefinition of %s", name); + free_list(defn, NULL); return sym; } break; diff --git a/scripts/genksyms/parse.y b/scripts/genksyms/parse.y index 8e9b5e69e8f0..840371d01bf4 100644 --- a/scripts/genksyms/parse.y +++ b/scripts/genksyms/parse.y @@ -152,14 +152,19 @@ simple_declaration: ; init_declarator_list_opt: - /* empty */ { $$ = NULL; } - | init_declarator_list + /* empty */ { $$ = NULL; } + | init_declarator_list { free_list(decl_spec, NULL); $$ = $1; } ; init_declarator_list: init_declarator { struct string_list *decl = *$1; *$1 = NULL; + + /* avoid sharing among multiple init_declarators */ + if (decl_spec) + decl_spec = copy_list_range(decl_spec, NULL); + add_symbol(current_name, is_typedef ? SYM_TYPEDEF : SYM_NORMAL, decl, is_extern); current_name = NULL; @@ -170,6 +175,11 @@ init_declarator_list: *$3 = NULL; free_list(*$2, NULL); *$2 = decl_spec; + + /* avoid sharing among multiple init_declarators */ + if (decl_spec) + decl_spec = copy_list_range(decl_spec, NULL); + add_symbol(current_name, is_typedef ? SYM_TYPEDEF : SYM_NORMAL, decl, is_extern); current_name = NULL; -- cgit v1.2.3 From be2fa44b5180a1f021efb40c55fdf63c249c3209 Mon Sep 17 00:00:00 2001 From: Masahiro Yamada <masahiroy@kernel.org> Date: Fri, 3 Jan 2025 16:30:39 +0900 Subject: genksyms: fix memory leak when the same symbol is read from *.symref file When a symbol that is already registered is read again from *.symref file, __add_symbol() removes the previous one from the hash table without freeing it. [Test Case] $ cat foo.c #include <linux/export.h> void foo(void); void foo(void) {} EXPORT_SYMBOL(foo); $ cat foo.symref foo void foo ( void ) foo void foo ( void ) When a symbol is removed from the hash table, it must be freed along with its ->name and ->defn members. However, sym->name cannot be freed because it is sometimes shared with node->string, but not always. If sym->name and node->string share the same memory, free(sym->name) could lead to a double-free bug. To resolve this issue, always assign a strdup'ed string to sym->name. Fixes: 64e6c1e12372 ("genksyms: track symbol checksum changes") Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> --- scripts/genksyms/genksyms.c | 8 ++++++-- scripts/genksyms/genksyms.h | 2 +- scripts/genksyms/parse.y | 4 ++-- 3 files changed, 9 insertions(+), 5 deletions(-) (limited to 'scripts') diff --git a/scripts/genksyms/genksyms.c b/scripts/genksyms/genksyms.c index 8ca46f807b57..c5e8e0e0f949 100644 --- a/scripts/genksyms/genksyms.c +++ b/scripts/genksyms/genksyms.c @@ -272,11 +272,15 @@ static struct symbol *__add_symbol(const char *name, enum symbol_type type, break; } } + + free_list(sym->defn, NULL); + free(sym->name); + free(sym); --nsyms; } sym = xmalloc(sizeof(*sym)); - sym->name = name; + sym->name = xstrdup(name); sym->type = type; sym->defn = defn; sym->expansion_trail = NULL; @@ -483,7 +487,7 @@ static void read_reference(FILE *f) defn = def; def = read_node(f); } - subsym = add_reference_symbol(xstrdup(sym->string), sym->tag, + subsym = add_reference_symbol(sym->string, sym->tag, defn, is_extern); subsym->is_override = is_override; free_node(sym); diff --git a/scripts/genksyms/genksyms.h b/scripts/genksyms/genksyms.h index 21ed2ec2d98c..5621533dcb8e 100644 --- a/scripts/genksyms/genksyms.h +++ b/scripts/genksyms/genksyms.h @@ -32,7 +32,7 @@ struct string_list { struct symbol { struct symbol *hash_next; - const char *name; + char *name; enum symbol_type type; struct string_list *defn; struct symbol *expansion_trail; diff --git a/scripts/genksyms/parse.y b/scripts/genksyms/parse.y index 840371d01bf4..689cb6bb40b6 100644 --- a/scripts/genksyms/parse.y +++ b/scripts/genksyms/parse.y @@ -482,12 +482,12 @@ enumerator_list: enumerator: IDENT { - const char *name = strdup((*$1)->string); + const char *name = (*$1)->string; add_symbol(name, SYM_ENUM_CONST, NULL, 0); } | IDENT '=' EXPRESSION_PHRASE { - const char *name = strdup((*$1)->string); + const char *name = (*$1)->string; struct string_list *expr = copy_list_range(*$3, *$2); add_symbol(name, SYM_ENUM_CONST, expr, 0); } -- cgit v1.2.3 From f034d186bf9e2857079815e5490e2810a1a287a6 Mon Sep 17 00:00:00 2001 From: Masahiro Yamada <masahiroy@kernel.org> Date: Fri, 3 Jan 2025 16:30:40 +0900 Subject: genksyms: reduce the indentation in the for-loop in __add_symbol() To improve readability, reduce the indentation as follows: - Use 'continue' earlier when the symbol does not match - flip !sym->is_declared to flatten the if-else chain No functional changes are intended. Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> --- scripts/genksyms/genksyms.c | 63 +++++++++++++++++++++------------------------ 1 file changed, 30 insertions(+), 33 deletions(-) (limited to 'scripts') diff --git a/scripts/genksyms/genksyms.c b/scripts/genksyms/genksyms.c index c5e8e0e0f949..5a90acd693f4 100644 --- a/scripts/genksyms/genksyms.c +++ b/scripts/genksyms/genksyms.c @@ -226,41 +226,38 @@ static struct symbol *__add_symbol(const char *name, enum symbol_type type, h = crc32(name) % HASH_BUCKETS; for (sym = symtab[h]; sym; sym = sym->hash_next) { - if (map_to_ns(sym->type) == map_to_ns(type) && - strcmp(name, sym->name) == 0) { - if (is_reference) - /* fall through */ ; - else if (sym->type == type && - equal_list(sym->defn, defn)) { - if (!sym->is_declared && sym->is_override) { - print_location(); - print_type_name(type, name); - fprintf(stderr, " modversion is " - "unchanged\n"); - } - sym->is_declared = 1; - free_list(defn, NULL); - return sym; - } else if (!sym->is_declared) { - if (sym->is_override && flag_preserve) { - print_location(); - fprintf(stderr, "ignoring "); - print_type_name(type, name); - fprintf(stderr, " modversion change\n"); - sym->is_declared = 1; - free_list(defn, NULL); - return sym; - } else { - status = is_unknown_symbol(sym) ? - STATUS_DEFINED : STATUS_MODIFIED; - } - } else { - error_with_pos("redefinition of %s", name); - free_list(defn, NULL); - return sym; + if (map_to_ns(sym->type) != map_to_ns(type) || + strcmp(name, sym->name)) + continue; + + if (is_reference) { + /* fall through */ ; + } else if (sym->type == type && equal_list(sym->defn, defn)) { + if (!sym->is_declared && sym->is_override) { + print_location(); + print_type_name(type, name); + fprintf(stderr, " modversion is unchanged\n"); } - break; + sym->is_declared = 1; + free_list(defn, NULL); + return sym; + } else if (sym->is_declared) { + error_with_pos("redefinition of %s", name); + free_list(defn, NULL); + return sym; + } else if (sym->is_override && flag_preserve) { + print_location(); + fprintf(stderr, "ignoring "); + print_type_name(type, name); + fprintf(stderr, " modversion change\n"); + sym->is_declared = 1; + free_list(defn, NULL); + return sym; + } else { + status = is_unknown_symbol(sym) ? + STATUS_DEFINED : STATUS_MODIFIED; } + break; } if (sym) { -- cgit v1.2.3 From 2480f53f21b21eb24a33815d4623f54fdb30cf27 Mon Sep 17 00:00:00 2001 From: Masahiro Yamada <masahiroy@kernel.org> Date: Fri, 3 Jan 2025 16:30:41 +0900 Subject: genksyms: refactor the return points in the for-loop in __add_symbol() free_list() must be called before returning from this for-loop. Swap 'break' and the combination of free_list() and 'return'. This reduces the code and minimizes the risk of introducing memory leaks in future changes. Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> --- scripts/genksyms/genksyms.c | 12 ++++-------- 1 file changed, 4 insertions(+), 8 deletions(-) (limited to 'scripts') diff --git a/scripts/genksyms/genksyms.c b/scripts/genksyms/genksyms.c index 5a90acd693f4..41d6cfce0088 100644 --- a/scripts/genksyms/genksyms.c +++ b/scripts/genksyms/genksyms.c @@ -231,7 +231,7 @@ static struct symbol *__add_symbol(const char *name, enum symbol_type type, continue; if (is_reference) { - /* fall through */ ; + break; } else if (sym->type == type && equal_list(sym->defn, defn)) { if (!sym->is_declared && sym->is_override) { print_location(); @@ -239,25 +239,21 @@ static struct symbol *__add_symbol(const char *name, enum symbol_type type, fprintf(stderr, " modversion is unchanged\n"); } sym->is_declared = 1; - free_list(defn, NULL); - return sym; } else if (sym->is_declared) { error_with_pos("redefinition of %s", name); - free_list(defn, NULL); - return sym; } else if (sym->is_override && flag_preserve) { print_location(); fprintf(stderr, "ignoring "); print_type_name(type, name); fprintf(stderr, " modversion change\n"); sym->is_declared = 1; - free_list(defn, NULL); - return sym; } else { status = is_unknown_symbol(sym) ? STATUS_DEFINED : STATUS_MODIFIED; + break; } - break; + free_list(defn, NULL); + return sym; } if (sym) { -- cgit v1.2.3 From 2759bd908f3cc8d286e1fa64ec7ee7f5d1124837 Mon Sep 17 00:00:00 2001 From: Masahiro Yamada <masahiroy@kernel.org> Date: Fri, 3 Jan 2025 16:30:42 +0900 Subject: genksyms: use generic macros for hash table implementation Use macros provided by hashtable.h Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> --- scripts/genksyms/genksyms.c | 32 ++++++++++++-------------------- scripts/genksyms/genksyms.h | 4 +++- 2 files changed, 15 insertions(+), 21 deletions(-) (limited to 'scripts') diff --git a/scripts/genksyms/genksyms.c b/scripts/genksyms/genksyms.c index 41d6cfce0088..e2cd3dcb469f 100644 --- a/scripts/genksyms/genksyms.c +++ b/scripts/genksyms/genksyms.c @@ -18,12 +18,12 @@ #include <stdarg.h> #include <getopt.h> +#include <hashtable.h> + #include "genksyms.h" /*----------------------------------------------------------------------*/ -#define HASH_BUCKETS 4096 - -static struct symbol *symtab[HASH_BUCKETS]; +static HASHTABLE_DEFINE(symbol_hashtable, 1U << 12); static FILE *debugfile; int cur_line = 1; @@ -151,14 +151,14 @@ static enum symbol_type map_to_ns(enum symbol_type t) struct symbol *find_symbol(const char *name, enum symbol_type ns, int exact) { - unsigned long h = crc32(name) % HASH_BUCKETS; struct symbol *sym; - for (sym = symtab[h]; sym; sym = sym->hash_next) + hash_for_each_possible(symbol_hashtable, sym, hnode, crc32(name)) { if (map_to_ns(sym->type) == map_to_ns(ns) && strcmp(name, sym->name) == 0 && sym->is_declared) break; + } if (exact && sym && sym->type != ns) return NULL; @@ -224,8 +224,8 @@ static struct symbol *__add_symbol(const char *name, enum symbol_type type, return NULL; } - h = crc32(name) % HASH_BUCKETS; - for (sym = symtab[h]; sym; sym = sym->hash_next) { + h = crc32(name); + hash_for_each_possible(symbol_hashtable, sym, hnode, h) { if (map_to_ns(sym->type) != map_to_ns(type) || strcmp(name, sym->name)) continue; @@ -257,14 +257,7 @@ static struct symbol *__add_symbol(const char *name, enum symbol_type type, } if (sym) { - struct symbol **psym; - - for (psym = &symtab[h]; *psym; psym = &(*psym)->hash_next) { - if (*psym == sym) { - *psym = sym->hash_next; - break; - } - } + hash_del(&sym->hnode); free_list(sym->defn, NULL); free(sym->name); @@ -280,8 +273,7 @@ static struct symbol *__add_symbol(const char *name, enum symbol_type type, sym->visited = NULL; sym->is_extern = is_extern; - sym->hash_next = symtab[h]; - symtab[h] = sym; + hash_add(symbol_hashtable, &sym->hnode, h); sym->is_declared = !is_reference; sym->status = status; @@ -832,9 +824,9 @@ int main(int argc, char **argv) } if (flag_debug) { - fprintf(debugfile, "Hash table occupancy %d/%d = %g\n", - nsyms, HASH_BUCKETS, - (double)nsyms / (double)HASH_BUCKETS); + fprintf(debugfile, "Hash table occupancy %d/%zd = %g\n", + nsyms, HASH_SIZE(symbol_hashtable), + (double)nsyms / HASH_SIZE(symbol_hashtable)); } if (dumpfile) diff --git a/scripts/genksyms/genksyms.h b/scripts/genksyms/genksyms.h index 5621533dcb8e..8c45ada59ece 100644 --- a/scripts/genksyms/genksyms.h +++ b/scripts/genksyms/genksyms.h @@ -14,6 +14,8 @@ #include <stdio.h> +#include <list_types.h> + enum symbol_type { SYM_NORMAL, SYM_TYPEDEF, SYM_ENUM, SYM_STRUCT, SYM_UNION, SYM_ENUM_CONST @@ -31,7 +33,7 @@ struct string_list { }; struct symbol { - struct symbol *hash_next; + struct hlist_node hnode; char *name; enum symbol_type type; struct string_list *defn; -- cgit v1.2.3 From a56fece7f302ff1eb49535e66bdd5d03ced0ca20 Mon Sep 17 00:00:00 2001 From: Masahiro Yamada <masahiroy@kernel.org> Date: Fri, 3 Jan 2025 16:30:43 +0900 Subject: genksyms: use uint32_t instead of unsigned long for calculating CRC Currently, 'unsigned long' is used for intermediate variables when calculating CRCs. The size of 'long' differs depending on the architecture: it is 32 bits on 32-bit architectures and 64 bits on 64-bit architectures. The CRC values generated by genksyms represent the compatibility of exported symbols. Therefore, reproducibility is important. In other words, we need to ensure that the output is the same when the kernel source is identical, regardless of whether genksyms is running on a 32-bit or 64-bit build machine. Fortunately, the output from genksyms is not affected by the build machine's architecture because only the lower 32 bits of the 'unsigned long' variables are used. To make it even clearer that the CRC calculation is independent of the build machine's architecture, this commit explicitly uses the fixed-width type, uint32_t. Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> --- scripts/genksyms/genksyms.c | 15 ++++++++------- 1 file changed, 8 insertions(+), 7 deletions(-) (limited to 'scripts') diff --git a/scripts/genksyms/genksyms.c b/scripts/genksyms/genksyms.c index e2cd3dcb469f..8b0d7ac73dbb 100644 --- a/scripts/genksyms/genksyms.c +++ b/scripts/genksyms/genksyms.c @@ -12,6 +12,7 @@ #include <stdio.h> #include <string.h> +#include <stdint.h> #include <stdlib.h> #include <unistd.h> #include <assert.h> @@ -60,7 +61,7 @@ static void print_type_name(enum symbol_type type, const char *name); /*----------------------------------------------------------------------*/ -static const unsigned int crctab32[] = { +static const uint32_t crctab32[] = { 0x00000000U, 0x77073096U, 0xee0e612cU, 0x990951baU, 0x076dc419U, 0x706af48fU, 0xe963a535U, 0x9e6495a3U, 0x0edb8832U, 0x79dcb8a4U, 0xe0d5e91eU, 0x97d2d988U, 0x09b64c2bU, 0x7eb17cbdU, 0xe7b82d07U, @@ -115,19 +116,19 @@ static const unsigned int crctab32[] = { 0x2d02ef8dU }; -static unsigned long partial_crc32_one(unsigned char c, unsigned long crc) +static uint32_t partial_crc32_one(uint8_t c, uint32_t crc) { return crctab32[(crc ^ c) & 0xff] ^ (crc >> 8); } -static unsigned long partial_crc32(const char *s, unsigned long crc) +static uint32_t partial_crc32(const char *s, uint32_t crc) { while (*s) crc = partial_crc32_one(*s++, crc); return crc; } -static unsigned long crc32(const char *s) +static uint32_t crc32(const char *s) { return partial_crc32(s, 0xffffffff) ^ 0xffffffff; } @@ -517,7 +518,7 @@ static void print_list(FILE * f, struct string_list *list) } } -static unsigned long expand_and_crc_sym(struct symbol *sym, unsigned long crc) +static uint32_t expand_and_crc_sym(struct symbol *sym, uint32_t crc) { struct string_list *list = sym->defn; struct string_list **e, **b; @@ -624,7 +625,7 @@ static unsigned long expand_and_crc_sym(struct symbol *sym, unsigned long crc) void export_symbol(const char *name) { struct symbol *sym; - unsigned long crc; + uint32_t crc; int has_changed = 0; sym = find_symbol(name, SYM_NORMAL, 0); @@ -672,7 +673,7 @@ void export_symbol(const char *name) if (flag_dump_defs) fputs(">\n", debugfile); - printf("#SYMVER %s 0x%08lx\n", name, crc); + printf("#SYMVER %s 0x%08lx\n", name, (unsigned long)crc); } /*----------------------------------------------------------------------*/ -- cgit v1.2.3 From f28568841ae0a0dd48dfc5400aaebedf10a54d10 Mon Sep 17 00:00:00 2001 From: Sami Tolvanen <samitolvanen@google.com> Date: Fri, 3 Jan 2025 20:45:23 +0000 Subject: tools: Add gendwarfksyms Add a basic DWARF parser, which uses libdw to traverse the debugging information in an object file and looks for functions and variables. In follow-up patches, this will be expanded to produce symbol versions for CONFIG_MODVERSIONS from DWARF. Signed-off-by: Sami Tolvanen <samitolvanen@google.com> Reviewed-by: Petr Pavlu <petr.pavlu@suse.com> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> --- MAINTAINERS | 7 ++ kernel/module/Kconfig | 8 ++ scripts/Makefile | 1 + scripts/gendwarfksyms/.gitignore | 2 + scripts/gendwarfksyms/Makefile | 8 ++ scripts/gendwarfksyms/dwarf.c | 166 ++++++++++++++++++++++++++++++++++ scripts/gendwarfksyms/gendwarfksyms.c | 128 ++++++++++++++++++++++++++ scripts/gendwarfksyms/gendwarfksyms.h | 95 +++++++++++++++++++ scripts/gendwarfksyms/symbols.c | 98 ++++++++++++++++++++ 9 files changed, 513 insertions(+) create mode 100644 scripts/gendwarfksyms/.gitignore create mode 100644 scripts/gendwarfksyms/Makefile create mode 100644 scripts/gendwarfksyms/dwarf.c create mode 100644 scripts/gendwarfksyms/gendwarfksyms.c create mode 100644 scripts/gendwarfksyms/gendwarfksyms.h create mode 100644 scripts/gendwarfksyms/symbols.c (limited to 'scripts') diff --git a/MAINTAINERS b/MAINTAINERS index 30cbc3d44cd5..1ec4753b1457 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -9550,6 +9550,13 @@ W: https://linuxtv.org T: git git://linuxtv.org/media.git F: drivers/media/radio/radio-gemtek* +GENDWARFKSYMS +M: Sami Tolvanen <samitolvanen@google.com> +L: linux-modules@vger.kernel.org +L: linux-kbuild@vger.kernel.org +S: Maintained +F: scripts/gendwarfksyms/ + GENERIC ARCHITECTURE TOPOLOGY M: Sudeep Holla <sudeep.holla@arm.com> L: linux-kernel@vger.kernel.org diff --git a/kernel/module/Kconfig b/kernel/module/Kconfig index 7b329057997a..4637f063d0fc 100644 --- a/kernel/module/Kconfig +++ b/kernel/module/Kconfig @@ -169,6 +169,14 @@ config MODVERSIONS make them incompatible with the kernel you are running. If unsure, say N. +config GENDWARFKSYMS + bool "gendwarfksyms (from debugging information)" + depends on DEBUG_INFO + # Requires full debugging information, split DWARF not supported. + depends on !DEBUG_INFO_REDUCED && !DEBUG_INFO_SPLIT + # Requires ELF object files. + depends on !LTO + config ASM_MODVERSIONS bool default HAVE_ASM_MODVERSIONS && MODVERSIONS diff --git a/scripts/Makefile b/scripts/Makefile index 6bcda4b9d054..d7fec46d38c0 100644 --- a/scripts/Makefile +++ b/scripts/Makefile @@ -54,6 +54,7 @@ targets += module.lds subdir-$(CONFIG_GCC_PLUGINS) += gcc-plugins subdir-$(CONFIG_MODVERSIONS) += genksyms +subdir-$(CONFIG_GENDWARFKSYMS) += gendwarfksyms subdir-$(CONFIG_SECURITY_SELINUX) += selinux subdir-$(CONFIG_SECURITY_IPE) += ipe diff --git a/scripts/gendwarfksyms/.gitignore b/scripts/gendwarfksyms/.gitignore new file mode 100644 index 000000000000..0927f8d3cd96 --- /dev/null +++ b/scripts/gendwarfksyms/.gitignore @@ -0,0 +1,2 @@ +# SPDX-License-Identifier: GPL-2.0 +/gendwarfksyms diff --git a/scripts/gendwarfksyms/Makefile b/scripts/gendwarfksyms/Makefile new file mode 100644 index 000000000000..9f8fec4fd39b --- /dev/null +++ b/scripts/gendwarfksyms/Makefile @@ -0,0 +1,8 @@ +# SPDX-License-Identifier: GPL-2.0 +hostprogs-always-y += gendwarfksyms + +gendwarfksyms-objs += gendwarfksyms.o +gendwarfksyms-objs += dwarf.o +gendwarfksyms-objs += symbols.o + +HOSTLDLIBS_gendwarfksyms := -ldw -lelf diff --git a/scripts/gendwarfksyms/dwarf.c b/scripts/gendwarfksyms/dwarf.c new file mode 100644 index 000000000000..81df3e2ad3ae --- /dev/null +++ b/scripts/gendwarfksyms/dwarf.c @@ -0,0 +1,166 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Copyright (C) 2024 Google LLC + */ + +#include "gendwarfksyms.h" + +static bool get_ref_die_attr(Dwarf_Die *die, unsigned int id, Dwarf_Die *value) +{ + Dwarf_Attribute da; + + /* dwarf_formref_die returns a pointer instead of an error value. */ + return dwarf_attr(die, id, &da) && dwarf_formref_die(&da, value); +} + +#define DEFINE_GET_STRING_ATTR(attr) \ + static const char *get_##attr##_attr(Dwarf_Die *die) \ + { \ + Dwarf_Attribute da; \ + if (dwarf_attr(die, DW_AT_##attr, &da)) \ + return dwarf_formstring(&da); \ + return NULL; \ + } + +DEFINE_GET_STRING_ATTR(name) +DEFINE_GET_STRING_ATTR(linkage_name) + +static const char *get_symbol_name(Dwarf_Die *die) +{ + const char *name; + + /* rustc uses DW_AT_linkage_name for exported symbols */ + name = get_linkage_name_attr(die); + if (!name) + name = get_name_attr(die); + + return name; +} + +static bool match_export_symbol(struct state *state, Dwarf_Die *die) +{ + Dwarf_Die *source = die; + Dwarf_Die origin; + + /* If the DIE has an abstract origin, use it for type information. */ + if (get_ref_die_attr(die, DW_AT_abstract_origin, &origin)) + source = &origin; + + state->sym = symbol_get(get_symbol_name(die)); + + /* Look up using the origin name if there are no matches. */ + if (!state->sym && source != die) + state->sym = symbol_get(get_symbol_name(source)); + + state->die = *source; + return !!state->sym; +} + +/* + * Type string processing + */ +static void process(const char *s) +{ + s = s ?: "<null>"; + + if (dump_dies) + fputs(s, stderr); +} + +bool match_all(Dwarf_Die *die) +{ + return true; +} + +int process_die_container(struct state *state, Dwarf_Die *die, + die_callback_t func, die_match_callback_t match) +{ + Dwarf_Die current; + int res; + + res = checkp(dwarf_child(die, ¤t)); + while (!res) { + if (match(¤t)) { + /* <0 = error, 0 = continue, >0 = stop */ + res = checkp(func(state, ¤t)); + if (res) + return res; + } + + res = checkp(dwarf_siblingof(¤t, ¤t)); + } + + return 0; +} + +/* + * Exported symbol processing + */ +static void process_symbol(struct state *state, Dwarf_Die *die, + die_callback_t process_func) +{ + debug("%s", state->sym->name); + check(process_func(state, die)); + if (dump_dies) + fputs("\n", stderr); +} + +static int __process_subprogram(struct state *state, Dwarf_Die *die) +{ + process("subprogram"); + return 0; +} + +static void process_subprogram(struct state *state, Dwarf_Die *die) +{ + process_symbol(state, die, __process_subprogram); +} + +static int __process_variable(struct state *state, Dwarf_Die *die) +{ + process("variable "); + return 0; +} + +static void process_variable(struct state *state, Dwarf_Die *die) +{ + process_symbol(state, die, __process_variable); +} + +static int process_exported_symbols(struct state *unused, Dwarf_Die *die) +{ + int tag = dwarf_tag(die); + + switch (tag) { + /* Possible containers of exported symbols */ + case DW_TAG_namespace: + case DW_TAG_class_type: + case DW_TAG_structure_type: + return check(process_die_container( + NULL, die, process_exported_symbols, match_all)); + + /* Possible exported symbols */ + case DW_TAG_subprogram: + case DW_TAG_variable: { + struct state state; + + if (!match_export_symbol(&state, die)) + return 0; + + if (tag == DW_TAG_subprogram) + process_subprogram(&state, &state.die); + else + process_variable(&state, &state.die); + + return 0; + } + default: + return 0; + } +} + +void process_cu(Dwarf_Die *cudie) +{ + check(process_die_container(NULL, cudie, process_exported_symbols, + match_all)); +} diff --git a/scripts/gendwarfksyms/gendwarfksyms.c b/scripts/gendwarfksyms/gendwarfksyms.c new file mode 100644 index 000000000000..a1d13353c6bc --- /dev/null +++ b/scripts/gendwarfksyms/gendwarfksyms.c @@ -0,0 +1,128 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Copyright (C) 2024 Google LLC + */ + +#include <fcntl.h> +#include <getopt.h> +#include <errno.h> +#include <stdarg.h> +#include <string.h> +#include <unistd.h> +#include "gendwarfksyms.h" + +/* + * Options + */ + +/* Print debugging information to stderr */ +int debug; +/* Dump DIE contents */ +int dump_dies; + +static void usage(void) +{ + fputs("Usage: gendwarfksyms [options] elf-object-file ... < symbol-list\n\n" + "Options:\n" + " -d, --debug Print debugging information\n" + " --dump-dies Dump DWARF DIE contents\n" + " -h, --help Print this message\n" + "\n", + stderr); +} + +static int process_module(Dwfl_Module *mod, void **userdata, const char *name, + Dwarf_Addr base, void *arg) +{ + Dwarf_Addr dwbias; + Dwarf_Die cudie; + Dwarf_CU *cu = NULL; + Dwarf *dbg; + int res; + + debug("%s", name); + dbg = dwfl_module_getdwarf(mod, &dwbias); + + do { + res = dwarf_get_units(dbg, cu, &cu, NULL, NULL, &cudie, NULL); + if (res < 0) + error("dwarf_get_units failed: no debugging information?"); + if (res == 1) + break; /* No more units */ + + process_cu(&cudie); + } while (cu); + + return DWARF_CB_OK; +} + +static const Dwfl_Callbacks callbacks = { + .section_address = dwfl_offline_section_address, + .find_debuginfo = dwfl_standard_find_debuginfo, +}; + +int main(int argc, char **argv) +{ + unsigned int n; + int opt; + + static const struct option opts[] = { + { "debug", 0, NULL, 'd' }, + { "dump-dies", 0, &dump_dies, 1 }, + { "help", 0, NULL, 'h' }, + { 0, 0, NULL, 0 } + }; + + while ((opt = getopt_long(argc, argv, "dh", opts, NULL)) != EOF) { + switch (opt) { + case 0: + break; + case 'd': + debug = 1; + break; + case 'h': + usage(); + return 0; + default: + usage(); + return 1; + } + } + + if (optind >= argc) { + usage(); + error("no input files?"); + } + + symbol_read_exports(stdin); + + for (n = optind; n < argc; n++) { + Dwfl *dwfl; + int fd; + + fd = open(argv[n], O_RDONLY); + if (fd == -1) + error("open failed for '%s': %s", argv[n], + strerror(errno)); + + dwfl = dwfl_begin(&callbacks); + if (!dwfl) + error("dwfl_begin failed for '%s': %s", argv[n], + dwarf_errmsg(-1)); + + if (!dwfl_report_offline(dwfl, argv[n], argv[n], fd)) + error("dwfl_report_offline failed for '%s': %s", + argv[n], dwarf_errmsg(-1)); + + dwfl_report_end(dwfl, NULL, NULL); + + if (dwfl_getmodules(dwfl, &process_module, NULL, 0)) + error("dwfl_getmodules failed for '%s'", argv[n]); + + dwfl_end(dwfl); + } + + symbol_free(); + + return 0; +} diff --git a/scripts/gendwarfksyms/gendwarfksyms.h b/scripts/gendwarfksyms/gendwarfksyms.h new file mode 100644 index 000000000000..5c8288c71fdd --- /dev/null +++ b/scripts/gendwarfksyms/gendwarfksyms.h @@ -0,0 +1,95 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Copyright (C) 2024 Google LLC + */ + +#include <dwarf.h> +#include <elfutils/libdw.h> +#include <elfutils/libdwfl.h> +#include <stdlib.h> +#include <stdio.h> + +#include <hash.h> +#include <hashtable.h> +#include <xalloc.h> + +#ifndef __GENDWARFKSYMS_H +#define __GENDWARFKSYMS_H + +/* + * Options -- in gendwarfksyms.c + */ +extern int debug; +extern int dump_dies; + +/* + * Output helpers + */ +#define __PREFIX "gendwarfksyms: " +#define __println(prefix, format, ...) \ + fprintf(stderr, prefix __PREFIX "%s: " format "\n", __func__, \ + ##__VA_ARGS__) + +#define debug(format, ...) \ + do { \ + if (debug) \ + __println("", format, ##__VA_ARGS__); \ + } while (0) + +#define warn(format, ...) __println("warning: ", format, ##__VA_ARGS__) +#define error(format, ...) \ + do { \ + __println("error: ", format, ##__VA_ARGS__); \ + exit(1); \ + } while (0) + +/* + * Error handling helpers + */ +#define __check(expr, test) \ + ({ \ + int __res = expr; \ + if (test) \ + error("`%s` failed: %d", #expr, __res); \ + __res; \ + }) + +/* Error == non-zero values */ +#define check(expr) __check(expr, __res) +/* Error == negative values */ +#define checkp(expr) __check(expr, __res < 0) + +/* + * symbols.c + */ + +struct symbol { + const char *name; + struct hlist_node name_hash; +}; + +typedef void (*symbol_callback_t)(struct symbol *, void *arg); + +void symbol_read_exports(FILE *file); +struct symbol *symbol_get(const char *name); +void symbol_free(void); + +/* + * dwarf.c + */ + +struct state { + struct symbol *sym; + Dwarf_Die die; +}; + +typedef int (*die_callback_t)(struct state *state, Dwarf_Die *die); +typedef bool (*die_match_callback_t)(Dwarf_Die *die); +bool match_all(Dwarf_Die *die); + +int process_die_container(struct state *state, Dwarf_Die *die, + die_callback_t func, die_match_callback_t match); + +void process_cu(Dwarf_Die *cudie); + +#endif /* __GENDWARFKSYMS_H */ diff --git a/scripts/gendwarfksyms/symbols.c b/scripts/gendwarfksyms/symbols.c new file mode 100644 index 000000000000..592eacf72694 --- /dev/null +++ b/scripts/gendwarfksyms/symbols.c @@ -0,0 +1,98 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Copyright (C) 2024 Google LLC + */ + +#include "gendwarfksyms.h" + +#define SYMBOL_HASH_BITS 12 + +/* name -> struct symbol */ +static HASHTABLE_DEFINE(symbol_names, 1 << SYMBOL_HASH_BITS); + +static unsigned int for_each(const char *name, symbol_callback_t func, + void *data) +{ + struct hlist_node *tmp; + struct symbol *match; + + if (!name || !*name) + return 0; + + hash_for_each_possible_safe(symbol_names, match, tmp, name_hash, + hash_str(name)) { + if (strcmp(match->name, name)) + continue; + + if (func) + func(match, data); + + return 1; + } + + return 0; +} + +static bool is_exported(const char *name) +{ + return for_each(name, NULL, NULL) > 0; +} + +void symbol_read_exports(FILE *file) +{ + struct symbol *sym; + char *line = NULL; + char *name = NULL; + size_t size = 0; + int nsym = 0; + + while (getline(&line, &size, file) > 0) { + if (sscanf(line, "%ms\n", &name) != 1) + error("malformed input line: %s", line); + + if (is_exported(name)) { + /* Ignore duplicates */ + free(name); + continue; + } + + sym = xcalloc(1, sizeof(struct symbol)); + sym->name = name; + + hash_add(symbol_names, &sym->name_hash, hash_str(sym->name)); + ++nsym; + + debug("%s", sym->name); + } + + free(line); + debug("%d exported symbols", nsym); +} + +static void get_symbol(struct symbol *sym, void *arg) +{ + struct symbol **res = arg; + + *res = sym; +} + +struct symbol *symbol_get(const char *name) +{ + struct symbol *sym = NULL; + + for_each(name, get_symbol, &sym); + return sym; +} + +void symbol_free(void) +{ + struct hlist_node *tmp; + struct symbol *sym; + + hash_for_each_safe(symbol_names, sym, tmp, name_hash) { + free((void *)sym->name); + free(sym); + } + + hash_init(symbol_names); +} -- cgit v1.2.3 From e982abf43749529687dd1d07fa4f495902910cf2 Mon Sep 17 00:00:00 2001 From: Sami Tolvanen <samitolvanen@google.com> Date: Fri, 3 Jan 2025 20:45:24 +0000 Subject: gendwarfksyms: Add address matching The compiler may choose not to emit type information in DWARF for all aliases, but it's possible for each alias to be exported separately. To ensure we find type information for the aliases as well, read {section, address} tuples from the symbol table and match symbols also by address. Signed-off-by: Sami Tolvanen <samitolvanen@google.com> Reviewed-by: Petr Pavlu <petr.pavlu@suse.com> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> --- scripts/gendwarfksyms/gendwarfksyms.c | 2 + scripts/gendwarfksyms/gendwarfksyms.h | 13 +++ scripts/gendwarfksyms/symbols.c | 161 ++++++++++++++++++++++++++++++++++ 3 files changed, 176 insertions(+) (limited to 'scripts') diff --git a/scripts/gendwarfksyms/gendwarfksyms.c b/scripts/gendwarfksyms/gendwarfksyms.c index a1d13353c6bc..cd8bfe973a5c 100644 --- a/scripts/gendwarfksyms/gendwarfksyms.c +++ b/scripts/gendwarfksyms/gendwarfksyms.c @@ -105,6 +105,8 @@ int main(int argc, char **argv) error("open failed for '%s': %s", argv[n], strerror(errno)); + symbol_read_symtab(fd); + dwfl = dwfl_begin(&callbacks); if (!dwfl) error("dwfl_begin failed for '%s': %s", argv[n], diff --git a/scripts/gendwarfksyms/gendwarfksyms.h b/scripts/gendwarfksyms/gendwarfksyms.h index 5c8288c71fdd..cb9fd78a58da 100644 --- a/scripts/gendwarfksyms/gendwarfksyms.h +++ b/scripts/gendwarfksyms/gendwarfksyms.h @@ -63,14 +63,27 @@ extern int dump_dies; * symbols.c */ +static inline unsigned int addr_hash(uintptr_t addr) +{ + return hash_ptr((const void *)addr); +} + +struct symbol_addr { + uint32_t section; + Elf64_Addr address; +}; + struct symbol { const char *name; + struct symbol_addr addr; + struct hlist_node addr_hash; struct hlist_node name_hash; }; typedef void (*symbol_callback_t)(struct symbol *, void *arg); void symbol_read_exports(FILE *file); +void symbol_read_symtab(int fd); struct symbol *symbol_get(const char *name); void symbol_free(void); diff --git a/scripts/gendwarfksyms/symbols.c b/scripts/gendwarfksyms/symbols.c index 592eacf72694..98febb524dd5 100644 --- a/scripts/gendwarfksyms/symbols.c +++ b/scripts/gendwarfksyms/symbols.c @@ -7,9 +7,38 @@ #define SYMBOL_HASH_BITS 12 +/* struct symbol_addr -> struct symbol */ +static HASHTABLE_DEFINE(symbol_addrs, 1 << SYMBOL_HASH_BITS); /* name -> struct symbol */ static HASHTABLE_DEFINE(symbol_names, 1 << SYMBOL_HASH_BITS); +static inline unsigned int symbol_addr_hash(const struct symbol_addr *addr) +{ + return hash_32(addr->section ^ addr_hash(addr->address)); +} + +static unsigned int __for_each_addr(struct symbol *sym, symbol_callback_t func, + void *data) +{ + struct hlist_node *tmp; + struct symbol *match = NULL; + unsigned int processed = 0; + + hash_for_each_possible_safe(symbol_addrs, match, tmp, addr_hash, + symbol_addr_hash(&sym->addr)) { + if (match == sym) + continue; /* Already processed */ + + if (match->addr.section == sym->addr.section && + match->addr.address == sym->addr.address) { + func(match, data); + ++processed; + } + } + + return processed; +} + static unsigned int for_each(const char *name, symbol_callback_t func, void *data) { @@ -24,9 +53,13 @@ static unsigned int for_each(const char *name, symbol_callback_t func, if (strcmp(match->name, name)) continue; + /* Call func for the match, and all address matches */ if (func) func(match, data); + if (match->addr.section != SHN_UNDEF) + return __for_each_addr(match, func, data) + 1; + return 1; } @@ -58,6 +91,7 @@ void symbol_read_exports(FILE *file) sym = xcalloc(1, sizeof(struct symbol)); sym->name = name; + sym->addr.section = SHN_UNDEF; hash_add(symbol_names, &sym->name_hash, hash_str(sym->name)); ++nsym; @@ -84,6 +118,132 @@ struct symbol *symbol_get(const char *name) return sym; } +typedef void (*elf_symbol_callback_t)(const char *name, GElf_Sym *sym, + Elf32_Word xndx, void *arg); + +static void elf_for_each_global(int fd, elf_symbol_callback_t func, void *arg) +{ + size_t sym_size; + GElf_Shdr shdr_mem; + GElf_Shdr *shdr; + Elf_Data *xndx_data = NULL; + Elf_Scn *scn; + Elf *elf; + + if (elf_version(EV_CURRENT) != EV_CURRENT) + error("elf_version failed: %s", elf_errmsg(-1)); + + elf = elf_begin(fd, ELF_C_READ_MMAP, NULL); + if (!elf) + error("elf_begin failed: %s", elf_errmsg(-1)); + + scn = elf_nextscn(elf, NULL); + + while (scn) { + shdr = gelf_getshdr(scn, &shdr_mem); + if (!shdr) + error("gelf_getshdr failed: %s", elf_errmsg(-1)); + + if (shdr->sh_type == SHT_SYMTAB_SHNDX) { + xndx_data = elf_getdata(scn, NULL); + if (!xndx_data) + error("elf_getdata failed: %s", elf_errmsg(-1)); + break; + } + + scn = elf_nextscn(elf, scn); + } + + sym_size = gelf_fsize(elf, ELF_T_SYM, 1, EV_CURRENT); + scn = elf_nextscn(elf, NULL); + + while (scn) { + shdr = gelf_getshdr(scn, &shdr_mem); + if (!shdr) + error("gelf_getshdr failed: %s", elf_errmsg(-1)); + + if (shdr->sh_type == SHT_SYMTAB) { + unsigned int nsyms; + unsigned int n; + Elf_Data *data = elf_getdata(scn, NULL); + + if (!data) + error("elf_getdata failed: %s", elf_errmsg(-1)); + + if (shdr->sh_entsize != sym_size) + error("expected sh_entsize (%lu) to be %zu", + shdr->sh_entsize, sym_size); + + nsyms = shdr->sh_size / shdr->sh_entsize; + + for (n = 1; n < nsyms; ++n) { + const char *name = NULL; + Elf32_Word xndx = 0; + GElf_Sym sym_mem; + GElf_Sym *sym; + + sym = gelf_getsymshndx(data, xndx_data, n, + &sym_mem, &xndx); + if (!sym) + error("gelf_getsymshndx failed: %s", + elf_errmsg(-1)); + + if (GELF_ST_BIND(sym->st_info) == STB_LOCAL) + continue; + + if (sym->st_shndx != SHN_XINDEX) + xndx = sym->st_shndx; + + name = elf_strptr(elf, shdr->sh_link, + sym->st_name); + if (!name) + error("elf_strptr failed: %s", + elf_errmsg(-1)); + + /* Skip empty symbol names */ + if (*name) + func(name, sym, xndx, arg); + } + } + + scn = elf_nextscn(elf, scn); + } + + check(elf_end(elf)); +} + +static void set_symbol_addr(struct symbol *sym, void *arg) +{ + struct symbol_addr *addr = arg; + + if (sym->addr.section == SHN_UNDEF) { + sym->addr = *addr; + hash_add(symbol_addrs, &sym->addr_hash, + symbol_addr_hash(&sym->addr)); + + debug("%s -> { %u, %lx }", sym->name, sym->addr.section, + sym->addr.address); + } else if (sym->addr.section != addr->section || + sym->addr.address != addr->address) { + warn("multiple addresses for symbol %s?", sym->name); + } +} + +static void elf_set_symbol_addr(const char *name, GElf_Sym *sym, + Elf32_Word xndx, void *arg) +{ + struct symbol_addr addr = { .section = xndx, .address = sym->st_value }; + + /* Set addresses for exported symbols */ + if (addr.section != SHN_UNDEF) + for_each(name, set_symbol_addr, &addr); +} + +void symbol_read_symtab(int fd) +{ + elf_for_each_global(fd, elf_set_symbol_addr, NULL); +} + void symbol_free(void) { struct hlist_node *tmp; @@ -94,5 +254,6 @@ void symbol_free(void) free(sym); } + hash_init(symbol_addrs); hash_init(symbol_names); } -- cgit v1.2.3 From 5b7780e86857f70249df2c4f8982cad3ba931eee Mon Sep 17 00:00:00 2001 From: Sami Tolvanen <samitolvanen@google.com> Date: Fri, 3 Jan 2025 20:45:25 +0000 Subject: gendwarfksyms: Expand base_type Start making gendwarfksyms more useful by adding support for expanding DW_TAG_base_type types and basic DWARF attributes. Example: $ echo loops_per_jiffy | \ scripts/gendwarfksyms/gendwarfksyms \ --debug --dump-dies vmlinux.o ... gendwarfksyms: process_symbol: loops_per_jiffy variable base_type unsigned long byte_size(8) encoding(7) ... Signed-off-by: Sami Tolvanen <samitolvanen@google.com> Reviewed-by: Petr Pavlu <petr.pavlu@suse.com> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> --- scripts/gendwarfksyms/dwarf.c | 160 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 160 insertions(+) (limited to 'scripts') diff --git a/scripts/gendwarfksyms/dwarf.c b/scripts/gendwarfksyms/dwarf.c index 81df3e2ad3ae..74e75b8ec891 100644 --- a/scripts/gendwarfksyms/dwarf.c +++ b/scripts/gendwarfksyms/dwarf.c @@ -3,8 +3,21 @@ * Copyright (C) 2024 Google LLC */ +#include <inttypes.h> +#include <stdarg.h> #include "gendwarfksyms.h" +#define DEFINE_GET_ATTR(attr, type) \ + static bool get_##attr##_attr(Dwarf_Die *die, unsigned int id, \ + type *value) \ + { \ + Dwarf_Attribute da; \ + return dwarf_attr(die, id, &da) && \ + !dwarf_form##attr(&da, value); \ + } + +DEFINE_GET_ATTR(udata, Dwarf_Word) + static bool get_ref_die_attr(Dwarf_Die *die, unsigned int id, Dwarf_Die *value) { Dwarf_Attribute da; @@ -67,6 +80,109 @@ static void process(const char *s) fputs(s, stderr); } +#define MAX_FMT_BUFFER_SIZE 128 + +static void process_fmt(const char *fmt, ...) +{ + char buf[MAX_FMT_BUFFER_SIZE]; + va_list args; + + va_start(args, fmt); + + if (checkp(vsnprintf(buf, sizeof(buf), fmt, args)) >= sizeof(buf)) + error("vsnprintf overflow: increase MAX_FMT_BUFFER_SIZE"); + + process(buf); + va_end(args); +} + +#define MAX_FQN_SIZE 64 + +/* Get a fully qualified name from DWARF scopes */ +static char *get_fqn(Dwarf_Die *die) +{ + const char *list[MAX_FQN_SIZE]; + Dwarf_Die *scopes = NULL; + bool has_name = false; + char *fqn = NULL; + char *p; + int count = 0; + int len = 0; + int res; + int i; + + res = checkp(dwarf_getscopes_die(die, &scopes)); + if (!res) { + list[count] = get_name_attr(die); + + if (!list[count]) + return NULL; + + len += strlen(list[count]); + count++; + + goto done; + } + + for (i = res - 1; i >= 0 && count < MAX_FQN_SIZE; i--) { + if (dwarf_tag(&scopes[i]) == DW_TAG_compile_unit) + continue; + + list[count] = get_name_attr(&scopes[i]); + + if (list[count]) { + has_name = true; + } else { + list[count] = "<anonymous>"; + has_name = false; + } + + len += strlen(list[count]); + count++; + + if (i > 0) { + list[count++] = "::"; + len += 2; + } + } + + free(scopes); + + if (count == MAX_FQN_SIZE) + warn("increase MAX_FQN_SIZE: reached the maximum"); + + /* Consider the DIE unnamed if the last scope doesn't have a name */ + if (!has_name) + return NULL; +done: + fqn = xmalloc(len + 1); + *fqn = '\0'; + + p = fqn; + for (i = 0; i < count; i++) + p = stpcpy(p, list[i]); + + return fqn; +} + +static void process_fqn(Dwarf_Die *die) +{ + process(" "); + process(get_fqn(die) ?: ""); +} + +#define DEFINE_PROCESS_UDATA_ATTRIBUTE(attribute) \ + static void process_##attribute##_attr(Dwarf_Die *die) \ + { \ + Dwarf_Word value; \ + if (get_udata_attr(die, DW_AT_##attribute, &value)) \ + process_fmt(" " #attribute "(%" PRIu64 ")", value); \ + } + +DEFINE_PROCESS_UDATA_ATTRIBUTE(alignment) +DEFINE_PROCESS_UDATA_ATTRIBUTE(byte_size) +DEFINE_PROCESS_UDATA_ATTRIBUTE(encoding) + bool match_all(Dwarf_Die *die) { return true; @@ -93,6 +209,49 @@ int process_die_container(struct state *state, Dwarf_Die *die, return 0; } +static int process_type(struct state *state, Dwarf_Die *die); + +static void process_type_attr(struct state *state, Dwarf_Die *die) +{ + Dwarf_Die type; + + if (get_ref_die_attr(die, DW_AT_type, &type)) { + check(process_type(state, &type)); + return; + } + + /* Compilers can omit DW_AT_type -- print out 'void' to clarify */ + process("base_type void"); +} + +static void process_base_type(struct state *state, Dwarf_Die *die) +{ + process("base_type"); + process_fqn(die); + process_byte_size_attr(die); + process_encoding_attr(die); + process_alignment_attr(die); +} + +#define PROCESS_TYPE(type) \ + case DW_TAG_##type##_type: \ + process_##type##_type(state, die); \ + break; + +static int process_type(struct state *state, Dwarf_Die *die) +{ + int tag = dwarf_tag(die); + + switch (tag) { + PROCESS_TYPE(base) + default: + debug("unimplemented type: %x", tag); + break; + } + + return 0; +} + /* * Exported symbol processing */ @@ -119,6 +278,7 @@ static void process_subprogram(struct state *state, Dwarf_Die *die) static int __process_variable(struct state *state, Dwarf_Die *die) { process("variable "); + process_type_attr(state, die); return 0; } -- cgit v1.2.3 From 0c1c76274e88c420779c3aea077f9812bd16edaa Mon Sep 17 00:00:00 2001 From: Sami Tolvanen <samitolvanen@google.com> Date: Fri, 3 Jan 2025 20:45:26 +0000 Subject: gendwarfksyms: Add a cache for processed DIEs Basic types in DWARF repeat frequently and traversing the DIEs using libdw is relatively slow. Add a simple hashtable based cache for the processed DIEs. Signed-off-by: Sami Tolvanen <samitolvanen@google.com> Reviewed-by: Petr Pavlu <petr.pavlu@suse.com> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> --- scripts/gendwarfksyms/Makefile | 1 + scripts/gendwarfksyms/die.c | 143 ++++++++++++++++++++++++++++++++++ scripts/gendwarfksyms/dwarf.c | 136 +++++++++++++++++++++++--------- scripts/gendwarfksyms/gendwarfksyms.c | 6 ++ scripts/gendwarfksyms/gendwarfksyms.h | 63 ++++++++++++++- 5 files changed, 308 insertions(+), 41 deletions(-) create mode 100644 scripts/gendwarfksyms/die.c (limited to 'scripts') diff --git a/scripts/gendwarfksyms/Makefile b/scripts/gendwarfksyms/Makefile index 9f8fec4fd39b..c0d4ce50fc27 100644 --- a/scripts/gendwarfksyms/Makefile +++ b/scripts/gendwarfksyms/Makefile @@ -2,6 +2,7 @@ hostprogs-always-y += gendwarfksyms gendwarfksyms-objs += gendwarfksyms.o +gendwarfksyms-objs += die.o gendwarfksyms-objs += dwarf.o gendwarfksyms-objs += symbols.o diff --git a/scripts/gendwarfksyms/die.c b/scripts/gendwarfksyms/die.c new file mode 100644 index 000000000000..b7d900c6a9c8 --- /dev/null +++ b/scripts/gendwarfksyms/die.c @@ -0,0 +1,143 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Copyright (C) 2024 Google LLC + */ + +#include <string.h> +#include "gendwarfksyms.h" + +#define DIE_HASH_BITS 15 + +/* {die->addr, state} -> struct die * */ +static HASHTABLE_DEFINE(die_map, 1 << DIE_HASH_BITS); + +static unsigned int map_hits; +static unsigned int map_misses; + +static inline unsigned int die_hash(uintptr_t addr, enum die_state state) +{ + return hash_32(addr_hash(addr) ^ (unsigned int)state); +} + +static void init_die(struct die *cd) +{ + cd->state = DIE_INCOMPLETE; + cd->fqn = NULL; + cd->tag = -1; + cd->addr = 0; + INIT_LIST_HEAD(&cd->fragments); +} + +static struct die *create_die(Dwarf_Die *die, enum die_state state) +{ + struct die *cd; + + cd = xmalloc(sizeof(struct die)); + init_die(cd); + cd->addr = (uintptr_t)die->addr; + + hash_add(die_map, &cd->hash, die_hash(cd->addr, state)); + return cd; +} + +int __die_map_get(uintptr_t addr, enum die_state state, struct die **res) +{ + struct die *cd; + + hash_for_each_possible(die_map, cd, hash, die_hash(addr, state)) { + if (cd->addr == addr && cd->state == state) { + *res = cd; + return 0; + } + } + + return -1; +} + +struct die *die_map_get(Dwarf_Die *die, enum die_state state) +{ + struct die *cd; + + if (__die_map_get((uintptr_t)die->addr, state, &cd) == 0) { + map_hits++; + return cd; + } + + map_misses++; + return create_die(die, state); +} + +static void reset_die(struct die *cd) +{ + struct die_fragment *tmp; + struct die_fragment *df; + + list_for_each_entry_safe(df, tmp, &cd->fragments, list) { + if (df->type == FRAGMENT_STRING) + free(df->data.str); + free(df); + } + + if (cd->fqn && *cd->fqn) + free(cd->fqn); + init_die(cd); +} + +void die_map_free(void) +{ + struct hlist_node *tmp; + unsigned int stats[DIE_LAST + 1]; + struct die *cd; + int i; + + memset(stats, 0, sizeof(stats)); + + hash_for_each_safe(die_map, cd, tmp, hash) { + stats[cd->state]++; + reset_die(cd); + free(cd); + } + hash_init(die_map); + + if (map_hits + map_misses > 0) + debug("hits %u, misses %u (hit rate %.02f%%)", map_hits, + map_misses, + (100.0f * map_hits) / (map_hits + map_misses)); + + for (i = 0; i <= DIE_LAST; i++) + debug("%s: %u entries", die_state_name(i), stats[i]); +} + +static struct die_fragment *append_item(struct die *cd) +{ + struct die_fragment *df; + + df = xmalloc(sizeof(struct die_fragment)); + df->type = FRAGMENT_EMPTY; + list_add_tail(&df->list, &cd->fragments); + return df; +} + +void die_map_add_string(struct die *cd, const char *str) +{ + struct die_fragment *df; + + if (!cd) + return; + + df = append_item(cd); + df->data.str = xstrdup(str); + df->type = FRAGMENT_STRING; +} + +void die_map_add_die(struct die *cd, struct die *child) +{ + struct die_fragment *df; + + if (!cd) + return; + + df = append_item(cd); + df->data.addr = child->addr; + df->type = FRAGMENT_DIE; +} diff --git a/scripts/gendwarfksyms/dwarf.c b/scripts/gendwarfksyms/dwarf.c index 74e75b8ec891..f40e23a547da 100644 --- a/scripts/gendwarfksyms/dwarf.c +++ b/scripts/gendwarfksyms/dwarf.c @@ -72,17 +72,19 @@ static bool match_export_symbol(struct state *state, Dwarf_Die *die) /* * Type string processing */ -static void process(const char *s) +static void process(struct die *cache, const char *s) { s = s ?: "<null>"; if (dump_dies) fputs(s, stderr); + + die_map_add_string(cache, s); } #define MAX_FMT_BUFFER_SIZE 128 -static void process_fmt(const char *fmt, ...) +static void process_fmt(struct die *cache, const char *fmt, ...) { char buf[MAX_FMT_BUFFER_SIZE]; va_list args; @@ -92,7 +94,7 @@ static void process_fmt(const char *fmt, ...) if (checkp(vsnprintf(buf, sizeof(buf), fmt, args)) >= sizeof(buf)) error("vsnprintf overflow: increase MAX_FMT_BUFFER_SIZE"); - process(buf); + process(cache, buf); va_end(args); } @@ -165,18 +167,28 @@ done: return fqn; } -static void process_fqn(Dwarf_Die *die) +static void update_fqn(struct die *cache, Dwarf_Die *die) +{ + if (!cache->fqn) + cache->fqn = get_fqn(die) ?: ""; +} + +static void process_fqn(struct die *cache, Dwarf_Die *die) { - process(" "); - process(get_fqn(die) ?: ""); + update_fqn(cache, die); + if (*cache->fqn) + process(cache, " "); + process(cache, cache->fqn); } -#define DEFINE_PROCESS_UDATA_ATTRIBUTE(attribute) \ - static void process_##attribute##_attr(Dwarf_Die *die) \ - { \ - Dwarf_Word value; \ - if (get_udata_attr(die, DW_AT_##attribute, &value)) \ - process_fmt(" " #attribute "(%" PRIu64 ")", value); \ +#define DEFINE_PROCESS_UDATA_ATTRIBUTE(attribute) \ + static void process_##attribute##_attr(struct die *cache, \ + Dwarf_Die *die) \ + { \ + Dwarf_Word value; \ + if (get_udata_attr(die, DW_AT_##attribute, &value)) \ + process_fmt(cache, " " #attribute "(%" PRIu64 ")", \ + value); \ } DEFINE_PROCESS_UDATA_ATTRIBUTE(alignment) @@ -188,8 +200,9 @@ bool match_all(Dwarf_Die *die) return true; } -int process_die_container(struct state *state, Dwarf_Die *die, - die_callback_t func, die_match_callback_t match) +int process_die_container(struct state *state, struct die *cache, + Dwarf_Die *die, die_callback_t func, + die_match_callback_t match) { Dwarf_Die current; int res; @@ -198,7 +211,7 @@ int process_die_container(struct state *state, Dwarf_Die *die, while (!res) { if (match(¤t)) { /* <0 = error, 0 = continue, >0 = stop */ - res = checkp(func(state, ¤t)); + res = checkp(func(state, cache, ¤t)); if (res) return res; } @@ -209,39 +222,78 @@ int process_die_container(struct state *state, Dwarf_Die *die, return 0; } -static int process_type(struct state *state, Dwarf_Die *die); +static int process_type(struct state *state, struct die *parent, + Dwarf_Die *die); -static void process_type_attr(struct state *state, Dwarf_Die *die) +static void process_type_attr(struct state *state, struct die *cache, + Dwarf_Die *die) { Dwarf_Die type; if (get_ref_die_attr(die, DW_AT_type, &type)) { - check(process_type(state, &type)); + check(process_type(state, cache, &type)); return; } /* Compilers can omit DW_AT_type -- print out 'void' to clarify */ - process("base_type void"); + process(cache, "base_type void"); +} + +static void process_base_type(struct state *state, struct die *cache, + Dwarf_Die *die) +{ + process(cache, "base_type"); + process_fqn(cache, die); + process_byte_size_attr(cache, die); + process_encoding_attr(cache, die); + process_alignment_attr(cache, die); } -static void process_base_type(struct state *state, Dwarf_Die *die) +static void process_cached(struct state *state, struct die *cache, + Dwarf_Die *die) { - process("base_type"); - process_fqn(die); - process_byte_size_attr(die); - process_encoding_attr(die); - process_alignment_attr(die); + struct die_fragment *df; + Dwarf_Die child; + + list_for_each_entry(df, &cache->fragments, list) { + switch (df->type) { + case FRAGMENT_STRING: + process(NULL, df->data.str); + break; + case FRAGMENT_DIE: + if (!dwarf_die_addr_die(dwarf_cu_getdwarf(die->cu), + (void *)df->data.addr, &child)) + error("dwarf_die_addr_die failed"); + check(process_type(state, NULL, &child)); + break; + default: + error("empty die_fragment"); + } + } } -#define PROCESS_TYPE(type) \ - case DW_TAG_##type##_type: \ - process_##type##_type(state, die); \ +#define PROCESS_TYPE(type) \ + case DW_TAG_##type##_type: \ + process_##type##_type(state, cache, die); \ break; -static int process_type(struct state *state, Dwarf_Die *die) +static int process_type(struct state *state, struct die *parent, Dwarf_Die *die) { + struct die *cache; int tag = dwarf_tag(die); + /* + * If we have the DIE already cached, use it instead of walking + * through DWARF. + */ + cache = die_map_get(die, DIE_COMPLETE); + + if (cache->state == DIE_COMPLETE) { + process_cached(state, cache, die); + die_map_add_die(parent, cache); + return 0; + } + switch (tag) { PROCESS_TYPE(base) default: @@ -249,6 +301,11 @@ static int process_type(struct state *state, Dwarf_Die *die) break; } + /* Update cache state and append to the parent (if any) */ + cache->tag = tag; + cache->state = DIE_COMPLETE; + die_map_add_die(parent, cache); + return 0; } @@ -259,14 +316,15 @@ static void process_symbol(struct state *state, Dwarf_Die *die, die_callback_t process_func) { debug("%s", state->sym->name); - check(process_func(state, die)); + check(process_func(state, NULL, die)); if (dump_dies) fputs("\n", stderr); } -static int __process_subprogram(struct state *state, Dwarf_Die *die) +static int __process_subprogram(struct state *state, struct die *cache, + Dwarf_Die *die) { - process("subprogram"); + process(cache, "subprogram"); return 0; } @@ -275,10 +333,11 @@ static void process_subprogram(struct state *state, Dwarf_Die *die) process_symbol(state, die, __process_subprogram); } -static int __process_variable(struct state *state, Dwarf_Die *die) +static int __process_variable(struct state *state, struct die *cache, + Dwarf_Die *die) { - process("variable "); - process_type_attr(state, die); + process(cache, "variable "); + process_type_attr(state, cache, die); return 0; } @@ -287,7 +346,8 @@ static void process_variable(struct state *state, Dwarf_Die *die) process_symbol(state, die, __process_variable); } -static int process_exported_symbols(struct state *unused, Dwarf_Die *die) +static int process_exported_symbols(struct state *unused, struct die *cache, + Dwarf_Die *die) { int tag = dwarf_tag(die); @@ -297,7 +357,7 @@ static int process_exported_symbols(struct state *unused, Dwarf_Die *die) case DW_TAG_class_type: case DW_TAG_structure_type: return check(process_die_container( - NULL, die, process_exported_symbols, match_all)); + NULL, cache, die, process_exported_symbols, match_all)); /* Possible exported symbols */ case DW_TAG_subprogram: @@ -321,6 +381,6 @@ static int process_exported_symbols(struct state *unused, Dwarf_Die *die) void process_cu(Dwarf_Die *cudie) { - check(process_die_container(NULL, cudie, process_exported_symbols, + check(process_die_container(NULL, NULL, cudie, process_exported_symbols, match_all)); } diff --git a/scripts/gendwarfksyms/gendwarfksyms.c b/scripts/gendwarfksyms/gendwarfksyms.c index cd8bfe973a5c..3809db840c06 100644 --- a/scripts/gendwarfksyms/gendwarfksyms.c +++ b/scripts/gendwarfksyms/gendwarfksyms.c @@ -43,6 +43,10 @@ static int process_module(Dwfl_Module *mod, void **userdata, const char *name, debug("%s", name); dbg = dwfl_module_getdwarf(mod, &dwbias); + /* + * Look for exported symbols in each CU, follow the DIE tree, and add + * the entries to die_map. + */ do { res = dwarf_get_units(dbg, cu, &cu, NULL, NULL, &cudie, NULL); if (res < 0) @@ -53,6 +57,8 @@ static int process_module(Dwfl_Module *mod, void **userdata, const char *name, process_cu(&cudie); } while (cu); + die_map_free(); + return DWARF_CB_OK; } diff --git a/scripts/gendwarfksyms/gendwarfksyms.h b/scripts/gendwarfksyms/gendwarfksyms.h index cb9fd78a58da..601f877bc8ca 100644 --- a/scripts/gendwarfksyms/gendwarfksyms.h +++ b/scripts/gendwarfksyms/gendwarfksyms.h @@ -87,6 +87,61 @@ void symbol_read_symtab(int fd); struct symbol *symbol_get(const char *name); void symbol_free(void); +/* + * die.c + */ + +enum die_state { + DIE_INCOMPLETE, + DIE_COMPLETE, + DIE_LAST = DIE_COMPLETE +}; + +enum die_fragment_type { + FRAGMENT_EMPTY, + FRAGMENT_STRING, + FRAGMENT_DIE +}; + +struct die_fragment { + enum die_fragment_type type; + union { + char *str; + uintptr_t addr; + } data; + struct list_head list; +}; + +#define CASE_CONST_TO_STR(name) \ + case name: \ + return #name; + +static inline const char *die_state_name(enum die_state state) +{ + switch (state) { + CASE_CONST_TO_STR(DIE_INCOMPLETE) + CASE_CONST_TO_STR(DIE_COMPLETE) + } + + error("unexpected die_state: %d", state); +} + +struct die { + enum die_state state; + char *fqn; + int tag; + uintptr_t addr; + struct list_head fragments; + struct hlist_node hash; +}; + +int __die_map_get(uintptr_t addr, enum die_state state, struct die **res); +struct die *die_map_get(Dwarf_Die *die, enum die_state state); +void die_map_add_string(struct die *pd, const char *str); +void die_map_add_linebreak(struct die *pd, int linebreak); +void die_map_add_die(struct die *pd, struct die *child); +void die_map_free(void); + /* * dwarf.c */ @@ -96,12 +151,14 @@ struct state { Dwarf_Die die; }; -typedef int (*die_callback_t)(struct state *state, Dwarf_Die *die); +typedef int (*die_callback_t)(struct state *state, struct die *cache, + Dwarf_Die *die); typedef bool (*die_match_callback_t)(Dwarf_Die *die); bool match_all(Dwarf_Die *die); -int process_die_container(struct state *state, Dwarf_Die *die, - die_callback_t func, die_match_callback_t match); +int process_die_container(struct state *state, struct die *cache, + Dwarf_Die *die, die_callback_t func, + die_match_callback_t match); void process_cu(Dwarf_Die *cudie); -- cgit v1.2.3 From 06b8b036ab9c1e70a562705a398bcd271e0b5ebf Mon Sep 17 00:00:00 2001 From: Sami Tolvanen <samitolvanen@google.com> Date: Fri, 3 Jan 2025 20:45:27 +0000 Subject: gendwarfksyms: Expand type modifiers and typedefs Add support for expanding DWARF type modifiers, such as pointers, const values etc., and typedefs. These types all have DW_AT_type attribute pointing to the underlying type, and thus produce similar output. Also add linebreaks and indentation to debugging output to make it more readable. Signed-off-by: Sami Tolvanen <samitolvanen@google.com> Reviewed-by: Petr Pavlu <petr.pavlu@suse.com> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> --- scripts/gendwarfksyms/die.c | 12 +++++++ scripts/gendwarfksyms/dwarf.c | 67 +++++++++++++++++++++++++++++++++++ scripts/gendwarfksyms/gendwarfksyms.h | 5 +++ 3 files changed, 84 insertions(+) (limited to 'scripts') diff --git a/scripts/gendwarfksyms/die.c b/scripts/gendwarfksyms/die.c index b7d900c6a9c8..0d70e02d02b5 100644 --- a/scripts/gendwarfksyms/die.c +++ b/scripts/gendwarfksyms/die.c @@ -130,6 +130,18 @@ void die_map_add_string(struct die *cd, const char *str) df->type = FRAGMENT_STRING; } +void die_map_add_linebreak(struct die *cd, int linebreak) +{ + struct die_fragment *df; + + if (!cd) + return; + + df = append_item(cd); + df->data.linebreak = linebreak; + df->type = FRAGMENT_LINEBREAK; +} + void die_map_add_die(struct die *cd, struct die *child) { struct die_fragment *df; diff --git a/scripts/gendwarfksyms/dwarf.c b/scripts/gendwarfksyms/dwarf.c index f40e23a547da..3e08a32b7b16 100644 --- a/scripts/gendwarfksyms/dwarf.c +++ b/scripts/gendwarfksyms/dwarf.c @@ -7,6 +7,17 @@ #include <stdarg.h> #include "gendwarfksyms.h" +static bool do_linebreak; +static int indentation_level; + +/* Line breaks and indentation for pretty-printing */ +static void process_linebreak(struct die *cache, int n) +{ + indentation_level += n; + do_linebreak = true; + die_map_add_linebreak(cache, n); +} + #define DEFINE_GET_ATTR(attr, type) \ static bool get_##attr##_attr(Dwarf_Die *die, unsigned int id, \ type *value) \ @@ -76,6 +87,12 @@ static void process(struct die *cache, const char *s) { s = s ?: "<null>"; + if (dump_dies && do_linebreak) { + fputs("\n", stderr); + for (int i = 0; i < indentation_level; i++) + fputs(" ", stderr); + do_linebreak = false; + } if (dump_dies) fputs(s, stderr); @@ -239,6 +256,40 @@ static void process_type_attr(struct state *state, struct die *cache, process(cache, "base_type void"); } +/* Container types with DW_AT_type */ +static void __process_type(struct state *state, struct die *cache, + Dwarf_Die *die, const char *type) +{ + process(cache, type); + process_fqn(cache, die); + process(cache, " {"); + process_linebreak(cache, 1); + process_type_attr(state, cache, die); + process_linebreak(cache, -1); + process(cache, "}"); + process_byte_size_attr(cache, die); + process_alignment_attr(cache, die); +} + +#define DEFINE_PROCESS_TYPE(type) \ + static void process_##type##_type(struct state *state, \ + struct die *cache, Dwarf_Die *die) \ + { \ + __process_type(state, cache, die, #type "_type"); \ + } + +DEFINE_PROCESS_TYPE(atomic) +DEFINE_PROCESS_TYPE(const) +DEFINE_PROCESS_TYPE(immutable) +DEFINE_PROCESS_TYPE(packed) +DEFINE_PROCESS_TYPE(pointer) +DEFINE_PROCESS_TYPE(reference) +DEFINE_PROCESS_TYPE(restrict) +DEFINE_PROCESS_TYPE(rvalue_reference) +DEFINE_PROCESS_TYPE(shared) +DEFINE_PROCESS_TYPE(volatile) +DEFINE_PROCESS_TYPE(typedef) + static void process_base_type(struct state *state, struct die *cache, Dwarf_Die *die) { @@ -260,6 +311,9 @@ static void process_cached(struct state *state, struct die *cache, case FRAGMENT_STRING: process(NULL, df->data.str); break; + case FRAGMENT_LINEBREAK: + process_linebreak(NULL, df->data.linebreak); + break; case FRAGMENT_DIE: if (!dwarf_die_addr_die(dwarf_cu_getdwarf(die->cu), (void *)df->data.addr, &child)) @@ -295,7 +349,20 @@ static int process_type(struct state *state, struct die *parent, Dwarf_Die *die) } switch (tag) { + /* Type modifiers */ + PROCESS_TYPE(atomic) + PROCESS_TYPE(const) + PROCESS_TYPE(immutable) + PROCESS_TYPE(packed) + PROCESS_TYPE(pointer) + PROCESS_TYPE(reference) + PROCESS_TYPE(restrict) + PROCESS_TYPE(rvalue_reference) + PROCESS_TYPE(shared) + PROCESS_TYPE(volatile) + /* Other types */ PROCESS_TYPE(base) + PROCESS_TYPE(typedef) default: debug("unimplemented type: %x", tag); break; diff --git a/scripts/gendwarfksyms/gendwarfksyms.h b/scripts/gendwarfksyms/gendwarfksyms.h index 601f877bc8ca..832d05b4fc1c 100644 --- a/scripts/gendwarfksyms/gendwarfksyms.h +++ b/scripts/gendwarfksyms/gendwarfksyms.h @@ -59,6 +59,9 @@ extern int dump_dies; /* Error == negative values */ #define checkp(expr) __check(expr, __res < 0) +/* Consistent aliases (DW_TAG_<type>_type) for DWARF tags */ +#define DW_TAG_typedef_type DW_TAG_typedef + /* * symbols.c */ @@ -100,6 +103,7 @@ enum die_state { enum die_fragment_type { FRAGMENT_EMPTY, FRAGMENT_STRING, + FRAGMENT_LINEBREAK, FRAGMENT_DIE }; @@ -107,6 +111,7 @@ struct die_fragment { enum die_fragment_type type; union { char *str; + int linebreak; uintptr_t addr; } data; struct list_head list; -- cgit v1.2.3 From 220a0857f3a89e0dce3fc7c38d981df41c4537a7 Mon Sep 17 00:00:00 2001 From: Sami Tolvanen <samitolvanen@google.com> Date: Fri, 3 Jan 2025 20:45:28 +0000 Subject: gendwarfksyms: Expand subroutine_type Add support for expanding DW_TAG_subroutine_type and the parameters in DW_TAG_formal_parameter. Use this to also expand subprograms. Example output with --dump-dies: subprogram ( formal_parameter pointer_type { const_type { base_type char byte_size(1) encoding(6) } } ) -> base_type unsigned long byte_size(8) encoding(7) Signed-off-by: Sami Tolvanen <samitolvanen@google.com> Reviewed-by: Petr Pavlu <petr.pavlu@suse.com> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> --- scripts/gendwarfksyms/dwarf.c | 84 +++++++++++++++++++++++++++++++++-- scripts/gendwarfksyms/gendwarfksyms.h | 4 ++ 2 files changed, 85 insertions(+), 3 deletions(-) (limited to 'scripts') diff --git a/scripts/gendwarfksyms/dwarf.c b/scripts/gendwarfksyms/dwarf.c index 3e08a32b7b16..7d8a4eb6c387 100644 --- a/scripts/gendwarfksyms/dwarf.c +++ b/scripts/gendwarfksyms/dwarf.c @@ -212,6 +212,15 @@ DEFINE_PROCESS_UDATA_ATTRIBUTE(alignment) DEFINE_PROCESS_UDATA_ATTRIBUTE(byte_size) DEFINE_PROCESS_UDATA_ATTRIBUTE(encoding) +/* Match functions -- die_match_callback_t */ +#define DEFINE_MATCH(type) \ + static bool match_##type##_type(Dwarf_Die *die) \ + { \ + return dwarf_tag(die) == DW_TAG_##type##_type; \ + } + +DEFINE_MATCH(formal_parameter) + bool match_all(Dwarf_Die *die) { return true; @@ -224,19 +233,28 @@ int process_die_container(struct state *state, struct die *cache, Dwarf_Die current; int res; + /* Track the first item in lists. */ + if (state) + state->first_list_item = true; + res = checkp(dwarf_child(die, ¤t)); while (!res) { if (match(¤t)) { /* <0 = error, 0 = continue, >0 = stop */ res = checkp(func(state, cache, ¤t)); if (res) - return res; + goto out; } res = checkp(dwarf_siblingof(¤t, ¤t)); } - return 0; + res = 0; +out: + if (state) + state->first_list_item = false; + + return res; } static int process_type(struct state *state, struct die *parent, @@ -256,6 +274,40 @@ static void process_type_attr(struct state *state, struct die *cache, process(cache, "base_type void"); } +static void process_list_comma(struct state *state, struct die *cache) +{ + if (state->first_list_item) { + state->first_list_item = false; + } else { + process(cache, " ,"); + process_linebreak(cache, 0); + } +} + +/* Comma-separated with DW_AT_type */ +static void __process_list_type(struct state *state, struct die *cache, + Dwarf_Die *die, const char *type) +{ + const char *name = get_name_attr(die); + + process_list_comma(state, cache); + process(cache, type); + process_type_attr(state, cache, die); + if (name) { + process(cache, " "); + process(cache, name); + } +} + +#define DEFINE_PROCESS_LIST_TYPE(type) \ + static void process_##type##_type(struct state *state, \ + struct die *cache, Dwarf_Die *die) \ + { \ + __process_list_type(state, cache, die, #type " "); \ + } + +DEFINE_PROCESS_LIST_TYPE(formal_parameter) + /* Container types with DW_AT_type */ static void __process_type(struct state *state, struct die *cache, Dwarf_Die *die, const char *type) @@ -290,6 +342,29 @@ DEFINE_PROCESS_TYPE(shared) DEFINE_PROCESS_TYPE(volatile) DEFINE_PROCESS_TYPE(typedef) +static void __process_subroutine_type(struct state *state, struct die *cache, + Dwarf_Die *die, const char *type) +{ + process(cache, type); + process(cache, " ("); + process_linebreak(cache, 1); + /* Parameters */ + check(process_die_container(state, cache, die, process_type, + match_formal_parameter_type)); + process_linebreak(cache, -1); + process(cache, ")"); + process_linebreak(cache, 0); + /* Return type */ + process(cache, "-> "); + process_type_attr(state, cache, die); +} + +static void process_subroutine_type(struct state *state, struct die *cache, + Dwarf_Die *die) +{ + __process_subroutine_type(state, cache, die, "subroutine_type"); +} + static void process_base_type(struct state *state, struct die *cache, Dwarf_Die *die) { @@ -360,8 +435,11 @@ static int process_type(struct state *state, struct die *parent, Dwarf_Die *die) PROCESS_TYPE(rvalue_reference) PROCESS_TYPE(shared) PROCESS_TYPE(volatile) + /* Subtypes */ + PROCESS_TYPE(formal_parameter) /* Other types */ PROCESS_TYPE(base) + PROCESS_TYPE(subroutine) PROCESS_TYPE(typedef) default: debug("unimplemented type: %x", tag); @@ -391,7 +469,7 @@ static void process_symbol(struct state *state, Dwarf_Die *die, static int __process_subprogram(struct state *state, struct die *cache, Dwarf_Die *die) { - process(cache, "subprogram"); + __process_subroutine_type(state, cache, die, "subprogram"); return 0; } diff --git a/scripts/gendwarfksyms/gendwarfksyms.h b/scripts/gendwarfksyms/gendwarfksyms.h index 832d05b4fc1c..0746a36f4924 100644 --- a/scripts/gendwarfksyms/gendwarfksyms.h +++ b/scripts/gendwarfksyms/gendwarfksyms.h @@ -60,6 +60,7 @@ extern int dump_dies; #define checkp(expr) __check(expr, __res < 0) /* Consistent aliases (DW_TAG_<type>_type) for DWARF tags */ +#define DW_TAG_formal_parameter_type DW_TAG_formal_parameter #define DW_TAG_typedef_type DW_TAG_typedef /* @@ -154,6 +155,9 @@ void die_map_free(void); struct state { struct symbol *sym; Dwarf_Die die; + + /* List expansion */ + bool first_list_item; }; typedef int (*die_callback_t)(struct state *state, struct die *cache, -- cgit v1.2.3 From c772f1d1eaac608c083ee79fd5cfbe879958eb3e Mon Sep 17 00:00:00 2001 From: Sami Tolvanen <samitolvanen@google.com> Date: Fri, 3 Jan 2025 20:45:29 +0000 Subject: gendwarfksyms: Expand array_type Add support for expanding DW_TAG_array_type, and the subrange type indicating array size. Example source code: const char *s[34]; Output with --dump-dies: variable array_type[34] { pointer_type { const_type { base_type char byte_size(1) encoding(6) } } byte_size(8) } Signed-off-by: Sami Tolvanen <samitolvanen@google.com> Reviewed-by: Petr Pavlu <petr.pavlu@suse.com> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> --- scripts/gendwarfksyms/dwarf.c | 30 ++++++++++++++++++++++++++++++ 1 file changed, 30 insertions(+) (limited to 'scripts') diff --git a/scripts/gendwarfksyms/dwarf.c b/scripts/gendwarfksyms/dwarf.c index 7d8a4eb6c387..46ce17b2459b 100644 --- a/scripts/gendwarfksyms/dwarf.c +++ b/scripts/gendwarfksyms/dwarf.c @@ -220,6 +220,7 @@ DEFINE_PROCESS_UDATA_ATTRIBUTE(encoding) } DEFINE_MATCH(formal_parameter) +DEFINE_MATCH(subrange) bool match_all(Dwarf_Die *die) { @@ -342,6 +343,33 @@ DEFINE_PROCESS_TYPE(shared) DEFINE_PROCESS_TYPE(volatile) DEFINE_PROCESS_TYPE(typedef) +static void process_subrange_type(struct state *state, struct die *cache, + Dwarf_Die *die) +{ + Dwarf_Word count = 0; + + if (get_udata_attr(die, DW_AT_count, &count)) + process_fmt(cache, "[%" PRIu64 "]", count); + else if (get_udata_attr(die, DW_AT_upper_bound, &count)) + process_fmt(cache, "[%" PRIu64 "]", count + 1); + else + process(cache, "[]"); +} + +static void process_array_type(struct state *state, struct die *cache, + Dwarf_Die *die) +{ + process(cache, "array_type"); + /* Array size */ + check(process_die_container(state, cache, die, process_type, + match_subrange_type)); + process(cache, " {"); + process_linebreak(cache, 1); + process_type_attr(state, cache, die); + process_linebreak(cache, -1); + process(cache, "}"); +} + static void __process_subroutine_type(struct state *state, struct die *cache, Dwarf_Die *die, const char *type) { @@ -437,7 +465,9 @@ static int process_type(struct state *state, struct die *parent, Dwarf_Die *die) PROCESS_TYPE(volatile) /* Subtypes */ PROCESS_TYPE(formal_parameter) + PROCESS_TYPE(subrange) /* Other types */ + PROCESS_TYPE(array) PROCESS_TYPE(base) PROCESS_TYPE(subroutine) PROCESS_TYPE(typedef) -- cgit v1.2.3 From f6bb92455a5e5b2241d2e1f3e240c5fc036c55cb Mon Sep 17 00:00:00 2001 From: Sami Tolvanen <samitolvanen@google.com> Date: Fri, 3 Jan 2025 20:45:30 +0000 Subject: gendwarfksyms: Expand structure types Recursively expand DWARF structure types, i.e. structs, unions, and enums. Also include relevant DWARF attributes in type strings to encode structure layout, for example. Example output with --dump-dies: subprogram ( formal_parameter structure_type &str { member pointer_type { base_type u8 byte_size(1) encoding(7) } data_ptr data_member_location(0) , member base_type usize byte_size(8) encoding(7) length data_member_location(8) } byte_size(16) alignment(8) msg ) -> base_type void Signed-off-by: Sami Tolvanen <samitolvanen@google.com> Reviewed-by: Petr Pavlu <petr.pavlu@suse.com> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> --- scripts/gendwarfksyms/dwarf.c | 138 +++++++++++++++++++++++++++++++++- scripts/gendwarfksyms/gendwarfksyms.h | 5 ++ 2 files changed, 141 insertions(+), 2 deletions(-) (limited to 'scripts') diff --git a/scripts/gendwarfksyms/dwarf.c b/scripts/gendwarfksyms/dwarf.c index 46ce17b2459b..6ec1138c459f 100644 --- a/scripts/gendwarfksyms/dwarf.c +++ b/scripts/gendwarfksyms/dwarf.c @@ -208,9 +208,14 @@ static void process_fqn(struct die *cache, Dwarf_Die *die) value); \ } +DEFINE_PROCESS_UDATA_ATTRIBUTE(accessibility) DEFINE_PROCESS_UDATA_ATTRIBUTE(alignment) +DEFINE_PROCESS_UDATA_ATTRIBUTE(bit_size) DEFINE_PROCESS_UDATA_ATTRIBUTE(byte_size) DEFINE_PROCESS_UDATA_ATTRIBUTE(encoding) +DEFINE_PROCESS_UDATA_ATTRIBUTE(data_bit_offset) +DEFINE_PROCESS_UDATA_ATTRIBUTE(data_member_location) +DEFINE_PROCESS_UDATA_ATTRIBUTE(discr_value) /* Match functions -- die_match_callback_t */ #define DEFINE_MATCH(type) \ @@ -219,7 +224,9 @@ DEFINE_PROCESS_UDATA_ATTRIBUTE(encoding) return dwarf_tag(die) == DW_TAG_##type##_type; \ } +DEFINE_MATCH(enumerator) DEFINE_MATCH(formal_parameter) +DEFINE_MATCH(member) DEFINE_MATCH(subrange) bool match_all(Dwarf_Die *die) @@ -298,6 +305,10 @@ static void __process_list_type(struct state *state, struct die *cache, process(cache, " "); process(cache, name); } + process_accessibility_attr(cache, die); + process_bit_size_attr(cache, die); + process_data_bit_offset_attr(cache, die); + process_data_member_location_attr(cache, die); } #define DEFINE_PROCESS_LIST_TYPE(type) \ @@ -308,6 +319,7 @@ static void __process_list_type(struct state *state, struct die *cache, } DEFINE_PROCESS_LIST_TYPE(formal_parameter) +DEFINE_PROCESS_LIST_TYPE(member) /* Container types with DW_AT_type */ static void __process_type(struct state *state, struct die *cache, @@ -340,6 +352,7 @@ DEFINE_PROCESS_TYPE(reference) DEFINE_PROCESS_TYPE(restrict) DEFINE_PROCESS_TYPE(rvalue_reference) DEFINE_PROCESS_TYPE(shared) +DEFINE_PROCESS_TYPE(template_type_parameter) DEFINE_PROCESS_TYPE(volatile) DEFINE_PROCESS_TYPE(typedef) @@ -393,6 +406,107 @@ static void process_subroutine_type(struct state *state, struct die *cache, __process_subroutine_type(state, cache, die, "subroutine_type"); } +static void process_variant_type(struct state *state, struct die *cache, + Dwarf_Die *die) +{ + process_list_comma(state, cache); + process(cache, "variant {"); + process_linebreak(cache, 1); + check(process_die_container(state, cache, die, process_type, + match_member_type)); + process_linebreak(cache, -1); + process(cache, "}"); + process_discr_value_attr(cache, die); +} + +static void process_variant_part_type(struct state *state, struct die *cache, + Dwarf_Die *die) +{ + process_list_comma(state, cache); + process(cache, "variant_part {"); + process_linebreak(cache, 1); + check(process_die_container(state, cache, die, process_type, + match_all)); + process_linebreak(cache, -1); + process(cache, "}"); +} + +static int ___process_structure_type(struct state *state, struct die *cache, + Dwarf_Die *die) +{ + switch (dwarf_tag(die)) { + case DW_TAG_member: + case DW_TAG_variant_part: + return check(process_type(state, cache, die)); + case DW_TAG_class_type: + case DW_TAG_enumeration_type: + case DW_TAG_structure_type: + case DW_TAG_template_type_parameter: + case DW_TAG_union_type: + case DW_TAG_subprogram: + /* Skip non-member types, including member functions */ + return 0; + default: + error("unexpected structure_type child: %x", dwarf_tag(die)); + } +} + +static void __process_structure_type(struct state *state, struct die *cache, + Dwarf_Die *die, const char *type, + die_callback_t process_func, + die_match_callback_t match_func) +{ + process(cache, type); + process_fqn(cache, die); + process(cache, " {"); + process_linebreak(cache, 1); + + check(process_die_container(state, cache, die, process_func, + match_func)); + + process_linebreak(cache, -1); + process(cache, "}"); + + process_byte_size_attr(cache, die); + process_alignment_attr(cache, die); +} + +#define DEFINE_PROCESS_STRUCTURE_TYPE(structure) \ + static void process_##structure##_type( \ + struct state *state, struct die *cache, Dwarf_Die *die) \ + { \ + __process_structure_type(state, cache, die, \ + #structure "_type", \ + ___process_structure_type, \ + match_all); \ + } + +DEFINE_PROCESS_STRUCTURE_TYPE(class) +DEFINE_PROCESS_STRUCTURE_TYPE(structure) +DEFINE_PROCESS_STRUCTURE_TYPE(union) + +static void process_enumerator_type(struct state *state, struct die *cache, + Dwarf_Die *die) +{ + Dwarf_Word value; + + process_list_comma(state, cache); + process(cache, "enumerator"); + process_fqn(cache, die); + + if (get_udata_attr(die, DW_AT_const_value, &value)) { + process(cache, " = "); + process_fmt(cache, "%" PRIu64, value); + } +} + +static void process_enumeration_type(struct state *state, struct die *cache, + Dwarf_Die *die) +{ + __process_structure_type(state, cache, die, "enumeration_type", + process_type, match_enumerator_type); +} + static void process_base_type(struct state *state, struct die *cache, Dwarf_Die *die) { @@ -403,6 +517,16 @@ static void process_base_type(struct state *state, struct die *cache, process_alignment_attr(cache, die); } +static void process_unspecified_type(struct state *state, struct die *cache, + Dwarf_Die *die) +{ + /* + * These can be emitted for stand-alone assembly code, which means we + * might run into them in vmlinux.o. + */ + process(cache, "unspecified_type"); +} + static void process_cached(struct state *state, struct die *cache, Dwarf_Die *die) { @@ -463,17 +587,27 @@ static int process_type(struct state *state, struct die *parent, Dwarf_Die *die) PROCESS_TYPE(rvalue_reference) PROCESS_TYPE(shared) PROCESS_TYPE(volatile) + /* Container types */ + PROCESS_TYPE(class) + PROCESS_TYPE(structure) + PROCESS_TYPE(union) + PROCESS_TYPE(enumeration) /* Subtypes */ + PROCESS_TYPE(enumerator) PROCESS_TYPE(formal_parameter) + PROCESS_TYPE(member) PROCESS_TYPE(subrange) + PROCESS_TYPE(template_type_parameter) + PROCESS_TYPE(variant) + PROCESS_TYPE(variant_part) /* Other types */ PROCESS_TYPE(array) PROCESS_TYPE(base) PROCESS_TYPE(subroutine) PROCESS_TYPE(typedef) + PROCESS_TYPE(unspecified) default: - debug("unimplemented type: %x", tag); - break; + error("unexpected type: %x", tag); } /* Update cache state and append to the parent (if any) */ diff --git a/scripts/gendwarfksyms/gendwarfksyms.h b/scripts/gendwarfksyms/gendwarfksyms.h index 0746a36f4924..1796f71b3a34 100644 --- a/scripts/gendwarfksyms/gendwarfksyms.h +++ b/scripts/gendwarfksyms/gendwarfksyms.h @@ -60,8 +60,13 @@ extern int dump_dies; #define checkp(expr) __check(expr, __res < 0) /* Consistent aliases (DW_TAG_<type>_type) for DWARF tags */ +#define DW_TAG_enumerator_type DW_TAG_enumerator #define DW_TAG_formal_parameter_type DW_TAG_formal_parameter +#define DW_TAG_member_type DW_TAG_member +#define DW_TAG_template_type_parameter_type DW_TAG_template_type_parameter #define DW_TAG_typedef_type DW_TAG_typedef +#define DW_TAG_variant_part_type DW_TAG_variant_part +#define DW_TAG_variant_type DW_TAG_variant /* * symbols.c -- cgit v1.2.3 From f936c129fd4c3ce495768374ea48e5b736655046 Mon Sep 17 00:00:00 2001 From: Sami Tolvanen <samitolvanen@google.com> Date: Fri, 3 Jan 2025 20:45:31 +0000 Subject: gendwarfksyms: Limit structure expansion Expand each structure type only once per exported symbol. This is necessary to support self-referential structures, which would otherwise result in infinite recursion, and it's sufficient for catching ABI changes. Types defined in .c files are opaque to external users and thus cannot affect the ABI. Consider type definitions in .c files to be declarations to prevent opaque types from changing symbol versions. Signed-off-by: Sami Tolvanen <samitolvanen@google.com> Reviewed-by: Petr Pavlu <petr.pavlu@suse.com> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> --- scripts/gendwarfksyms/Makefile | 1 + scripts/gendwarfksyms/cache.c | 51 ++++++++++++++ scripts/gendwarfksyms/dwarf.c | 125 +++++++++++++++++++++++++++++++--- scripts/gendwarfksyms/gendwarfksyms.h | 46 +++++++++++++ 4 files changed, 215 insertions(+), 8 deletions(-) create mode 100644 scripts/gendwarfksyms/cache.c (limited to 'scripts') diff --git a/scripts/gendwarfksyms/Makefile b/scripts/gendwarfksyms/Makefile index c0d4ce50fc27..c06145d84df8 100644 --- a/scripts/gendwarfksyms/Makefile +++ b/scripts/gendwarfksyms/Makefile @@ -2,6 +2,7 @@ hostprogs-always-y += gendwarfksyms gendwarfksyms-objs += gendwarfksyms.o +gendwarfksyms-objs += cache.o gendwarfksyms-objs += die.o gendwarfksyms-objs += dwarf.o gendwarfksyms-objs += symbols.o diff --git a/scripts/gendwarfksyms/cache.c b/scripts/gendwarfksyms/cache.c new file mode 100644 index 000000000000..c9c19b86a686 --- /dev/null +++ b/scripts/gendwarfksyms/cache.c @@ -0,0 +1,51 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Copyright (C) 2024 Google LLC + */ + +#include "gendwarfksyms.h" + +struct cache_item { + unsigned long key; + int value; + struct hlist_node hash; +}; + +void cache_set(struct cache *cache, unsigned long key, int value) +{ + struct cache_item *ci; + + ci = xmalloc(sizeof(struct cache_item)); + ci->key = key; + ci->value = value; + hash_add(cache->cache, &ci->hash, hash_32(key)); +} + +int cache_get(struct cache *cache, unsigned long key) +{ + struct cache_item *ci; + + hash_for_each_possible(cache->cache, ci, hash, hash_32(key)) { + if (ci->key == key) + return ci->value; + } + + return -1; +} + +void cache_init(struct cache *cache) +{ + hash_init(cache->cache); +} + +void cache_free(struct cache *cache) +{ + struct hlist_node *tmp; + struct cache_item *ci; + + hash_for_each_safe(cache->cache, ci, tmp, hash) { + free(ci); + } + + hash_init(cache->cache); +} diff --git a/scripts/gendwarfksyms/dwarf.c b/scripts/gendwarfksyms/dwarf.c index 6ec1138c459f..6b30e45a4e82 100644 --- a/scripts/gendwarfksyms/dwarf.c +++ b/scripts/gendwarfksyms/dwarf.c @@ -27,6 +27,7 @@ static void process_linebreak(struct die *cache, int n) !dwarf_form##attr(&da, value); \ } +DEFINE_GET_ATTR(flag, bool) DEFINE_GET_ATTR(udata, Dwarf_Word) static bool get_ref_die_attr(Dwarf_Die *die, unsigned int id, Dwarf_Die *value) @@ -80,6 +81,55 @@ static bool match_export_symbol(struct state *state, Dwarf_Die *die) return !!state->sym; } +/* DW_AT_decl_file -> struct srcfile */ +static struct cache srcfile_cache; + +static bool is_definition_private(Dwarf_Die *die) +{ + Dwarf_Word filenum; + Dwarf_Files *files; + Dwarf_Die cudie; + const char *s; + int res; + + /* + * Definitions in .c files cannot change the public ABI, + * so consider them private. + */ + if (!get_udata_attr(die, DW_AT_decl_file, &filenum)) + return false; + + res = cache_get(&srcfile_cache, filenum); + if (res >= 0) + return !!res; + + if (!dwarf_cu_die(die->cu, &cudie, NULL, NULL, NULL, NULL, NULL, NULL)) + error("dwarf_cu_die failed: '%s'", dwarf_errmsg(-1)); + + if (dwarf_getsrcfiles(&cudie, &files, NULL)) + error("dwarf_getsrcfiles failed: '%s'", dwarf_errmsg(-1)); + + s = dwarf_filesrc(files, filenum, NULL, NULL); + if (!s) + error("dwarf_filesrc failed: '%s'", dwarf_errmsg(-1)); + + s = strrchr(s, '.'); + res = s && !strcmp(s, ".c"); + cache_set(&srcfile_cache, filenum, res); + + return !!res; +} + +static bool is_kabi_definition(Dwarf_Die *die) +{ + bool value; + + if (get_flag_attr(die, DW_AT_declaration, &value) && value) + return false; + + return !is_definition_private(die); +} + /* * Type string processing */ @@ -456,19 +506,27 @@ static void __process_structure_type(struct state *state, struct die *cache, die_callback_t process_func, die_match_callback_t match_func) { + bool expand; + process(cache, type); process_fqn(cache, die); process(cache, " {"); process_linebreak(cache, 1); - check(process_die_container(state, cache, die, process_func, - match_func)); + expand = state->expand.expand && is_kabi_definition(die); + + if (expand) { + check(process_die_container(state, cache, die, process_func, + match_func)); + } process_linebreak(cache, -1); process(cache, "}"); - process_byte_size_attr(cache, die); - process_alignment_attr(cache, die); + if (expand) { + process_byte_size_attr(cache, die); + process_alignment_attr(cache, die); + } } #define DEFINE_PROCESS_STRUCTURE_TYPE(structure) \ @@ -553,6 +611,30 @@ static void process_cached(struct state *state, struct die *cache, } } +static void state_init(struct state *state) +{ + state->expand.expand = true; + cache_init(&state->expansion_cache); +} + +static void expansion_state_restore(struct expansion_state *state, + struct expansion_state *saved) +{ + state->expand = saved->expand; +} + +static void expansion_state_save(struct expansion_state *state, + struct expansion_state *saved) +{ + expansion_state_restore(saved, state); +} + +static bool is_expanded_type(int tag) +{ + return tag == DW_TAG_class_type || tag == DW_TAG_structure_type || + tag == DW_TAG_union_type || tag == DW_TAG_enumeration_type; +} + #define PROCESS_TYPE(type) \ case DW_TAG_##type##_type: \ process_##type##_type(state, cache, die); \ @@ -560,18 +642,39 @@ static void process_cached(struct state *state, struct die *cache, static int process_type(struct state *state, struct die *parent, Dwarf_Die *die) { + enum die_state want_state = DIE_COMPLETE; struct die *cache; + struct expansion_state saved; int tag = dwarf_tag(die); + expansion_state_save(&state->expand, &saved); + /* - * If we have the DIE already cached, use it instead of walking + * Structures and enumeration types are expanded only once per + * exported symbol. This is sufficient for detecting ABI changes + * within the structure. + */ + if (is_expanded_type(tag)) { + if (cache_was_expanded(&state->expansion_cache, die->addr)) + state->expand.expand = false; + + if (state->expand.expand) + cache_mark_expanded(&state->expansion_cache, die->addr); + else + want_state = DIE_UNEXPANDED; + } + + /* + * If we have want_state already cached, use it instead of walking * through DWARF. */ - cache = die_map_get(die, DIE_COMPLETE); + cache = die_map_get(die, want_state); - if (cache->state == DIE_COMPLETE) { + if (cache->state == want_state) { process_cached(state, cache, die); die_map_add_die(parent, cache); + + expansion_state_restore(&state->expand, &saved); return 0; } @@ -612,9 +715,10 @@ static int process_type(struct state *state, struct die *parent, Dwarf_Die *die) /* Update cache state and append to the parent (if any) */ cache->tag = tag; - cache->state = DIE_COMPLETE; + cache->state = want_state; die_map_add_die(parent, cache); + expansion_state_restore(&state->expand, &saved); return 0; } @@ -676,11 +780,14 @@ static int process_exported_symbols(struct state *unused, struct die *cache, if (!match_export_symbol(&state, die)) return 0; + state_init(&state); + if (tag == DW_TAG_subprogram) process_subprogram(&state, &state.die); else process_variable(&state, &state.die); + cache_free(&state.expansion_cache); return 0; } default: @@ -692,4 +799,6 @@ void process_cu(Dwarf_Die *cudie) { check(process_die_container(NULL, NULL, cudie, process_exported_symbols, match_all)); + + cache_free(&srcfile_cache); } diff --git a/scripts/gendwarfksyms/gendwarfksyms.h b/scripts/gendwarfksyms/gendwarfksyms.h index 1796f71b3a34..941c4134da8e 100644 --- a/scripts/gendwarfksyms/gendwarfksyms.h +++ b/scripts/gendwarfksyms/gendwarfksyms.h @@ -102,6 +102,7 @@ void symbol_free(void); enum die_state { DIE_INCOMPLETE, + DIE_UNEXPANDED, DIE_COMPLETE, DIE_LAST = DIE_COMPLETE }; @@ -131,6 +132,7 @@ static inline const char *die_state_name(enum die_state state) { switch (state) { CASE_CONST_TO_STR(DIE_INCOMPLETE) + CASE_CONST_TO_STR(DIE_UNEXPANDED) CASE_CONST_TO_STR(DIE_COMPLETE) } @@ -153,16 +155,60 @@ void die_map_add_linebreak(struct die *pd, int linebreak); void die_map_add_die(struct die *pd, struct die *child); void die_map_free(void); +/* + * cache.c + */ + +#define CACHE_HASH_BITS 10 + +/* A cache for addresses we've already seen. */ +struct cache { + HASHTABLE_DECLARE(cache, 1 << CACHE_HASH_BITS); +}; + +void cache_set(struct cache *cache, unsigned long key, int value); +int cache_get(struct cache *cache, unsigned long key); +void cache_init(struct cache *cache); +void cache_free(struct cache *cache); + +static inline void __cache_mark_expanded(struct cache *cache, uintptr_t addr) +{ + cache_set(cache, addr, 1); +} + +static inline bool __cache_was_expanded(struct cache *cache, uintptr_t addr) +{ + return cache_get(cache, addr) == 1; +} + +static inline void cache_mark_expanded(struct cache *cache, void *addr) +{ + __cache_mark_expanded(cache, (uintptr_t)addr); +} + +static inline bool cache_was_expanded(struct cache *cache, void *addr) +{ + return __cache_was_expanded(cache, (uintptr_t)addr); +} + /* * dwarf.c */ +struct expansion_state { + bool expand; +}; + struct state { struct symbol *sym; Dwarf_Die die; /* List expansion */ bool first_list_item; + + /* Structure expansion */ + struct expansion_state expand; + struct cache expansion_cache; }; typedef int (*die_callback_t)(struct state *state, struct die *cache, -- cgit v1.2.3 From d2ffdc1c9a0ee71b30e25fbe3e2a37bf4c146085 Mon Sep 17 00:00:00 2001 From: Sami Tolvanen <samitolvanen@google.com> Date: Fri, 3 Jan 2025 20:45:32 +0000 Subject: gendwarfksyms: Add die_map debugging Debugging the DWARF processing can be somewhat challenging, so add more detailed debugging output for die_map operations. Add the --dump-die-map flag, which adds color coded tags to the output for die_map changes. Signed-off-by: Sami Tolvanen <samitolvanen@google.com> Reviewed-by: Petr Pavlu <petr.pavlu@suse.com> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> --- scripts/gendwarfksyms/dwarf.c | 15 +++++++++++++++ scripts/gendwarfksyms/gendwarfksyms.c | 7 +++++++ scripts/gendwarfksyms/gendwarfksyms.h | 13 +++++++++++++ 3 files changed, 35 insertions(+) (limited to 'scripts') diff --git a/scripts/gendwarfksyms/dwarf.c b/scripts/gendwarfksyms/dwarf.c index 6b30e45a4e82..364ff4892d5c 100644 --- a/scripts/gendwarfksyms/dwarf.c +++ b/scripts/gendwarfksyms/dwarf.c @@ -146,6 +146,8 @@ static void process(struct die *cache, const char *s) if (dump_dies) fputs(s, stderr); + if (cache) + die_debug_r("cache %p string '%s'", cache, s); die_map_add_string(cache, s); } @@ -594,6 +596,8 @@ static void process_cached(struct state *state, struct die *cache, list_for_each_entry(df, &cache->fragments, list) { switch (df->type) { case FRAGMENT_STRING: + die_debug_b("cache %p STRING '%s'", cache, + df->data.str); process(NULL, df->data.str); break; case FRAGMENT_LINEBREAK: @@ -603,6 +607,8 @@ static void process_cached(struct state *state, struct die *cache, if (!dwarf_die_addr_die(dwarf_cu_getdwarf(die->cu), (void *)df->data.addr, &child)) error("dwarf_die_addr_die failed"); + die_debug_b("cache %p DIE addr %" PRIxPTR " tag %x", + cache, df->data.addr, dwarf_tag(&child)); check(process_type(state, NULL, &child)); break; default: @@ -671,6 +677,9 @@ static int process_type(struct state *state, struct die *parent, Dwarf_Die *die) cache = die_map_get(die, want_state); if (cache->state == want_state) { + die_debug_g("cached addr %p tag %x -- %s", die->addr, tag, + die_state_name(cache->state)); + process_cached(state, cache, die); die_map_add_die(parent, cache); @@ -678,6 +687,9 @@ static int process_type(struct state *state, struct die *parent, Dwarf_Die *die) return 0; } + die_debug_g("addr %p tag %x -- %s -> %s", die->addr, tag, + die_state_name(cache->state), die_state_name(want_state)); + switch (tag) { /* Type modifiers */ PROCESS_TYPE(atomic) @@ -713,6 +725,9 @@ static int process_type(struct state *state, struct die *parent, Dwarf_Die *die) error("unexpected type: %x", tag); } + die_debug_r("parent %p cache %p die addr %p tag %x", parent, cache, + die->addr, tag); + /* Update cache state and append to the parent (if any) */ cache->tag = tag; cache->state = want_state; diff --git a/scripts/gendwarfksyms/gendwarfksyms.c b/scripts/gendwarfksyms/gendwarfksyms.c index 3809db840c06..bf282e33e00c 100644 --- a/scripts/gendwarfksyms/gendwarfksyms.c +++ b/scripts/gendwarfksyms/gendwarfksyms.c @@ -19,6 +19,8 @@ int debug; /* Dump DIE contents */ int dump_dies; +/* Print debugging information about die_map changes */ +int dump_die_map; static void usage(void) { @@ -26,6 +28,7 @@ static void usage(void) "Options:\n" " -d, --debug Print debugging information\n" " --dump-dies Dump DWARF DIE contents\n" + " --dump-die-map Print debugging information about die_map changes\n" " -h, --help Print this message\n" "\n", stderr); @@ -75,6 +78,7 @@ int main(int argc, char **argv) static const struct option opts[] = { { "debug", 0, NULL, 'd' }, { "dump-dies", 0, &dump_dies, 1 }, + { "dump-die-map", 0, &dump_die_map, 1 }, { "help", 0, NULL, 'h' }, { 0, 0, NULL, 0 } }; @@ -95,6 +99,9 @@ int main(int argc, char **argv) } } + if (dump_die_map) + dump_dies = 1; + if (optind >= argc) { usage(); error("no input files?"); diff --git a/scripts/gendwarfksyms/gendwarfksyms.h b/scripts/gendwarfksyms/gendwarfksyms.h index 941c4134da8e..251832dac599 100644 --- a/scripts/gendwarfksyms/gendwarfksyms.h +++ b/scripts/gendwarfksyms/gendwarfksyms.h @@ -21,6 +21,7 @@ */ extern int debug; extern int dump_dies; +extern int dump_die_map; /* * Output helpers @@ -43,6 +44,18 @@ extern int dump_dies; exit(1); \ } while (0) +#define __die_debug(color, format, ...) \ + do { \ + if (dump_dies && dump_die_map) \ + fprintf(stderr, \ + "\033[" #color "m<" format ">\033[39m", \ + __VA_ARGS__); \ + } while (0) + +#define die_debug_r(format, ...) __die_debug(91, format, __VA_ARGS__) +#define die_debug_g(format, ...) __die_debug(92, format, __VA_ARGS__) +#define die_debug_b(format, ...) __die_debug(94, format, __VA_ARGS__) + /* * Error handling helpers */ -- cgit v1.2.3 From ab4439981f8549b013f4ea0b274b7c77c88ab4bc Mon Sep 17 00:00:00 2001 From: Sami Tolvanen <samitolvanen@google.com> Date: Fri, 3 Jan 2025 20:45:33 +0000 Subject: gendwarfksyms: Add symtypes output Add support for producing genksyms-style symtypes files. Process die_map to find the longest expansions for each type, and use symtypes references in type definitions. The basic file format is similar to genksyms, with two notable exceptions: 1. Type names with spaces (common with Rust) in references are wrapped in single quotes. E.g.: s#'core::result::Result<u8, core::num::error::ParseIntError>' 2. The actual type definition is the simple parsed DWARF format we output with --dump-dies, not the preprocessed C-style format genksyms produces. Signed-off-by: Sami Tolvanen <samitolvanen@google.com> Reviewed-by: Petr Pavlu <petr.pavlu@suse.com> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> --- scripts/gendwarfksyms/Makefile | 1 + scripts/gendwarfksyms/die.c | 11 ++ scripts/gendwarfksyms/dwarf.c | 1 + scripts/gendwarfksyms/gendwarfksyms.c | 33 +++- scripts/gendwarfksyms/gendwarfksyms.h | 19 ++ scripts/gendwarfksyms/symbols.c | 4 +- scripts/gendwarfksyms/types.c | 363 ++++++++++++++++++++++++++++++++++ 7 files changed, 429 insertions(+), 3 deletions(-) create mode 100644 scripts/gendwarfksyms/types.c (limited to 'scripts') diff --git a/scripts/gendwarfksyms/Makefile b/scripts/gendwarfksyms/Makefile index c06145d84df8..6540282dc746 100644 --- a/scripts/gendwarfksyms/Makefile +++ b/scripts/gendwarfksyms/Makefile @@ -6,5 +6,6 @@ gendwarfksyms-objs += cache.o gendwarfksyms-objs += die.o gendwarfksyms-objs += dwarf.o gendwarfksyms-objs += symbols.o +gendwarfksyms-objs += types.o HOSTLDLIBS_gendwarfksyms := -ldw -lelf diff --git a/scripts/gendwarfksyms/die.c b/scripts/gendwarfksyms/die.c index 0d70e02d02b5..66bd4c9bc952 100644 --- a/scripts/gendwarfksyms/die.c +++ b/scripts/gendwarfksyms/die.c @@ -22,6 +22,7 @@ static inline unsigned int die_hash(uintptr_t addr, enum die_state state) static void init_die(struct die *cd) { cd->state = DIE_INCOMPLETE; + cd->mapped = false; cd->fqn = NULL; cd->tag = -1; cd->addr = 0; @@ -83,6 +84,16 @@ static void reset_die(struct die *cd) init_die(cd); } +void die_map_for_each(die_map_callback_t func, void *arg) +{ + struct hlist_node *tmp; + struct die *cd; + + hash_for_each_safe(die_map, cd, tmp, hash) { + func(cd, arg); + } +} + void die_map_free(void) { struct hlist_node *tmp; diff --git a/scripts/gendwarfksyms/dwarf.c b/scripts/gendwarfksyms/dwarf.c index 364ff4892d5c..a9966a23167a 100644 --- a/scripts/gendwarfksyms/dwarf.c +++ b/scripts/gendwarfksyms/dwarf.c @@ -745,6 +745,7 @@ static void process_symbol(struct state *state, Dwarf_Die *die, { debug("%s", state->sym->name); check(process_func(state, NULL, die)); + state->sym->state = SYMBOL_MAPPED; if (dump_dies) fputs("\n", stderr); } diff --git a/scripts/gendwarfksyms/gendwarfksyms.c b/scripts/gendwarfksyms/gendwarfksyms.c index bf282e33e00c..1d30f42cbd14 100644 --- a/scripts/gendwarfksyms/gendwarfksyms.c +++ b/scripts/gendwarfksyms/gendwarfksyms.c @@ -21,6 +21,11 @@ int debug; int dump_dies; /* Print debugging information about die_map changes */ int dump_die_map; +/* Print out type strings (i.e. type_map) */ +int dump_types; +/* Write a symtypes file */ +int symtypes; +static const char *symtypes_file; static void usage(void) { @@ -29,6 +34,8 @@ static void usage(void) " -d, --debug Print debugging information\n" " --dump-dies Dump DWARF DIE contents\n" " --dump-die-map Print debugging information about die_map changes\n" + " --dump-types Dump type strings\n" + " -T, --symtypes file Write a symtypes file\n" " -h, --help Print this message\n" "\n", stderr); @@ -41,6 +48,7 @@ static int process_module(Dwfl_Module *mod, void **userdata, const char *name, Dwarf_Die cudie; Dwarf_CU *cu = NULL; Dwarf *dbg; + FILE *symfile = arg; int res; debug("%s", name); @@ -60,6 +68,10 @@ static int process_module(Dwfl_Module *mod, void **userdata, const char *name, process_cu(&cudie); } while (cu); + /* + * Use die_map to expand type strings and write them to `symfile`. + */ + generate_symtypes(symfile); die_map_free(); return DWARF_CB_OK; @@ -72,6 +84,7 @@ static const Dwfl_Callbacks callbacks = { int main(int argc, char **argv) { + FILE *symfile = NULL; unsigned int n; int opt; @@ -79,17 +92,23 @@ int main(int argc, char **argv) { "debug", 0, NULL, 'd' }, { "dump-dies", 0, &dump_dies, 1 }, { "dump-die-map", 0, &dump_die_map, 1 }, + { "dump-types", 0, &dump_types, 1 }, + { "symtypes", 1, NULL, 'T' }, { "help", 0, NULL, 'h' }, { 0, 0, NULL, 0 } }; - while ((opt = getopt_long(argc, argv, "dh", opts, NULL)) != EOF) { + while ((opt = getopt_long(argc, argv, "dT:h", opts, NULL)) != EOF) { switch (opt) { case 0: break; case 'd': debug = 1; break; + case 'T': + symtypes = 1; + symtypes_file = optarg; + break; case 'h': usage(); return 0; @@ -109,6 +128,13 @@ int main(int argc, char **argv) symbol_read_exports(stdin); + if (symtypes_file) { + symfile = fopen(symtypes_file, "w"); + if (!symfile) + error("fopen failed for '%s': %s", symtypes_file, + strerror(errno)); + } + for (n = optind; n < argc; n++) { Dwfl *dwfl; int fd; @@ -131,12 +157,15 @@ int main(int argc, char **argv) dwfl_report_end(dwfl, NULL, NULL); - if (dwfl_getmodules(dwfl, &process_module, NULL, 0)) + if (dwfl_getmodules(dwfl, &process_module, symfile, 0)) error("dwfl_getmodules failed for '%s'", argv[n]); dwfl_end(dwfl); } + if (symfile) + check(fclose(symfile)); + symbol_free(); return 0; diff --git a/scripts/gendwarfksyms/gendwarfksyms.h b/scripts/gendwarfksyms/gendwarfksyms.h index 251832dac599..98d5b2315f21 100644 --- a/scripts/gendwarfksyms/gendwarfksyms.h +++ b/scripts/gendwarfksyms/gendwarfksyms.h @@ -22,6 +22,8 @@ extern int debug; extern int dump_dies; extern int dump_die_map; +extern int dump_types; +extern int symtypes; /* * Output helpers @@ -90,6 +92,11 @@ static inline unsigned int addr_hash(uintptr_t addr) return hash_ptr((const void *)addr); } +enum symbol_state { + SYMBOL_UNPROCESSED, + SYMBOL_MAPPED, +}; + struct symbol_addr { uint32_t section; Elf64_Addr address; @@ -100,6 +107,8 @@ struct symbol { struct symbol_addr addr; struct hlist_node addr_hash; struct hlist_node name_hash; + enum symbol_state state; + uintptr_t die_addr; }; typedef void (*symbol_callback_t)(struct symbol *, void *arg); @@ -154,6 +163,7 @@ static inline const char *die_state_name(enum die_state state) struct die { enum die_state state; + bool mapped; char *fqn; int tag; uintptr_t addr; @@ -161,10 +171,13 @@ struct die { struct hlist_node hash; }; +typedef void (*die_map_callback_t)(struct die *, void *arg); + int __die_map_get(uintptr_t addr, enum die_state state, struct die **res); struct die *die_map_get(Dwarf_Die *die, enum die_state state); void die_map_add_string(struct die *pd, const char *str); void die_map_add_linebreak(struct die *pd, int linebreak); +void die_map_for_each(die_map_callback_t func, void *arg); void die_map_add_die(struct die *pd, struct die *child); void die_map_free(void); @@ -235,4 +248,10 @@ int process_die_container(struct state *state, struct die *cache, void process_cu(Dwarf_Die *cudie); +/* + * types.c + */ + +void generate_symtypes(FILE *file); + #endif /* __GENDWARFKSYMS_H */ diff --git a/scripts/gendwarfksyms/symbols.c b/scripts/gendwarfksyms/symbols.c index 98febb524dd5..0d2ce7284a53 100644 --- a/scripts/gendwarfksyms/symbols.c +++ b/scripts/gendwarfksyms/symbols.c @@ -92,6 +92,7 @@ void symbol_read_exports(FILE *file) sym = xcalloc(1, sizeof(struct symbol)); sym->name = name; sym->addr.section = SHN_UNDEF; + sym->state = SYMBOL_UNPROCESSED; hash_add(symbol_names, &sym->name_hash, hash_str(sym->name)); ++nsym; @@ -107,7 +108,8 @@ static void get_symbol(struct symbol *sym, void *arg) { struct symbol **res = arg; - *res = sym; + if (sym->state == SYMBOL_UNPROCESSED) + *res = sym; } struct symbol *symbol_get(const char *name) diff --git a/scripts/gendwarfksyms/types.c b/scripts/gendwarfksyms/types.c new file mode 100644 index 000000000000..21d7a34228eb --- /dev/null +++ b/scripts/gendwarfksyms/types.c @@ -0,0 +1,363 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Copyright (C) 2024 Google LLC + */ + +#define _GNU_SOURCE +#include <inttypes.h> +#include <stdio.h> + +#include "gendwarfksyms.h" + +static struct cache expansion_cache; + +/* + * A simple linked list of shared or owned strings to avoid copying strings + * around when not necessary. + */ +struct type_list_entry { + const char *str; + void *owned; + struct list_head list; +}; + +static void type_list_free(struct list_head *list) +{ + struct type_list_entry *entry; + struct type_list_entry *tmp; + + list_for_each_entry_safe(entry, tmp, list, list) { + if (entry->owned) + free(entry->owned); + free(entry); + } + + INIT_LIST_HEAD(list); +} + +static int type_list_append(struct list_head *list, const char *s, void *owned) +{ + struct type_list_entry *entry; + + if (!s) + return 0; + + entry = xmalloc(sizeof(struct type_list_entry)); + entry->str = s; + entry->owned = owned; + list_add_tail(&entry->list, list); + + return strlen(entry->str); +} + +static void type_list_write(struct list_head *list, FILE *file) +{ + struct type_list_entry *entry; + + list_for_each_entry(entry, list, list) { + if (entry->str) + checkp(fputs(entry->str, file)); + } +} + +/* + * An expanded type string in symtypes format. + */ +struct type_expansion { + char *name; + size_t len; + struct list_head expanded; + struct hlist_node hash; +}; + +static void type_expansion_init(struct type_expansion *type) +{ + type->name = NULL; + type->len = 0; + INIT_LIST_HEAD(&type->expanded); +} + +static inline void type_expansion_free(struct type_expansion *type) +{ + free(type->name); + type->name = NULL; + type->len = 0; + type_list_free(&type->expanded); +} + +static void type_expansion_append(struct type_expansion *type, const char *s, + void *owned) +{ + type->len += type_list_append(&type->expanded, s, owned); +} + +/* + * type_map -- the longest expansions for each type. + * + * const char *name -> struct type_expansion * + */ +#define TYPE_HASH_BITS 12 +static HASHTABLE_DEFINE(type_map, 1 << TYPE_HASH_BITS); + +static int type_map_get(const char *name, struct type_expansion **res) +{ + struct type_expansion *e; + + hash_for_each_possible(type_map, e, hash, hash_str(name)) { + if (!strcmp(name, e->name)) { + *res = e; + return 0; + } + } + + return -1; +} + +static void type_map_add(const char *name, struct type_expansion *type) +{ + struct type_expansion *e; + + if (type_map_get(name, &e)) { + e = xmalloc(sizeof(struct type_expansion)); + type_expansion_init(e); + e->name = xstrdup(name); + + hash_add(type_map, &e->hash, hash_str(e->name)); + + if (dump_types) + debug("adding %s", e->name); + } else { + /* Use the longest available expansion */ + if (type->len <= e->len) + return; + + type_list_free(&e->expanded); + + if (dump_types) + debug("replacing %s", e->name); + } + + /* Take ownership of type->expanded */ + list_replace_init(&type->expanded, &e->expanded); + e->len = type->len; + + if (dump_types) { + checkp(fputs(e->name, stderr)); + checkp(fputs(" ", stderr)); + type_list_write(&e->expanded, stderr); + checkp(fputs("\n", stderr)); + } +} + +static void type_map_write(FILE *file) +{ + struct type_expansion *e; + struct hlist_node *tmp; + + if (!file) + return; + + hash_for_each_safe(type_map, e, tmp, hash) { + checkp(fputs(e->name, file)); + checkp(fputs(" ", file)); + type_list_write(&e->expanded, file); + checkp(fputs("\n", file)); + } +} + +static void type_map_free(void) +{ + struct type_expansion *e; + struct hlist_node *tmp; + + hash_for_each_safe(type_map, e, tmp, hash) { + type_expansion_free(e); + free(e); + } + + hash_init(type_map); +} + +/* + * Type reference format: <prefix>#<name>, where prefix: + * s -> structure + * u -> union + * e -> enum + * t -> typedef + * + * Names with spaces are additionally wrapped in single quotes. + */ +static char get_type_prefix(int tag) +{ + switch (tag) { + case DW_TAG_class_type: + case DW_TAG_structure_type: + return 's'; + case DW_TAG_union_type: + return 'u'; + case DW_TAG_enumeration_type: + return 'e'; + case DW_TAG_typedef_type: + return 't'; + default: + return 0; + } +} + +static char *get_type_name(struct die *cache) +{ + const char *quote; + char prefix; + char *name; + + if (cache->state == DIE_INCOMPLETE) { + warn("found incomplete cache entry: %p", cache); + return NULL; + } + if (!cache->fqn || !*cache->fqn) + return NULL; + + prefix = get_type_prefix(cache->tag); + if (!prefix) + return NULL; + + /* Wrap names with spaces in single quotes */ + quote = strstr(cache->fqn, " ") ? "'" : ""; + + /* <prefix>#<type_name>\0 */ + if (asprintf(&name, "%c#%s%s%s", prefix, quote, cache->fqn, quote) < 0) + error("asprintf failed for '%s'", cache->fqn); + + return name; +} + +static void __type_expand(struct die *cache, struct type_expansion *type, + bool recursive); + +static void type_expand_child(struct die *cache, struct type_expansion *type, + bool recursive) +{ + struct type_expansion child; + char *name; + + name = get_type_name(cache); + if (!name) { + __type_expand(cache, type, recursive); + return; + } + + if (recursive && !__cache_was_expanded(&expansion_cache, cache->addr)) { + __cache_mark_expanded(&expansion_cache, cache->addr); + type_expansion_init(&child); + __type_expand(cache, &child, true); + type_map_add(name, &child); + type_expansion_free(&child); + } + + type_expansion_append(type, name, name); +} + +static void __type_expand(struct die *cache, struct type_expansion *type, + bool recursive) +{ + struct die_fragment *df; + struct die *child; + + list_for_each_entry(df, &cache->fragments, list) { + switch (df->type) { + case FRAGMENT_STRING: + type_expansion_append(type, df->data.str, NULL); + break; + case FRAGMENT_DIE: + /* Use a complete die_map expansion if available */ + if (__die_map_get(df->data.addr, DIE_COMPLETE, + &child) && + __die_map_get(df->data.addr, DIE_UNEXPANDED, + &child)) + error("unknown child: %" PRIxPTR, + df->data.addr); + + type_expand_child(child, type, recursive); + break; + case FRAGMENT_LINEBREAK: + /* + * Keep whitespace in the symtypes format, but avoid + * repeated spaces. + */ + if (list_is_last(&df->list, &cache->fragments) || + list_next_entry(df, list)->type != + FRAGMENT_LINEBREAK) + type_expansion_append(type, " ", NULL); + break; + default: + error("empty die_fragment in %p", cache); + } + } +} + +static void type_expand(struct die *cache, struct type_expansion *type, + bool recursive) +{ + type_expansion_init(type); + __type_expand(cache, type, recursive); + cache_free(&expansion_cache); +} + +static void expand_type(struct die *cache, void *arg) +{ + struct type_expansion type; + char *name; + + if (cache->mapped) + return; + + cache->mapped = true; + + /* + * Skip unexpanded die_map entries if there's a complete + * expansion available for this DIE. + */ + if (cache->state == DIE_UNEXPANDED && + !__die_map_get(cache->addr, DIE_COMPLETE, &cache)) { + if (cache->mapped) + return; + + cache->mapped = true; + } + + name = get_type_name(cache); + if (!name) + return; + + debug("%s", name); + type_expand(cache, &type, true); + type_map_add(name, &type); + + type_expansion_free(&type); + free(name); +} + +void generate_symtypes(FILE *file) +{ + cache_init(&expansion_cache); + + /* + * die_map processing: + * + * 1. die_map contains all types referenced in exported symbol + * signatures, but can contain duplicates just like the original + * DWARF, and some references may not be fully expanded depending + * on how far we processed the DIE tree for that specific symbol. + * + * For each die_map entry, find the longest available expansion, + * and add it to type_map. + */ + die_map_for_each(expand_type, NULL); + + /* + * 2. If a symtypes file is requested, write type_map contents to + * the file. + */ + type_map_write(file); + type_map_free(); +} -- cgit v1.2.3 From 71378888018833a1cdcbf72f1e95d7c010542d8b Mon Sep 17 00:00:00 2001 From: Sami Tolvanen <samitolvanen@google.com> Date: Fri, 3 Jan 2025 20:45:34 +0000 Subject: gendwarfksyms: Add symbol versioning Calculate symbol versions from the fully expanded type strings in type_map, and output the versions in a genksyms-compatible format. Signed-off-by: Sami Tolvanen <samitolvanen@google.com> Reviewed-by: Petr Pavlu <petr.pavlu@suse.com> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> --- scripts/gendwarfksyms/Makefile | 2 +- scripts/gendwarfksyms/dwarf.c | 25 ++++++- scripts/gendwarfksyms/gendwarfksyms.c | 10 ++- scripts/gendwarfksyms/gendwarfksyms.h | 13 +++- scripts/gendwarfksyms/symbols.c | 53 +++++++++++++++ scripts/gendwarfksyms/types.c | 122 +++++++++++++++++++++++++++++++++- 6 files changed, 216 insertions(+), 9 deletions(-) (limited to 'scripts') diff --git a/scripts/gendwarfksyms/Makefile b/scripts/gendwarfksyms/Makefile index 6540282dc746..e889b958957b 100644 --- a/scripts/gendwarfksyms/Makefile +++ b/scripts/gendwarfksyms/Makefile @@ -8,4 +8,4 @@ gendwarfksyms-objs += dwarf.o gendwarfksyms-objs += symbols.o gendwarfksyms-objs += types.o -HOSTLDLIBS_gendwarfksyms := -ldw -lelf +HOSTLDLIBS_gendwarfksyms := -ldw -lelf -lz diff --git a/scripts/gendwarfksyms/dwarf.c b/scripts/gendwarfksyms/dwarf.c index a9966a23167a..bdf899d60707 100644 --- a/scripts/gendwarfksyms/dwarf.c +++ b/scripts/gendwarfksyms/dwarf.c @@ -740,12 +740,33 @@ static int process_type(struct state *state, struct die *parent, Dwarf_Die *die) /* * Exported symbol processing */ +static struct die *get_symbol_cache(struct state *state, Dwarf_Die *die) +{ + struct die *cache; + + cache = die_map_get(die, DIE_SYMBOL); + + if (cache->state != DIE_INCOMPLETE) + return NULL; /* We already processed a symbol for this DIE */ + + cache->tag = dwarf_tag(die); + return cache; +} + static void process_symbol(struct state *state, Dwarf_Die *die, die_callback_t process_func) { + struct die *cache; + + symbol_set_die(state->sym, die); + + cache = get_symbol_cache(state, die); + if (!cache) + return; + debug("%s", state->sym->name); - check(process_func(state, NULL, die)); - state->sym->state = SYMBOL_MAPPED; + check(process_func(state, cache, die)); + cache->state = DIE_SYMBOL; if (dump_dies) fputs("\n", stderr); } diff --git a/scripts/gendwarfksyms/gendwarfksyms.c b/scripts/gendwarfksyms/gendwarfksyms.c index 1d30f42cbd14..b0e13c37c6c2 100644 --- a/scripts/gendwarfksyms/gendwarfksyms.c +++ b/scripts/gendwarfksyms/gendwarfksyms.c @@ -23,6 +23,8 @@ int dump_dies; int dump_die_map; /* Print out type strings (i.e. type_map) */ int dump_types; +/* Print out expanded type strings used for symbol versions */ +int dump_versions; /* Write a symtypes file */ int symtypes; static const char *symtypes_file; @@ -35,6 +37,7 @@ static void usage(void) " --dump-dies Dump DWARF DIE contents\n" " --dump-die-map Print debugging information about die_map changes\n" " --dump-types Dump type strings\n" + " --dump-versions Dump expanded type strings used for symbol versions\n" " -T, --symtypes file Write a symtypes file\n" " -h, --help Print this message\n" "\n", @@ -69,9 +72,10 @@ static int process_module(Dwfl_Module *mod, void **userdata, const char *name, } while (cu); /* - * Use die_map to expand type strings and write them to `symfile`. + * Use die_map to expand type strings, write them to `symfile`, and + * calculate symbol versions. */ - generate_symtypes(symfile); + generate_symtypes_and_versions(symfile); die_map_free(); return DWARF_CB_OK; @@ -93,6 +97,7 @@ int main(int argc, char **argv) { "dump-dies", 0, &dump_dies, 1 }, { "dump-die-map", 0, &dump_die_map, 1 }, { "dump-types", 0, &dump_types, 1 }, + { "dump-versions", 0, &dump_versions, 1 }, { "symtypes", 1, NULL, 'T' }, { "help", 0, NULL, 'h' }, { 0, 0, NULL, 0 } @@ -166,6 +171,7 @@ int main(int argc, char **argv) if (symfile) check(fclose(symfile)); + symbol_print_versions(); symbol_free(); return 0; diff --git a/scripts/gendwarfksyms/gendwarfksyms.h b/scripts/gendwarfksyms/gendwarfksyms.h index 98d5b2315f21..203534abcd35 100644 --- a/scripts/gendwarfksyms/gendwarfksyms.h +++ b/scripts/gendwarfksyms/gendwarfksyms.h @@ -23,6 +23,7 @@ extern int debug; extern int dump_dies; extern int dump_die_map; extern int dump_types; +extern int dump_versions; extern int symtypes; /* @@ -95,6 +96,7 @@ static inline unsigned int addr_hash(uintptr_t addr) enum symbol_state { SYMBOL_UNPROCESSED, SYMBOL_MAPPED, + SYMBOL_PROCESSED }; struct symbol_addr { @@ -109,6 +111,7 @@ struct symbol { struct hlist_node name_hash; enum symbol_state state; uintptr_t die_addr; + unsigned long crc; }; typedef void (*symbol_callback_t)(struct symbol *, void *arg); @@ -116,6 +119,10 @@ typedef void (*symbol_callback_t)(struct symbol *, void *arg); void symbol_read_exports(FILE *file); void symbol_read_symtab(int fd); struct symbol *symbol_get(const char *name); +void symbol_set_die(struct symbol *sym, Dwarf_Die *die); +void symbol_set_crc(struct symbol *sym, unsigned long crc); +void symbol_for_each(symbol_callback_t func, void *arg); +void symbol_print_versions(void); void symbol_free(void); /* @@ -126,7 +133,8 @@ enum die_state { DIE_INCOMPLETE, DIE_UNEXPANDED, DIE_COMPLETE, - DIE_LAST = DIE_COMPLETE + DIE_SYMBOL, + DIE_LAST = DIE_SYMBOL }; enum die_fragment_type { @@ -156,6 +164,7 @@ static inline const char *die_state_name(enum die_state state) CASE_CONST_TO_STR(DIE_INCOMPLETE) CASE_CONST_TO_STR(DIE_UNEXPANDED) CASE_CONST_TO_STR(DIE_COMPLETE) + CASE_CONST_TO_STR(DIE_SYMBOL) } error("unexpected die_state: %d", state); @@ -252,6 +261,6 @@ void process_cu(Dwarf_Die *cudie); * types.c */ -void generate_symtypes(FILE *file); +void generate_symtypes_and_versions(FILE *file); #endif /* __GENDWARFKSYMS_H */ diff --git a/scripts/gendwarfksyms/symbols.c b/scripts/gendwarfksyms/symbols.c index 0d2ce7284a53..4c499ba6c86d 100644 --- a/scripts/gendwarfksyms/symbols.c +++ b/scripts/gendwarfksyms/symbols.c @@ -66,6 +66,36 @@ static unsigned int for_each(const char *name, symbol_callback_t func, return 0; } +static void set_crc(struct symbol *sym, void *data) +{ + unsigned long *crc = data; + + if (sym->state == SYMBOL_PROCESSED && sym->crc != *crc) + warn("overriding version for symbol %s (crc %lx vs. %lx)", + sym->name, sym->crc, *crc); + + sym->state = SYMBOL_PROCESSED; + sym->crc = *crc; +} + +void symbol_set_crc(struct symbol *sym, unsigned long crc) +{ + if (for_each(sym->name, set_crc, &crc) == 0) + error("no matching symbols: '%s'", sym->name); +} + +static void set_die(struct symbol *sym, void *data) +{ + sym->die_addr = (uintptr_t)((Dwarf_Die *)data)->addr; + sym->state = SYMBOL_MAPPED; +} + +void symbol_set_die(struct symbol *sym, Dwarf_Die *die) +{ + if (for_each(sym->name, set_die, die) == 0) + error("no matching symbols: '%s'", sym->name); +} + static bool is_exported(const char *name) { return for_each(name, NULL, NULL) > 0; @@ -120,6 +150,16 @@ struct symbol *symbol_get(const char *name) return sym; } +void symbol_for_each(symbol_callback_t func, void *arg) +{ + struct hlist_node *tmp; + struct symbol *sym; + + hash_for_each_safe(symbol_names, sym, tmp, name_hash) { + func(sym, arg); + } +} + typedef void (*elf_symbol_callback_t)(const char *name, GElf_Sym *sym, Elf32_Word xndx, void *arg); @@ -246,6 +286,19 @@ void symbol_read_symtab(int fd) elf_for_each_global(fd, elf_set_symbol_addr, NULL); } +void symbol_print_versions(void) +{ + struct hlist_node *tmp; + struct symbol *sym; + + hash_for_each_safe(symbol_names, sym, tmp, name_hash) { + if (sym->state != SYMBOL_PROCESSED) + warn("no information for symbol %s", sym->name); + + printf("#SYMVER %s 0x%08lx\n", sym->name, sym->crc); + } +} + void symbol_free(void) { struct hlist_node *tmp; diff --git a/scripts/gendwarfksyms/types.c b/scripts/gendwarfksyms/types.c index 21d7a34228eb..6c03265f4d10 100644 --- a/scripts/gendwarfksyms/types.c +++ b/scripts/gendwarfksyms/types.c @@ -6,6 +6,7 @@ #define _GNU_SOURCE #include <inttypes.h> #include <stdio.h> +#include <zlib.h> #include "gendwarfksyms.h" @@ -178,6 +179,33 @@ static void type_map_free(void) hash_init(type_map); } +/* + * CRC for a type, with an optional fully expanded type string for + * debugging. + */ +struct version { + struct type_expansion type; + unsigned long crc; +}; + +static void version_init(struct version *version) +{ + version->crc = crc32(0, NULL, 0); + type_expansion_init(&version->type); +} + +static void version_free(struct version *version) +{ + type_expansion_free(&version->type); +} + +static void version_add(struct version *version, const char *s) +{ + version->crc = crc32(version->crc, (void *)s, strlen(s)); + if (dump_versions) + type_expansion_append(&version->type, s, NULL); +} + /* * Type reference format: <prefix>#<name>, where prefix: * s -> structure @@ -187,6 +215,12 @@ static void type_map_free(void) * * Names with spaces are additionally wrapped in single quotes. */ +static inline bool is_type_prefix(const char *s) +{ + return (s[0] == 's' || s[0] == 'u' || s[0] == 'e' || s[0] == 't') && + s[1] == '#'; +} + static char get_type_prefix(int tag) { switch (tag) { @@ -214,6 +248,8 @@ static char *get_type_name(struct die *cache) warn("found incomplete cache entry: %p", cache); return NULL; } + if (cache->state == DIE_SYMBOL) + return NULL; if (!cache->fqn || !*cache->fqn) return NULL; @@ -231,6 +267,39 @@ static char *get_type_name(struct die *cache) return name; } +static void __calculate_version(struct version *version, struct list_head *list) +{ + struct type_list_entry *entry; + struct type_expansion *e; + + /* Calculate a CRC over an expanded type string */ + list_for_each_entry(entry, list, list) { + if (is_type_prefix(entry->str)) { + check(type_map_get(entry->str, &e)); + + /* + * It's sufficient to expand each type reference just + * once to detect changes. + */ + if (cache_was_expanded(&expansion_cache, e)) { + version_add(version, entry->str); + } else { + cache_mark_expanded(&expansion_cache, e); + __calculate_version(version, &e->expanded); + } + } else { + version_add(version, entry->str); + } + } +} + +static void calculate_version(struct version *version, struct list_head *list) +{ + version_init(version); + __calculate_version(version, list); + cache_free(&expansion_cache); +} + static void __type_expand(struct die *cache, struct type_expansion *type, bool recursive); @@ -337,7 +406,49 @@ static void expand_type(struct die *cache, void *arg) free(name); } -void generate_symtypes(FILE *file) +static void expand_symbol(struct symbol *sym, void *arg) +{ + struct type_expansion type; + struct version version; + struct die *cache; + + /* + * No need to expand again unless we want a symtypes file entry + * for the symbol. Note that this means `sym` has the same address + * as another symbol that was already processed. + */ + if (!symtypes && sym->state == SYMBOL_PROCESSED) + return; + + if (__die_map_get(sym->die_addr, DIE_SYMBOL, &cache)) + return; /* We'll warn about missing CRCs later. */ + + type_expand(cache, &type, false); + + /* If the symbol already has a version, don't calculate it again. */ + if (sym->state != SYMBOL_PROCESSED) { + calculate_version(&version, &type.expanded); + symbol_set_crc(sym, version.crc); + debug("%s = %lx", sym->name, version.crc); + + if (dump_versions) { + checkp(fputs(sym->name, stderr)); + checkp(fputs(" ", stderr)); + type_list_write(&version.type.expanded, stderr); + checkp(fputs("\n", stderr)); + } + + version_free(&version); + } + + /* These aren't needed in type_map unless we want a symtypes file. */ + if (symtypes) + type_map_add(sym->name, &type); + + type_expansion_free(&type); +} + +void generate_symtypes_and_versions(FILE *file) { cache_init(&expansion_cache); @@ -355,7 +466,14 @@ void generate_symtypes(FILE *file) die_map_for_each(expand_type, NULL); /* - * 2. If a symtypes file is requested, write type_map contents to + * 2. For each exported symbol, expand the die_map type, and use + * type_map expansions to calculate a symbol version from the + * fully expanded type string. + */ + symbol_for_each(expand_symbol, NULL); + + /* + * 3. If a symtypes file is requested, write type_map contents to * the file. */ type_map_write(file); -- cgit v1.2.3 From 936cf61c3ef5d6dad714d6c01a85704027dddeb9 Mon Sep 17 00:00:00 2001 From: Sami Tolvanen <samitolvanen@google.com> Date: Fri, 3 Jan 2025 20:45:35 +0000 Subject: gendwarfksyms: Add support for kABI rules Distributions that want to maintain a stable kABI need the ability to make ABI compatible changes to kernel without affecting symbol versions, either because of LTS updates or backports. With genksyms, developers would typically hide these changes from version calculation with #ifndef __GENKSYMS__, which would result in the symbol version not changing even though the actual type has changed. When we process precompiled object files, this isn't an option. To support this use case, add a --stable command line flag that gates kABI stability features that are not needed in mainline kernels, but can be useful for distributions, and add support for kABI rules, which can be used to restrict gendwarfksyms output. The rules are specified as a set of null-terminated strings stored in the .discard.gendwarfksyms.kabi_rules section. Each rule consists of four strings as follows: "version\0type\0target\0value" The version string ensures the structure can be changed in a backwards compatible way. The type string indicates the type of the rule, and target and value strings contain rule-specific data. Initially support two simple rules: 1. Declaration-only types A type declaration can change into a full definition when additional includes are pulled in to the TU, which changes the versions of any symbol that references the type. Add support for defining declaration-only types whose definition is not expanded during versioning. 2. Ignored enumerators It's possible to add new enum fields without changing the ABI, but as the fields are included in symbol versioning, this would change the versions. Add support for ignoring specific fields. 3. Overridden enumerator values Add support for overriding enumerator values when calculating versions. This may be needed when the last field of the enum is used as a sentinel and new fields must be added before it. Add examples for using the rules under the examples/ directory. Signed-off-by: Sami Tolvanen <samitolvanen@google.com> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> --- scripts/gendwarfksyms/Makefile | 1 + scripts/gendwarfksyms/dwarf.c | 25 ++- scripts/gendwarfksyms/examples/kabi.h | 70 +++++++ scripts/gendwarfksyms/examples/kabi_ex.c | 14 ++ scripts/gendwarfksyms/examples/kabi_ex.h | 64 ++++++ scripts/gendwarfksyms/gendwarfksyms.c | 11 +- scripts/gendwarfksyms/gendwarfksyms.h | 14 ++ scripts/gendwarfksyms/kabi.c | 336 +++++++++++++++++++++++++++++++ 8 files changed, 531 insertions(+), 4 deletions(-) create mode 100644 scripts/gendwarfksyms/examples/kabi.h create mode 100644 scripts/gendwarfksyms/examples/kabi_ex.c create mode 100644 scripts/gendwarfksyms/examples/kabi_ex.h create mode 100644 scripts/gendwarfksyms/kabi.c (limited to 'scripts') diff --git a/scripts/gendwarfksyms/Makefile b/scripts/gendwarfksyms/Makefile index e889b958957b..6334c7d3c4d5 100644 --- a/scripts/gendwarfksyms/Makefile +++ b/scripts/gendwarfksyms/Makefile @@ -5,6 +5,7 @@ gendwarfksyms-objs += gendwarfksyms.o gendwarfksyms-objs += cache.o gendwarfksyms-objs += die.o gendwarfksyms-objs += dwarf.o +gendwarfksyms-objs += kabi.o gendwarfksyms-objs += symbols.o gendwarfksyms-objs += types.o diff --git a/scripts/gendwarfksyms/dwarf.c b/scripts/gendwarfksyms/dwarf.c index bdf899d60707..17f7e6b9a7ff 100644 --- a/scripts/gendwarfksyms/dwarf.c +++ b/scripts/gendwarfksyms/dwarf.c @@ -120,13 +120,16 @@ static bool is_definition_private(Dwarf_Die *die) return !!res; } -static bool is_kabi_definition(Dwarf_Die *die) +static bool is_kabi_definition(struct die *cache, Dwarf_Die *die) { bool value; if (get_flag_attr(die, DW_AT_declaration, &value) && value) return false; + if (kabi_is_declonly(cache->fqn)) + return false; + return !is_definition_private(die); } @@ -515,9 +518,10 @@ static void __process_structure_type(struct state *state, struct die *cache, process(cache, " {"); process_linebreak(cache, 1); - expand = state->expand.expand && is_kabi_definition(die); + expand = state->expand.expand && is_kabi_definition(cache, die); if (expand) { + state->expand.current_fqn = cache->fqn; check(process_die_container(state, cache, die, process_func, match_func)); } @@ -548,13 +552,26 @@ DEFINE_PROCESS_STRUCTURE_TYPE(union) static void process_enumerator_type(struct state *state, struct die *cache, Dwarf_Die *die) { + bool overridden = false; Dwarf_Word value; + if (stable) { + /* Get the fqn before we process anything */ + update_fqn(cache, die); + + if (kabi_is_enumerator_ignored(state->expand.current_fqn, + cache->fqn)) + return; + + overridden = kabi_get_enumerator_value( + state->expand.current_fqn, cache->fqn, &value); + } + process_list_comma(state, cache); process(cache, "enumerator"); process_fqn(cache, die); - if (get_udata_attr(die, DW_AT_const_value, &value)) { + if (overridden || get_udata_attr(die, DW_AT_const_value, &value)) { process(cache, " = "); process_fmt(cache, "%" PRIu64, value); } @@ -620,6 +637,7 @@ static void process_cached(struct state *state, struct die *cache, static void state_init(struct state *state) { state->expand.expand = true; + state->expand.current_fqn = NULL; cache_init(&state->expansion_cache); } @@ -627,6 +645,7 @@ static void expansion_state_restore(struct expansion_state *state, struct expansion_state *saved) { state->expand = saved->expand; + state->current_fqn = saved->current_fqn; } static void expansion_state_save(struct expansion_state *state, diff --git a/scripts/gendwarfksyms/examples/kabi.h b/scripts/gendwarfksyms/examples/kabi.h new file mode 100644 index 000000000000..fcd0300e5b58 --- /dev/null +++ b/scripts/gendwarfksyms/examples/kabi.h @@ -0,0 +1,70 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Copyright (C) 2024 Google LLC + * + * Example macros for maintaining kABI stability. + * + * This file is based on android_kabi.h, which has the following notice: + * + * Heavily influenced by rh_kabi.h which came from the RHEL/CENTOS kernel + * and was: + * Copyright (c) 2014 Don Zickus + * Copyright (c) 2015-2018 Jiri Benc + * Copyright (c) 2015 Sabrina Dubroca, Hannes Frederic Sowa + * Copyright (c) 2016-2018 Prarit Bhargava + * Copyright (c) 2017 Paolo Abeni, Larry Woodman + */ + +#ifndef __KABI_H__ +#define __KABI_H__ + +/* Kernel macros for userspace testing. */ +#ifndef __aligned +#define __aligned(x) __attribute__((__aligned__(x))) +#endif +#ifndef __used +#define __used __attribute__((__used__)) +#endif +#ifndef __section +#define __section(section) __attribute__((__section__(section))) +#endif +#ifndef __PASTE +#define ___PASTE(a, b) a##b +#define __PASTE(a, b) ___PASTE(a, b) +#endif +#ifndef __stringify +#define __stringify_1(x...) #x +#define __stringify(x...) __stringify_1(x) +#endif + +#define __KABI_RULE(hint, target, value) \ + static const char __PASTE(__gendwarfksyms_rule_, \ + __COUNTER__)[] __used __aligned(1) \ + __section(".discard.gendwarfksyms.kabi_rules") = \ + "1\0" #hint "\0" #target "\0" #value + +/* + * KABI_DECLONLY(fqn) + * Treat the struct/union/enum fqn as a declaration, i.e. even if + * a definition is available, don't expand the contents. + */ +#define KABI_DECLONLY(fqn) __KABI_RULE(declonly, fqn, ) + +/* + * KABI_ENUMERATOR_IGNORE(fqn, field) + * When expanding enum fqn, skip the provided field. This makes it + * possible to hide added enum fields from versioning. + */ +#define KABI_ENUMERATOR_IGNORE(fqn, field) \ + __KABI_RULE(enumerator_ignore, fqn field, ) + +/* + * KABI_ENUMERATOR_VALUE(fqn, field, value) + * When expanding enum fqn, use the provided value for the + * specified field. This makes it possible to override enumerator + * values when calculating versions. + */ +#define KABI_ENUMERATOR_VALUE(fqn, field, value) \ + __KABI_RULE(enumerator_value, fqn field, value) + +#endif /* __KABI_H__ */ diff --git a/scripts/gendwarfksyms/examples/kabi_ex.c b/scripts/gendwarfksyms/examples/kabi_ex.c new file mode 100644 index 000000000000..799552ea6679 --- /dev/null +++ b/scripts/gendwarfksyms/examples/kabi_ex.c @@ -0,0 +1,14 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * kabi_ex.c + * + * Copyright (C) 2024 Google LLC + * + * Examples for kABI stability features with --stable. See kabi_ex.h + * for details. + */ + +#include "kabi_ex.h" + +struct s e0; +enum e e1; diff --git a/scripts/gendwarfksyms/examples/kabi_ex.h b/scripts/gendwarfksyms/examples/kabi_ex.h new file mode 100644 index 000000000000..fca1e07c78e2 --- /dev/null +++ b/scripts/gendwarfksyms/examples/kabi_ex.h @@ -0,0 +1,64 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * kabi_ex.h + * + * Copyright (C) 2024 Google LLC + * + * Examples for kABI stability features with --stable. + */ + +/* + * The comments below each example contain the expected gendwarfksyms + * output, which can be verified using LLVM's FileCheck tool: + * + * https://llvm.org/docs/CommandGuide/FileCheck.html + * + * Usage: + * + * $ gcc -g -c examples/kabi_ex.c -o examples/kabi_ex.o + * + * $ nm examples/kabi_ex.o | awk '{ print $NF }' | \ + * ./gendwarfksyms --stable --dump-dies \ + * examples/kabi_ex.o 2>&1 >/dev/null | \ + * FileCheck examples/kabi_ex.h --check-prefix=STABLE + */ + +#ifndef __KABI_EX_H__ +#define __KABI_EX_H__ + +#include "kabi.h" + +/* + * Example: kABI rules + */ + +struct s { + int a; +}; + +KABI_DECLONLY(s); + +/* + * STABLE: variable structure_type s { + * STABLE-NEXT: } + */ + +enum e { + A, + B, + C, + D, +}; + +KABI_ENUMERATOR_IGNORE(e, B); +KABI_ENUMERATOR_IGNORE(e, C); +KABI_ENUMERATOR_VALUE(e, D, 123456789); + +/* + * STABLE: variable enumeration_type e { + * STABLE-NEXT: enumerator A = 0 , + * STABLE-NEXT: enumerator D = 123456789 + * STABLE-NEXT: } byte_size(4) + */ + +#endif /* __KABI_EX_H__ */ diff --git a/scripts/gendwarfksyms/gendwarfksyms.c b/scripts/gendwarfksyms/gendwarfksyms.c index b0e13c37c6c2..08ae61eb327e 100644 --- a/scripts/gendwarfksyms/gendwarfksyms.c +++ b/scripts/gendwarfksyms/gendwarfksyms.c @@ -25,6 +25,8 @@ int dump_die_map; int dump_types; /* Print out expanded type strings used for symbol versions */ int dump_versions; +/* Support kABI stability features */ +int stable; /* Write a symtypes file */ int symtypes; static const char *symtypes_file; @@ -38,6 +40,7 @@ static void usage(void) " --dump-die-map Print debugging information about die_map changes\n" " --dump-types Dump type strings\n" " --dump-versions Dump expanded type strings used for symbol versions\n" + " -s, --stable Support kABI stability features\n" " -T, --symtypes file Write a symtypes file\n" " -h, --help Print this message\n" "\n", @@ -98,18 +101,22 @@ int main(int argc, char **argv) { "dump-die-map", 0, &dump_die_map, 1 }, { "dump-types", 0, &dump_types, 1 }, { "dump-versions", 0, &dump_versions, 1 }, + { "stable", 0, NULL, 's' }, { "symtypes", 1, NULL, 'T' }, { "help", 0, NULL, 'h' }, { 0, 0, NULL, 0 } }; - while ((opt = getopt_long(argc, argv, "dT:h", opts, NULL)) != EOF) { + while ((opt = getopt_long(argc, argv, "dsT:h", opts, NULL)) != EOF) { switch (opt) { case 0: break; case 'd': debug = 1; break; + case 's': + stable = 1; + break; case 'T': symtypes = 1; symtypes_file = optarg; @@ -150,6 +157,7 @@ int main(int argc, char **argv) strerror(errno)); symbol_read_symtab(fd); + kabi_read_rules(fd); dwfl = dwfl_begin(&callbacks); if (!dwfl) @@ -166,6 +174,7 @@ int main(int argc, char **argv) error("dwfl_getmodules failed for '%s'", argv[n]); dwfl_end(dwfl); + kabi_free(); } if (symfile) diff --git a/scripts/gendwarfksyms/gendwarfksyms.h b/scripts/gendwarfksyms/gendwarfksyms.h index 203534abcd35..c0207ca10e19 100644 --- a/scripts/gendwarfksyms/gendwarfksyms.h +++ b/scripts/gendwarfksyms/gendwarfksyms.h @@ -24,6 +24,7 @@ extern int dump_dies; extern int dump_die_map; extern int dump_types; extern int dump_versions; +extern int stable; extern int symtypes; /* @@ -232,6 +233,7 @@ static inline bool cache_was_expanded(struct cache *cache, void *addr) struct expansion_state { bool expand; + const char *current_fqn; }; struct state { @@ -263,4 +265,16 @@ void process_cu(Dwarf_Die *cudie); void generate_symtypes_and_versions(FILE *file); +/* + * kabi.c + */ + +bool kabi_is_enumerator_ignored(const char *fqn, const char *field); +bool kabi_get_enumerator_value(const char *fqn, const char *field, + unsigned long *value); +bool kabi_is_declonly(const char *fqn); + +void kabi_read_rules(int fd); +void kabi_free(void); + #endif /* __GENDWARFKSYMS_H */ diff --git a/scripts/gendwarfksyms/kabi.c b/scripts/gendwarfksyms/kabi.c new file mode 100644 index 000000000000..66f01fcd1607 --- /dev/null +++ b/scripts/gendwarfksyms/kabi.c @@ -0,0 +1,336 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Copyright (C) 2024 Google LLC + */ + +#define _GNU_SOURCE +#include <errno.h> +#include <stdio.h> + +#include "gendwarfksyms.h" + +#define KABI_RULE_SECTION ".discard.gendwarfksyms.kabi_rules" +#define KABI_RULE_VERSION "1" + +/* + * The rule section consists of four null-terminated strings per + * entry: + * + * 1. version + * Entry format version. Must match KABI_RULE_VERSION. + * + * 2. type + * Type of the kABI rule. Must be one of the tags defined below. + * + * 3. target + * Rule-dependent target, typically the fully qualified name of + * the target DIE. + * + * 4. value + * Rule-dependent value. + */ +#define KABI_RULE_MIN_ENTRY_SIZE \ + (/* version\0 */ 2 + /* type\0 */ 2 + /* target\0" */ 1 + \ + /* value\0 */ 1) +#define KABI_RULE_EMPTY_VALUE "" + +/* + * Rule: declonly + * - For the struct/enum/union in the target field, treat it as a + * declaration only even if a definition is available. + */ +#define KABI_RULE_TAG_DECLONLY "declonly" + +/* + * Rule: enumerator_ignore + * - For the enum_field in the target field, ignore the enumerator. + */ +#define KABI_RULE_TAG_ENUMERATOR_IGNORE "enumerator_ignore" + +/* + * Rule: enumerator_value + * - For the fqn_field in the target field, set the value to the + * unsigned integer in the value field. + */ +#define KABI_RULE_TAG_ENUMERATOR_VALUE "enumerator_value" + +enum kabi_rule_type { + KABI_RULE_TYPE_UNKNOWN, + KABI_RULE_TYPE_DECLONLY, + KABI_RULE_TYPE_ENUMERATOR_IGNORE, + KABI_RULE_TYPE_ENUMERATOR_VALUE, +}; + +#define RULE_HASH_BITS 7 + +struct rule { + enum kabi_rule_type type; + const char *target; + const char *value; + struct hlist_node hash; +}; + +/* { type, target } -> struct rule */ +static HASHTABLE_DEFINE(rules, 1 << RULE_HASH_BITS); + +static inline unsigned int rule_values_hash(enum kabi_rule_type type, + const char *target) +{ + return hash_32(type) ^ hash_str(target); +} + +static inline unsigned int rule_hash(const struct rule *rule) +{ + return rule_values_hash(rule->type, rule->target); +} + +static inline const char *get_rule_field(const char **pos, ssize_t *left) +{ + const char *start = *pos; + size_t len; + + if (*left <= 0) + error("unexpected end of kABI rules"); + + len = strnlen(start, *left) + 1; + *pos += len; + *left -= len; + + return start; +} + +void kabi_read_rules(int fd) +{ + GElf_Shdr shdr_mem; + GElf_Shdr *shdr; + Elf_Data *rule_data = NULL; + Elf_Scn *scn; + Elf *elf; + size_t shstrndx; + const char *rule_str; + ssize_t left; + int i; + + const struct { + enum kabi_rule_type type; + const char *tag; + } rule_types[] = { + { + .type = KABI_RULE_TYPE_DECLONLY, + .tag = KABI_RULE_TAG_DECLONLY, + }, + { + .type = KABI_RULE_TYPE_ENUMERATOR_IGNORE, + .tag = KABI_RULE_TAG_ENUMERATOR_IGNORE, + }, + { + .type = KABI_RULE_TYPE_ENUMERATOR_VALUE, + .tag = KABI_RULE_TAG_ENUMERATOR_VALUE, + }, + }; + + if (!stable) + return; + + if (elf_version(EV_CURRENT) != EV_CURRENT) + error("elf_version failed: %s", elf_errmsg(-1)); + + elf = elf_begin(fd, ELF_C_READ_MMAP, NULL); + if (!elf) + error("elf_begin failed: %s", elf_errmsg(-1)); + + if (elf_getshdrstrndx(elf, &shstrndx) < 0) + error("elf_getshdrstrndx failed: %s", elf_errmsg(-1)); + + scn = elf_nextscn(elf, NULL); + + while (scn) { + const char *sname; + + shdr = gelf_getshdr(scn, &shdr_mem); + if (!shdr) + error("gelf_getshdr failed: %s", elf_errmsg(-1)); + + sname = elf_strptr(elf, shstrndx, shdr->sh_name); + if (!sname) + error("elf_strptr failed: %s", elf_errmsg(-1)); + + if (!strcmp(sname, KABI_RULE_SECTION)) { + rule_data = elf_getdata(scn, NULL); + if (!rule_data) + error("elf_getdata failed: %s", elf_errmsg(-1)); + break; + } + + scn = elf_nextscn(elf, scn); + } + + if (!rule_data) { + debug("kABI rules not found"); + check(elf_end(elf)); + return; + } + + rule_str = rule_data->d_buf; + left = shdr->sh_size; + + if (left < KABI_RULE_MIN_ENTRY_SIZE) + error("kABI rule section too small: %zd bytes", left); + + if (rule_str[left - 1] != '\0') + error("kABI rules are not null-terminated"); + + while (left > KABI_RULE_MIN_ENTRY_SIZE) { + enum kabi_rule_type type = KABI_RULE_TYPE_UNKNOWN; + const char *field; + struct rule *rule; + + /* version */ + field = get_rule_field(&rule_str, &left); + + if (strcmp(field, KABI_RULE_VERSION)) + error("unsupported kABI rule version: '%s'", field); + + /* type */ + field = get_rule_field(&rule_str, &left); + + for (i = 0; i < ARRAY_SIZE(rule_types); i++) { + if (!strcmp(field, rule_types[i].tag)) { + type = rule_types[i].type; + break; + } + } + + if (type == KABI_RULE_TYPE_UNKNOWN) + error("unsupported kABI rule type: '%s'", field); + + rule = xmalloc(sizeof(struct rule)); + + rule->type = type; + rule->target = xstrdup(get_rule_field(&rule_str, &left)); + rule->value = xstrdup(get_rule_field(&rule_str, &left)); + + hash_add(rules, &rule->hash, rule_hash(rule)); + + debug("kABI rule: type: '%s', target: '%s', value: '%s'", field, + rule->target, rule->value); + } + + if (left > 0) + warn("unexpected data at the end of the kABI rules section"); + + check(elf_end(elf)); +} + +bool kabi_is_declonly(const char *fqn) +{ + struct rule *rule; + + if (!stable) + return false; + if (!fqn || !*fqn) + return false; + + hash_for_each_possible(rules, rule, hash, + rule_values_hash(KABI_RULE_TYPE_DECLONLY, fqn)) { + if (rule->type == KABI_RULE_TYPE_DECLONLY && + !strcmp(fqn, rule->target)) + return true; + } + + return false; +} + +static char *get_enumerator_target(const char *fqn, const char *field) +{ + char *target = NULL; + + if (asprintf(&target, "%s %s", fqn, field) < 0) + error("asprintf failed for '%s %s'", fqn, field); + + return target; +} + +static unsigned long get_ulong_value(const char *value) +{ + unsigned long result = 0; + char *endptr = NULL; + + errno = 0; + result = strtoul(value, &endptr, 10); + + if (errno || *endptr) + error("invalid unsigned value '%s'", value); + + return result; +} + +bool kabi_is_enumerator_ignored(const char *fqn, const char *field) +{ + bool match = false; + struct rule *rule; + char *target; + + if (!stable) + return false; + if (!fqn || !*fqn || !field || !*field) + return false; + + target = get_enumerator_target(fqn, field); + + hash_for_each_possible( + rules, rule, hash, + rule_values_hash(KABI_RULE_TYPE_ENUMERATOR_IGNORE, target)) { + if (rule->type == KABI_RULE_TYPE_ENUMERATOR_IGNORE && + !strcmp(target, rule->target)) { + match = true; + break; + } + } + + free(target); + return match; +} + +bool kabi_get_enumerator_value(const char *fqn, const char *field, + unsigned long *value) +{ + bool match = false; + struct rule *rule; + char *target; + + if (!stable) + return false; + if (!fqn || !*fqn || !field || !*field) + return false; + + target = get_enumerator_target(fqn, field); + + hash_for_each_possible(rules, rule, hash, + rule_values_hash(KABI_RULE_TYPE_ENUMERATOR_VALUE, + target)) { + if (rule->type == KABI_RULE_TYPE_ENUMERATOR_VALUE && + !strcmp(target, rule->target)) { + *value = get_ulong_value(rule->value); + match = true; + break; + } + } + + free(target); + return match; +} + +void kabi_free(void) +{ + struct hlist_node *tmp; + struct rule *rule; + + hash_for_each_safe(rules, rule, tmp, hash) { + free((void *)rule->target); + free((void *)rule->value); + free(rule); + } + + hash_init(rules); +} -- cgit v1.2.3 From a93694188127a5f7ba3baa2f98b275ce388a5246 Mon Sep 17 00:00:00 2001 From: Sami Tolvanen <samitolvanen@google.com> Date: Fri, 3 Jan 2025 20:45:36 +0000 Subject: gendwarfksyms: Add support for reserved and ignored fields Distributions that want to maintain a stable kABI need the ability to make ABI compatible changes to kernel data structures without affecting symbol versions, either because of LTS updates or backports. With genksyms, developers would typically hide these changes from version calculation with #ifndef __GENKSYMS__, which would result in the symbol version not changing even though the actual type has changed. When we process precompiled object files, this isn't an option. Change union processing to recognize field name prefixes that allow the user to ignore the union completely during symbol versioning with a __kabi_ignored prefix in a field name, or to replace the type of a placeholder field using a __kabi_reserved field name prefix. For example, assume we want to add a new field to an existing alignment hole in a data structure, and ignore the new field when calculating symbol versions: struct struct1 { int a; /* a 4-byte alignment hole */ unsigned long b; }; To add `int n` to the alignment hole, we can add a union that includes a __kabi_ignored field that causes gendwarfksyms to ignore the entire union: struct struct1 { int a; union { char __kabi_ignored_0; int n; }; unsigned long b; }; With --stable, both structs produce the same symbol version. Alternatively, when a distribution expects future modification to a data structure, they can explicitly add reserved fields: struct struct2 { long a; long __kabi_reserved_0; /* reserved for future use */ }; To take the field into use, we can again replace it with a union, with one of the fields keeping the __kabi_reserved name prefix to indicate the original type: struct struct2 { long a; union { long __kabi_reserved_0; struct { int b; int v; }; }; Here gendwarfksyms --stable replaces the union with the type of the placeholder field when calculating versions. Signed-off-by: Sami Tolvanen <samitolvanen@google.com> Reviewed-by: Petr Pavlu <petr.pavlu@suse.com> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> --- scripts/gendwarfksyms/dwarf.c | 248 ++++++++++++++++++++++++++++++- scripts/gendwarfksyms/examples/kabi.h | 87 +++++++++++ scripts/gendwarfksyms/examples/kabi_ex.c | 16 ++ scripts/gendwarfksyms/examples/kabi_ex.h | 199 +++++++++++++++++++++++++ scripts/gendwarfksyms/gendwarfksyms.h | 9 ++ 5 files changed, 558 insertions(+), 1 deletion(-) (limited to 'scripts') diff --git a/scripts/gendwarfksyms/dwarf.c b/scripts/gendwarfksyms/dwarf.c index 17f7e6b9a7ff..746a89d9e3d4 100644 --- a/scripts/gendwarfksyms/dwarf.c +++ b/scripts/gendwarfksyms/dwarf.c @@ -3,10 +3,33 @@ * Copyright (C) 2024 Google LLC */ +#include <assert.h> #include <inttypes.h> #include <stdarg.h> #include "gendwarfksyms.h" +/* See get_union_kabi_status */ +#define KABI_PREFIX "__kabi_" +#define KABI_PREFIX_LEN (sizeof(KABI_PREFIX) - 1) +#define KABI_RESERVED_PREFIX "reserved" +#define KABI_RESERVED_PREFIX_LEN (sizeof(KABI_RESERVED_PREFIX) - 1) +#define KABI_RENAMED_PREFIX "renamed" +#define KABI_RENAMED_PREFIX_LEN (sizeof(KABI_RENAMED_PREFIX) - 1) +#define KABI_IGNORED_PREFIX "ignored" +#define KABI_IGNORED_PREFIX_LEN (sizeof(KABI_IGNORED_PREFIX) - 1) + +static inline bool is_kabi_prefix(const char *name) +{ + return name && !strncmp(name, KABI_PREFIX, KABI_PREFIX_LEN); +} + +enum kabi_status { + /* >0 to stop DIE processing */ + KABI_NORMAL = 1, + KABI_RESERVED, + KABI_IGNORED, +}; + static bool do_linebreak; static int indentation_level; @@ -353,13 +376,23 @@ static void __process_list_type(struct state *state, struct die *cache, { const char *name = get_name_attr(die); + if (stable) { + if (is_kabi_prefix(name)) + name = NULL; + state->kabi.orig_name = NULL; + } + process_list_comma(state, cache); process(cache, type); process_type_attr(state, cache, die); + + if (stable && state->kabi.orig_name) + name = state->kabi.orig_name; if (name) { process(cache, " "); process(cache, name); } + process_accessibility_attr(cache, die); process_bit_size_attr(cache, die); process_data_bit_offset_attr(cache, die); @@ -486,11 +519,208 @@ static void process_variant_part_type(struct state *state, struct die *cache, process(cache, "}"); } +static int get_kabi_status(Dwarf_Die *die, const char **suffix) +{ + const char *name = get_name_attr(die); + + if (suffix) + *suffix = NULL; + + if (is_kabi_prefix(name)) { + name += KABI_PREFIX_LEN; + + if (!strncmp(name, KABI_RESERVED_PREFIX, + KABI_RESERVED_PREFIX_LEN)) + return KABI_RESERVED; + if (!strncmp(name, KABI_IGNORED_PREFIX, + KABI_IGNORED_PREFIX_LEN)) + return KABI_IGNORED; + + if (!strncmp(name, KABI_RENAMED_PREFIX, + KABI_RENAMED_PREFIX_LEN)) { + if (suffix) { + name += KABI_RENAMED_PREFIX_LEN; + *suffix = name; + } + return KABI_RESERVED; + } + } + + return KABI_NORMAL; +} + +static int check_struct_member_kabi_status(struct state *state, + struct die *__unused, Dwarf_Die *die) +{ + int res; + + assert(dwarf_tag(die) == DW_TAG_member_type); + + /* + * If the union member is a struct, expect the __kabi field to + * be the first member of the structure, i.e..: + * + * union { + * type new_member; + * struct { + * type __kabi_field; + * } + * }; + */ + res = get_kabi_status(die, &state->kabi.orig_name); + + if (res == KABI_RESERVED && + !get_ref_die_attr(die, DW_AT_type, &state->kabi.placeholder)) + error("structure member missing a type?"); + + return res; +} + +static int check_union_member_kabi_status(struct state *state, + struct die *__unused, Dwarf_Die *die) +{ + Dwarf_Die type; + int res; + + assert(dwarf_tag(die) == DW_TAG_member_type); + + if (!get_ref_die_attr(die, DW_AT_type, &type)) + error("union member missing a type?"); + + /* + * We expect a union with two members. Check if either of them + * has a __kabi name prefix, i.e.: + * + * union { + * ... + * type memberN; // <- type, N = {0,1} + * ... + * }; + * + * The member can also be a structure type, in which case we'll + * check the first structure member. + * + * In any case, stop processing after we've seen two members. + */ + res = get_kabi_status(die, &state->kabi.orig_name); + + if (res == KABI_RESERVED) + state->kabi.placeholder = type; + if (res != KABI_NORMAL) + return res; + + if (dwarf_tag(&type) == DW_TAG_structure_type) + res = checkp(process_die_container( + state, NULL, &type, check_struct_member_kabi_status, + match_member_type)); + + if (res <= KABI_NORMAL && ++state->kabi.members < 2) + return 0; /* Continue */ + + return res; +} + +static int get_union_kabi_status(Dwarf_Die *die, Dwarf_Die *placeholder, + const char **orig_name) +{ + struct state state; + int res; + + if (!stable) + return KABI_NORMAL; + + /* + * To maintain a stable kABI, distributions may choose to reserve + * space in structs for later use by adding placeholder members, + * for example: + * + * struct s { + * u32 a; + * // an 8-byte placeholder for future use + * u64 __kabi_reserved_0; + * }; + * + * When the reserved member is taken into use, the type change + * would normally cause the symbol version to change as well, but + * if the replacement uses the following convention, gendwarfksyms + * continues to use the placeholder type for versioning instead, + * thus maintaining the same symbol version: + * + * struct s { + * u32 a; + * union { + * // placeholder replaced with a new member `b` + * struct t b; + * struct { + * // the placeholder type that is still + * // used for versioning + * u64 __kabi_reserved_0; + * }; + * }; + * }; + * + * I.e., as long as the replaced member is in a union, and the + * placeholder has a __kabi_reserved name prefix, we'll continue + * to use the placeholder type (here u64) for version calculation + * instead of the union type. + * + * It's also possible to ignore new members from versioning if + * they've been added to alignment holes, for example, by + * including them in a union with another member that uses the + * __kabi_ignored name prefix: + * + * struct s { + * u32 a; + * // an alignment hole is used to add `n` + * union { + * u32 n; + * // hide the entire union member from versioning + * u8 __kabi_ignored_0; + * }; + * u64 b; + * }; + * + * Note that the user of this feature is responsible for ensuring + * that the structure actually remains ABI compatible. + */ + memset(&state.kabi, 0, sizeof(struct kabi_state)); + + res = checkp(process_die_container(&state, NULL, die, + check_union_member_kabi_status, + match_member_type)); + + if (res == KABI_RESERVED) { + if (placeholder) + *placeholder = state.kabi.placeholder; + if (orig_name) + *orig_name = state.kabi.orig_name; + } + + return res; +} + +static bool is_kabi_ignored(Dwarf_Die *die) +{ + Dwarf_Die type; + + if (!stable) + return false; + + if (!get_ref_die_attr(die, DW_AT_type, &type)) + error("member missing a type?"); + + return dwarf_tag(&type) == DW_TAG_union_type && + checkp(get_union_kabi_status(&type, NULL, NULL)) == KABI_IGNORED; +} + static int ___process_structure_type(struct state *state, struct die *cache, Dwarf_Die *die) { switch (dwarf_tag(die)) { case DW_TAG_member: + if (is_kabi_ignored(die)) + return 0; + return check(process_type(state, cache, die)); case DW_TAG_variant_part: return check(process_type(state, cache, die)); case DW_TAG_class_type: @@ -547,7 +777,23 @@ static void __process_structure_type(struct state *state, struct die *cache, DEFINE_PROCESS_STRUCTURE_TYPE(class) DEFINE_PROCESS_STRUCTURE_TYPE(structure) -DEFINE_PROCESS_STRUCTURE_TYPE(union) + +static void process_union_type(struct state *state, struct die *cache, + Dwarf_Die *die) +{ + Dwarf_Die placeholder; + + int res = checkp(get_union_kabi_status(die, &placeholder, + &state->kabi.orig_name)); + + if (res == KABI_RESERVED) + check(process_type(state, cache, &placeholder)); + if (res > KABI_NORMAL) + return; + + __process_structure_type(state, cache, die, "union_type", + ___process_structure_type, match_all); +} static void process_enumerator_type(struct state *state, struct die *cache, Dwarf_Die *die) diff --git a/scripts/gendwarfksyms/examples/kabi.h b/scripts/gendwarfksyms/examples/kabi.h index fcd0300e5b58..97a5669b083d 100644 --- a/scripts/gendwarfksyms/examples/kabi.h +++ b/scripts/gendwarfksyms/examples/kabi.h @@ -43,6 +43,28 @@ __section(".discard.gendwarfksyms.kabi_rules") = \ "1\0" #hint "\0" #target "\0" #value +#define __KABI_NORMAL_SIZE_ALIGN(_orig, _new) \ + union { \ + _Static_assert( \ + sizeof(struct { _new; }) <= sizeof(struct { _orig; }), \ + __FILE__ ":" __stringify(__LINE__) ": " __stringify( \ + _new) " is larger than " __stringify(_orig)); \ + _Static_assert( \ + __alignof__(struct { _new; }) <= \ + __alignof__(struct { _orig; }), \ + __FILE__ ":" __stringify(__LINE__) ": " __stringify( \ + _orig) " is not aligned the same as " __stringify(_new)); \ + } + +#define __KABI_REPLACE(_orig, _new) \ + union { \ + _new; \ + struct { \ + _orig; \ + }; \ + __KABI_NORMAL_SIZE_ALIGN(_orig, _new); \ + } + /* * KABI_DECLONLY(fqn) * Treat the struct/union/enum fqn as a declaration, i.e. even if @@ -67,4 +89,69 @@ #define KABI_ENUMERATOR_VALUE(fqn, field, value) \ __KABI_RULE(enumerator_value, fqn field, value) +/* + * KABI_RESERVE + * Reserve some "padding" in a structure for use by LTS backports. + * This is normally placed at the end of a structure. + * number: the "number" of the padding variable in the structure. Start with + * 1 and go up. + */ +#define KABI_RESERVE(n) unsigned long __kabi_reserved##n + +/* + * KABI_RESERVE_ARRAY + * Same as _BACKPORT_RESERVE but allocates an array with the specified + * size in bytes. + */ +#define KABI_RESERVE_ARRAY(n, s) \ + unsigned char __aligned(8) __kabi_reserved##n[s] + +/* + * KABI_IGNORE + * Add a new field that's ignored in versioning. + */ +#define KABI_IGNORE(n, _new) \ + union { \ + _new; \ + unsigned char __kabi_ignored##n; \ + } + +/* + * KABI_REPLACE + * Replace a field with a compatible new field. + */ +#define KABI_REPLACE(_oldtype, _oldname, _new) \ + __KABI_REPLACE(_oldtype __kabi_renamed##_oldname, struct { _new; }) + +/* + * KABI_USE(number, _new) + * Use a previous padding entry that was defined with KABI_RESERVE + * number: the previous "number" of the padding variable + * _new: the variable to use now instead of the padding variable + */ +#define KABI_USE(number, _new) __KABI_REPLACE(KABI_RESERVE(number), _new) + +/* + * KABI_USE2(number, _new1, _new2) + * Use a previous padding entry that was defined with KABI_RESERVE for + * two new variables that fit into 64 bits. This is good for when you do not + * want to "burn" a 64bit padding variable for a smaller variable size if not + * needed. + */ +#define KABI_USE2(number, _new1, _new2) \ + __KABI_REPLACE( \ + KABI_RESERVE(number), struct { \ + _new1; \ + _new2; \ + }) +/* + * KABI_USE_ARRAY(number, bytes, _new) + * Use a previous padding entry that was defined with KABI_RESERVE_ARRAY + * number: the previous "number" of the padding variable + * bytes: the size in bytes reserved for the array + * _new: the variable to use now instead of the padding variable + */ +#define KABI_USE_ARRAY(number, bytes, _new) \ + __KABI_REPLACE(KABI_RESERVE_ARRAY(number, bytes), _new) + #endif /* __KABI_H__ */ diff --git a/scripts/gendwarfksyms/examples/kabi_ex.c b/scripts/gendwarfksyms/examples/kabi_ex.c index 799552ea6679..0b7ffd830541 100644 --- a/scripts/gendwarfksyms/examples/kabi_ex.c +++ b/scripts/gendwarfksyms/examples/kabi_ex.c @@ -12,3 +12,19 @@ struct s e0; enum e e1; + +struct ex0a ex0a; +struct ex0b ex0b; +struct ex0c ex0c; + +struct ex1a ex1a; +struct ex1b ex1b; +struct ex1c ex1c; + +struct ex2a ex2a; +struct ex2b ex2b; +struct ex2c ex2c; + +struct ex3a ex3a; +struct ex3b ex3b; +struct ex3c ex3c; diff --git a/scripts/gendwarfksyms/examples/kabi_ex.h b/scripts/gendwarfksyms/examples/kabi_ex.h index fca1e07c78e2..1736e0f65208 100644 --- a/scripts/gendwarfksyms/examples/kabi_ex.h +++ b/scripts/gendwarfksyms/examples/kabi_ex.h @@ -59,6 +59,205 @@ KABI_ENUMERATOR_VALUE(e, D, 123456789); * STABLE-NEXT: enumerator A = 0 , * STABLE-NEXT: enumerator D = 123456789 * STABLE-NEXT: } byte_size(4) +*/ + +/* + * Example: Reserved fields + */ +struct ex0a { + int a; + KABI_RESERVE(0); + KABI_RESERVE(1); +}; + +/* + * STABLE: variable structure_type ex0a { + * STABLE-NEXT: member base_type int byte_size(4) encoding(5) a data_member_location(0) , + * STABLE-NEXT: member base_type [[ULONG:long unsigned int|unsigned long]] byte_size(8) encoding(7) data_member_location(8) , + * STABLE-NEXT: member base_type [[ULONG]] byte_size(8) encoding(7) data_member_location(16) + * STABLE-NEXT: } byte_size(24) + */ + +struct ex0b { + int a; + KABI_RESERVE(0); + KABI_USE2(1, int b, int c); +}; + +/* + * STABLE: variable structure_type ex0b { + * STABLE-NEXT: member base_type int byte_size(4) encoding(5) a data_member_location(0) , + * STABLE-NEXT: member base_type [[ULONG]] byte_size(8) encoding(7) data_member_location(8) , + * STABLE-NEXT: member base_type [[ULONG]] byte_size(8) encoding(7) data_member_location(16) + * STABLE-NEXT: } byte_size(24) + */ + +struct ex0c { + int a; + KABI_USE(0, void *p); + KABI_USE2(1, int b, int c); +}; + +/* + * STABLE: variable structure_type ex0c { + * STABLE-NEXT: member base_type int byte_size(4) encoding(5) a data_member_location(0) , + * STABLE-NEXT: member base_type [[ULONG]] byte_size(8) encoding(7) data_member_location(8) , + * STABLE-NEXT: member base_type [[ULONG]] byte_size(8) encoding(7) data_member_location(16) + * STABLE-NEXT: } byte_size(24) + */ + +/* + * Example: A reserved array + */ + +struct ex1a { + unsigned int a; + KABI_RESERVE_ARRAY(0, 64); +}; + +/* + * STABLE: variable structure_type ex1a { + * STABLE-NEXT: member base_type unsigned int byte_size(4) encoding(7) a data_member_location(0) , + * STABLE-NEXT: member array_type[64] { + * STABLE-NEXT: base_type unsigned char byte_size(1) encoding(8) + * STABLE-NEXT: } data_member_location(8) + * STABLE-NEXT: } byte_size(72) + */ + +struct ex1b { + unsigned int a; + KABI_USE_ARRAY( + 0, 64, struct { + void *p; + KABI_RESERVE_ARRAY(1, 56); + }); +}; + +/* + * STABLE: variable structure_type ex1b { + * STABLE-NEXT: member base_type unsigned int byte_size(4) encoding(7) a data_member_location(0) , + * STABLE-NEXT: member array_type[64] { + * STABLE-NEXT: base_type unsigned char byte_size(1) encoding(8) + * STABLE-NEXT: } data_member_location(8) + * STABLE-NEXT: } byte_size(72) + */ + +struct ex1c { + unsigned int a; + KABI_USE_ARRAY(0, 64, void *p[8]); +}; + +/* + * STABLE: variable structure_type ex1c { + * STABLE-NEXT: member base_type unsigned int byte_size(4) encoding(7) a data_member_location(0) , + * STABLE-NEXT: member array_type[64] { + * STABLE-NEXT: base_type unsigned char byte_size(1) encoding(8) + * STABLE-NEXT: } data_member_location(8) + * STABLE-NEXT: } byte_size(72) + */ + +/* + * Example: An ignored field added to an alignment hole + */ + +struct ex2a { + int a; + unsigned long b; + int c; + unsigned long d; +}; + +/* + * STABLE: variable structure_type ex2a { + * STABLE-NEXT: member base_type int byte_size(4) encoding(5) a data_member_location(0) , + * STABLE-NEXT: member base_type [[ULONG:long unsigned int|unsigned long]] byte_size(8) encoding(7) b data_member_location(8) + * STABLE-NEXT: member base_type int byte_size(4) encoding(5) c data_member_location(16) , + * STABLE-NEXT: member base_type [[ULONG]] byte_size(8) encoding(7) d data_member_location(24) + * STABLE-NEXT: } byte_size(32) + */ + +struct ex2b { + int a; + KABI_IGNORE(0, unsigned int n); + unsigned long b; + int c; + unsigned long d; +}; + +_Static_assert(sizeof(struct ex2a) == sizeof(struct ex2b), "ex2a size doesn't match ex2b"); + +/* + * STABLE: variable structure_type ex2b { + * STABLE-NEXT: member base_type int byte_size(4) encoding(5) a data_member_location(0) , + * STABLE-NEXT: member base_type [[ULONG]] byte_size(8) encoding(7) b data_member_location(8) + * STABLE-NEXT: member base_type int byte_size(4) encoding(5) c data_member_location(16) , + * STABLE-NEXT: member base_type [[ULONG]] byte_size(8) encoding(7) d data_member_location(24) + * STABLE-NEXT: } byte_size(32) + */ + +struct ex2c { + int a; + KABI_IGNORE(0, unsigned int n); + unsigned long b; + int c; + KABI_IGNORE(1, unsigned int m); + unsigned long d; +}; + +_Static_assert(sizeof(struct ex2a) == sizeof(struct ex2c), "ex2a size doesn't match ex2c"); + +/* + * STABLE: variable structure_type ex2c { + * STABLE-NEXT: member base_type int byte_size(4) encoding(5) a data_member_location(0) , + * STABLE-NEXT: member base_type [[ULONG]] byte_size(8) encoding(7) b data_member_location(8) + * STABLE-NEXT: member base_type int byte_size(4) encoding(5) c data_member_location(16) , + * STABLE-NEXT: member base_type [[ULONG]] byte_size(8) encoding(7) d data_member_location(24) + * STABLE-NEXT: } byte_size(32) + */ + + +/* + * Example: A replaced field + */ + +struct ex3a { + unsigned long a; + unsigned long unused; +}; + +/* + * STABLE: variable structure_type ex3a { + * STABLE-NEXT: member base_type [[ULONG:long unsigned int|unsigned long]] byte_size(8) encoding(7) a data_member_location(0) + * STABLE-NEXT: member base_type [[ULONG]] byte_size(8) encoding(7) unused data_member_location(8) + * STABLE-NEXT: } byte_size(16) + */ + +struct ex3b { + unsigned long a; + KABI_REPLACE(unsigned long, unused, unsigned long renamed); +}; + +_Static_assert(sizeof(struct ex3a) == sizeof(struct ex3b), "ex3a size doesn't match ex3b"); + +/* + * STABLE: variable structure_type ex3b { + * STABLE-NEXT: member base_type [[ULONG]] byte_size(8) encoding(7) a data_member_location(0) + * STABLE-NEXT: member base_type [[ULONG]] byte_size(8) encoding(7) unused data_member_location(8) + * STABLE-NEXT: } byte_size(16) + */ + +struct ex3c { + unsigned long a; + KABI_REPLACE(unsigned long, unused, long replaced); +}; + +_Static_assert(sizeof(struct ex3a) == sizeof(struct ex3c), "ex3a size doesn't match ex3c"); + +/* + * STABLE: variable structure_type ex3c { + * STABLE-NEXT: member base_type [[ULONG]] byte_size(8) encoding(7) a data_member_location(0) + * STABLE-NEXT: member base_type [[ULONG]] byte_size(8) encoding(7) unused data_member_location(8) + * STABLE-NEXT: } byte_size(16) */ #endif /* __KABI_EX_H__ */ diff --git a/scripts/gendwarfksyms/gendwarfksyms.h b/scripts/gendwarfksyms/gendwarfksyms.h index c0207ca10e19..fe49730fe623 100644 --- a/scripts/gendwarfksyms/gendwarfksyms.h +++ b/scripts/gendwarfksyms/gendwarfksyms.h @@ -236,6 +236,12 @@ struct expansion_state { const char *current_fqn; }; +struct kabi_state { + int members; + Dwarf_Die placeholder; + const char *orig_name; +}; + struct state { struct symbol *sym; Dwarf_Die die; @@ -246,6 +252,9 @@ struct state { /* Structure expansion */ struct expansion_state expand; struct cache expansion_cache; + + /* Reserved or ignored members */ + struct kabi_state kabi; }; typedef int (*die_callback_t)(struct state *state, struct die *cache, -- cgit v1.2.3 From fa624569b70d8015775592ae7e2c514009367541 Mon Sep 17 00:00:00 2001 From: Sami Tolvanen <samitolvanen@google.com> Date: Fri, 3 Jan 2025 20:45:37 +0000 Subject: gendwarfksyms: Add support for symbol type pointers The compiler may choose not to emit type information in DWARF for external symbols. Clang, for example, does this for symbols not defined in the current TU. To provide a way to work around this issue, add support for __gendwarfksyms_ptr_<symbol> pointers that force the compiler to emit the necessary type information in DWARF also for the missing symbols. Example usage: #define GENDWARFKSYMS_PTR(sym) \ static typeof(sym) *__gendwarfksyms_ptr_##sym __used \ __section(".discard.gendwarfksyms") = &sym; extern int external_symbol(void); GENDWARFKSYMS_PTR(external_symbol); Signed-off-by: Sami Tolvanen <samitolvanen@google.com> Reviewed-by: Petr Pavlu <petr.pavlu@suse.com> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> --- scripts/gendwarfksyms/dwarf.c | 55 +++++++++++++++++++++++++++++- scripts/gendwarfksyms/examples/symbolptr.c | 33 ++++++++++++++++++ scripts/gendwarfksyms/gendwarfksyms.h | 7 ++++ scripts/gendwarfksyms/symbols.c | 27 +++++++++++++++ 4 files changed, 121 insertions(+), 1 deletion(-) create mode 100644 scripts/gendwarfksyms/examples/symbolptr.c (limited to 'scripts') diff --git a/scripts/gendwarfksyms/dwarf.c b/scripts/gendwarfksyms/dwarf.c index 746a89d9e3d4..534d9aa7c114 100644 --- a/scripts/gendwarfksyms/dwarf.c +++ b/scripts/gendwarfksyms/dwarf.c @@ -1061,6 +1061,31 @@ static void process_variable(struct state *state, Dwarf_Die *die) process_symbol(state, die, __process_variable); } +static void save_symbol_ptr(struct state *state) +{ + Dwarf_Die ptr_type; + Dwarf_Die type; + + if (!get_ref_die_attr(&state->die, DW_AT_type, &ptr_type) || + dwarf_tag(&ptr_type) != DW_TAG_pointer_type) + error("%s must be a pointer type!", + get_symbol_name(&state->die)); + + if (!get_ref_die_attr(&ptr_type, DW_AT_type, &type)) + error("%s pointer missing a type attribute?", + get_symbol_name(&state->die)); + + /* + * Save the symbol pointer DIE in case the actual symbol is + * missing from the DWARF. Clang, for example, intentionally + * omits external symbols from the debugging information. + */ + if (dwarf_tag(&type) == DW_TAG_subroutine_type) + symbol_set_ptr(state->sym, &type); + else + symbol_set_ptr(state->sym, &ptr_type); +} + static int process_exported_symbols(struct state *unused, struct die *cache, Dwarf_Die *die) { @@ -1084,7 +1109,9 @@ static int process_exported_symbols(struct state *unused, struct die *cache, state_init(&state); - if (tag == DW_TAG_subprogram) + if (is_symbol_ptr(get_symbol_name(&state.die))) + save_symbol_ptr(&state); + else if (tag == DW_TAG_subprogram) process_subprogram(&state, &state.die); else process_variable(&state, &state.die); @@ -1097,10 +1124,36 @@ static int process_exported_symbols(struct state *unused, struct die *cache, } } +static void process_symbol_ptr(struct symbol *sym, void *arg) +{ + struct state state; + Dwarf *dwarf = arg; + + if (sym->state != SYMBOL_UNPROCESSED || !sym->ptr_die_addr) + return; + + debug("%s", sym->name); + state_init(&state); + state.sym = sym; + + if (!dwarf_die_addr_die(dwarf, (void *)sym->ptr_die_addr, &state.die)) + error("dwarf_die_addr_die failed for symbol ptr: '%s'", + sym->name); + + if (dwarf_tag(&state.die) == DW_TAG_subroutine_type) + process_subprogram(&state, &state.die); + else + process_variable(&state, &state.die); + + cache_free(&state.expansion_cache); +} + void process_cu(Dwarf_Die *cudie) { check(process_die_container(NULL, NULL, cudie, process_exported_symbols, match_all)); + symbol_for_each(process_symbol_ptr, dwarf_cu_getdwarf(cudie->cu)); + cache_free(&srcfile_cache); } diff --git a/scripts/gendwarfksyms/examples/symbolptr.c b/scripts/gendwarfksyms/examples/symbolptr.c new file mode 100644 index 000000000000..88bc1bd60da8 --- /dev/null +++ b/scripts/gendwarfksyms/examples/symbolptr.c @@ -0,0 +1,33 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Copyright (C) 2024 Google LLC + * + * Example for symbol pointers. When compiled with Clang, gendwarfkyms + * uses a symbol pointer for `f`. + * + * $ clang -g -c examples/symbolptr.c -o examples/symbolptr.o + * $ echo -e "f\ng\np" | ./gendwarfksyms -d examples/symbolptr.o + */ + +/* Kernel macros for userspace testing. */ +#ifndef __used +#define __used __attribute__((__used__)) +#endif +#ifndef __section +#define __section(section) __attribute__((__section__(section))) +#endif + +#define __GENDWARFKSYMS_EXPORT(sym) \ + static typeof(sym) *__gendwarfksyms_ptr_##sym __used \ + __section(".discard.gendwarfksyms") = &sym; + +extern void f(unsigned int arg); +void g(int *arg); +void g(int *arg) {} + +struct s; +extern struct s *p; + +__GENDWARFKSYMS_EXPORT(f); +__GENDWARFKSYMS_EXPORT(g); +__GENDWARFKSYMS_EXPORT(p); diff --git a/scripts/gendwarfksyms/gendwarfksyms.h b/scripts/gendwarfksyms/gendwarfksyms.h index fe49730fe623..197a1a8123c6 100644 --- a/scripts/gendwarfksyms/gendwarfksyms.h +++ b/scripts/gendwarfksyms/gendwarfksyms.h @@ -89,6 +89,10 @@ extern int symtypes; * symbols.c */ +/* See symbols.c:is_symbol_ptr */ +#define SYMBOL_PTR_PREFIX "__gendwarfksyms_ptr_" +#define SYMBOL_PTR_PREFIX_LEN (sizeof(SYMBOL_PTR_PREFIX) - 1) + static inline unsigned int addr_hash(uintptr_t addr) { return hash_ptr((const void *)addr); @@ -112,14 +116,17 @@ struct symbol { struct hlist_node name_hash; enum symbol_state state; uintptr_t die_addr; + uintptr_t ptr_die_addr; unsigned long crc; }; typedef void (*symbol_callback_t)(struct symbol *, void *arg); +bool is_symbol_ptr(const char *name); void symbol_read_exports(FILE *file); void symbol_read_symtab(int fd); struct symbol *symbol_get(const char *name); +void symbol_set_ptr(struct symbol *sym, Dwarf_Die *ptr); void symbol_set_die(struct symbol *sym, Dwarf_Die *die); void symbol_set_crc(struct symbol *sym, unsigned long crc); void symbol_for_each(symbol_callback_t func, void *arg); diff --git a/scripts/gendwarfksyms/symbols.c b/scripts/gendwarfksyms/symbols.c index 4c499ba6c86d..327f87389c34 100644 --- a/scripts/gendwarfksyms/symbols.c +++ b/scripts/gendwarfksyms/symbols.c @@ -39,6 +39,20 @@ static unsigned int __for_each_addr(struct symbol *sym, symbol_callback_t func, return processed; } +/* + * For symbols without debugging information (e.g. symbols defined in other + * TUs), we also match __gendwarfksyms_ptr_<symbol_name> symbols, which the + * kernel uses to ensure type information is present in the TU that exports + * the symbol. A __gendwarfksyms_ptr pointer must have the same type as the + * exported symbol, e.g.: + * + * typeof(symname) *__gendwarf_ptr_symname = &symname; + */ +bool is_symbol_ptr(const char *name) +{ + return name && !strncmp(name, SYMBOL_PTR_PREFIX, SYMBOL_PTR_PREFIX_LEN); +} + static unsigned int for_each(const char *name, symbol_callback_t func, void *data) { @@ -47,6 +61,8 @@ static unsigned int for_each(const char *name, symbol_callback_t func, if (!name || !*name) return 0; + if (is_symbol_ptr(name)) + name += SYMBOL_PTR_PREFIX_LEN; hash_for_each_possible_safe(symbol_names, match, tmp, name_hash, hash_str(name)) { @@ -84,6 +100,17 @@ void symbol_set_crc(struct symbol *sym, unsigned long crc) error("no matching symbols: '%s'", sym->name); } +static void set_ptr(struct symbol *sym, void *data) +{ + sym->ptr_die_addr = (uintptr_t)((Dwarf_Die *)data)->addr; +} + +void symbol_set_ptr(struct symbol *sym, Dwarf_Die *ptr) +{ + if (for_each(sym->name, set_ptr, ptr) == 0) + error("no matching symbols: '%s'", sym->name); +} + static void set_die(struct symbol *sym, void *data) { sym->die_addr = (uintptr_t)((Dwarf_Die *)data)->addr; -- cgit v1.2.3 From 9c3681f9b9fd12cdbc4a542df599f1837512f3d5 Mon Sep 17 00:00:00 2001 From: Sami Tolvanen <samitolvanen@google.com> Date: Fri, 3 Jan 2025 20:45:39 +0000 Subject: kbuild: Add gendwarfksyms as an alternative to genksyms When MODVERSIONS is enabled, allow selecting gendwarfksyms as the implementation, but default to genksyms. Signed-off-by: Sami Tolvanen <samitolvanen@google.com> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> --- kernel/module/Kconfig | 22 ++++++++++++++++++++++ scripts/Makefile | 2 +- scripts/Makefile.build | 35 +++++++++++++++++++++++++++++------ 3 files changed, 52 insertions(+), 7 deletions(-) (limited to 'scripts') diff --git a/kernel/module/Kconfig b/kernel/module/Kconfig index 4637f063d0fc..d443fc504ffc 100644 --- a/kernel/module/Kconfig +++ b/kernel/module/Kconfig @@ -169,6 +169,22 @@ config MODVERSIONS make them incompatible with the kernel you are running. If unsure, say N. +choice + prompt "Module versioning implementation" + depends on MODVERSIONS + help + Select the tool used to calculate symbol versions for modules. + + If unsure, select GENKSYMS. + +config GENKSYMS + bool "genksyms (from source code)" + help + Calculate symbol versions from pre-processed source code using + genksyms. + + If unsure, say Y. + config GENDWARFKSYMS bool "gendwarfksyms (from debugging information)" depends on DEBUG_INFO @@ -176,6 +192,12 @@ config GENDWARFKSYMS depends on !DEBUG_INFO_REDUCED && !DEBUG_INFO_SPLIT # Requires ELF object files. depends on !LTO + help + Calculate symbol versions from DWARF debugging information using + gendwarfksyms. Requires DEBUG_INFO to be enabled. + + If unsure, say N. +endchoice config ASM_MODVERSIONS bool diff --git a/scripts/Makefile b/scripts/Makefile index d7fec46d38c0..8533f4498885 100644 --- a/scripts/Makefile +++ b/scripts/Makefile @@ -53,7 +53,7 @@ hostprogs += unifdef targets += module.lds subdir-$(CONFIG_GCC_PLUGINS) += gcc-plugins -subdir-$(CONFIG_MODVERSIONS) += genksyms +subdir-$(CONFIG_GENKSYMS) += genksyms subdir-$(CONFIG_GENDWARFKSYMS) += gendwarfksyms subdir-$(CONFIG_SECURITY_SELINUX) += selinux subdir-$(CONFIG_SECURITY_IPE) += ipe diff --git a/scripts/Makefile.build b/scripts/Makefile.build index c16e4cf54d77..81d9dacad03c 100644 --- a/scripts/Makefile.build +++ b/scripts/Makefile.build @@ -107,13 +107,24 @@ cmd_cpp_i_c = $(CPP) $(c_flags) -o $@ $< $(obj)/%.i: $(obj)/%.c FORCE $(call if_changed_dep,cpp_i_c) +getexportsymbols = $(NM) $@ | sed -n 's/.* __export_symbol_\(.*\)/$(1)/p' + +gendwarfksyms = $(objtree)/scripts/gendwarfksyms/gendwarfksyms \ + $(if $(KBUILD_SYMTYPES), --symtypes $(@:.o=.symtypes)) \ + $(if $(KBUILD_GENDWARFKSYMS_STABLE), --stable) + genksyms = $(objtree)/scripts/genksyms/genksyms \ $(if $(KBUILD_SYMTYPES), -T $(@:.o=.symtypes)) \ $(if $(KBUILD_PRESERVE), -p) \ $(addprefix -r , $(wildcard $(@:.o=.symref))) # These mirror gensymtypes_S and co below, keep them in synch. +ifdef CONFIG_GENDWARFKSYMS +cmd_gensymtypes_c = $(if $(skip_gendwarfksyms),, \ + $(call getexportsymbols,\1) | $(gendwarfksyms) $@) +else cmd_gensymtypes_c = $(CPP) -D__GENKSYMS__ $(c_flags) $< | $(genksyms) +endif # CONFIG_GENDWARFKSYMS # LLVM assembly # Generate .ll files from .c @@ -286,14 +297,26 @@ $(obj)/%.rs: $(obj)/%.rs.S FORCE # This is convoluted. The .S file must first be preprocessed to run guards and # expand names, then the resulting exports must be constructed into plain # EXPORT_SYMBOL(symbol); to build our dummy C file, and that gets preprocessed -# to make the genksyms input. +# to make the genksyms input or compiled into an object for gendwarfksyms. # # These mirror gensymtypes_c and co above, keep them in synch. -cmd_gensymtypes_S = \ - { echo "\#include <linux/kernel.h>" ; \ - echo "\#include <asm/asm-prototypes.h>" ; \ - $(NM) $@ | sed -n 's/.* __export_symbol_\(.*\)/EXPORT_SYMBOL(\1);/p' ; } | \ - $(CPP) -D__GENKSYMS__ $(c_flags) -xc - | $(genksyms) +getasmexports = \ + { echo "\#include <linux/kernel.h>" ; \ + echo "\#include <linux/string.h>" ; \ + echo "\#include <asm/asm-prototypes.h>" ; \ + $(call getexportsymbols,EXPORT_SYMBOL(\1);) ; } + +ifdef CONFIG_GENDWARFKSYMS +cmd_gensymtypes_S = \ + $(getasmexports) | \ + $(CC) $(c_flags) -c -o $(@:.o=.gendwarfksyms.o) -xc -; \ + $(call getexportsymbols,\1) | \ + $(gendwarfksyms) $(@:.o=.gendwarfksyms.o) +else +cmd_gensymtypes_S = \ + $(getasmexports) | \ + $(CPP) -D__GENKSYMS__ $(c_flags) -xc - | $(genksyms) +endif # CONFIG_GENDWARFKSYMS quiet_cmd_cpp_s_S = CPP $(quiet_modtag) $@ cmd_cpp_s_S = $(CPP) $(a_flags) -o $@ $< -- cgit v1.2.3 From fc7d5e3210ae083a29ce224ffce18eaf3d1c645a Mon Sep 17 00:00:00 2001 From: Matthew Maurer <mmaurer@google.com> Date: Fri, 3 Jan 2025 17:37:02 +0000 Subject: modpost: Produce extended MODVERSIONS information Generate both the existing modversions format and the new extended one when running modpost. Presence of this metadata in the final .ko is guarded by CONFIG_EXTENDED_MODVERSIONS. We no longer generate an error on long symbols in modpost if CONFIG_EXTENDED_MODVERSIONS is set, as they can now be appropriately encoded in the extended section. These symbols will be skipped in the previous encoding. An error will still be generated if CONFIG_EXTENDED_MODVERSIONS is not set. Reviewed-by: Sami Tolvanen <samitolvanen@google.com> Signed-off-by: Matthew Maurer <mmaurer@google.com> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> --- kernel/module/Kconfig | 10 ++++++++ scripts/Makefile.modpost | 1 + scripts/mod/modpost.c | 62 ++++++++++++++++++++++++++++++++++++++++++++---- 3 files changed, 69 insertions(+), 4 deletions(-) (limited to 'scripts') diff --git a/kernel/module/Kconfig b/kernel/module/Kconfig index d443fc504ffc..9568b629a03c 100644 --- a/kernel/module/Kconfig +++ b/kernel/module/Kconfig @@ -207,6 +207,16 @@ config ASM_MODVERSIONS assembly. This can be enabled only when the target architecture supports it. +config EXTENDED_MODVERSIONS + bool "Extended Module Versioning Support" + depends on MODVERSIONS + help + This enables extended MODVERSIONs support, allowing long symbol + names to be versioned. + + The most likely reason you would enable this is to enable Rust + support. If unsure, say N. + config MODULE_SRCVERSION_ALL bool "Source checksum for all modules" help diff --git a/scripts/Makefile.modpost b/scripts/Makefile.modpost index ab0e94ea6249..40426fc63509 100644 --- a/scripts/Makefile.modpost +++ b/scripts/Makefile.modpost @@ -43,6 +43,7 @@ MODPOST = $(objtree)/scripts/mod/modpost modpost-args = \ $(if $(CONFIG_MODULES),-M) \ $(if $(CONFIG_MODVERSIONS),-m) \ + $(if $(CONFIG_EXTENDED_MODVERSIONS),-x) \ $(if $(CONFIG_MODULE_SRCVERSION_ALL),-a) \ $(if $(CONFIG_SECTION_MISMATCH_WARN_ONLY),,-E) \ $(if $(KBUILD_MODPOST_WARN),-w) \ diff --git a/scripts/mod/modpost.c b/scripts/mod/modpost.c index dc907014108b..38ff3dd4a9a1 100644 --- a/scripts/mod/modpost.c +++ b/scripts/mod/modpost.c @@ -33,6 +33,8 @@ static bool module_enabled; static bool modversions; /* Is CONFIG_MODULE_SRCVERSION_ALL set? */ static bool all_versions; +/* Is CONFIG_EXTENDED_MODVERSIONS set? */ +static bool extended_modversions; /* If we are modposting external module set to 1 */ static bool external_module; /* Only warn about unresolved symbols */ @@ -1805,6 +1807,49 @@ static void add_exported_symbols(struct buffer *buf, struct module *mod) } } +/** + * Record CRCs for unresolved symbols, supporting long names + */ +static void add_extended_versions(struct buffer *b, struct module *mod) +{ + struct symbol *s; + + if (!extended_modversions) + return; + + buf_printf(b, "\n"); + buf_printf(b, "static const u32 ____version_ext_crcs[]\n"); + buf_printf(b, "__used __section(\"__version_ext_crcs\") = {\n"); + list_for_each_entry(s, &mod->unresolved_symbols, list) { + if (!s->module) + continue; + if (!s->crc_valid) { + warn("\"%s\" [%s.ko] has no CRC!\n", + s->name, mod->name); + continue; + } + buf_printf(b, "\t0x%08x,\n", s->crc); + } + buf_printf(b, "};\n"); + + buf_printf(b, "static const char ____version_ext_names[]\n"); + buf_printf(b, "__used __section(\"__version_ext_names\") =\n"); + list_for_each_entry(s, &mod->unresolved_symbols, list) { + if (!s->module) + continue; + if (!s->crc_valid) + /* + * We already warned on this when producing the crc + * table. + * We need to skip its name too, as the indexes in + * both tables need to align. + */ + continue; + buf_printf(b, "\t\"%s\\0\"\n", s->name); + } + buf_printf(b, ";\n"); +} + /** * Record CRCs for unresolved symbols **/ @@ -1828,9 +1873,14 @@ static void add_versions(struct buffer *b, struct module *mod) continue; } if (strlen(s->name) >= MODULE_NAME_LEN) { - error("too long symbol \"%s\" [%s.ko]\n", - s->name, mod->name); - break; + if (extended_modversions) { + /* this symbol will only be in the extended info */ + continue; + } else { + error("too long symbol \"%s\" [%s.ko]\n", + s->name, mod->name); + break; + } } buf_printf(b, "\t{ 0x%08x, \"%s\" },\n", s->crc, s->name); @@ -1961,6 +2011,7 @@ static void write_mod_c_file(struct module *mod) add_header(&buf, mod); add_exported_symbols(&buf, mod); add_versions(&buf, mod); + add_extended_versions(&buf, mod); add_depends(&buf, mod); buf_printf(&buf, "\n"); @@ -2126,7 +2177,7 @@ int main(int argc, char **argv) LIST_HEAD(dump_lists); struct dump_list *dl, *dl2; - while ((opt = getopt(argc, argv, "ei:MmnT:to:au:WwENd:")) != -1) { + while ((opt = getopt(argc, argv, "ei:MmnT:to:au:WwENd:x")) != -1) { switch (opt) { case 'e': external_module = true; @@ -2175,6 +2226,9 @@ int main(int argc, char **argv) case 'd': missing_namespace_deps = optarg; break; + case 'x': + extended_modversions = true; + break; default: exit(1); } -- cgit v1.2.3 From a937f384c9da493e526ad896ef4e8054526d2941 Mon Sep 17 00:00:00 2001 From: Peter Zijlstra <peterz@infradead.org> Date: Mon, 6 Jan 2025 11:26:48 +0100 Subject: cleanup, tags: Create tags for the cleanup primitives Oleg reported that it is hard to find the definition of things like: __free(argv) without having to do 'git grep "DEFINE_FREE(argv,"'. Add tag generation for the various macros in cleanup.h. Notably 'DEFINE_FREE(argv, ...)' will now generate a 'cleanup_argv' tag, while all the others, eg. 'DEFINE_GUARD(mutex, ...)' will generate 'class_mutex' like tags. Reported-by: Oleg Nesterov <oleg@redhat.com> Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org> Link: https://lkml.kernel.org/r/20250106102647.GB20870@noisy.programming.kicks-ass.net --- scripts/tags.sh | 7 +++++++ 1 file changed, 7 insertions(+) (limited to 'scripts') diff --git a/scripts/tags.sh b/scripts/tags.sh index b21236377998..7939aea731f1 100755 --- a/scripts/tags.sh +++ b/scripts/tags.sh @@ -212,6 +212,13 @@ regex_c=( '/^SEQCOUNT_LOCKTYPE(\([^,]*\),[[:space:]]*\([^,]*\),[^)]*)/seqcount_\2_init/' '/^\<DECLARE_IDTENTRY[[:alnum:]_]*([^,)]*,[[:space:]]*\([[:alnum:]_]\+\)/\1/' '/^\<DEFINE_IDTENTRY[[:alnum:]_]*([[:space:]]*\([[:alnum:]_]\+\)/\1/' + '/^\<DEFINE_FREE(\([[:alnum:]_]\+\)/cleanup_\1/' + '/^\<DEFINE_CLASS(\([[:alnum:]_]\+\)/class_\1/' + '/^\<EXTEND_CLASS(\([[:alnum:]_]\+\),[[:space:]]*\([[:alnum:]_]\+\)/class_\1\2/' + '/^\<DEFINE_GUARD(\([[:alnum:]_]\+\)/class_\1/' + '/^\<DEFINE_GUARD_COND(\([[:alnum:]_]\+\),[[:space:]]*\([[:alnum:]_]\+\)/class_\1\2/' + '/^\<DEFINE_LOCK_GUARD_[[:digit:]](\([[:alnum:]_]\+\)/class_\1/' + '/^\<DEFINE_LOCK_GUARD_[[:digit:]]_COND(\([[:alnum:]_]\+\),[[:space:]]*\([[:alnum:]_]\+\)/class_\1\2/' ) regex_kconfig=( '/^[[:blank:]]*\(menu\|\)config[[:blank:]]\+\([[:alnum:]_]\+\)/\2/' -- cgit v1.2.3 From e8639b7ef0f871753b4262ec0eacd3da29eebcee Mon Sep 17 00:00:00 2001 From: Matthew Maurer <mmaurer@google.com> Date: Fri, 3 Jan 2025 17:37:03 +0000 Subject: modpost: Allow extended modversions without basic MODVERSIONS If you know that your kernel modules will only ever be loaded by a newer kernel, you can disable BASIC_MODVERSIONS to save space. This also allows easy creation of test modules to see how tooling will respond to modules that only have the new format. Signed-off-by: Matthew Maurer <mmaurer@google.com> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> --- kernel/module/Kconfig | 15 +++++++++++++++ scripts/Makefile.modpost | 1 + scripts/mod/modpost.c | 9 +++++++-- 3 files changed, 23 insertions(+), 2 deletions(-) (limited to 'scripts') diff --git a/kernel/module/Kconfig b/kernel/module/Kconfig index 9568b629a03c..4538f3af63e1 100644 --- a/kernel/module/Kconfig +++ b/kernel/module/Kconfig @@ -217,6 +217,21 @@ config EXTENDED_MODVERSIONS The most likely reason you would enable this is to enable Rust support. If unsure, say N. +config BASIC_MODVERSIONS + bool "Basic Module Versioning Support" + depends on MODVERSIONS + default y + help + This enables basic MODVERSIONS support, allowing older tools or + kernels to potentially load modules. + + Disabling this may cause older `modprobe` or `kmod` to be unable + to read MODVERSIONS information from built modules. With this + disabled, older kernels may treat this module as unversioned. + + This is enabled by default when MODVERSIONS are enabled. + If unsure, say Y. + config MODULE_SRCVERSION_ALL bool "Source checksum for all modules" help diff --git a/scripts/Makefile.modpost b/scripts/Makefile.modpost index 40426fc63509..d7d45067d08b 100644 --- a/scripts/Makefile.modpost +++ b/scripts/Makefile.modpost @@ -43,6 +43,7 @@ MODPOST = $(objtree)/scripts/mod/modpost modpost-args = \ $(if $(CONFIG_MODULES),-M) \ $(if $(CONFIG_MODVERSIONS),-m) \ + $(if $(CONFIG_BASIC_MODVERSIONS),-b) \ $(if $(CONFIG_EXTENDED_MODVERSIONS),-x) \ $(if $(CONFIG_MODULE_SRCVERSION_ALL),-a) \ $(if $(CONFIG_SECTION_MISMATCH_WARN_ONLY),,-E) \ diff --git a/scripts/mod/modpost.c b/scripts/mod/modpost.c index 38ff3dd4a9a1..e18ae7dc8140 100644 --- a/scripts/mod/modpost.c +++ b/scripts/mod/modpost.c @@ -33,6 +33,8 @@ static bool module_enabled; static bool modversions; /* Is CONFIG_MODULE_SRCVERSION_ALL set? */ static bool all_versions; +/* Is CONFIG_BASIC_MODVERSIONS set? */ +static bool basic_modversions; /* Is CONFIG_EXTENDED_MODVERSIONS set? */ static bool extended_modversions; /* If we are modposting external module set to 1 */ @@ -1857,7 +1859,7 @@ static void add_versions(struct buffer *b, struct module *mod) { struct symbol *s; - if (!modversions) + if (!basic_modversions) return; buf_printf(b, "\n"); @@ -2177,7 +2179,7 @@ int main(int argc, char **argv) LIST_HEAD(dump_lists); struct dump_list *dl, *dl2; - while ((opt = getopt(argc, argv, "ei:MmnT:to:au:WwENd:x")) != -1) { + while ((opt = getopt(argc, argv, "ei:MmnT:to:au:WwENd:xb")) != -1) { switch (opt) { case 'e': external_module = true; @@ -2226,6 +2228,9 @@ int main(int argc, char **argv) case 'd': missing_namespace_deps = optarg; break; + case 'b': + basic_modversions = true; + break; case 'x': extended_modversions = true; break; -- cgit v1.2.3 From 4093676f8358be23ee5536b8a32dda381862a3cb Mon Sep 17 00:00:00 2001 From: Colin Ian King <colin.i.king@gmail.com> Date: Wed, 13 Nov 2024 10:21:06 +0000 Subject: scripts/spelling.txt: add more spellings to spelling.txt Add some of the more common spelling mistakes and typos that I've found while fixing up spelling mistakes in the kernel over the past year. Link: https://lkml.kernel.org/r/20241113102106.1163050-1-colin.i.king@gmail.com Signed-off-by: Colin Ian King <colin.i.king@gmail.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> --- scripts/spelling.txt | 36 ++++++++++++++++++++++++++++++++++++ 1 file changed, 36 insertions(+) (limited to 'scripts') diff --git a/scripts/spelling.txt b/scripts/spelling.txt index 05bd9ca1fbfa..2decc50f5a6e 100644 --- a/scripts/spelling.txt +++ b/scripts/spelling.txt @@ -222,6 +222,7 @@ autonymous||autonomous auxillary||auxiliary auxilliary||auxiliary avaiable||available +avaialable||available avaible||available availabe||available availabled||available @@ -267,6 +268,7 @@ broadcase||broadcast broadcat||broadcast bufer||buffer bufferred||buffered +bufferur||buffer bufufer||buffer cacluated||calculated caculate||calculate @@ -405,6 +407,7 @@ configutation||configuration congiuration||configuration conider||consider conjuction||conjunction +connction||connection connecetd||connected connectinos||connections connetor||connector @@ -413,6 +416,7 @@ connnections||connections consistancy||consistency consistant||consistent consits||consists +constructred||constructed containes||contains containts||contains contaisn||contains @@ -450,6 +454,7 @@ creationg||creating cryptocraphic||cryptographic cummulative||cumulative cunter||counter +curent||current curently||currently cylic||cyclic dafault||default @@ -461,6 +466,7 @@ decendant||descendant decendants||descendants decompres||decompress decsribed||described +decrese||decrease decription||description detault||default dectected||detected @@ -485,6 +491,7 @@ delare||declare delares||declares delaring||declaring delemiter||delimiter +deley||delay delibrately||deliberately delievered||delivered demodualtor||demodulator @@ -551,6 +558,7 @@ disgest||digest disired||desired dispalying||displaying dissable||disable +dissapeared||disappeared diplay||display directon||direction direcly||directly @@ -606,6 +614,7 @@ eigth||eight elementry||elementary eletronic||electronic embeded||embedded +emtpy||empty enabledi||enabled enbale||enable enble||enable @@ -723,10 +732,12 @@ followign||following followings||following follwing||following fonud||found +forcebly||forcibly forseeable||foreseeable forse||force fortan||fortran forwardig||forwarding +forwared||forwarded frambuffer||framebuffer framming||framing framwork||framework @@ -767,6 +778,7 @@ grahpical||graphical granularty||granularity grapic||graphic grranted||granted +grups||groups guage||gauge guarenteed||guaranteed guarentee||guarantee @@ -780,6 +792,7 @@ hardare||hardware harware||hardware hardward||hardware havind||having +heigth||height heirarchically||hierarchically heirarchy||hierarchy heirachy||hierarchy @@ -788,9 +801,11 @@ hearbeat||heartbeat heterogenous||heterogeneous hexdecimal||hexadecimal hybernate||hibernate +hiearchy||hierarchy hierachy||hierarchy hierarchie||hierarchy homogenous||homogeneous +horizental||horizontal howver||however hsould||should hypervior||hypervisor @@ -842,6 +857,7 @@ independed||independent indiate||indicate indicat||indicate inexpect||inexpected +infalte||inflate inferface||interface infinit||infinite infomation||information @@ -861,6 +877,7 @@ initators||initiators initialiazation||initialization initializationg||initialization initializiation||initialization +initializtion||initialization initialze||initialize initialzed||initialized initialzing||initializing @@ -877,6 +894,7 @@ instanciate||instantiate instanciated||instantiated instuments||instruments insufficent||insufficient +intead||instead inteface||interface integreated||integrated integrety||integrity @@ -1081,6 +1099,7 @@ notications||notifications notifcations||notifications notifed||notified notity||notify +notfify||notify nubmer||number numebr||number numer||number @@ -1122,6 +1141,7 @@ orientatied||orientated orientied||oriented orignal||original originial||original +orphanded||orphaned otherise||otherwise ouput||output oustanding||outstanding @@ -1184,9 +1204,11 @@ peroid||period persistance||persistence persistant||persistent phoneticly||phonetically +pipline||pipeline plaform||platform plalform||platform platfoem||platform +platfomr||platform platfrom||platform plattform||platform pleaes||please @@ -1211,6 +1233,7 @@ preceeding||preceding preceed||precede precendence||precedence precission||precision +predicition||prediction preemptable||preemptible prefered||preferred prefferably||preferably @@ -1289,6 +1312,7 @@ querrying||querying queus||queues randomally||randomly raoming||roaming +readyness||readiness reasearcher||researcher reasearchers||researchers reasearch||research @@ -1305,8 +1329,10 @@ recieves||receives recieving||receiving recogniced||recognised recognizeable||recognizable +recompte||recompute recommanded||recommended recyle||recycle +redect||reject redircet||redirect redirectrion||redirection redundacy||redundancy @@ -1314,6 +1340,7 @@ reename||rename refcounf||refcount refence||reference refered||referred +referencce||reference referenace||reference refererence||reference refering||referring @@ -1348,11 +1375,13 @@ replys||replies reponse||response representaion||representation repsonse||response +reqested||requested reqeust||request reqister||register requed||requeued requestied||requested requiere||require +requieres||requires requirment||requirement requred||required requried||required @@ -1440,6 +1469,7 @@ sequencial||sequential serivce||service serveral||several servive||service +sesion||session setts||sets settting||setting shapshot||snapshot @@ -1602,11 +1632,13 @@ trys||tries thses||these tiggers||triggers tiggered||triggered +tiggerring||triggering tipically||typically timeing||timing timming||timing timout||timeout tmis||this +tolarance||tolerance toogle||toggle torerable||tolerable torlence||tolerance @@ -1633,6 +1665,7 @@ trasfer||transfer trasmission||transmission trasmitter||transmitter treshold||threshold +trigged||triggered triggerd||triggered trigerred||triggered trigerring||triggering @@ -1648,6 +1681,7 @@ uknown||unknown usccess||success uncommited||uncommitted uncompatible||incompatible +uncomressed||uncompressed unconditionaly||unconditionally undeflow||underflow undelying||underlying @@ -1715,6 +1749,7 @@ utitity||utility utitlty||utility vaid||valid vaild||valid +validationg||validating valide||valid variantions||variations varible||variable @@ -1724,6 +1759,7 @@ verbse||verbose veify||verify verfication||verification veriosn||version +versoin||version verisons||versions verison||version veritical||vertical -- cgit v1.2.3 From 10d2711105bd0da774f57bdcd272021b8c453bd3 Mon Sep 17 00:00:00 2001 From: Geert Uytterhoeven <geert+renesas@glider.be> Date: Thu, 5 Dec 2024 14:20:42 +0100 Subject: checkpatch: update reference to include/asm-<arch> Patch series "Update reference to include/asm-<arch>". Despite "include/asm-<arch>" having been replaced by "arch/<arch>/include/asm" 15 years ago, there are still several references left. This patch series updates the most visible ones. This patch (of 3): "include/asm-<arch>" was replaced by "arch/<arch>/include/asm" a long time ago. Link: https://lkml.kernel.org/r/cover.1733404444.git.geert+renesas@glider.be Link: https://lkml.kernel.org/r/2c4a75726a976d117055055b68a31c40dcab044e.1733404444.git.geert+renesas@glider.be Signed-off-by: Geert Uytterhoeven <geert+renesas@glider.be> Cc: Andy Whitcroft <apw@canonical.com> Cc: Arnd Bergmann <arnd@arndb.de> Cc: Dwaipayan Ray <dwaipayanray1@gmail.com> Cc: Joe Perches <joe@perches.com> Cc: Lukas Bulwahn <lukas.bulwahn@gmail.com> Cc: Masahiro Yamada <masahiroy@kernel.org> Cc: Nathan Chancellor <nathan@kernel.org> Cc: Nicolas Schier <nicolas@fjasle.eu> Cc: Oleg Nesterov <oleg@redhat.com> Cc: Rasmus Villemoes <linux@rasmusvillemoes.dk> Cc: Yury Norov <yury.norov@gmail.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> --- scripts/checkpatch.pl | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'scripts') diff --git a/scripts/checkpatch.pl b/scripts/checkpatch.pl index 9eed3683ad76..dbb9c3c6fe30 100755 --- a/scripts/checkpatch.pl +++ b/scripts/checkpatch.pl @@ -2875,7 +2875,7 @@ sub process { if ($realfile =~ m@^include/asm/@) { ERROR("MODIFIED_INCLUDE_ASM", - "do not modify files in include/asm, change architecture specific files in include/asm-<architecture>\n" . "$here$rawline\n"); + "do not modify files in include/asm, change architecture specific files in arch/<architecture>/include/asm\n" . "$here$rawline\n"); } $found_file = 1; } -- cgit v1.2.3 From 3735c5225b97cb57745afd151904d08847b09cc7 Mon Sep 17 00:00:00 2001 From: Tamir Duberstein <tamird@gmail.com> Date: Wed, 4 Dec 2024 13:41:06 -0500 Subject: checkpatch: check return of `git_commit_info` Avoid string concatenation with an undefined variable when a reference to a missing commit is contained in a `Fixes` tag. Given this patch: : From: Tamir Duberstein <tamird@gmail.com> : Subject: Test patch : Date: Fri, 25 Oct 2024 19:30:51 -0400 : : This is a test patch. : : Fixes: deadbeef111 : Signed-off-by: Tamir Duberstein <tamird@gmail.com> : --- /dev/null : +++ b/new-file : @@ -0,0 +1 @@ : +Test. Before: WARNING: Please use correct Fixes: style 'Fixes: <12 chars of sha1> ("<title line>")' - ie: 'Fixes: ("commit title")' WARNING: Unknown commit id 'deadbeef111', maybe rebased or not pulled? Use of uninitialized value $cid in concatenation (.) or string at scripts/checkpatch.pl line 3242. After: WARNING: Unknown commit id 'deadbeef111', maybe rebased or not pulled? This patch also reduce duplication slightly. [akpm@linux-foundation.org: s/12 chars of sha1/12+ chars of sha1/, per Jon] Link: https://lkml.kernel.org/r/87o70kt232.fsf@trenco.lwn.net Link: https://lkml.kernel.org/r/20241204-checkpatch-missing-commit-v1-1-68b34c94944e@gmail.com Signed-off-by: Tamir Duberstein <tamird@gmail.com> Cc: Andy Whitcroft <apw@canonical.com> Cc: Dwaipayan Ray <dwaipayanray1@gmail.com> Cc: Joe Perches <joe@perches.com> Cc: Lukas Bulwahn <lukas.bulwahn@gmail.com> Cc: Jonathan Corbet <corbet@lwn.net> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> --- scripts/checkpatch.pl | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) (limited to 'scripts') diff --git a/scripts/checkpatch.pl b/scripts/checkpatch.pl index dbb9c3c6fe30..2bdc3d169af5 100755 --- a/scripts/checkpatch.pl +++ b/scripts/checkpatch.pl @@ -3237,12 +3237,12 @@ sub process { my ($cid, $ctitle) = git_commit_info($orig_commit, $id, $title); - if ($ctitle ne $title || $tag_case || $tag_space || - $id_length || $id_case || !$title_has_quotes) { + if (defined($cid) && ($ctitle ne $title || $tag_case || $tag_space || $id_length || $id_case || !$title_has_quotes)) { + my $fixed = "Fixes: $cid (\"$ctitle\")"; if (WARN("BAD_FIXES_TAG", - "Please use correct Fixes: style 'Fixes: <12 chars of sha1> (\"<title line>\")' - ie: 'Fixes: $cid (\"$ctitle\")'\n" . $herecurr) && + "Please use correct Fixes: style 'Fixes: <12+ chars of sha1> (\"<title line>\")' - ie: '$fixed'\n" . $herecurr) && $fix) { - $fixed[$fixlinenr] = "Fixes: $cid (\"$ctitle\")"; + $fixed[$fixlinenr] = $fixed; } } } -- cgit v1.2.3 From 551dbd1ec7ff10e06ec077eb264cec0c971b66eb Mon Sep 17 00:00:00 2001 From: Easwar Hariharan <eahariha@linux.microsoft.com> Date: Tue, 10 Dec 2024 22:02:33 +0000 Subject: coccinelle: misc: add secs_to_jiffies script MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit This script finds and suggests conversions of timeout patterns that result in seconds-denominated timeouts to use the new secs_to_jiffies() API in include/linux/jiffies.h for better readability. Link: https://lkml.kernel.org/r/20241210-converge-secs-to-jiffies-v3-2-ddfefd7e9f2a@linux.microsoft.com Signed-off-by: Easwar Hariharan <eahariha@linux.microsoft.com> Suggested-by: Anna-Maria Behnsen <anna-maria@linutronix.de> Cc: Alexander Gordeev <agordeev@linux.ibm.com> Cc: Andrew Lunn <andrew+netdev@lunn.ch> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Christian Borntraeger <borntraeger@linux.ibm.com> Cc: Christophe Leroy <christophe.leroy@csgroup.eu> Cc: Daniel Mack <daniel@zonque.org> Cc: David Airlie <airlied@gmail.com> Cc: David S. Miller <davem@davemloft.net> Cc: Dick Kennedy <dick.kennedy@broadcom.com> Cc: Eric Dumazet <edumazet@google.com> Cc: Florian Fainelli <florian.fainelli@broadcom.com> Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org> Cc: Haojian Zhuang <haojian.zhuang@gmail.com> Cc: Heiko Carstens <hca@linux.ibm.com> Cc: Ilya Dryomov <idryomov@gmail.com> Cc: Jack Wang <jinpu.wang@cloud.ionos.com> Cc: Jakub Kicinski <kuba@kernel.org> Cc: James Bottomley <James.Bottomley@HansenPartnership.com> Cc: James Smart <james.smart@broadcom.com> Cc: Jaroslav Kysela <perex@perex.cz> Cc: Jeff Johnson <jjohnson@kernel.org> Cc: Jeff Johnson <quic_jjohnson@quicinc.com> Cc: Jens Axboe <axboe@kernel.dk> Cc: Jeroen de Borst <jeroendb@google.com> Cc: Jiri Kosina <jikos@kernel.org> Cc: Joe Lawrence <joe.lawrence@redhat.com> Cc: Johan Hedberg <johan.hedberg@gmail.com> Cc: Josh Poimboeuf <jpoimboe@kernel.org> Cc: Jozsef Kadlecsik <kadlec@netfilter.org> Cc: Julia Lawall <julia.lawall@inria.fr> Cc: Kalle Valo <kvalo@kernel.org> Cc: Louis Peens <louis.peens@corigine.com> Cc: Lucas De Marchi <lucas.demarchi@intel.com> Cc: Luiz Augusto von Dentz <luiz.dentz@gmail.com> Cc: Maarten Lankhorst <maarten.lankhorst@linux.intel.com> Cc: Madhavan Srinivasan <maddy@linux.ibm.com> Cc: Marcel Holtmann <marcel@holtmann.org> Cc: Martin K. Petersen <martin.petersen@oracle.com> Cc: Maxime Ripard <mripard@kernel.org> Cc: Michael Ellerman <mpe@ellerman.id.au> Cc: Miroslav Benes <mbenes@suse.cz> Cc: Naveen N Rao <naveen@kernel.org> Cc: Nicholas Piggin <npiggin@gmail.com> Cc: Nicolas Palix <nicolas.palix@imag.fr> Cc: Oded Gabbay <ogabbay@kernel.org> Cc: Ofir Bitton <obitton@habana.ai> Cc: Pablo Neira Ayuso <pablo@netfilter.org> Cc: Paolo Abeni <pabeni@redhat.com> Cc: Petr Mladek <pmladek@suse.com> Cc: Praveen Kaligineedi <pkaligineedi@google.com> Cc: Ray Jui <rjui@broadcom.com> Cc: Robert Jarzmik <robert.jarzmik@free.fr> Cc: Rodrigo Vivi <rodrigo.vivi@intel.com> Cc: Roger Pau Monné <roger.pau@citrix.com> Cc: Russell King <linux@armlinux.org.uk> Cc: Scott Branden <sbranden@broadcom.com> Cc: Shailend Chand <shailend@google.com> Cc: Simona Vetter <simona@ffwll.ch> Cc: Simon Horman <horms@kernel.org> Cc: Sven Schnelle <svens@linux.ibm.com> Cc: Takashi Iwai <tiwai@suse.com> Cc: Thomas Hellström <thomas.hellstrom@linux.intel.com> Cc: Thomas Zimmermann <tzimmermann@suse.de> Cc: Vasily Gorbik <gor@linux.ibm.com> Cc: Xiubo Li <xiubli@redhat.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> --- scripts/coccinelle/misc/secs_to_jiffies.cocci | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) create mode 100644 scripts/coccinelle/misc/secs_to_jiffies.cocci (limited to 'scripts') diff --git a/scripts/coccinelle/misc/secs_to_jiffies.cocci b/scripts/coccinelle/misc/secs_to_jiffies.cocci new file mode 100644 index 000000000000..8bbb2884ea5d --- /dev/null +++ b/scripts/coccinelle/misc/secs_to_jiffies.cocci @@ -0,0 +1,22 @@ +// SPDX-License-Identifier: GPL-2.0-only +/// +/// Find usages of: +/// - msecs_to_jiffies(value*1000) +/// - msecs_to_jiffies(value*MSEC_PER_SEC) +/// +// Confidence: High +// Copyright: (C) 2024 Easwar Hariharan, Microsoft +// Keywords: secs, seconds, jiffies +// + +virtual patch + +@depends on patch@ constant C; @@ + +- msecs_to_jiffies(C * 1000) ++ secs_to_jiffies(C) + +@depends on patch@ constant C; @@ + +- msecs_to_jiffies(C * MSEC_PER_SEC) ++ secs_to_jiffies(C) -- cgit v1.2.3 From 93b6bd40688ba17225ba8c5f28e8ccb713359b05 Mon Sep 17 00:00:00 2001 From: Shivam Chaudhary <cvam0000@gmail.com> Date: Wed, 11 Dec 2024 21:19:03 +0530 Subject: kernel-wide: add explicity||explicitly to spelling.txt Correct the spelling dictionary so that future instances will be caught by checkpatch, and fix the instances found. Link: https://lkml.kernel.org/r/20241211154903.47027-1-cvam0000@gmail.com Signed-off-by: Shivam Chaudhary <cvam0000@gmail.com> Cc: Al Viro <viro@zeniv.linux.org.uk> Cc: Christophe Leroy <christophe.leroy@csgroup.eu> Cc: Dennis Dalessandro <dennis.dalessandro@cornelisnetworks.com> Cc: Jason Gunthorpe <jgg@ziepe.ca> Cc: Leon Romanovsky <leon@kernel.org> Cc: Madhavan Srinivasan <maddy@linux.ibm.com> Cc: Michael Ellerman <mpe@ellerman.id.au> Cc: Naveen N Rao <naveen@kernel.org> Cc: Nicholas Piggin <npiggin@gmail.com> Cc: Shivam Chaudhary <cvam0000@gmail.com> Cc: Colin Ian King <colin.i.king@gmail.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> --- arch/powerpc/kvm/book3s_hv.c | 2 +- drivers/infiniband/hw/hfi1/iowait.h | 2 +- drivers/infiniband/hw/usnic/usnic_abi.h | 2 +- drivers/net/wireless/broadcom/brcm80211/brcmfmac/fwsignal.c | 2 +- drivers/scsi/cxlflash/superpipe.c | 2 +- scripts/spelling.txt | 1 + tools/testing/selftests/pidfd/pidfd_test.c | 2 +- 7 files changed, 7 insertions(+), 6 deletions(-) (limited to 'scripts') diff --git a/arch/powerpc/kvm/book3s_hv.c b/arch/powerpc/kvm/book3s_hv.c index 25429905ae90..86bff159c51e 100644 --- a/arch/powerpc/kvm/book3s_hv.c +++ b/arch/powerpc/kvm/book3s_hv.c @@ -4957,7 +4957,7 @@ int kvmhv_run_single_vcpu(struct kvm_vcpu *vcpu, u64 time_limit, * states are synchronized from L0 to L1. L1 needs to inform L0 about * MER=1 only when there are pending external interrupts. * In the above if check, MER bit is set if there are pending - * external interrupts. Hence, explicity mask off MER bit + * external interrupts. Hence, explicitly mask off MER bit * here as otherwise it may generate spurious interrupts in L2 KVM * causing an endless loop, which results in L2 guest getting hung. */ diff --git a/drivers/infiniband/hw/hfi1/iowait.h b/drivers/infiniband/hw/hfi1/iowait.h index 49805a24bb0a..7259f4f55700 100644 --- a/drivers/infiniband/hw/hfi1/iowait.h +++ b/drivers/infiniband/hw/hfi1/iowait.h @@ -92,7 +92,7 @@ struct iowait_work { * * The lock field is used by waiters to record * the seqlock_t that guards the list head. - * Waiters explicity know that, but the destroy + * Waiters explicitly know that, but the destroy * code that unwaits QPs does not. */ struct iowait { diff --git a/drivers/infiniband/hw/usnic/usnic_abi.h b/drivers/infiniband/hw/usnic/usnic_abi.h index 7fe9502ce8d3..86a82a4da0aa 100644 --- a/drivers/infiniband/hw/usnic/usnic_abi.h +++ b/drivers/infiniband/hw/usnic/usnic_abi.h @@ -72,7 +72,7 @@ struct usnic_ib_create_qp_resp { u64 bar_bus_addr; u32 bar_len; /* - * WQ, RQ, CQ are explicity specified bc exposing a generic resources inteface + * WQ, RQ, CQ are explicitly specified bc exposing a generic resources inteface * expands the scope of ABI to many files. */ u32 wq_cnt; diff --git a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/fwsignal.c b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/fwsignal.c index 0949e7975ff1..b70d20128f98 100644 --- a/drivers/net/wireless/broadcom/brcm80211/brcmfmac/fwsignal.c +++ b/drivers/net/wireless/broadcom/brcm80211/brcmfmac/fwsignal.c @@ -1810,7 +1810,7 @@ void brcmf_fws_rxreorder(struct brcmf_if *ifp, struct sk_buff *pkt) rfi->cur_idx = cur_idx; } } else { - /* explicity window move updating the expected index */ + /* explicitly window move updating the expected index */ exp_idx = reorder_data[BRCMF_RXREORDER_EXPIDX_OFFSET]; brcmf_dbg(DATA, "flow-%d (0x%x): change expected: %d -> %d\n", diff --git a/drivers/scsi/cxlflash/superpipe.c b/drivers/scsi/cxlflash/superpipe.c index b375509d1470..97631f48e19d 100644 --- a/drivers/scsi/cxlflash/superpipe.c +++ b/drivers/scsi/cxlflash/superpipe.c @@ -966,7 +966,7 @@ static int cxlflash_disk_detach(struct scsi_device *sdev, void *detach) * * This routine is the release handler for the fops registered with * the CXL services on an initial attach for a context. It is called - * when a close (explicity by the user or as part of a process tear + * when a close (explicitly by the user or as part of a process tear * down) is performed on the adapter file descriptor returned to the * user. The user should be aware that explicitly performing a close * considered catastrophic and subsequent usage of the superpipe API diff --git a/scripts/spelling.txt b/scripts/spelling.txt index 2decc50f5a6e..a290db720b0f 100644 --- a/scripts/spelling.txt +++ b/scripts/spelling.txt @@ -678,6 +678,7 @@ exmaple||example expecially||especially experies||expires explicite||explicit +explicity||explicitly explicitely||explicitly explict||explicit explictely||explicitly diff --git a/tools/testing/selftests/pidfd/pidfd_test.c b/tools/testing/selftests/pidfd/pidfd_test.c index 9faa686f90e4..e9728e86b4f2 100644 --- a/tools/testing/selftests/pidfd/pidfd_test.c +++ b/tools/testing/selftests/pidfd/pidfd_test.c @@ -497,7 +497,7 @@ static int child_poll_leader_exit_test(void *args) pthread_create(&t2, NULL, test_pidfd_poll_leader_exit_thread, NULL); /* - * glibc exit calls exit_group syscall, so explicity call exit only + * glibc exit calls exit_group syscall, so explicitly call exit only * so that only the group leader exits, leaving the threads alone. */ *child_exit_secs = time(NULL); -- cgit v1.2.3 From 2217573f4c8797eb5ed764a728f986aa80bd403c Mon Sep 17 00:00:00 2001 From: Costa Shulyupin <costa.shul@redhat.com> Date: Mon, 13 Jan 2025 10:55:47 +0200 Subject: scripts/tags.sh: Don't tag usages of DECLARE_BITMAP For all bitmap declarations like DECLARE_BITMAP(x, y); ctags generates multiple DECLARE_BITMAP tags for each usage because it doesn't expand the DECLARE_BITMAP macro. Configure ctags to skip generating tags for DECLARE_BITMAP in such cases. The #define DECLARE_BITMAP itself and declared bitmaps are tagged correctly. Signed-off-by: Costa Shulyupin <costa.shul@redhat.com> Link: https://lore.kernel.org/r/20250113085554.649141-1-costa.shul@redhat.com Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> --- scripts/tags.sh | 1 + 1 file changed, 1 insertion(+) (limited to 'scripts') diff --git a/scripts/tags.sh b/scripts/tags.sh index 7102f14fc775..dba0bb0213eb 100755 --- a/scripts/tags.sh +++ b/scripts/tags.sh @@ -261,6 +261,7 @@ exuberant() # identifiers to ignore by ctags local ign=( ACPI_EXPORT_SYMBOL + DECLARE_BITMAP DEFINE_{TRACE,MUTEX,TIMER} EXPORT_SYMBOL EXPORT_SYMBOL_GPL EXPORT_TRACEPOINT_SYMBOL EXPORT_TRACEPOINT_SYMBOL_GPL -- cgit v1.2.3 From 25ff08aa43e373a61c3e36fc7d7cae88ed0fc2d7 Mon Sep 17 00:00:00 2001 From: Torsten Hilbrich <torsten.hilbrich@secunet.com> Date: Mon, 13 Jan 2025 07:01:29 +0100 Subject: kbuild: Fix signing issue for external modules When running the sign script the kernel is within the source directory of external modules. This caused issues when the kernel uses relative paths, like: make[5]: Entering directory '/build/client/devel/kernel/work/linux-2.6' make[6]: Entering directory '/build/client/devel/addmodules/vtx/work/vtx' INSTALL /build/client/devel/addmodules/vtx/_/lib/modules/6.13.0-devel+/extra/vtx.ko SIGN /build/client/devel/addmodules/vtx/_/lib/modules/6.13.0-devel+/extra/vtx.ko /bin/sh: 1: scripts/sign-file: not found DEPMOD /build/client/devel/addmodules/vtx/_/lib/modules/6.13.0-devel+ Working around it by using absolute pathes here. Fixes: 13b25489b6f8 ("kbuild: change working directory to external module directory with M=") Signed-off-by: Torsten Hilbrich <torsten.hilbrich@secunet.com> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> --- scripts/Makefile.modinst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'scripts') diff --git a/scripts/Makefile.modinst b/scripts/Makefile.modinst index f97c9926ed31..1628198f3e83 100644 --- a/scripts/Makefile.modinst +++ b/scripts/Makefile.modinst @@ -105,7 +105,7 @@ else sig-key := $(CONFIG_MODULE_SIG_KEY) endif quiet_cmd_sign = SIGN $@ - cmd_sign = scripts/sign-file $(CONFIG_MODULE_SIG_HASH) "$(sig-key)" certs/signing_key.x509 $@ \ + cmd_sign = $(objtree)/scripts/sign-file $(CONFIG_MODULE_SIG_HASH) "$(sig-key)" $(objtree)/certs/signing_key.x509 $@ \ $(if $(KBUILD_EXTMOD),|| true) ifeq ($(sign-only),) -- cgit v1.2.3 From 015b0bfe754ae157cfccd7a13c41391124b115f2 Mon Sep 17 00:00:00 2001 From: Masahiro Yamada <masahiroy@kernel.org> Date: Tue, 14 Jan 2025 00:00:39 +0900 Subject: genksyms: rename m_abstract_declarator to abstract_declarator This is called "abstract-declarator" in K&R. [1] I am not sure what "m_" stands for, but the name is clear enough without it. No functional changes are intended. [1] https://cs.wmich.edu/~gupta/teaching/cs4850/sumII06/The%20syntax%20of%20C%20in%20Backus-Naur%20form.htm Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Acked-by: Nicolas Schier <n.schier@avm.de> --- scripts/genksyms/parse.y | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) (limited to 'scripts') diff --git a/scripts/genksyms/parse.y b/scripts/genksyms/parse.y index 689cb6bb40b6..02f2f713ec5a 100644 --- a/scripts/genksyms/parse.y +++ b/scripts/genksyms/parse.y @@ -367,17 +367,17 @@ parameter_declaration_list: ; parameter_declaration: - decl_specifier_seq m_abstract_declarator + decl_specifier_seq abstract_declarator { $$ = $2 ? $2 : $1; } ; -m_abstract_declarator: - ptr_operator m_abstract_declarator +abstract_declarator: + ptr_operator abstract_declarator { $$ = $2 ? $2 : $1; } - | direct_m_abstract_declarator + | direct_abstract_declarator ; -direct_m_abstract_declarator: +direct_abstract_declarator: /* empty */ { $$ = NULL; } | IDENT { /* For version 2 checksums, we don't want to remember @@ -391,13 +391,13 @@ direct_m_abstract_declarator: { remove_node($1); $$ = $1; } - | direct_m_abstract_declarator '(' parameter_declaration_clause ')' + | direct_abstract_declarator '(' parameter_declaration_clause ')' { $$ = $4; } - | direct_m_abstract_declarator '(' error ')' + | direct_abstract_declarator '(' error ')' { $$ = $4; } - | direct_m_abstract_declarator BRACKET_PHRASE + | direct_abstract_declarator BRACKET_PHRASE { $$ = $2; } - | '(' m_abstract_declarator ')' + | '(' abstract_declarator ')' { $$ = $3; } | '(' error ')' { $$ = $3; } -- cgit v1.2.3 From f33bfbd171a03c5d3f64ce956ccdfbece7114da4 Mon Sep 17 00:00:00 2001 From: Masahiro Yamada <masahiroy@kernel.org> Date: Tue, 14 Jan 2025 00:00:40 +0900 Subject: genksyms: rename cvar_qualifier to type_qualifier I believe "cvar" stands for "Const, Volatile, Attribute, or Restrict". This is called "type-qualifier" in K&R. [1] Adopt this more generic naming. No functional changes are intended. [1] https://cs.wmich.edu/~gupta/teaching/cs4850/sumII06/The%20syntax%20of%20C%20in%20Backus-Naur%20form.htm Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Acked-by: Nicolas Schier <n.schier@avm.de> --- scripts/genksyms/parse.y | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) (limited to 'scripts') diff --git a/scripts/genksyms/parse.y b/scripts/genksyms/parse.y index 02f2f713ec5a..8f62b9f0d99c 100644 --- a/scripts/genksyms/parse.y +++ b/scripts/genksyms/parse.y @@ -223,7 +223,7 @@ storage_class_specifier: type_specifier: simple_type_specifier - | cvar_qualifier + | type_qualifier | TYPEOF_KEYW '(' parameter_declaration ')' | TYPEOF_PHRASE @@ -270,21 +270,21 @@ simple_type_specifier: ; ptr_operator: - '*' cvar_qualifier_seq_opt + '*' type_qualifier_seq_opt { $$ = $2 ? $2 : $1; } ; -cvar_qualifier_seq_opt: +type_qualifier_seq_opt: /* empty */ { $$ = NULL; } - | cvar_qualifier_seq + | type_qualifier_seq ; -cvar_qualifier_seq: - cvar_qualifier - | cvar_qualifier_seq cvar_qualifier { $$ = $2; } +type_qualifier_seq: + type_qualifier + | type_qualifier_seq type_qualifier { $$ = $2; } ; -cvar_qualifier: +type_qualifier: CONST_KEYW | VOLATILE_KEYW | ATTRIBUTE_PHRASE | RESTRICT_KEYW { /* restrict has no effect in prototypes so ignore it */ -- cgit v1.2.3 From bc3a812b751ae1a4d91b3ea667ed77e76398bf46 Mon Sep 17 00:00:00 2001 From: Masahiro Yamada <masahiroy@kernel.org> Date: Tue, 14 Jan 2025 00:00:41 +0900 Subject: genksyms: reduce type_qualifier directly to decl_specifier A type_qualifier (const, volatile, etc.) is not a type_specifier. According to K&R [1], a type-qualifier should be directly reduced to a declaration-specifier. <declaration-specifier> ::= <storage-class-specifier> | <type-specifier> | <type-qualifier> [1]: https://cs.wmich.edu/~gupta/teaching/cs4850/sumII06/The%20syntax%20of%20C%20in%20Backus-Naur%20form.htm Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Acked-by: Nicolas Schier <n.schier@avm.de> --- scripts/genksyms/parse.y | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'scripts') diff --git a/scripts/genksyms/parse.y b/scripts/genksyms/parse.y index 8f62b9f0d99c..20cb3db7f149 100644 --- a/scripts/genksyms/parse.y +++ b/scripts/genksyms/parse.y @@ -211,6 +211,7 @@ decl_specifier: $$ = $1; } | type_specifier + | type_qualifier ; storage_class_specifier: @@ -223,7 +224,6 @@ storage_class_specifier: type_specifier: simple_type_specifier - | type_qualifier | TYPEOF_KEYW '(' parameter_declaration ')' | TYPEOF_PHRASE -- cgit v1.2.3 From 3ccda63a3af5f12c9e0b01c06561285227d2f79c Mon Sep 17 00:00:00 2001 From: Masahiro Yamada <masahiroy@kernel.org> Date: Tue, 14 Jan 2025 00:00:42 +0900 Subject: genksyms: fix 6 shift/reduce conflicts and 5 reduce/reduce conflicts The genksyms parser has ambiguities in its grammar, which are currently suppressed by a workaround in scripts/genksyms/Makefile. Building genksyms with W=1 generates the following warnings: YACC scripts/genksyms/parse.tab.[ch] scripts/genksyms/parse.y: warning: 9 shift/reduce conflicts [-Wconflicts-sr] scripts/genksyms/parse.y: warning: 5 reduce/reduce conflicts [-Wconflicts-rr] scripts/genksyms/parse.y: note: rerun with option '-Wcounterexamples' to generate conflict counterexamples The comment in the parser describes the current problem: /* This wasn't really a typedef name but an identifier that shadows one. */ Consider the following simple C code: typedef int foo; void my_func(foo foo) {} In the function parameter list (foo foo), the first 'foo' is a type specifier (typedef'ed as 'int'), while the second 'foo' is an identifier. However, the lexer cannot distinguish between the two. Since 'foo' is already typedef'ed, the lexer returns TYPE for both instances, instead of returning IDENT for the second one. To support shadowed identifiers, TYPE can be reduced to either a simple_type_specifier or a direct_abstract_declarator, which creates a grammatical ambiguity. Without analyzing the grammar context, it is very difficult to resolve this correctly. This commit introduces a flag, dont_want_type_specifier, which allows the parser to inform the lexer whether an identifier is expected. When dont_want_type_specifier is true, the type lookup is suppressed, and the lexer returns IDENT regardless of any preceding typedef. After this commit, only 3 shift/reduce conflicts will remain. Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Acked-by: Nicolas Schier <n.schier@avm.de> --- scripts/genksyms/genksyms.h | 3 +++ scripts/genksyms/lex.l | 9 ++++++++- scripts/genksyms/parse.y | 37 +++++++++++++++---------------------- 3 files changed, 26 insertions(+), 23 deletions(-) (limited to 'scripts') diff --git a/scripts/genksyms/genksyms.h b/scripts/genksyms/genksyms.h index 8c45ada59ece..0c355075f0e6 100644 --- a/scripts/genksyms/genksyms.h +++ b/scripts/genksyms/genksyms.h @@ -12,6 +12,7 @@ #ifndef MODUTILS_GENKSYMS_H #define MODUTILS_GENKSYMS_H 1 +#include <stdbool.h> #include <stdio.h> #include <list_types.h> @@ -66,6 +67,8 @@ struct string_list *copy_list_range(struct string_list *start, int yylex(void); int yyparse(void); +extern bool dont_want_type_specifier; + void error_with_pos(const char *, ...) __attribute__ ((format(printf, 1, 2))); /*----------------------------------------------------------------------*/ diff --git a/scripts/genksyms/lex.l b/scripts/genksyms/lex.l index a4d7495eaf75..e886133af578 100644 --- a/scripts/genksyms/lex.l +++ b/scripts/genksyms/lex.l @@ -12,6 +12,7 @@ %{ #include <limits.h> +#include <stdbool.h> #include <stdlib.h> #include <string.h> #include <ctype.h> @@ -113,6 +114,12 @@ MC_TOKEN ([~%^&*+=|<>/-]=)|(&&)|("||")|(->)|(<<)|(>>) /* The second stage lexer. Here we incorporate knowledge of the state of the parser to tailor the tokens that are returned. */ +/* + * The lexer cannot distinguish whether a typedef'ed string is a TYPE or an + * IDENT. We need a hint from the parser to handle this accurately. + */ +bool dont_want_type_specifier; + int yylex(void) { @@ -207,7 +214,7 @@ repeat: goto repeat; } } - if (!suppress_type_lookup) + if (!suppress_type_lookup && !dont_want_type_specifier) { if (find_symbol(yytext, SYM_TYPEDEF, 1)) token = TYPE; diff --git a/scripts/genksyms/parse.y b/scripts/genksyms/parse.y index 20cb3db7f149..dc575d467bbf 100644 --- a/scripts/genksyms/parse.y +++ b/scripts/genksyms/parse.y @@ -12,6 +12,7 @@ %{ #include <assert.h> +#include <stdbool.h> #include <stdlib.h> #include <string.h> #include "genksyms.h" @@ -148,6 +149,7 @@ simple_declaration: current_name = NULL; } $$ = $3; + dont_want_type_specifier = false; } ; @@ -169,6 +171,7 @@ init_declarator_list: is_typedef ? SYM_TYPEDEF : SYM_NORMAL, decl, is_extern); current_name = NULL; $$ = $1; + dont_want_type_specifier = true; } | init_declarator_list ',' init_declarator { struct string_list *decl = *$3; @@ -184,6 +187,7 @@ init_declarator_list: is_typedef ? SYM_TYPEDEF : SYM_NORMAL, decl, is_extern); current_name = NULL; $$ = $3; + dont_want_type_specifier = true; } ; @@ -210,7 +214,7 @@ decl_specifier: remove_node($1); $$ = $1; } - | type_specifier + | type_specifier { dont_want_type_specifier = true; $$ = $1; } | type_qualifier ; @@ -307,15 +311,7 @@ direct_declarator: current_name = (*$1)->string; $$ = $1; } - } - | TYPE - { if (current_name != NULL) { - error_with_pos("unexpected second declaration name"); - YYERROR; - } else { - current_name = (*$1)->string; - $$ = $1; - } + dont_want_type_specifier = false; } | direct_declarator '(' parameter_declaration_clause ')' { $$ = $4; } @@ -335,8 +331,7 @@ nested_declarator: ; direct_nested_declarator: - IDENT - | TYPE + IDENT { $$ = $1; dont_want_type_specifier = false; } | direct_nested_declarator '(' parameter_declaration_clause ')' { $$ = $4; } | direct_nested_declarator '(' error ')' @@ -362,8 +357,9 @@ parameter_declaration_list_opt: parameter_declaration_list: parameter_declaration + { $$ = $1; dont_want_type_specifier = false; } | parameter_declaration_list ',' parameter_declaration - { $$ = $3; } + { $$ = $3; dont_want_type_specifier = false; } ; parameter_declaration: @@ -375,6 +371,7 @@ abstract_declarator: ptr_operator abstract_declarator { $$ = $2 ? $2 : $1; } | direct_abstract_declarator + { $$ = $1; dont_want_type_specifier = false; } ; direct_abstract_declarator: @@ -385,12 +382,6 @@ direct_abstract_declarator: remove_node($1); $$ = $1; } - /* This wasn't really a typedef name but an identifier that - shadows one. */ - | TYPE - { remove_node($1); - $$ = $1; - } | direct_abstract_declarator '(' parameter_declaration_clause ')' { $$ = $4; } | direct_abstract_declarator '(' error ')' @@ -440,9 +431,9 @@ member_specification: member_declaration: decl_specifier_seq_opt member_declarator_list_opt ';' - { $$ = $3; } + { $$ = $3; dont_want_type_specifier = false; } | error ';' - { $$ = $2; } + { $$ = $2; dont_want_type_specifier = false; } ; member_declarator_list_opt: @@ -452,7 +443,9 @@ member_declarator_list_opt: member_declarator_list: member_declarator - | member_declarator_list ',' member_declarator { $$ = $3; } + { $$ = $1; dont_want_type_specifier = true; } + | member_declarator_list ',' member_declarator + { $$ = $3; dont_want_type_specifier = true; } ; member_declarator: -- cgit v1.2.3 From 668de2b9d48dccdc1b992e07287f15459515fefb Mon Sep 17 00:00:00 2001 From: Masahiro Yamada <masahiroy@kernel.org> Date: Tue, 14 Jan 2025 00:00:43 +0900 Subject: genksyms: fix last 3 shift/reduce conflicts The genksyms parser has ambiguities in its grammar, which are currently suppressed by a workaround in scripts/genksyms/Makefile. Building genksyms with W=1 generates the following warnings: YACC scripts/genksyms/parse.tab.[ch] scripts/genksyms/parse.y: warning: 3 shift/reduce conflicts [-Wconflicts-sr] scripts/genksyms/parse.y: note: rerun with option '-Wcounterexamples' to generate conflict counterexamples The ambiguity arises when decl_specifier_seq is followed by '(' because the following two interpretations are possible: - decl_specifier_seq direct_abstract_declarator '(' parameter_declaration_clause ')' - decl_specifier_seq '(' abstract_declarator ')' This issue occurs because the current parser allows an empty string to be reduced to direct_abstract_declarator, which is incorrect. K&R [1] explains the correct grammar: <parameter-declaration> ::= {<declaration-specifier>}+ <declarator> | {<declaration-specifier>}+ <abstract-declarator> | {<declaration-specifier>}+ <abstract-declarator> ::= <pointer> | <pointer> <direct-abstract-declarator> | <direct-abstract-declarator> <direct-abstract-declarator> ::= ( <abstract-declarator> ) | {<direct-abstract-declarator>}? [ {<constant-expression>}? ] | {<direct-abstract-declarator>}? ( {<parameter-type-list>}? ) This commit resolves all remaining conflicts. We need to consider the difference between the following two examples: [Example 1] ( <abstract-declarator> ) can become <direct-abstract-declarator> void my_func(int (foo)); ... is equivalent to: void my_func(int foo); [Example 2] ( <parameter-type-list> ) can become <direct-abstract-declarator> typedef int foo; void my_func(int (foo)); ... is equivalent to: void my_func(int (*callback)(int)); Please note that the function declaration is identical in both examples, but the preceding typedef creates the distinction. I introduced a new term, open_paren, to enable the type lookup immediately after the '(' token. Without this, we cannot distinguish between [Example 1] and [Example 2]. [1]: https://cs.wmich.edu/~gupta/teaching/cs4850/sumII06/The%20syntax%20of%20C%20in%20Backus-Naur%20form.htm Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Acked-by: Nicolas Schier <n.schier@avm.de> --- scripts/genksyms/parse.y | 28 ++++++++++++++++++++-------- 1 file changed, 20 insertions(+), 8 deletions(-) (limited to 'scripts') diff --git a/scripts/genksyms/parse.y b/scripts/genksyms/parse.y index dc575d467bbf..fafce939c32f 100644 --- a/scripts/genksyms/parse.y +++ b/scripts/genksyms/parse.y @@ -363,35 +363,47 @@ parameter_declaration_list: ; parameter_declaration: - decl_specifier_seq abstract_declarator + decl_specifier_seq abstract_declarator_opt { $$ = $2 ? $2 : $1; } ; +abstract_declarator_opt: + /* empty */ { $$ = NULL; } + | abstract_declarator + ; + abstract_declarator: - ptr_operator abstract_declarator + ptr_operator + | ptr_operator abstract_declarator { $$ = $2 ? $2 : $1; } | direct_abstract_declarator { $$ = $1; dont_want_type_specifier = false; } ; direct_abstract_declarator: - /* empty */ { $$ = NULL; } - | IDENT + IDENT { /* For version 2 checksums, we don't want to remember private parameter names. */ remove_node($1); $$ = $1; } - | direct_abstract_declarator '(' parameter_declaration_clause ')' + | direct_abstract_declarator open_paren parameter_declaration_clause ')' { $$ = $4; } - | direct_abstract_declarator '(' error ')' + | direct_abstract_declarator open_paren error ')' { $$ = $4; } | direct_abstract_declarator BRACKET_PHRASE { $$ = $2; } - | '(' abstract_declarator ')' + | open_paren parameter_declaration_clause ')' { $$ = $3; } - | '(' error ')' + | open_paren abstract_declarator ')' { $$ = $3; } + | open_paren error ')' + { $$ = $3; } + | BRACKET_PHRASE + ; + +open_paren: + '(' { $$ = $1; dont_want_type_specifier = false; } ; function_definition: -- cgit v1.2.3 From a95298656c434357b38bec242412c65dcf6114d1 Mon Sep 17 00:00:00 2001 From: Masahiro Yamada <masahiroy@kernel.org> Date: Tue, 14 Jan 2025 00:00:44 +0900 Subject: genksyms: remove Makefile hack This workaround was introduced for suppressing the reduce/reduce conflict warnings because the %expect-rr directive, which is applicable only to GLR parsers, cannot be used for genksyms. Since there are no longer any conflicts, this Makefile hack is now unnecessary. Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Acked-by: Nicolas Schier <n.schier@avm.de> --- scripts/genksyms/Makefile | 18 ------------------ 1 file changed, 18 deletions(-) (limited to 'scripts') diff --git a/scripts/genksyms/Makefile b/scripts/genksyms/Makefile index 312edccda736..4350311fb7b3 100644 --- a/scripts/genksyms/Makefile +++ b/scripts/genksyms/Makefile @@ -4,24 +4,6 @@ hostprogs-always-y += genksyms genksyms-objs := genksyms.o parse.tab.o lex.lex.o -# FIXME: fix the ambiguous grammar in parse.y and delete this hack -# -# Suppress shift/reduce, reduce/reduce conflicts warnings -# unless W=1 is specified. -# -# Just in case, run "$(YACC) --version" without suppressing stderr -# so that 'bison: not found' will be displayed if it is missing. -ifeq ($(findstring 1,$(KBUILD_EXTRA_WARN)),) - -quiet_cmd_bison_no_warn = $(quiet_cmd_bison) - cmd_bison_no_warn = $(YACC) --version >/dev/null; \ - $(cmd_bison) 2>/dev/null - -$(obj)/pars%.tab.c $(obj)/pars%.tab.h: $(src)/pars%.y FORCE - $(call if_changed,bison_no_warn) - -endif - # -I needed for generated C source to include headers in source tree HOSTCFLAGS_parse.tab.o := -I $(src) HOSTCFLAGS_lex.lex.o := -I $(src) -- cgit v1.2.3 From c2f1846ba87ead7ac544be624c13249d6b90eca0 Mon Sep 17 00:00:00 2001 From: Masahiro Yamada <masahiroy@kernel.org> Date: Tue, 14 Jan 2025 00:00:45 +0900 Subject: genksyms: restrict direct-abstract-declarator to take one parameter-type-list While there is no more grammatical ambiguity in genksyms, the parser logic is still inaccurate. For example, genksyms accepts the following invalid C code: void my_func(int ()(int)); This should result in a syntax error because () cannot be reduced to <direct-abstract-declarator>. ( <abstract-declarator> ) can be reduced, but <abstract-declarator> must not be empty in the following grammar from K&R [1]: <direct-abstract-declarator> ::= ( <abstract-declarator> ) | {<direct-abstract-declarator>}? [ {<constant-expression>}? ] | {<direct-abstract-declarator>}? ( {<parameter-type-list>}? ) Furthermore, genksyms accepts the following weird code: void my_func(int (*callback)(int)(int)(int)); The parser allows <direct-abstract-declarator> to recursively absorb multiple ( {<parameter-type-list>}? ), but this behavior is incorrect. In the example above, (*callback) should be followed by at most one (int). [1]: https://cs.wmich.edu/~gupta/teaching/cs4850/sumII06/The%20syntax%20of%20C%20in%20Backus-Naur%20form.htm Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Acked-by: Nicolas Schier <n.schier@avm.de> --- scripts/genksyms/parse.y | 16 ++++++++++------ 1 file changed, 10 insertions(+), 6 deletions(-) (limited to 'scripts') diff --git a/scripts/genksyms/parse.y b/scripts/genksyms/parse.y index fafce939c32f..03cdd8d53c13 100644 --- a/scripts/genksyms/parse.y +++ b/scripts/genksyms/parse.y @@ -381,20 +381,24 @@ abstract_declarator: ; direct_abstract_declarator: + direct_abstract_declarator1 + | direct_abstract_declarator1 open_paren parameter_declaration_clause ')' + { $$ = $4; } + | open_paren parameter_declaration_clause ')' + { $$ = $3; } + ; + +direct_abstract_declarator1: IDENT { /* For version 2 checksums, we don't want to remember private parameter names. */ remove_node($1); $$ = $1; } - | direct_abstract_declarator open_paren parameter_declaration_clause ')' - { $$ = $4; } - | direct_abstract_declarator open_paren error ')' + | direct_abstract_declarator1 open_paren error ')' { $$ = $4; } - | direct_abstract_declarator BRACKET_PHRASE + | direct_abstract_declarator1 BRACKET_PHRASE { $$ = $2; } - | open_paren parameter_declaration_clause ')' - { $$ = $3; } | open_paren abstract_declarator ')' { $$ = $3; } | open_paren error ')' -- cgit v1.2.3 From aa710cee0d677043f49a447c4665df51a553a2ba Mon Sep 17 00:00:00 2001 From: Masahiro Yamada <masahiroy@kernel.org> Date: Tue, 14 Jan 2025 00:00:46 +0900 Subject: genksyms: restrict direct-declarator to take one parameter-type-list Similar to the previous commit, this change makes the parser logic a little more accurate. Currently, genksyms accepts the following invalid code: struct foo { int (*callback)(int)(int)(int); }; A direct-declarator should not recursively absorb multiple ( parameter-type-list ) constructs. In the example above, (*callback) should be followed by at most one (int). Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Acked-by: Nicolas Schier <n.schier@avm.de> --- scripts/genksyms/parse.y | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) (limited to 'scripts') diff --git a/scripts/genksyms/parse.y b/scripts/genksyms/parse.y index 03cdd8d53c13..33a6aab53b69 100644 --- a/scripts/genksyms/parse.y +++ b/scripts/genksyms/parse.y @@ -331,12 +331,16 @@ nested_declarator: ; direct_nested_declarator: - IDENT { $$ = $1; dont_want_type_specifier = false; } - | direct_nested_declarator '(' parameter_declaration_clause ')' + direct_nested_declarator1 + | direct_nested_declarator1 '(' parameter_declaration_clause ')' { $$ = $4; } - | direct_nested_declarator '(' error ')' + ; + +direct_nested_declarator1: + IDENT { $$ = $1; dont_want_type_specifier = false; } + | direct_nested_declarator1 '(' error ')' { $$ = $4; } - | direct_nested_declarator BRACKET_PHRASE + | direct_nested_declarator1 BRACKET_PHRASE { $$ = $2; } | '(' nested_declarator ')' { $$ = $3; } -- cgit v1.2.3 From ccc11a195c69b0c01ee140aecadfbdcdcdd03605 Mon Sep 17 00:00:00 2001 From: Masahiro Yamada <masahiroy@kernel.org> Date: Tue, 14 Jan 2025 00:00:47 +0900 Subject: genksyms: record attributes consistently for init-declarator I believe the missing action here is a bug. For rules with no explicit action, the following default is used: { $$ = $1; } However, in this case, $1 is the value of attribute_opt itself. As a result, the value of attribute_opt is always NULL. The following test code demonstrates inconsistent behavior. int x __attribute__((__aligned__(4))); int y __attribute__((__aligned__(4))) = 0; The attribute is recorded only when followed by an initializer. This commit adds the correct action to propagate the value of the ATTRIBUTE_PHRASE token. With this change, the attribute in the example above is consistently recorded for both 'x' and 'y'. [Before] $ cat <<EOF | scripts/genksyms/genksyms -d int x __attribute__((__aligned__(4))); int y __attribute__((__aligned__(4))) = 0; EOF Defn for type0 x == <int x > Defn for type0 y == <int y __attribute__ ( ( __aligned__ ( 4 ) ) ) > Hash table occupancy 2/4096 = 0.000488281 [After] $ cat <<EOF | scripts/genksyms/genksyms -d int x __attribute__((__aligned__(4))); int y __attribute__((__aligned__(4))) = 0; EOF Defn for type0 x == <int x __attribute__ ( ( __aligned__ ( 4 ) ) ) > Defn for type0 y == <int y __attribute__ ( ( __aligned__ ( 4 ) ) ) > Hash table occupancy 2/4096 = 0.000488281 Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Acked-by: Nicolas Schier <n.schier@avm.de> --- scripts/genksyms/parse.y | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'scripts') diff --git a/scripts/genksyms/parse.y b/scripts/genksyms/parse.y index 33a6aab53b69..e3c160046143 100644 --- a/scripts/genksyms/parse.y +++ b/scripts/genksyms/parse.y @@ -480,7 +480,7 @@ member_bitfield_declarator: attribute_opt: /* empty */ { $$ = NULL; } - | attribute_opt ATTRIBUTE_PHRASE + | attribute_opt ATTRIBUTE_PHRASE { $$ = $2; } ; enum_body: -- cgit v1.2.3 From ec28bfff83c49b65527f0055e313d9d7c8c04a31 Mon Sep 17 00:00:00 2001 From: Masahiro Yamada <masahiroy@kernel.org> Date: Tue, 14 Jan 2025 00:00:48 +0900 Subject: genksyms: decouple ATTRIBUTE_PHRASE from type-qualifier The __attribute__ keyword can appear in more contexts than 'const' or 'volatile'. To avoid grammatical conflicts with future changes, ATTRIBUTE_PHRASE should not be reduced into type_qualifier. No functional changes are intended. Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Acked-by: Nicolas Schier <n.schier@avm.de> --- scripts/genksyms/parse.y | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) (limited to 'scripts') diff --git a/scripts/genksyms/parse.y b/scripts/genksyms/parse.y index e3c160046143..cd933a95548d 100644 --- a/scripts/genksyms/parse.y +++ b/scripts/genksyms/parse.y @@ -216,6 +216,7 @@ decl_specifier: } | type_specifier { dont_want_type_specifier = true; $$ = $1; } | type_qualifier + | ATTRIBUTE_PHRASE ; storage_class_specifier: @@ -285,11 +286,13 @@ type_qualifier_seq_opt: type_qualifier_seq: type_qualifier + | ATTRIBUTE_PHRASE | type_qualifier_seq type_qualifier { $$ = $2; } + | type_qualifier_seq ATTRIBUTE_PHRASE { $$ = $2; } ; type_qualifier: - CONST_KEYW | VOLATILE_KEYW | ATTRIBUTE_PHRASE + CONST_KEYW | VOLATILE_KEYW | RESTRICT_KEYW { /* restrict has no effect in prototypes so ignore it */ remove_node($1); -- cgit v1.2.3 From 2966b66c94a2b0d897f8626b8f2c50a0fd4878a9 Mon Sep 17 00:00:00 2001 From: Masahiro Yamada <masahiroy@kernel.org> Date: Tue, 14 Jan 2025 00:00:49 +0900 Subject: genksyms: fix syntax error for attribute before abstract_declarator A longstanding issue with genksyms is that it has hidden syntax errors. When a syntax error occurs, yyerror() is called. However, error_with_pos() is a no-op unless the -w option is provided. You can observe syntax errors by manually passing the -w option. For example, with CONFIG_MODVERSIONS=y on v6.13-rc1: $ make -s KCFLAGS=-D__GENKSYMS__ init/main.i $ cat init/main.i | scripts/genksyms/genksyms -w [ snip ] ./include/linux/efi.h:1225: syntax error The syntax error occurs in the following code in include/linux/efi.h: efi_status_t efi_call_acpi_prm_handler(efi_status_t (__efiapi *handler_addr)(u64, void *), u64 param_buffer_addr, void *context); The issue arises from __efiapi, which is defined as either __attribute__((ms_abi)) or __attribute__((regparm(0))). This commit allows abstract_declarator to be prefixed with attributes. To avoid conflicts, I tweaked the rule for decl_specifier_seq. Due to this change, a standalone attribute cannot become decl_specifier_seq. Otherwise, I do not know how to resolve the conflicts. The following code, which was previously accepted by genksyms, will now result in a syntax error: void my_func(__attribute__((unused))x); I do not think it is a big deal because GCC also fails to parse it. $ echo 'void my_func(__attribute__((unused))x);' | gcc -c -x c - <stdin>:1:37: error: unknown type name 'x' Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Acked-by: Nicolas Schier <n.schier@avm.de> --- scripts/genksyms/parse.y | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) (limited to 'scripts') diff --git a/scripts/genksyms/parse.y b/scripts/genksyms/parse.y index cd933a95548d..54e16c2e0b4b 100644 --- a/scripts/genksyms/parse.y +++ b/scripts/genksyms/parse.y @@ -203,8 +203,9 @@ decl_specifier_seq_opt: ; decl_specifier_seq: - decl_specifier { decl_spec = *$1; } + attribute_opt decl_specifier { decl_spec = *$2; } | decl_specifier_seq decl_specifier { decl_spec = *$2; } + | decl_specifier_seq ATTRIBUTE_PHRASE { decl_spec = *$2; } ; decl_specifier: @@ -216,7 +217,6 @@ decl_specifier: } | type_specifier { dont_want_type_specifier = true; $$ = $1; } | type_qualifier - | ATTRIBUTE_PHRASE ; storage_class_specifier: @@ -406,8 +406,8 @@ direct_abstract_declarator1: { $$ = $4; } | direct_abstract_declarator1 BRACKET_PHRASE { $$ = $2; } - | open_paren abstract_declarator ')' - { $$ = $3; } + | open_paren attribute_opt abstract_declarator ')' + { $$ = $4; } | open_paren error ')' { $$ = $3; } | BRACKET_PHRASE -- cgit v1.2.3 From a8b7d066f8626ec847d3e66aef1320968d1fe298 Mon Sep 17 00:00:00 2001 From: Masahiro Yamada <masahiroy@kernel.org> Date: Tue, 14 Jan 2025 00:00:50 +0900 Subject: genksyms: fix syntax error for attribute before nested_declarator A longstanding issue with genksyms is that it has hidden syntax errors. When a syntax error occurs, yyerror() is called. However, error_with_pos() is a no-op unless the -w option is provided. You can observe syntax errors by manually passing the -w option. For example, with CONFIG_MODVERSIONS=y on v6.13-rc1: $ make -s KCFLAGS=-D__GENKSYMS__ drivers/acpi/prmt.i $ cat drivers/acpi/prmt.i | scripts/genksyms/genksyms -w [ snip ] drivers/acpi/prmt.c:56: syntax error The syntax error occurs in the following code in drivers/acpi/prmt.c: struct prm_handler_info { [ snip ] efi_status_t (__efiapi *handler_addr)(u64, void *); [ snip ] }; The issue arises from __efiapi, which is defined as either __attribute__((ms_abi)) or __attribute__((regparm(0))). This commit allows nested_declarator to be prefixed with attributes. Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Acked-by: Nicolas Schier <n.schier@avm.de> --- scripts/genksyms/parse.y | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) (limited to 'scripts') diff --git a/scripts/genksyms/parse.y b/scripts/genksyms/parse.y index 54e16c2e0b4b..49d3e536b9a8 100644 --- a/scripts/genksyms/parse.y +++ b/scripts/genksyms/parse.y @@ -345,8 +345,8 @@ direct_nested_declarator1: { $$ = $4; } | direct_nested_declarator1 BRACKET_PHRASE { $$ = $2; } - | '(' nested_declarator ')' - { $$ = $3; } + | '(' attribute_opt nested_declarator ')' + { $$ = $4; } | '(' error ')' { $$ = $3; } ; -- cgit v1.2.3 From 2ac068cb0b366c61e7aebaccf0240eae8b2c1b43 Mon Sep 17 00:00:00 2001 From: Masahiro Yamada <masahiroy@kernel.org> Date: Tue, 14 Jan 2025 00:00:51 +0900 Subject: genksyms: fix syntax error for attribute after abstact_declarator A longstanding issue with genksyms is that it has hidden syntax errors. When a syntax error occurs, yyerror() is called. However, error_with_pos() is a no-op unless the -w option is provided. You can observe syntax errors by manually passing the -w option. For example, with CONFIG_MODVERSIONS=y on v6.13-rc1: $ make -s KCFLAGS=-D__GENKSYMS__ kernel/module/main.i $ cat kernel/module/main.i | scripts/genksyms/genksyms -w [ snip ] kernel/module/main.c:97: syntax error The syntax error occurs in the following code in kernel/module/main.c: static void __mod_update_bounds(enum mod_mem_type type __maybe_unused, void *base, unsigned int size, struct mod_tree_root *tree) { [ snip ] } The issue arises from __maybe_unused, which is defined as __attribute__((__unused__)). This commit allows direct_abstract_declarator to be followed with attributes. Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Acked-by: Nicolas Schier <n.schier@avm.de> --- scripts/genksyms/parse.y | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) (limited to 'scripts') diff --git a/scripts/genksyms/parse.y b/scripts/genksyms/parse.y index 49d3e536b9a8..82774df50642 100644 --- a/scripts/genksyms/parse.y +++ b/scripts/genksyms/parse.y @@ -383,8 +383,8 @@ abstract_declarator: ptr_operator | ptr_operator abstract_declarator { $$ = $2 ? $2 : $1; } - | direct_abstract_declarator - { $$ = $1; dont_want_type_specifier = false; } + | direct_abstract_declarator attribute_opt + { $$ = $2; dont_want_type_specifier = false; } ; direct_abstract_declarator: -- cgit v1.2.3 From 82db1c29103ebf581484c0b30805e68726121dcb Mon Sep 17 00:00:00 2001 From: Masahiro Yamada <masahiroy@kernel.org> Date: Tue, 14 Jan 2025 00:00:52 +0900 Subject: genksyms: fix syntax error for attribute after 'struct' A longstanding issue with genksyms is that it has hidden syntax errors. When a syntax error occurs, yyerror() is called. However, error_with_pos() is a no-op unless the -w option is provided. You can observe syntax errors by manually passing the -w option. For example, with CONFIG_MODVERSIONS=y on v6.13-rc1: $ make -s KCFLAGS=-D__GENKSYMS__ arch/x86/kernel/cpu/mshyperv.i $ cat arch/x86/kernel/cpu/mshyperv.i | scripts/genksyms/genksyms -w [ snip ] ./arch/x86/include/asm/svm.h:122: syntax error The syntax error occurs in the following code in arch/x86/include/asm/svm.h: struct __attribute__ ((__packed__)) vmcb_control_area { [ snip ] }; The issue arises from __attribute__ immediately after the 'struct' keyword. This commit allows the 'struct' keyword to be followed by attributes. The lexer must be adjusted because dont_want_brace_phase should not be decremented while processing attributes. Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Acked-by: Nicolas Schier <n.schier@avm.de> --- scripts/genksyms/lex.l | 7 ++++++- scripts/genksyms/parse.y | 10 +++++----- 2 files changed, 11 insertions(+), 6 deletions(-) (limited to 'scripts') diff --git a/scripts/genksyms/lex.l b/scripts/genksyms/lex.l index e886133af578..a1f969dcf24f 100644 --- a/scripts/genksyms/lex.l +++ b/scripts/genksyms/lex.l @@ -438,7 +438,12 @@ fini: if (suppress_type_lookup > 0) --suppress_type_lookup; - if (dont_want_brace_phrase > 0) + + /* + * __attribute__() can be placed immediately after the 'struct' keyword. + * e.g.) struct __attribute__((__packed__)) foo { ... }; + */ + if (token != ATTRIBUTE_PHRASE && dont_want_brace_phrase > 0) --dont_want_brace_phrase; yylval = &next_node->next; diff --git a/scripts/genksyms/parse.y b/scripts/genksyms/parse.y index 82774df50642..33639232a709 100644 --- a/scripts/genksyms/parse.y +++ b/scripts/genksyms/parse.y @@ -234,16 +234,16 @@ type_specifier: /* References to s/u/e's defined elsewhere. Rearrange things so that it is easier to expand the definition fully later. */ - | STRUCT_KEYW IDENT - { remove_node($1); (*$2)->tag = SYM_STRUCT; $$ = $2; } + | STRUCT_KEYW attribute_opt IDENT + { remove_node($1); (*$3)->tag = SYM_STRUCT; $$ = $3; } | UNION_KEYW IDENT { remove_node($1); (*$2)->tag = SYM_UNION; $$ = $2; } | ENUM_KEYW IDENT { remove_node($1); (*$2)->tag = SYM_ENUM; $$ = $2; } /* Full definitions of an s/u/e. Record it. */ - | STRUCT_KEYW IDENT class_body - { record_compound($1, $2, $3, SYM_STRUCT); $$ = $3; } + | STRUCT_KEYW attribute_opt IDENT class_body + { record_compound($1, $3, $4, SYM_STRUCT); $$ = $4; } | UNION_KEYW IDENT class_body { record_compound($1, $2, $3, SYM_UNION); $$ = $3; } | ENUM_KEYW IDENT enum_body @@ -254,7 +254,7 @@ type_specifier: | ENUM_KEYW enum_body { add_symbol(NULL, SYM_ENUM, NULL, 0); $$ = $2; } /* Anonymous s/u definitions. Nothing needs doing. */ - | STRUCT_KEYW class_body { $$ = $2; } + | STRUCT_KEYW attribute_opt class_body { $$ = $3; } | UNION_KEYW class_body { $$ = $2; } ; -- cgit v1.2.3 From c6da721f5889b650bc99a77ac3b6dcf86f4cb138 Mon Sep 17 00:00:00 2001 From: David Reaver <me@davidreaver.com> Date: Wed, 8 Jan 2025 11:24:54 -0800 Subject: checkpatch: remove migrated RCU APIs from deprecated_apis The deprecated_apis map was created in [1] so checkpatch would flag deprecated RCU APIs. These deprecated APIs have since been removed from the kernel. This patch removes them from this map so checkpatch doesn't waste time looking for them, and so readers of checkpatch looking for deprecated APIs don't waste time searching for them. Link: https://lore.kernel.org/all/20181111192904.3199-13-paulmck@linux.ibm.com/ [1] Link: https://lkml.kernel.org/r/20250108192456.47871-1-me@davidreaver.com Signed-off-by: David Reaver <me@davidreaver.com> Reviewed-by: Paul E. McKenney <paulmck@kernel.org> Reviewed-by: Kuan-Wei Chiu <visitorckw@gmail.com> Acked-by: Joe Perches <joe@perches.com> Cc: Andy Whitcroft <apw@canonical.com> Cc: Dwaipayan Ray <dwaipayanray1@gmail.com> Cc: Krister Johansen <kjlx@templeofstupid.com> Cc: Lukas Bulwahn <lukas.bulwahn@gmail.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> --- scripts/checkpatch.pl | 10 ---------- 1 file changed, 10 deletions(-) (limited to 'scripts') diff --git a/scripts/checkpatch.pl b/scripts/checkpatch.pl index 2bdc3d169af5..c625da28cdae 100755 --- a/scripts/checkpatch.pl +++ b/scripts/checkpatch.pl @@ -834,16 +834,6 @@ foreach my $entry (@mode_permission_funcs) { $mode_perms_search = "(?:${mode_perms_search})"; our %deprecated_apis = ( - "synchronize_rcu_bh" => "synchronize_rcu", - "synchronize_rcu_bh_expedited" => "synchronize_rcu_expedited", - "call_rcu_bh" => "call_rcu", - "rcu_barrier_bh" => "rcu_barrier", - "synchronize_sched" => "synchronize_rcu", - "synchronize_sched_expedited" => "synchronize_rcu_expedited", - "call_rcu_sched" => "call_rcu", - "rcu_barrier_sched" => "rcu_barrier", - "get_state_synchronize_sched" => "get_state_synchronize_rcu", - "cond_synchronize_sched" => "cond_synchronize_rcu", "kmap" => "kmap_local_page", "kunmap" => "kunmap_local", "kmap_atomic" => "kmap_local_page", -- cgit v1.2.3 From d22feb5b64a4ecb1f029b3266148823c9fe569ee Mon Sep 17 00:00:00 2001 From: Dan Carpenter <dan.carpenter@linaro.org> Date: Fri, 10 Jan 2025 10:12:17 +0300 Subject: checkpatch: don't warn about extra parentheses in staging/ This "Unnecessary parentheses" warning is disabled for drivers/staging unless the --strict option is used. Really, we don't want it at all even if the --strict option is used. Link: https://lkml.kernel.org/r/c7278d21-d96c-4c1e-b3bf-f82b8decc5df@stanley.mountain Signed-off-by: Dan Carpenter <dan.carpenter@linaro.org> Acked-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> Cc: Andy Whitcroft <apw@canonical.com> Cc: Dwaipayan Ray <dwaipayanray1@gmail.com> Cc: Joe Perches <joe@perches.com> Cc: Lukas Bulwahn <lukas.bulwahn@gmail.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> --- scripts/checkpatch.pl | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) (limited to 'scripts') diff --git a/scripts/checkpatch.pl b/scripts/checkpatch.pl index c625da28cdae..9d469c20871f 100755 --- a/scripts/checkpatch.pl +++ b/scripts/checkpatch.pl @@ -5503,9 +5503,9 @@ sub process { } } -# check for unnecessary parentheses around comparisons in if uses -# when !drivers/staging or command-line uses --strict - if (($realfile !~ m@^(?:drivers/staging/)@ || $check_orig) && +# check for unnecessary parentheses around comparisons +# except in drivers/staging + if (($realfile !~ m@^(?:drivers/staging/)@) && $perl_version_ok && defined($stat) && $stat =~ /(^.\s*if\s*($balanced_parens))/) { my $if_stat = $1; -- cgit v1.2.3 From 6494bd2d05f927fc0395c2ea11461517a9e0bb80 Mon Sep 17 00:00:00 2001 From: Masahiro Yamada <masahiroy@kernel.org> Date: Tue, 14 Jan 2025 00:00:53 +0900 Subject: genksyms: fix syntax error for attribute after 'union' A longstanding issue with genksyms is that it has hidden syntax errors. When a syntax error occurs, yyerror() is called. However, error_with_pos() is a no-op unless the -w option is provided. You can observe syntax errors by manually passing the -w option. For example, with CONFIG_MODVERSIONS=y on v6.13-rc1: $ make -s KCFLAGS=-D__GENKSYMS__ fs/lockd/svc.i $ cat fs/lockd/svc.i | scripts/genksyms/genksyms -w [ snip ] ./include/net/addrconf.h:35: syntax error The syntax error occurs in the following code in include/net/addrconf.h: union __packed { [ snip ] }; The issue arises from __packed, which is defined as __attribute__((__packed__)), immediately after the 'union' keyword. This commit allows the 'union' keyword to be followed by attributes. Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Acked-by: Nicolas Schier <n.schier@avm.de> --- scripts/genksyms/parse.y | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) (limited to 'scripts') diff --git a/scripts/genksyms/parse.y b/scripts/genksyms/parse.y index 33639232a709..a2cd035a78c9 100644 --- a/scripts/genksyms/parse.y +++ b/scripts/genksyms/parse.y @@ -236,16 +236,16 @@ type_specifier: so that it is easier to expand the definition fully later. */ | STRUCT_KEYW attribute_opt IDENT { remove_node($1); (*$3)->tag = SYM_STRUCT; $$ = $3; } - | UNION_KEYW IDENT - { remove_node($1); (*$2)->tag = SYM_UNION; $$ = $2; } + | UNION_KEYW attribute_opt IDENT + { remove_node($1); (*$3)->tag = SYM_UNION; $$ = $3; } | ENUM_KEYW IDENT { remove_node($1); (*$2)->tag = SYM_ENUM; $$ = $2; } /* Full definitions of an s/u/e. Record it. */ | STRUCT_KEYW attribute_opt IDENT class_body { record_compound($1, $3, $4, SYM_STRUCT); $$ = $4; } - | UNION_KEYW IDENT class_body - { record_compound($1, $2, $3, SYM_UNION); $$ = $3; } + | UNION_KEYW attribute_opt IDENT class_body + { record_compound($1, $3, $4, SYM_UNION); $$ = $4; } | ENUM_KEYW IDENT enum_body { record_compound($1, $2, $3, SYM_ENUM); $$ = $3; } /* @@ -255,7 +255,7 @@ type_specifier: { add_symbol(NULL, SYM_ENUM, NULL, 0); $$ = $2; } /* Anonymous s/u definitions. Nothing needs doing. */ | STRUCT_KEYW attribute_opt class_body { $$ = $3; } - | UNION_KEYW class_body { $$ = $2; } + | UNION_KEYW attribute_opt class_body { $$ = $3; } ; simple_type_specifier: -- cgit v1.2.3 From c825840527813582385edca3ddeee46886527258 Mon Sep 17 00:00:00 2001 From: Masahiro Yamada <masahiroy@kernel.org> Date: Tue, 14 Jan 2025 00:00:54 +0900 Subject: genksyms: fix syntax error for builtin (u)int*x*_t types A longstanding issue with genksyms is that it has hidden syntax errors. When a syntax error occurs, yyerror() is called. However, error_with_pos() is a no-op unless the -w option is provided. You can observe syntax errors by manually passing the -w option. For example, genksyms fails to parse the following code in arch/arm64/lib/xor-neon.c: static inline uint64x2_t eor3(uint64x2_t p, uint64x2_t q, uint64x2_t r) { [ snip ] } The syntax error occurs because genksyms does not recognize the uint64x2_t keyword. This commit adds support for builtin types described in Arm Neon Intrinsics Reference. Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Acked-by: Nicolas Schier <n.schier@avm.de> --- scripts/genksyms/lex.l | 1 + 1 file changed, 1 insertion(+) (limited to 'scripts') diff --git a/scripts/genksyms/lex.l b/scripts/genksyms/lex.l index a1f969dcf24f..22aeb57649d9 100644 --- a/scripts/genksyms/lex.l +++ b/scripts/genksyms/lex.l @@ -51,6 +51,7 @@ MC_TOKEN ([~%^&*+=|<>/-]=)|(&&)|("||")|(->)|(<<)|(>>) %% +u?int(8|16|32|64)x(1|2|4|8|16)_t return BUILTIN_INT_KEYW; /* Keep track of our location in the original source files. */ ^#[ \t]+{INT}[ \t]+\"[^\"\n]+\".*\n return FILENAME; -- cgit v1.2.3 From a23d4c2f5b80a8dc5f1e40658abbe5983af1a0e9 Mon Sep 17 00:00:00 2001 From: Masahiro Yamada <masahiroy@kernel.org> Date: Tue, 14 Jan 2025 00:00:55 +0900 Subject: genksyms: fix syntax error for attribute before init-declarator A longstanding issue with genksyms is that it has hidden syntax errors. For example, genksyms fails to parse the following valid code: int x, __attribute__((__section__(".init.data")))y; Here, only 'y' is annotated by the attribute, although I am not aware of actual uses of this pattern in the kernel tree. When a syntax error occurs, yyerror() is called. However, error_with_pos() is a no-op unless the -w option is provided. You can observe syntax errors by manually passing the -w option. $ echo 'int x, __attribute__((__section__(".init.data")))y;' | scripts/genksyms/genksyms -w <stdin>:1: syntax error This commit allows attributes to be placed between a comma and init_declarator. Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Acked-by: Nicolas Schier <n.schier@avm.de> --- scripts/genksyms/parse.y | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) (limited to 'scripts') diff --git a/scripts/genksyms/parse.y b/scripts/genksyms/parse.y index a2cd035a78c9..ee600a804fa1 100644 --- a/scripts/genksyms/parse.y +++ b/scripts/genksyms/parse.y @@ -173,9 +173,9 @@ init_declarator_list: $$ = $1; dont_want_type_specifier = true; } - | init_declarator_list ',' init_declarator - { struct string_list *decl = *$3; - *$3 = NULL; + | init_declarator_list ',' attribute_opt init_declarator + { struct string_list *decl = *$4; + *$4 = NULL; free_list(*$2, NULL); *$2 = decl_spec; @@ -186,7 +186,7 @@ init_declarator_list: add_symbol(current_name, is_typedef ? SYM_TYPEDEF : SYM_NORMAL, decl, is_extern); current_name = NULL; - $$ = $3; + $$ = $4; dont_want_type_specifier = true; } ; -- cgit v1.2.3 From a314f52a0210730d0d556de76bb7388e76d4597d Mon Sep 17 00:00:00 2001 From: Masahiro Yamada <masahiroy@kernel.org> Date: Mon, 20 Jan 2025 16:59:14 +0900 Subject: kconfig: fix file name in warnings when loading KCONFIG_DEFCONFIG_LIST Most 'make *config' commands use .config as the base configuration file. When .config does not exist, Kconfig tries to load a file listed in KCONFIG_DEFCONFIG_LIST instead. However, since commit b75b0a819af9 ("kconfig: change defconfig_list option to environment variable"), warning messages have displayed an incorrect file name in such cases. Below is a demonstration using Debian Trixie. While loading /boot/config-6.12.9-amd64, the warning messages incorrectly show .config as the file name. With this commit, the correct file name is displayed in warnings. [Before] $ rm -f .config $ make config # # using defaults found in /boot/config-6.12.9-amd64 # .config:6804:warning: symbol value 'm' invalid for FB_BACKLIGHT .config:9895:warning: symbol value 'm' invalid for ANDROID_BINDER_IPC [After] $ rm -f .config $ make config # # using defaults found in /boot/config-6.12.9-amd64 # /boot/config-6.12.9-amd64:6804:warning: symbol value 'm' invalid for FB_BACKLIGHT /boot/config-6.12.9-amd64:9895:warning: symbol value 'm' invalid for ANDROID_BINDER_IPC Fixes: b75b0a819af9 ("kconfig: change defconfig_list option to environment variable") Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> --- scripts/kconfig/confdata.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) (limited to 'scripts') diff --git a/scripts/kconfig/confdata.c b/scripts/kconfig/confdata.c index 4286d5e7f95d..3b55e7a4131d 100644 --- a/scripts/kconfig/confdata.c +++ b/scripts/kconfig/confdata.c @@ -360,10 +360,12 @@ int conf_read_simple(const char *name, int def) *p = '\0'; - in = zconf_fopen(env); + name = env; + + in = zconf_fopen(name); if (in) { conf_message("using defaults found in %s", - env); + name); goto load; } -- cgit v1.2.3 From a409fc1463d664002ea9bf700ae4674df03de111 Mon Sep 17 00:00:00 2001 From: Masahiro Yamada <masahiroy@kernel.org> Date: Mon, 20 Jan 2025 17:10:31 +0900 Subject: kconfig: fix memory leak in sym_warn_unmet_dep() The string allocated in sym_warn_unmet_dep() is never freed, leading to a memory leak when an unmet dependency is detected. Fixes: f8f69dc0b4e0 ("kconfig: make unmet dependency warnings readable") Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Reviewed-by: Petr Vorel <pvorel@suse.cz> --- scripts/kconfig/symbol.c | 1 + 1 file changed, 1 insertion(+) (limited to 'scripts') diff --git a/scripts/kconfig/symbol.c b/scripts/kconfig/symbol.c index 89b84bf8e21f..7beb59dec5a0 100644 --- a/scripts/kconfig/symbol.c +++ b/scripts/kconfig/symbol.c @@ -388,6 +388,7 @@ static void sym_warn_unmet_dep(const struct symbol *sym) " Selected by [m]:\n"); fputs(str_get(&gs), stderr); + str_free(&gs); sym_warnings++; } -- cgit v1.2.3 From 101971298be2aa4706c8602bd81066a0f6f2ced5 Mon Sep 17 00:00:00 2001 From: Yunhui Cui <cuiyunhui@bytedance.com> Date: Wed, 14 Aug 2024 14:26:25 +0800 Subject: riscv: add a warning when physical memory address overflows The part of physical memory that exceeds the size of the linear mapping will be discarded. When the system starts up normally, a warning message will be printed to prevent confusion caused by the mismatch between the system memory and the actual physical memory. Signed-off-by: Yunhui Cui <cuiyunhui@bytedance.com> Reviewed-by: Alexandre Ghiti <alexghiti@rivosinc.com> Tested-by: Alexandre Ghiti <alexghiti@rivosinc.com> Link: https://lore.kernel.org/r/20240814062625.19794-1-cuiyunhui@bytedance.com Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com> --- arch/riscv/mm/init.c | 8 ++++++-- scripts/selinux/genheaders/genheaders | Bin 0 -> 90112 bytes 2 files changed, 6 insertions(+), 2 deletions(-) create mode 100755 scripts/selinux/genheaders/genheaders (limited to 'scripts') diff --git a/arch/riscv/mm/init.c b/arch/riscv/mm/init.c index 0e8c20adcd98..9641e4ad387f 100644 --- a/arch/riscv/mm/init.c +++ b/arch/riscv/mm/init.c @@ -256,8 +256,12 @@ static void __init setup_bootmem(void) */ if (IS_ENABLED(CONFIG_64BIT) && IS_ENABLED(CONFIG_MMU)) { max_mapped_addr = __pa(PAGE_OFFSET) + KERN_VIRT_SIZE; - memblock_cap_memory_range(phys_ram_base, - max_mapped_addr - phys_ram_base); + if (memblock_end_of_DRAM() > max_mapped_addr) { + memblock_cap_memory_range(phys_ram_base, + max_mapped_addr - phys_ram_base); + pr_warn("Physical memory overflows the linear mapping size: region above %pa removed", + &max_mapped_addr); + } } /* diff --git a/scripts/selinux/genheaders/genheaders b/scripts/selinux/genheaders/genheaders new file mode 100755 index 000000000000..3fc32a664a79 Binary files /dev/null and b/scripts/selinux/genheaders/genheaders differ -- cgit v1.2.3 From dce4aab8441d285b9a78b33753e0bf583c1320ee Mon Sep 17 00:00:00 2001 From: Kees Cook <kees@kernel.org> Date: Mon, 27 Jan 2025 11:10:28 -0800 Subject: kbuild: Use -fzero-init-padding-bits=all GCC 15 introduces a regression in "= { 0 }" style initialization of unions that Linux has depended on for eliminating uninitialized variable contents. GCC does not seem likely to fix it[1], instead suggesting[2] that affected projects start using -fzero-init-padding-bits=unions. To avoid future surprises beyond just the current situation with unions, enable -fzero-init-padding-bits=all when available (GCC 15+). This will correctly zero padding bits in unions and structs that might have been left uninitialized, and will make sure there is no immediate regression in union initializations. As seen in the stackinit KUnit selftest union cases, which were passing before, were failing under GCC 15: not ok 18 test_small_start_old_zero ok 29 test_small_start_dynamic_partial # SKIP XFAIL uninit bytes: 63 ok 32 test_small_start_assigned_dynamic_partial # SKIP XFAIL uninit bytes: 63 ok 67 test_small_start_static_partial # SKIP XFAIL uninit bytes: 63 ok 70 test_small_start_static_all # SKIP XFAIL uninit bytes: 56 ok 73 test_small_start_dynamic_all # SKIP XFAIL uninit bytes: 56 ok 82 test_small_start_assigned_static_partial # SKIP XFAIL uninit bytes: 63 ok 85 test_small_start_assigned_static_all # SKIP XFAIL uninit bytes: 56 ok 88 test_small_start_assigned_dynamic_all # SKIP XFAIL uninit bytes: 56 The above all now pass again with -fzero-init-padding-bits=all added. This also fixes the following cases for struct initialization that had been XFAIL until now because there was no compiler support beyond the larger "-ftrivial-auto-var-init=zero" option: ok 38 test_small_hole_static_all # SKIP XFAIL uninit bytes: 3 ok 39 test_big_hole_static_all # SKIP XFAIL uninit bytes: 124 ok 40 test_trailing_hole_static_all # SKIP XFAIL uninit bytes: 7 ok 42 test_small_hole_dynamic_all # SKIP XFAIL uninit bytes: 3 ok 43 test_big_hole_dynamic_all # SKIP XFAIL uninit bytes: 124 ok 44 test_trailing_hole_dynamic_all # SKIP XFAIL uninit bytes: 7 ok 58 test_small_hole_assigned_static_all # SKIP XFAIL uninit bytes: 3 ok 59 test_big_hole_assigned_static_all # SKIP XFAIL uninit bytes: 124 ok 60 test_trailing_hole_assigned_static_all # SKIP XFAIL uninit bytes: 7 ok 62 test_small_hole_assigned_dynamic_all # SKIP XFAIL uninit bytes: 3 ok 63 test_big_hole_assigned_dynamic_all # SKIP XFAIL uninit bytes: 124 ok 64 test_trailing_hole_assigned_dynamic_all # SKIP XFAIL uninit bytes: 7 All of the above now pass when built under GCC 15. Tests can be seen with: ./tools/testing/kunit/kunit.py run stackinit --arch=x86_64 \ --make_option CC=gcc-15 Clang continues to fully initialize these kinds of variables[3] without additional flags. Suggested-by: Jakub Jelinek <jakub@redhat.com> Link: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=118403 [1] Link: https://lore.kernel.org/linux-toolchains/Z0hRrrNU3Q+ro2T7@tucnak/ [2] Link: https://github.com/llvm/llvm-project/commit/7a086e1b2dc05f54afae3591614feede727601fa [3] Reviewed-by: Nathan Chancellor <nathan@kernel.org> Acked-by: Masahiro Yamada <masahiroy@kernel.org> Link: https://lore.kernel.org/r/20250127191031.245214-3-kees@kernel.org Signed-off-by: Kees Cook <kees@kernel.org> --- scripts/Makefile.extrawarn | 3 +++ 1 file changed, 3 insertions(+) (limited to 'scripts') diff --git a/scripts/Makefile.extrawarn b/scripts/Makefile.extrawarn index 1d13cecc7cc7..eb719f6d8d53 100644 --- a/scripts/Makefile.extrawarn +++ b/scripts/Makefile.extrawarn @@ -77,6 +77,9 @@ KBUILD_CFLAGS += $(call cc-option,-Werror=designated-init) # Warn if there is an enum types mismatch KBUILD_CFLAGS += $(call cc-option,-Wenum-conversion) +# Explicitly clear padding bits during variable initialization +KBUILD_CFLAGS += $(call cc-option,-fzero-init-padding-bits=all) + KBUILD_CFLAGS += -Wextra KBUILD_CFLAGS += -Wunused -- cgit v1.2.3 From 71d815bf5dfd4f63f7557e0abe7f257c202863a1 Mon Sep 17 00:00:00 2001 From: Ard Biesheuvel <ardb@kernel.org> Date: Mon, 13 Jan 2025 16:53:07 +0100 Subject: kbuild: Strip runtime const RELA sections correctly Due to the fact that runtime const ELF sections are named without a leading period or double underscore, the RSTRIP logic that removes the static RELA sections from vmlinux fails to identify them. This results in a situation like below, where some sections that were supposed to get removed are left behind. [Nr] Name Type Address Off Size ES Flg Lk Inf Al [58] runtime_shift_d_hash_shift PROGBITS ffffffff83500f50 2900f50 000014 00 A 0 0 1 [59] .relaruntime_shift_d_hash_shift RELA 0000000000000000 55b6f00 000078 18 I 70 58 8 [60] runtime_ptr_dentry_hashtable PROGBITS ffffffff83500f68 2900f68 000014 00 A 0 0 1 [61] .relaruntime_ptr_dentry_hashtable RELA 0000000000000000 55b6f78 000078 18 I 70 60 8 [62] runtime_ptr_USER_PTR_MAX PROGBITS ffffffff83500f80 2900f80 000238 00 A 0 0 1 [63] .relaruntime_ptr_USER_PTR_MAX RELA 0000000000000000 55b6ff0 000d50 18 I 70 62 8 So tweak the match expression to strip all sections starting with .rel. While at it, consolidate the logic used by RISC-V, s390 and x86 into a single shared Makefile library command. Link: https://lore.kernel.org/all/CAHk-=wjk3ynjomNvFN8jf9A1k=qSc=JFF591W00uXj-qqNUxPQ@mail.gmail.com/ Signed-off-by: Ard Biesheuvel <ardb@kernel.org> Reviewed-by: Charlie Jenkins <charlie@rivosinc.com> Tested-by: Charlie Jenkins <charlie@rivosinc.com> Tested-by: Alexander Gordeev <agordeev@linux.ibm.com> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> --- arch/riscv/Makefile.postlink | 8 ++------ arch/s390/Makefile.postlink | 6 +----- arch/x86/Makefile.postlink | 6 +----- scripts/Makefile.lib | 3 +++ 4 files changed, 7 insertions(+), 16 deletions(-) (limited to 'scripts') diff --git a/arch/riscv/Makefile.postlink b/arch/riscv/Makefile.postlink index 829b9abc91f6..6b0580949b6a 100644 --- a/arch/riscv/Makefile.postlink +++ b/arch/riscv/Makefile.postlink @@ -10,6 +10,7 @@ __archpost: -include include/config/auto.conf include $(srctree)/scripts/Kbuild.include +include $(srctree)/scripts/Makefile.lib quiet_cmd_relocs_check = CHKREL $@ cmd_relocs_check = \ @@ -19,11 +20,6 @@ ifdef CONFIG_RELOCATABLE quiet_cmd_cp_vmlinux_relocs = CPREL vmlinux.relocs cmd_cp_vmlinux_relocs = cp vmlinux vmlinux.relocs -quiet_cmd_relocs_strip = STRIPREL $@ -cmd_relocs_strip = $(OBJCOPY) --remove-section='.rel.*' \ - --remove-section='.rel__*' \ - --remove-section='.rela.*' \ - --remove-section='.rela__*' $@ endif # `@true` prevents complaint when there is nothing to be done @@ -33,7 +29,7 @@ vmlinux: FORCE ifdef CONFIG_RELOCATABLE $(call if_changed,relocs_check) $(call if_changed,cp_vmlinux_relocs) - $(call if_changed,relocs_strip) + $(call if_changed,strip_relocs) endif clean: diff --git a/arch/s390/Makefile.postlink b/arch/s390/Makefile.postlink index df82f5410769..1ae5478cd6ac 100644 --- a/arch/s390/Makefile.postlink +++ b/arch/s390/Makefile.postlink @@ -11,6 +11,7 @@ __archpost: -include include/config/auto.conf include $(srctree)/scripts/Kbuild.include +include $(srctree)/scripts/Makefile.lib CMD_RELOCS=arch/s390/tools/relocs OUT_RELOCS = arch/s390/boot @@ -19,11 +20,6 @@ quiet_cmd_relocs = RELOCS $(OUT_RELOCS)/relocs.S mkdir -p $(OUT_RELOCS); \ $(CMD_RELOCS) $@ > $(OUT_RELOCS)/relocs.S -quiet_cmd_strip_relocs = RSTRIP $@ - cmd_strip_relocs = \ - $(OBJCOPY) --remove-section='.rel.*' --remove-section='.rel__*' \ - --remove-section='.rela.*' --remove-section='.rela__*' $@ - vmlinux: FORCE $(call cmd,relocs) $(call cmd,strip_relocs) diff --git a/arch/x86/Makefile.postlink b/arch/x86/Makefile.postlink index fef2e977cc7d..8b8a68162c94 100644 --- a/arch/x86/Makefile.postlink +++ b/arch/x86/Makefile.postlink @@ -11,6 +11,7 @@ __archpost: -include include/config/auto.conf include $(srctree)/scripts/Kbuild.include +include $(srctree)/scripts/Makefile.lib CMD_RELOCS = arch/x86/tools/relocs OUT_RELOCS = arch/x86/boot/compressed @@ -20,11 +21,6 @@ quiet_cmd_relocs = RELOCS $(OUT_RELOCS)/$@.relocs $(CMD_RELOCS) $@ > $(OUT_RELOCS)/$@.relocs; \ $(CMD_RELOCS) --abs-relocs $@ -quiet_cmd_strip_relocs = RSTRIP $@ - cmd_strip_relocs = \ - $(OBJCOPY) --remove-section='.rel.*' --remove-section='.rel__*' \ - --remove-section='.rela.*' --remove-section='.rela__*' $@ - # `@true` prevents complaint when there is nothing to be done vmlinux: FORCE diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib index 7395200538da..f604f51d23ca 100644 --- a/scripts/Makefile.lib +++ b/scripts/Makefile.lib @@ -374,6 +374,9 @@ quiet_cmd_ar = AR $@ quiet_cmd_objcopy = OBJCOPY $@ cmd_objcopy = $(OBJCOPY) $(OBJCOPYFLAGS) $(OBJCOPYFLAGS_$(@F)) $< $@ +quiet_cmd_strip_relocs = RSTRIP $@ +cmd_strip_relocs = $(OBJCOPY) --remove-section='.rel*' $@ + # Gzip # --------------------------------------------------------------------------- -- cgit v1.2.3 From 695ed93bb30e03e9f826ee70abdd83f970741a37 Mon Sep 17 00:00:00 2001 From: Masahiro Yamada <masahiroy@kernel.org> Date: Fri, 31 Jan 2025 23:04:01 +0900 Subject: kbuild: fix Clang LTO with CONFIG_OBJTOOL=n Since commit bede169618c6 ("kbuild: enable objtool for *.mod.o and additional kernel objects"), Clang LTO builds do not perform any optimizations when CONFIG_OBJTOOL is disabled (e.g., for ARCH=arm64). This is because every LLVM bitcode file is immediately converted to ELF format before the object files are linked together. This commit fixes the breakage. Fixes: bede169618c6 ("kbuild: enable objtool for *.mod.o and additional kernel objects") Reported-by: Yonghong Song <yonghong.song@linux.dev> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Tested-by: Yonghong Song <yonghong.song@linux.dev> --- scripts/Makefile.build | 2 ++ scripts/Makefile.lib | 10 ++++++---- 2 files changed, 8 insertions(+), 4 deletions(-) (limited to 'scripts') diff --git a/scripts/Makefile.build b/scripts/Makefile.build index 81d9dacad03c..993708d11874 100644 --- a/scripts/Makefile.build +++ b/scripts/Makefile.build @@ -194,7 +194,9 @@ endif # CONFIG_FTRACE_MCOUNT_USE_RECORDMCOUNT is-standard-object = $(if $(filter-out y%, $(OBJECT_FILES_NON_STANDARD_$(target-stem).o)$(OBJECT_FILES_NON_STANDARD)n),$(is-kernel-object)) +ifdef CONFIG_OBJTOOL $(obj)/%.o: private objtool-enabled = $(if $(is-standard-object),$(if $(delay-objtool),$(is-single-obj-m),y)) +endif ifneq ($(findstring 1, $(KBUILD_EXTRA_WARN)),) cmd_warn_shared_object = $(if $(word 2, $(modname-multi)),$(warning $(kbuild-file): $*.o is added to multiple modules: $(modname-multi))) diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib index f604f51d23ca..ad55ef201aac 100644 --- a/scripts/Makefile.lib +++ b/scripts/Makefile.lib @@ -287,6 +287,8 @@ delay-objtool := $(or $(CONFIG_LTO_CLANG),$(CONFIG_X86_KERNEL_IBT)) cmd_objtool = $(if $(objtool-enabled), ; $(objtool) $(objtool-args) $@) cmd_gen_objtooldep = $(if $(objtool-enabled), { echo ; echo '$@: $$(wildcard $(objtool))' ; } >> $(dot-target).cmd) +objtool-enabled := y + endif # CONFIG_OBJTOOL # Useful for describing the dependency of composite objects @@ -302,11 +304,11 @@ endef # =========================================================================== # These are shared by some Makefile.* files. -objtool-enabled := y - ifdef CONFIG_LTO_CLANG -# objtool cannot process LLVM IR. Make $(LD) covert LLVM IR to ELF here. -cmd_ld_single = $(if $(objtool-enabled), ; $(LD) $(ld_flags) -r -o $(tmp-target) $@; mv $(tmp-target) $@) +# Run $(LD) here to covert LLVM IR to ELF in the following cases: +# - when this object needs objtool processing, as objtool cannot process LLVM IR +# - when this is a single-object module, as modpost cannot process LLVM IR +cmd_ld_single = $(if $(objtool-enabled)$(is-single-obj-m), ; $(LD) $(ld_flags) -r -o $(tmp-target) $@; mv $(tmp-target) $@) endif quiet_cmd_cc_o_c = CC $(quiet_modtag) $@ -- cgit v1.2.3 From 04a3389b35357e9bf44533d20a80eb70d188adb8 Mon Sep 17 00:00:00 2001 From: Linus Torvalds <torvalds@linux-foundation.org> Date: Fri, 31 Jan 2025 19:49:17 -0800 Subject: Remove stale generated 'genheaders' file This bogus stale file was added in commit 101971298be2 ("riscv: add a warning when physical memory address overflows"). It's the old location for what is now 'security/selinux/genheaders'. It looks like it got incorrectly committed back when that file was in the old location, and then rebasing kept the bogus file alive. Reported-by: Eric Biggers <ebiggers@kernel.org> Link: https://lore.kernel.org/linux-riscv/20250201020003.GA77370@sol.localdomain/ Fixes: 101971298be2 ("riscv: add a warning when physical memory address overflows") Cc: Palmer Dabbelt <palmer@rivosinc.com> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org> --- scripts/selinux/genheaders/genheaders | Bin 90112 -> 0 bytes 1 file changed, 0 insertions(+), 0 deletions(-) delete mode 100755 scripts/selinux/genheaders/genheaders (limited to 'scripts') diff --git a/scripts/selinux/genheaders/genheaders b/scripts/selinux/genheaders/genheaders deleted file mode 100755 index 3fc32a664a79..000000000000 Binary files a/scripts/selinux/genheaders/genheaders and /dev/null differ -- cgit v1.2.3 From 4ebc417ef9cb34010a71270421fe320ec5d88aa2 Mon Sep 17 00:00:00 2001 From: Jan Kiszka <jan.kiszka@siemens.com> Date: Fri, 10 Jan 2025 11:36:33 +0100 Subject: scripts/gdb: fix aarch64 userspace detection in get_current_task At least recent gdb releases (seen with 14.2) return SP_EL0 as signed long which lets the right-shift always return 0. Link: https://lkml.kernel.org/r/dcd2fabc-9131-4b48-8419-6444e2d67454@siemens.com Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com> Cc: Barry Song <baohua@kernel.org> Cc: Kieran Bingham <kbingham@kernel.org> Cc: <stable@vger.kernel.org> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> --- scripts/gdb/linux/cpus.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'scripts') diff --git a/scripts/gdb/linux/cpus.py b/scripts/gdb/linux/cpus.py index 2f11c4f9c345..13eb8b3901b8 100644 --- a/scripts/gdb/linux/cpus.py +++ b/scripts/gdb/linux/cpus.py @@ -167,7 +167,7 @@ def get_current_task(cpu): var_ptr = gdb.parse_and_eval("&pcpu_hot.current_task") return per_cpu(var_ptr, cpu).dereference() elif utils.is_target_arch("aarch64"): - current_task_addr = gdb.parse_and_eval("$SP_EL0") + current_task_addr = gdb.parse_and_eval("(unsigned long)$SP_EL0") if (current_task_addr >> 63) != 0: current_task = current_task_addr.cast(task_ptr_type) return current_task.dereference() -- cgit v1.2.3 From 738fc998b639407346a9e026514f0562301462cd Mon Sep 17 00:00:00 2001 From: Nathan Chancellor <nathan@kernel.org> Date: Fri, 31 Jan 2025 15:55:28 -0700 Subject: scripts/Makefile.extrawarn: Do not show clang's non-kprintf warnings at W=1 Clang's -Wformat-overflow and -Wformat-truncation have chosen to check '%p' unlike GCC but it does not know about the kernel's pointer extensions in lib/vsprintf.c, so the developers split that part of the warning out for the kernel to disable because there will always be false positives. Commit 908dd508276d ("kbuild: enable -Wformat-truncation on clang") did disabled these warnings but only in a block that would be called when W=1 was not passed, so they would appear with W=1. Move the disabling of the non-kprintf warnings to a block that always runs so that they are never seen, regardless of warning level. Fixes: 908dd508276d ("kbuild: enable -Wformat-truncation on clang") Reported-by: kernel test robot <lkp@intel.com> Closes: https://lore.kernel.org/oe-kbuild-all/202501291646.VtwF98qd-lkp@intel.com/ Signed-off-by: Nathan Chancellor <nathan@kernel.org> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> --- scripts/Makefile.extrawarn | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) (limited to 'scripts') diff --git a/scripts/Makefile.extrawarn b/scripts/Makefile.extrawarn index eb719f6d8d53..e976790f84dc 100644 --- a/scripts/Makefile.extrawarn +++ b/scripts/Makefile.extrawarn @@ -31,6 +31,11 @@ KBUILD_CFLAGS-$(CONFIG_CC_NO_ARRAY_BOUNDS) += -Wno-array-bounds ifdef CONFIG_CC_IS_CLANG # The kernel builds with '-std=gnu11' so use of GNU extensions is acceptable. KBUILD_CFLAGS += -Wno-gnu + +# Clang checks for overflow/truncation with '%p', while GCC does not: +# https://gcc.gnu.org/bugzilla/show_bug.cgi?id=111219 +KBUILD_CFLAGS += $(call cc-disable-warning, format-overflow-non-kprintf) +KBUILD_CFLAGS += $(call cc-disable-warning, format-truncation-non-kprintf) else # gcc inanely warns about local variables called 'main' @@ -105,11 +110,6 @@ KBUILD_CFLAGS += $(call cc-disable-warning, packed-not-aligned) KBUILD_CFLAGS += $(call cc-disable-warning, format-overflow) ifdef CONFIG_CC_IS_GCC KBUILD_CFLAGS += $(call cc-disable-warning, format-truncation) -else -# Clang checks for overflow/truncation with '%p', while GCC does not: -# https://gcc.gnu.org/bugzilla/show_bug.cgi?id=111219 -KBUILD_CFLAGS += $(call cc-disable-warning, format-overflow-non-kprintf) -KBUILD_CFLAGS += $(call cc-disable-warning, format-truncation-non-kprintf) endif KBUILD_CFLAGS += $(call cc-disable-warning, stringop-truncation) -- cgit v1.2.3 From 4c56eb33e603c3b9eb4bd24efbfdd0283c1c37e4 Mon Sep 17 00:00:00 2001 From: Masahiro Yamada <masahiroy@kernel.org> Date: Sun, 2 Feb 2025 03:51:41 +0900 Subject: kbuild: keep symbols for symbol_get() even with CONFIG_TRIM_UNUSED_KSYMS Linus observed that the symbol_request(utf8_data_table) call fails when CONFIG_UNICODE=y and CONFIG_TRIM_UNUSED_KSYMS=y. symbol_get() relies on the symbol data being present in the ksymtab for symbol lookups. However, EXPORT_SYMBOL_GPL(utf8_data_table) is dropped due to CONFIG_TRIM_UNUSED_KSYMS, as no module references it in this case. Probably, this has been broken since commit dbacb0ef670d ("kconfig option for TRIM_UNUSED_KSYMS"). This commit addresses the issue by leveraging modpost. Symbol names passed to symbol_get() are recorded in the special .no_trim_symbol section, which is then parsed by modpost to forcibly keep such symbols. The .no_trim_symbol section is discarded by the linker scripts, so there is no impact on the size of the final vmlinux or modules. This commit cannot resolve the issue for direct calls to __symbol_get() because the symbol name is not known at compile-time. Although symbol_get() may eventually be deprecated, this workaround should be good enough meanwhile. Reported-by: Linus Torvalds <torvalds@linux-foundation.org> Suggested-by: Linus Torvalds <torvalds@linux-foundation.org> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> --- include/asm-generic/vmlinux.lds.h | 1 + include/linux/module.h | 5 ++++- scripts/mod/modpost.c | 35 +++++++++++++++++++++++++++++++++++ scripts/mod/modpost.h | 6 ++++++ scripts/module.lds.S | 1 + 5 files changed, 47 insertions(+), 1 deletion(-) (limited to 'scripts') diff --git a/include/asm-generic/vmlinux.lds.h b/include/asm-generic/vmlinux.lds.h index 54504013c749..02a4adb4a999 100644 --- a/include/asm-generic/vmlinux.lds.h +++ b/include/asm-generic/vmlinux.lds.h @@ -1038,6 +1038,7 @@ defined(CONFIG_AUTOFDO_CLANG) || defined(CONFIG_PROPELLER_CLANG) *(.discard) \ *(.discard.*) \ *(.export_symbol) \ + *(.no_trim_symbol) \ *(.modinfo) \ /* ld.bfd warns about .gnu.version* even when not emitted */ \ *(.gnu.version*) \ diff --git a/include/linux/module.h b/include/linux/module.h index 23792d5d7b74..30e5b19bafa9 100644 --- a/include/linux/module.h +++ b/include/linux/module.h @@ -306,7 +306,10 @@ extern int modules_disabled; /* for sysctl */ /* Get/put a kernel symbol (calls must be symmetric) */ void *__symbol_get(const char *symbol); void *__symbol_get_gpl(const char *symbol); -#define symbol_get(x) ((typeof(&x))(__symbol_get(__stringify(x)))) +#define symbol_get(x) ({ \ + static const char __notrim[] \ + __used __section(".no_trim_symbol") = __stringify(x); \ + (typeof(&x))(__symbol_get(__stringify(x))); }) /* modules using other modules: kdb wants to see this. */ struct module_use { diff --git a/scripts/mod/modpost.c b/scripts/mod/modpost.c index e18ae7dc8140..36b28987a2f0 100644 --- a/scripts/mod/modpost.c +++ b/scripts/mod/modpost.c @@ -507,6 +507,9 @@ static int parse_elf(struct elf_info *info, const char *filename) info->modinfo_len = sechdrs[i].sh_size; } else if (!strcmp(secname, ".export_symbol")) { info->export_symbol_secndx = i; + } else if (!strcmp(secname, ".no_trim_symbol")) { + info->no_trim_symbol = (void *)hdr + sechdrs[i].sh_offset; + info->no_trim_symbol_len = sechdrs[i].sh_size; } if (sechdrs[i].sh_type == SHT_SYMTAB) { @@ -1566,6 +1569,14 @@ static void read_symbols(const char *modname) /* strip trailing .o */ mod = new_module(modname, strlen(modname) - strlen(".o")); + /* save .no_trim_symbol section for later use */ + if (info.no_trim_symbol_len) { + mod->no_trim_symbol = xmalloc(info.no_trim_symbol_len); + memcpy(mod->no_trim_symbol, info.no_trim_symbol, + info.no_trim_symbol_len); + mod->no_trim_symbol_len = info.no_trim_symbol_len; + } + if (!mod->is_vmlinux) { license = get_modinfo(&info, "license"); if (!license) @@ -1728,6 +1739,28 @@ static void handle_white_list_exports(const char *white_list) free(buf); } +/* + * Keep symbols recorded in the .no_trim_symbol section. This is necessary to + * prevent CONFIG_TRIM_UNUSED_KSYMS from dropping EXPORT_SYMBOL because + * symbol_get() relies on the symbol being present in the ksymtab for lookups. + */ +static void keep_no_trim_symbols(struct module *mod) +{ + unsigned long size = mod->no_trim_symbol_len; + + for (char *s = mod->no_trim_symbol; s; s = next_string(s , &size)) { + struct symbol *sym; + + /* + * If find_symbol() returns NULL, this symbol is not provided + * by any module, and symbol_get() will fail. + */ + sym = find_symbol(s); + if (sym) + sym->used = true; + } +} + static void check_modname_len(struct module *mod) { const char *mod_name; @@ -2254,6 +2287,8 @@ int main(int argc, char **argv) read_symbols_from_files(files_source); list_for_each_entry(mod, &modules, list) { + keep_no_trim_symbols(mod); + if (mod->dump_file || mod->is_vmlinux) continue; diff --git a/scripts/mod/modpost.h b/scripts/mod/modpost.h index ffd0a52a606e..59366f456b76 100644 --- a/scripts/mod/modpost.h +++ b/scripts/mod/modpost.h @@ -111,6 +111,8 @@ struct module_alias { * * @dump_file: path to the .symvers file if loaded from a file * @aliases: list head for module_aliases + * @no_trim_symbol: .no_trim_symbol section data + * @no_trim_symbol_len: length of the .no_trim_symbol section */ struct module { struct list_head list; @@ -128,6 +130,8 @@ struct module { // Actual imported namespaces struct list_head imported_namespaces; struct list_head aliases; + char *no_trim_symbol; + unsigned int no_trim_symbol_len; char name[]; }; @@ -141,6 +145,8 @@ struct elf_info { char *strtab; char *modinfo; unsigned int modinfo_len; + char *no_trim_symbol; + unsigned int no_trim_symbol_len; /* support for 32bit section numbers */ diff --git a/scripts/module.lds.S b/scripts/module.lds.S index c2f80f9141d4..450f1088d5fd 100644 --- a/scripts/module.lds.S +++ b/scripts/module.lds.S @@ -16,6 +16,7 @@ SECTIONS { *(.discard) *(.discard.*) *(.export_symbol) + *(.no_trim_symbol) } __ksymtab 0 : ALIGN(8) { *(SORT(___ksymtab+*)) } -- cgit v1.2.3 From ba958ac74800573f7f54dbe2a7a7b9a9a523ed52 Mon Sep 17 00:00:00 2001 From: Oleh Zadorozhnyi <lesorubshayan@gmail.com> Date: Tue, 4 Feb 2025 07:17:30 +0200 Subject: kbuild: fix misspelling in scripts/Makefile.lib Signed-off-by: Oleh Zadorozhnyi <lesorubshayan@gmail.com> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> --- scripts/Makefile.lib | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'scripts') diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib index ad55ef201aac..cad20f0e66ee 100644 --- a/scripts/Makefile.lib +++ b/scripts/Makefile.lib @@ -305,7 +305,7 @@ endef # These are shared by some Makefile.* files. ifdef CONFIG_LTO_CLANG -# Run $(LD) here to covert LLVM IR to ELF in the following cases: +# Run $(LD) here to convert LLVM IR to ELF in the following cases: # - when this object needs objtool processing, as objtool cannot process LLVM IR # - when this is a single-object module, as modpost cannot process LLVM IR cmd_ld_single = $(if $(objtool-enabled)$(is-single-obj-m), ; $(LD) $(ld_flags) -r -o $(tmp-target) $@; mv $(tmp-target) $@) -- cgit v1.2.3 From 6273a058383e05465083b535ed9469f2c8a48321 Mon Sep 17 00:00:00 2001 From: Alice Ryhl <aliceryhl@google.com> Date: Mon, 3 Feb 2025 08:40:57 +0000 Subject: x86: rust: set rustc-abi=x86-softfloat on rustc>=1.86.0 When using Rust on the x86 architecture, we are currently using the unstable target.json feature to specify the compilation target. Rustc is going to change how softfloat is specified in the target.json file on x86, thus update generate_rust_target.rs to specify softfloat using the new option. Note that if you enable this parameter with a compiler that does not recognize it, then that triggers a warning but it does not break the build. [ For future reference, this solves the following error: RUSTC L rust/core.o error: Error loading target specification: target feature `soft-float` is incompatible with the ABI but gets enabled in target spec. Run `rustc --print target-list` for a list of built-in targets - Miguel ] Cc: <stable@vger.kernel.org> # Needed in 6.12.y and 6.13.y only (Rust is pinned in older LTSs). Link: https://github.com/rust-lang/rust/pull/136146 Signed-off-by: Alice Ryhl <aliceryhl@google.com> Acked-by: Dave Hansen <dave.hansen@linux.intel.com> # for x86 Link: https://lore.kernel.org/r/20250203-rustc-1-86-x86-softfloat-v1-1-220a72a5003e@google.com [ Added 6.13.y too to Cc: stable tag and added reasoning to avoid over-backporting. - Miguel ] Signed-off-by: Miguel Ojeda <ojeda@kernel.org> --- scripts/generate_rust_target.rs | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) (limited to 'scripts') diff --git a/scripts/generate_rust_target.rs b/scripts/generate_rust_target.rs index 0d00ac3723b5..4fd6b6ab3e32 100644 --- a/scripts/generate_rust_target.rs +++ b/scripts/generate_rust_target.rs @@ -165,6 +165,18 @@ impl KernelConfig { let option = "CONFIG_".to_owned() + option; self.0.contains_key(&option) } + + /// Is the rustc version at least `major.minor.patch`? + fn rustc_version_atleast(&self, major: u32, minor: u32, patch: u32) -> bool { + let check_version = 100000 * major + 100 * minor + patch; + let actual_version = self + .0 + .get("CONFIG_RUSTC_VERSION") + .unwrap() + .parse::<u32>() + .unwrap(); + check_version <= actual_version + } } fn main() { @@ -182,6 +194,9 @@ fn main() { } } else if cfg.has("X86_64") { ts.push("arch", "x86_64"); + if cfg.rustc_version_atleast(1, 86, 0) { + ts.push("rustc-abi", "x86-softfloat"); + } ts.push( "data-layout", "e-m:e-p270:32:32-p271:32:32-p272:64:64-i64:64-i128:128-f80:128-n8:16:32:64-S128", @@ -215,6 +230,9 @@ fn main() { panic!("32-bit x86 only works under UML"); } ts.push("arch", "x86"); + if cfg.rustc_version_atleast(1, 86, 0) { + ts.push("rustc-abi", "x86-softfloat"); + } ts.push( "data-layout", "e-m:e-p:32:32-p270:32:32-p271:32:32-p272:64:64-i128:128-f64:32:64-f80:32-n8:16:32-S128", -- cgit v1.2.3 From f354fc88a72ae83dacd68370f6fa040e5733bcfe Mon Sep 17 00:00:00 2001 From: WangYuli <wangyuli@uniontech.com> Date: Fri, 7 Feb 2025 15:08:55 +0800 Subject: kbuild: install-extmod-build: add missing quotation marks for CC variable While attempting to build a Debian packages with CC="ccache gcc", I saw the following error as builddeb builds linux-headers-$KERNELVERSION: make HOSTCC=ccache gcc VPATH= srcroot=. -f ./scripts/Makefile.build obj=debian/linux-headers-6.14.0-rc1/usr/src/linux-headers-6.14.0-rc1/scripts make[6]: *** No rule to make target 'gcc'. Stop. Upon investigation, it seems that one instance of $(CC) variable reference in ./scripts/package/install-extmod-build was missing quotation marks, causing the above error. Add the missing quotation marks around $(CC) to fix build. Fixes: 5f73e7d0386d ("kbuild: refactor cross-compiling linux-headers package") Co-developed-by: Mingcong Bai <jeffbai@aosc.io> Signed-off-by: Mingcong Bai <jeffbai@aosc.io> Tested-by: WangYuli <wangyuli@uniontech.com> Signed-off-by: WangYuli <wangyuli@uniontech.com> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> --- scripts/package/install-extmod-build | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'scripts') diff --git a/scripts/package/install-extmod-build b/scripts/package/install-extmod-build index bb6e23c1174e..b724626ea0ca 100755 --- a/scripts/package/install-extmod-build +++ b/scripts/package/install-extmod-build @@ -63,7 +63,7 @@ if [ "${CC}" != "${HOSTCC}" ]; then # Clear VPATH and srcroot because the source files reside in the output # directory. # shellcheck disable=SC2016 # $(MAKE), $(CC), and $(build) will be expanded by Make - "${MAKE}" run-command KBUILD_RUN_COMMAND='+$(MAKE) HOSTCC=$(CC) VPATH= srcroot=. $(build)='"${destdir}"/scripts + "${MAKE}" run-command KBUILD_RUN_COMMAND='+$(MAKE) HOSTCC="$(CC)" VPATH= srcroot=. $(build)='"${destdir}"/scripts rm -f "${destdir}/scripts/Kbuild" fi -- cgit v1.2.3 From 8f6629c004b193d23612641c3607e785819e97ab Mon Sep 17 00:00:00 2001 From: Nathan Chancellor <nathan@kernel.org> Date: Thu, 17 Oct 2024 10:09:22 -0700 Subject: kbuild: Move -Wenum-enum-conversion to W=2 -Wenum-enum-conversion was strengthened in clang-19 to warn for C, which caused the kernel to move it to W=1 in commit 75b5ab134bb5 ("kbuild: Move -Wenum-{compare-conditional,enum-conversion} into W=1") because there were numerous instances that would break builds with -Werror. Unfortunately, this is not a full solution, as more and more developers, subsystems, and distributors are building with W=1 as well, so they continue to see the numerous instances of this warning. Since the move to W=1, there have not been many new instances that have appeared through various build reports and the ones that have appeared seem to be following similar existing patterns, suggesting that most instances of this warning will not be real issues. The only alternatives for silencing this warning are adding casts (which is generally seen as an ugly practice) or refactoring the enums to macro defines or a unified enum (which may be undesirable because of type safety in other parts of the code). Move the warning to W=2, where warnings that occur frequently but may be relevant should reside. Cc: stable@vger.kernel.org Fixes: 75b5ab134bb5 ("kbuild: Move -Wenum-{compare-conditional,enum-conversion} into W=1") Link: https://lore.kernel.org/ZwRA9SOcOjjLJcpi@google.com/ Signed-off-by: Nathan Chancellor <nathan@kernel.org> Acked-by: Arnd Bergmann <arnd@arndb.de> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org> --- scripts/Makefile.extrawarn | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) (limited to 'scripts') diff --git a/scripts/Makefile.extrawarn b/scripts/Makefile.extrawarn index eb719f6d8d53..a7003c1e66c7 100644 --- a/scripts/Makefile.extrawarn +++ b/scripts/Makefile.extrawarn @@ -133,7 +133,6 @@ KBUILD_CFLAGS += $(call cc-disable-warning, pointer-to-enum-cast) KBUILD_CFLAGS += -Wno-tautological-constant-out-of-range-compare KBUILD_CFLAGS += $(call cc-disable-warning, unaligned-access) KBUILD_CFLAGS += -Wno-enum-compare-conditional -KBUILD_CFLAGS += -Wno-enum-enum-conversion endif endif @@ -157,6 +156,10 @@ KBUILD_CFLAGS += -Wno-missing-field-initializers KBUILD_CFLAGS += -Wno-type-limits KBUILD_CFLAGS += -Wno-shift-negative-value +ifdef CONFIG_CC_IS_CLANG +KBUILD_CFLAGS += -Wno-enum-enum-conversion +endif + ifdef CONFIG_CC_IS_GCC KBUILD_CFLAGS += -Wno-maybe-uninitialized endif -- cgit v1.2.3 From 31c89102cf39fb7a171283f35c41f57bf422a4df Mon Sep 17 00:00:00 2001 From: Eric Biggers <ebiggers@google.com> Date: Mon, 10 Feb 2025 09:26:44 -0800 Subject: scripts/gen-crc-consts: add gen-crc-consts.py Add a Python script that generates constants for computing the given CRC variant(s) using x86's pclmulqdq or vpclmulqdq instructions. This is specifically tuned for x86's crc-pclmul-template.S. However, other architectures with a 64x64 => 128-bit carryless multiplication instruction should be able to use the generated constants too. (Some tweaks may be warranted based on the exact instructions available on each arch, so the script may grow an arch argument in the future.) The script also supports generating the tables needed for table-based CRC computation. Thus, it can also be used to reproduce the tables like t10_dif_crc_table[] and crc16_table[] that are currently hardcoded in the source with no generation script explicitly documented. Python is used rather than C since it enables implementing the CRC math in the simplest way possible, using arbitrary precision integers. The outputs of this script are intended to be checked into the repo, so Python will continue to not be required to build the kernel, and the script has been optimized for simplicity rather than performance. Acked-by: Ard Biesheuvel <ardb@kernel.org> Acked-by: Keith Busch <kbusch@kernel.org> Link: https://lore.kernel.org/r/20250210174540.161705-3-ebiggers@kernel.org Signed-off-by: Eric Biggers <ebiggers@google.com> --- MAINTAINERS | 1 + scripts/gen-crc-consts.py | 238 ++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 239 insertions(+) create mode 100755 scripts/gen-crc-consts.py (limited to 'scripts') diff --git a/MAINTAINERS b/MAINTAINERS index 896a307fa065..3532167f3193 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -6132,6 +6132,7 @@ F: Documentation/staging/crc* F: arch/*/lib/crc* F: include/linux/crc* F: lib/crc* +F: scripts/gen-crc-consts.py CREATIVE SB0540 M: Bastien Nocera <hadess@hadess.net> diff --git a/scripts/gen-crc-consts.py b/scripts/gen-crc-consts.py new file mode 100755 index 000000000000..aa678a50897d --- /dev/null +++ b/scripts/gen-crc-consts.py @@ -0,0 +1,238 @@ +#!/usr/bin/env python3 +# SPDX-License-Identifier: GPL-2.0-or-later +# +# Script that generates constants for computing the given CRC variant(s). +# +# Copyright 2025 Google LLC +# +# Author: Eric Biggers <ebiggers@google.com> + +import sys + +# XOR (add) an iterable of polynomials. +def xor(iterable): + res = 0 + for val in iterable: + res ^= val + return res + +# Multiply two polynomials. +def clmul(a, b): + return xor(a << i for i in range(b.bit_length()) if (b & (1 << i)) != 0) + +# Polynomial division floor(a / b). +def div(a, b): + q = 0 + while a.bit_length() >= b.bit_length(): + q ^= 1 << (a.bit_length() - b.bit_length()) + a ^= b << (a.bit_length() - b.bit_length()) + return q + +# Reduce the polynomial 'a' modulo the polynomial 'b'. +def reduce(a, b): + return a ^ clmul(div(a, b), b) + +# Reflect the bits of a polynomial. +def bitreflect(poly, num_bits): + assert poly.bit_length() <= num_bits + return xor(((poly >> i) & 1) << (num_bits - 1 - i) for i in range(num_bits)) + +# Format a polynomial as hex. Bit-reflect it if the CRC is lsb-first. +def fmt_poly(variant, poly, num_bits): + if variant.lsb: + poly = bitreflect(poly, num_bits) + return f'0x{poly:0{2*num_bits//8}x}' + +# Print a pair of 64-bit polynomial multipliers. They are always passed in the +# order [HI64_TERMS, LO64_TERMS] but will be printed in the appropriate order. +def print_mult_pair(variant, mults): + mults = list(mults if variant.lsb else reversed(mults)) + terms = ['HI64_TERMS', 'LO64_TERMS'] if variant.lsb else ['LO64_TERMS', 'HI64_TERMS'] + for i in range(2): + print(f'\t\t{fmt_poly(variant, mults[i]["val"], 64)},\t/* {terms[i]}: {mults[i]["desc"]} */') + +# Pretty-print a polynomial. +def pprint_poly(prefix, poly): + terms = [f'x^{i}' for i in reversed(range(poly.bit_length())) + if (poly & (1 << i)) != 0] + j = 0 + while j < len(terms): + s = prefix + terms[j] + (' +' if j < len(terms) - 1 else '') + j += 1 + while j < len(terms) and len(s) < 73: + s += ' ' + terms[j] + (' +' if j < len(terms) - 1 else '') + j += 1 + print(s) + prefix = ' * ' + (' ' * (len(prefix) - 3)) + +# Print a comment describing constants generated for the given CRC variant. +def print_header(variant, what): + print('/*') + s = f'{"least" if variant.lsb else "most"}-significant-bit-first CRC-{variant.bits}' + print(f' * {what} generated for {s} using') + pprint_poly(' * G(x) = ', variant.G) + print(' */') + +class CrcVariant: + def __init__(self, bits, generator_poly, bit_order): + self.bits = bits + if bit_order not in ['lsb', 'msb']: + raise ValueError('Invalid value for bit_order') + self.lsb = bit_order == 'lsb' + self.name = f'crc{bits}_{bit_order}_0x{generator_poly:0{(2*bits+7)//8}x}' + if self.lsb: + generator_poly = bitreflect(generator_poly, bits) + self.G = generator_poly ^ (1 << bits) + +# Generate tables for CRC computation using the "slice-by-N" method. +# N=1 corresponds to the traditional byte-at-a-time table. +def gen_slicebyN_tables(variants, n): + for v in variants: + print('') + print_header(v, f'Slice-by-{n} CRC table') + print(f'static const u{v.bits} __maybe_unused {v.name}_table[{256*n}] = {{') + s = '' + for i in range(256 * n): + # The i'th table entry is the CRC of the message consisting of byte + # i % 256 followed by i // 256 zero bytes. + poly = (bitreflect(i % 256, 8) if v.lsb else i % 256) << (v.bits + 8*(i//256)) + next_entry = fmt_poly(v, reduce(poly, v.G), v.bits) + ',' + if len(s + next_entry) > 71: + print(f'\t{s}') + s = '' + s += (' ' if s else '') + next_entry + if s: + print(f'\t{s}') + print('};') + +# Generate constants for carryless multiplication based CRC computation. +def gen_x86_pclmul_consts(variants): + # These are the distances, in bits, to generate folding constants for. + FOLD_DISTANCES = [2048, 1024, 512, 256, 128] + + for v in variants: + (G, n, lsb) = (v.G, v.bits, v.lsb) + print('') + print_header(v, 'CRC folding constants') + print('static const struct {') + if not lsb: + print('\tu8 bswap_mask[16];') + for i in FOLD_DISTANCES: + print(f'\tu64 fold_across_{i}_bits_consts[2];') + print('\tu8 shuf_table[48];') + print('\tu64 barrett_reduction_consts[2];') + print(f'}} {v.name}_consts ____cacheline_aligned __maybe_unused = {{') + + # Byte-reflection mask, needed for msb-first CRCs + if not lsb: + print('\t.bswap_mask = {' + ', '.join(str(i) for i in reversed(range(16))) + '},') + + # Fold constants for all distances down to 128 bits + for i in FOLD_DISTANCES: + print(f'\t.fold_across_{i}_bits_consts = {{') + # Given 64x64 => 128 bit carryless multiplication instructions, two + # 64-bit fold constants are needed per "fold distance" i: one for + # HI64_TERMS that is basically x^(i+64) mod G and one for LO64_TERMS + # that is basically x^i mod G. The exact values however undergo a + # couple adjustments, described below. + mults = [] + for j in [64, 0]: + pow_of_x = i + j + if lsb: + # Each 64x64 => 128 bit carryless multiplication instruction + # actually generates a 127-bit product in physical bits 0 + # through 126, which in the lsb-first case represent the + # coefficients of x^1 through x^127, not x^0 through x^126. + # Thus in the lsb-first case, each such instruction + # implicitly adds an extra factor of x. The below removes a + # factor of x from each constant to compensate for this. + # For n < 64 the x could be removed from either the reduced + # part or unreduced part, but for n == 64 the reduced part + # is the only option. Just always use the reduced part. + pow_of_x -= 1 + # Make a factor of x^(64-n) be applied unreduced rather than + # reduced, to cause the product to use only the x^(64-n) and + # higher terms and always be zero in the lower terms. Usually + # this makes no difference as it does not affect the product's + # congruence class mod G and the constant remains 64-bit, but + # part of the final reduction from 128 bits does rely on this + # property when it reuses one of the constants. + pow_of_x -= 64 - n + mults.append({ 'val': reduce(1 << pow_of_x, G) << (64 - n), + 'desc': f'(x^{pow_of_x} mod G) * x^{64-n}' }) + print_mult_pair(v, mults) + print('\t},') + + # Shuffle table for handling 1..15 bytes at end + print('\t.shuf_table = {') + print('\t\t' + (16*'-1, ').rstrip()) + print('\t\t' + ''.join(f'{i:2}, ' for i in range(16)).rstrip()) + print('\t\t' + (16*'-1, ').rstrip()) + print('\t},') + + # Barrett reduction constants for reducing 128 bits to the final CRC + print('\t.barrett_reduction_consts = {') + mults = [] + + val = div(1 << (63+n), G) + desc = f'floor(x^{63+n} / G)' + if not lsb: + val = (val << 1) - (1 << 64) + desc = f'({desc} * x) - x^64' + mults.append({ 'val': val, 'desc': desc }) + + val = G - (1 << n) + desc = f'G - x^{n}' + if lsb and n == 64: + assert (val & 1) != 0 # The x^0 term should always be nonzero. + val >>= 1 + desc = f'({desc} - x^0) / x' + else: + pow_of_x = 64 - n - (1 if lsb else 0) + val <<= pow_of_x + desc = f'({desc}) * x^{pow_of_x}' + mults.append({ 'val': val, 'desc': desc }) + + print_mult_pair(v, mults) + print('\t},') + + print('};') + +def parse_crc_variants(vars_string): + variants = [] + for var_string in vars_string.split(','): + bits, bit_order, generator_poly = var_string.split('_') + assert bits.startswith('crc') + bits = int(bits.removeprefix('crc')) + assert generator_poly.startswith('0x') + generator_poly = generator_poly.removeprefix('0x') + assert len(generator_poly) % 2 == 0 + generator_poly = int(generator_poly, 16) + variants.append(CrcVariant(bits, generator_poly, bit_order)) + return variants + +if len(sys.argv) != 3: + sys.stderr.write(f'Usage: {sys.argv[0]} CONSTS_TYPE[,CONSTS_TYPE]... CRC_VARIANT[,CRC_VARIANT]...\n') + sys.stderr.write(' CONSTS_TYPE can be sliceby[1-8] or x86_pclmul\n') + sys.stderr.write(' CRC_VARIANT is crc${num_bits}_${bit_order}_${generator_poly_as_hex}\n') + sys.stderr.write(' E.g. crc16_msb_0x8bb7 or crc32_lsb_0xedb88320\n') + sys.stderr.write(' Polynomial must use the given bit_order and exclude x^{num_bits}\n') + sys.exit(1) + +print('/* SPDX-License-Identifier: GPL-2.0-or-later */') +print('/*') +print(' * CRC constants generated by:') +print(' *') +print(f' *\t{sys.argv[0]} {" ".join(sys.argv[1:])}') +print(' *') +print(' * Do not edit manually.') +print(' */') +consts_types = sys.argv[1].split(',') +variants = parse_crc_variants(sys.argv[2]) +for consts_type in consts_types: + if consts_type.startswith('sliceby'): + gen_slicebyN_tables(variants, int(consts_type.removeprefix('sliceby'))) + elif consts_type == 'x86_pclmul': + gen_x86_pclmul_consts(variants) + else: + raise ValueError(f'Unknown consts_type: {consts_type}') -- cgit v1.2.3 From 790ca8b0b5a343090b148d9c0c1ddb2d1a012952 Mon Sep 17 00:00:00 2001 From: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Date: Mon, 10 Feb 2025 11:17:58 +0100 Subject: scripts/documentation-file-ref-check: don't check perl/python scripts Such scripts may have regular expressions, which would make the parser confusing. Also, they shouldn't hardcode filenames there, so skipping them is OK. While here, also don't check references on extensions used for file backup and patch rej/orig. Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Link: https://lore.kernel.org/r/712bfc8412ee5ad8ab43dd21a8c30fc858eff5a6.1739182025.git.mchehab+huawei@kernel.org --- scripts/documentation-file-ref-check | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'scripts') diff --git a/scripts/documentation-file-ref-check b/scripts/documentation-file-ref-check index 68083f2f1122..408b1dbe7884 100755 --- a/scripts/documentation-file-ref-check +++ b/scripts/documentation-file-ref-check @@ -92,7 +92,7 @@ while (<IN>) { next if ($f =~ m,^Next/,); # Makefiles and scripts contain nasty expressions to parse docs - next if ($f =~ m/Makefile/ || $f =~ m/\.sh$/); + next if ($f =~ m/Makefile/ || $f =~ m/\.(sh|py|pl|~|rej|org|orig)$/); # It doesn't make sense to parse hidden files next if ($f =~ m#/\.#); -- cgit v1.2.3 From 484e9aa6efaf96a7a5b5fe3216f24973166fbfe3 Mon Sep 17 00:00:00 2001 From: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Date: Mon, 10 Feb 2025 11:17:59 +0100 Subject: scripts/get_abi.py: add a Python tool to generate ReST output The get_abi.pl script is requiring some care, but it seems that the number of changes on it since when I originally wrote it was not too high. Maintaining perl scripts without using classes requires a higher efforted than on python, due to global variables management. Also, it sounds easier to find python developer those days than perl ones. As a plus, using a Python class to handle ABI allows a better integration with Sphinx extensions, allowing, for instance, to let automarkup to generate cross-references for ABI symbols. With that in mind, rewrite the core of get_abi.pl in Python, using classes, to help producing documentation. This will allow a better integration in the future with the Sphinx ABI extension. The algorithms used there are the same as the ones in Perl, with a couple of cleanups to remove redundant variables and to help with cross-reference generation. While doing that, remove some code that were important in the past, where ABI files weren't using ReST format. Some minor improvements were added like using a fixed seed when generating ABI keys for duplicated names, making its results reproductible. The end script is a little bit faster than the original one (tested on a machine with ssd disks). That's probably because we're now using only pre-compiled regular expressions, and it is using string replacement methods instead of regex where possible. The new version is a little bit more conservative when converting text to cross-references to avoid adding them into literal blocks. To ensure that the ReST output is parsing all variables and files properly, the end result was compared using diff with the one produced by the perl script and showed no regressions. There are minor improvements at the results, as it now properly groups What on some special cases. It also better escape some XREF names. Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Link: https://lore.kernel.org/r/71a894211a8b69664711144d9c4f8a0e73d1ae3c.1739182025.git.mchehab+huawei@kernel.org --- scripts/get_abi.py | 118 ++++++++++ scripts/lib/abi/abi_parser.py | 512 ++++++++++++++++++++++++++++++++++++++++++ scripts/lib/abi/helpers.py | 28 +++ 3 files changed, 658 insertions(+) create mode 100755 scripts/get_abi.py create mode 100644 scripts/lib/abi/abi_parser.py create mode 100644 scripts/lib/abi/helpers.py (limited to 'scripts') diff --git a/scripts/get_abi.py b/scripts/get_abi.py new file mode 100755 index 000000000000..bb17c54feeff --- /dev/null +++ b/scripts/get_abi.py @@ -0,0 +1,118 @@ +#!/usr/bin/env python3 +# pylint: disable=R0903 +# Copyright(c) 2025: Mauro Carvalho Chehab <mchehab@kernel.org>. +# SPDX-License-Identifier: GPL-2.0 + +""" +Parse ABI documentation and produce results from it. +""" + +import argparse +import logging +import os +import sys + +# Import Python modules + +LIB_DIR = "lib/abi" +SRC_DIR = os.path.dirname(os.path.realpath(__file__)) + +sys.path.insert(0, os.path.join(SRC_DIR, LIB_DIR)) + +from abi_parser import AbiParser # pylint: disable=C0413 +from helpers import ABI_DIR, DEBUG_HELP # pylint: disable=C0413 + +# Command line classes + + +REST_DESC = """ +Produce output in ReST format. + +The output is done on two sections: + +- Symbols: show all parsed symbols in alphabetic order; +- Files: cross reference the content of each file with the symbols on it. +""" + +class AbiRest: + """Initialize an argparse subparser for rest output""" + + def __init__(self, subparsers): + """Initialize argparse subparsers""" + + parser = subparsers.add_parser("rest", + formatter_class=argparse.RawTextHelpFormatter, + description=REST_DESC) + + parser.add_argument("--enable-lineno", action="store_true", + help="enable lineno") + parser.add_argument("--raw", action="store_true", + help="output text as contained in the ABI files. " + "It not used, output will contain dynamically" + " generated cross references when possible.") + parser.add_argument("--no-file", action="store_true", + help="Don't the files section") + parser.add_argument("--show-hints", help="Show-hints") + + parser.set_defaults(func=self.run) + + def run(self, args): + """Run subparser""" + + parser = AbiParser(args.dir, debug=args.debug) + parser.parse_abi() + parser.check_issues() + parser.print_data(args.enable_lineno, args.raw, not args.no_file) + + +class AbiValidate: + """Initialize an argparse subparser for ABI validation""" + + def __init__(self, subparsers): + """Initialize argparse subparsers""" + + parser = subparsers.add_parser("validate", + formatter_class=argparse.ArgumentDefaultsHelpFormatter, + description="list events") + + parser.set_defaults(func=self.run) + + def run(self, args): + """Run subparser""" + + parser = AbiParser(args.dir, debug=args.debug) + parser.parse_abi() + parser.check_issues() + + +def main(): + """Main program""" + + parser = argparse.ArgumentParser(formatter_class=argparse.RawTextHelpFormatter) + + parser.add_argument("-d", "--debug", type=int, default=0, help="debug level") + parser.add_argument("-D", "--dir", default=ABI_DIR, help=DEBUG_HELP) + + subparsers = parser.add_subparsers() + + AbiRest(subparsers) + AbiValidate(subparsers) + + args = parser.parse_args() + + if args.debug: + level = logging.DEBUG + else: + level = logging.INFO + + logging.basicConfig(level=level, format="[%(levelname)s] %(message)s") + + if "func" in args: + args.func(args) + else: + sys.exit(f"Please specify a valid command for {sys.argv[0]}") + + +# Call main method +if __name__ == "__main__": + main() diff --git a/scripts/lib/abi/abi_parser.py b/scripts/lib/abi/abi_parser.py new file mode 100644 index 000000000000..b3fa70eee412 --- /dev/null +++ b/scripts/lib/abi/abi_parser.py @@ -0,0 +1,512 @@ +#!/usr/bin/env python3 +# pylint: disable=R0902,R0903,R0911,R0912,R0913,R0914,R0915,R0917,C0302 +# Copyright(c) 2025: Mauro Carvalho Chehab <mchehab@kernel.org>. +# SPDX-License-Identifier: GPL-2.0 + +""" +Parse ABI documentation and produce results from it. +""" + +from argparse import Namespace +import logging +import os +import re + +from glob import glob +from pprint import pformat +from random import randrange, seed + +# Import Python modules + +from helpers import AbiDebug, ABI_DIR + + +class AbiParser: + """Main class to parse ABI files""" + + TAGS = r"(what|where|date|kernelversion|contact|description|users)" + XREF = r"(?:^|\s|\()(\/(?:sys|config|proc|dev|kvd)\/[^,.:;\)\s]+)(?:[,.:;\)\s]|\Z)" + + def __init__(self, directory, logger=None, + enable_lineno=False, show_warnings=True, debug=0): + """Stores arguments for the class and initialize class vars""" + + self.directory = directory + self.enable_lineno = enable_lineno + self.show_warnings = show_warnings + self.debug = debug + + if not logger: + self.log = logging.getLogger("get_abi") + else: + self.log = logger + + self.data = {} + self.what_symbols = {} + self.file_refs = {} + self.what_refs = {} + + # Regular expressions used on parser + self.re_tag = re.compile(r"(\S+)(:\s*)(.*)", re.I) + self.re_valid = re.compile(self.TAGS) + self.re_start_spc = re.compile(r"(\s*)(\S.*)") + self.re_whitespace = re.compile(r"^\s+") + + # Regular used on print + self.re_what = re.compile(r"(\/?(?:[\w\-]+\/?){1,2})") + self.re_escape = re.compile(r"([\.\x01-\x08\x0e-\x1f\x21-\x2f\x3a-\x40\x7b-\xff])") + self.re_unprintable = re.compile(r"([\x00-\x2f\x3a-\x40\x5b-\x60\x7b-\xff]+)") + self.re_title_mark = re.compile(r"\n[\-\*\=\^\~]+\n") + self.re_doc = re.compile(r"Documentation/(?!devicetree)(\S+)\.rst") + self.re_abi = re.compile(r"(Documentation/ABI/)([\w\/\-]+)") + self.re_xref_node = re.compile(self.XREF) + + def warn(self, fdata, msg, extra=None): + """Displays a parse error if warning is enabled""" + + if not self.show_warnings: + return + + msg = f"{fdata.fname}:{fdata.ln}: {msg}" + if extra: + msg += "\n\t\t" + extra + + self.log.warning(msg) + + def add_symbol(self, what, fname, ln=None, xref=None): + """Create a reference table describing where each 'what' is located""" + + if what not in self.what_symbols: + self.what_symbols[what] = {"file": {}} + + if fname not in self.what_symbols[what]["file"]: + self.what_symbols[what]["file"][fname] = [] + + if ln and ln not in self.what_symbols[what]["file"][fname]: + self.what_symbols[what]["file"][fname].append(ln) + + if xref: + self.what_symbols[what]["xref"] = xref + + def _parse_line(self, fdata, line): + """Parse a single line of an ABI file""" + + new_what = False + new_tag = False + content = None + + match = self.re_tag.match(line) + if match: + new = match.group(1).lower() + sep = match.group(2) + content = match.group(3) + + match = self.re_valid.search(new) + if match: + new_tag = match.group(1) + else: + if fdata.tag == "description": + # New "tag" is actually part of description. + # Don't consider it a tag + new_tag = False + elif fdata.tag != "": + self.warn(fdata, f"tag '{fdata.tag}' is invalid", line) + + if new_tag: + # "where" is Invalid, but was a common mistake. Warn if found + if new_tag == "where": + self.warn(fdata, "tag 'Where' is invalid. Should be 'What:' instead") + new_tag = "what" + + if new_tag == "what": + fdata.space = None + + if content not in self.what_symbols: + self.add_symbol(what=content, fname=fdata.fname, ln=fdata.ln) + + if fdata.tag == "what": + fdata.what.append(content.strip("\n")) + else: + if fdata.key: + if "description" not in self.data.get(fdata.key, {}): + self.warn(fdata, f"{fdata.key} doesn't have a description") + + for w in fdata.what: + self.add_symbol(what=w, fname=fdata.fname, + ln=fdata.what_ln, xref=fdata.key) + + fdata.label = content + new_what = True + + key = "abi_" + content.lower() + fdata.key = self.re_unprintable.sub("_", key).strip("_") + + # Avoid duplicated keys but using a defined seed, to make + # the namespace identical if there aren't changes at the + # ABI symbols + seed(42) + + while fdata.key in self.data: + char = randrange(0, 51) + ord("A") + if char > ord("Z"): + char += ord("a") - ord("Z") - 1 + + fdata.key += chr(char) + + if fdata.key and fdata.key not in self.data: + self.data[fdata.key] = { + "what": [content], + "file": [fdata.file_ref], + "line_no": fdata.ln, + } + + fdata.what = self.data[fdata.key]["what"] + + self.what_refs[content] = fdata.key + fdata.tag = new_tag + fdata.what_ln = fdata.ln + + if fdata.nametag["what"]: + t = (content, fdata.key) + if t not in fdata.nametag["symbols"]: + fdata.nametag["symbols"].append(t) + + return + + if fdata.tag and new_tag: + fdata.tag = new_tag + + if new_what: + fdata.label = "" + + self.data[fdata.key]["type"] = fdata.ftype + + if "description" in self.data[fdata.key]: + self.data[fdata.key]["description"] += "\n\n" + + if fdata.file_ref not in self.data[fdata.key]["file"]: + self.data[fdata.key]["file"].append(fdata.file_ref) + + if self.debug == AbiDebug.WHAT_PARSING: + self.log.debug("what: %s", fdata.what) + + if not fdata.what: + self.warn(fdata, "'What:' should come first:", line) + return + + if new_tag == "description": + fdata.space = None + + if content: + sep = sep.replace(":", " ") + + c = " " * len(new_tag) + sep + content + c = c.expandtabs() + + match = self.re_start_spc.match(c) + if match: + # Preserve initial spaces for the first line + fdata.space = match.group(1) + content = match.group(2) + "\n" + + self.data[fdata.key][fdata.tag] = content + + return + + # Store any contents before tags at the database + if not fdata.tag and "what" in fdata.nametag: + fdata.nametag["description"] += line + return + + if fdata.tag == "description": + content = line.expandtabs() + + if self.re_whitespace.sub("", content) == "": + self.data[fdata.key][fdata.tag] += "\n" + return + + if fdata.space is None: + match = self.re_start_spc.match(content) + if match: + # Preserve initial spaces for the first line + fdata.space = match.group(1) + + content = match.group(2) + "\n" + else: + if content.startswith(fdata.space): + content = content[len(fdata.space):] + + else: + fdata.space = "" + + if fdata.tag == "what": + w = content.strip("\n") + if w: + self.data[fdata.key][fdata.tag].append(w) + else: + self.data[fdata.key][fdata.tag] += content + return + + content = line.strip() + if fdata.tag: + if fdata.tag == "what": + w = content.strip("\n") + if w: + self.data[fdata.key][fdata.tag].append(w) + else: + self.data[fdata.key][fdata.tag] += "\n" + content.rstrip("\n") + return + + # Everything else is error + if content: + self.warn(fdata, "Unexpected content", line) + + def parse_file(self, fname, path, basename): + """Parse a single file""" + + ref = f"abi_file_{path}_{basename}" + ref = self.re_unprintable.sub("_", ref).strip("_") + + # Store per-file state into a namespace variable. This will be used + # by the per-line parser state machine and by the warning function. + fdata = Namespace + + fdata.fname = fname + fdata.name = basename + + pos = fname.find(ABI_DIR) + if pos > 0: + f = fname[pos:] + else: + f = fname + + fdata.file_ref = (f, ref) + self.file_refs[f] = ref + + fdata.ln = 0 + fdata.what_ln = 0 + fdata.tag = "" + fdata.label = "" + fdata.what = [] + fdata.key = None + fdata.xrefs = None + fdata.space = None + fdata.ftype = path.split("/")[0] + + fdata.nametag = {} + fdata.nametag["what"] = [f"File {path}/{basename}"] + fdata.nametag["type"] = "File" + fdata.nametag["file"] = [fdata.file_ref] + fdata.nametag["line_no"] = 1 + fdata.nametag["description"] = "" + fdata.nametag["symbols"] = [] + + self.data[ref] = fdata.nametag + + if self.debug & AbiDebug.WHAT_OPEN: + self.log.debug("Opening file %s", fname) + + with open(fname, "r", encoding="utf8", errors="backslashreplace") as fp: + for line in fp: + fdata.ln += 1 + + self._parse_line(fdata, line) + + if "description" in fdata.nametag: + fdata.nametag["description"] = fdata.nametag["description"].lstrip("\n") + + if fdata.key: + if "description" not in self.data.get(fdata.key, {}): + self.warn(fdata, f"{fdata.key} doesn't have a description") + + for w in fdata.what: + self.add_symbol(what=w, fname=fname, xref=fdata.key) + + def parse_abi(self): + """Parse documentation ABI""" + + ignore_suffixes = ("rej", "org", "orig", "bak", "~") + re_abi = re.compile(r".*" + ABI_DIR) + + for fname in glob(os.path.join(self.directory, "**"), recursive=True): + if os.path.isdir(fname): + continue + + basename = os.path.basename(fname) + + if basename == "README": + continue + if basename.startswith(".") or basename.endswith(ignore_suffixes): + continue + + path = re_abi.sub("", os.path.dirname(fname)) + + self.parse_file(fname, path, basename) + + if self.debug & AbiDebug.DUMP_ABI_STRUCTS: + self.log.debug(pformat(self.data)) + + def print_desc_txt(self, desc): + """Print description as found inside ABI files""" + + desc = desc.strip(" \t\n") + + print(desc + "\n") + + def print_desc_rst(self, desc): + """Enrich ReST output by creating cross-references""" + + # Remove title markups from the description + # Having titles inside ABI files will only work if extra + # care would be taken in order to strictly follow the same + # level order for each markup. + desc = self.re_title_mark.sub("\n\n", "\n" + desc) + desc = desc.rstrip(" \t\n").lstrip("\n") + + # Python's regex performance for non-compiled expressions is a lot + # than Perl, as Perl automatically caches them at their + # first usage. Here, we'll need to do the same, as otherwise the + # performance penalty is be high + + new_desc = "" + for d in desc.split("\n"): + if d == "": + new_desc += "\n" + continue + + # Use cross-references for doc files where needed + d = self.re_doc.sub(r":doc:`/\1`", d) + + # Use cross-references for ABI generated docs where needed + matches = self.re_abi.findall(d) + for m in matches: + abi = m[0] + m[1] + + xref = self.file_refs.get(abi) + if not xref: + # This may happen if ABI is on a separate directory, + # like parsing ABI testing and symbol is at stable. + # The proper solution is to move this part of the code + # for it to be inside sphinx/kernel_abi.py + self.log.info("Didn't find ABI reference for '%s'", abi) + else: + new = self.re_escape.sub(r"\\\1", m[1]) + d = re.sub(fr"\b{abi}\b", f":ref:`{new} <{xref}>`", d) + + # Seek for cross reference symbols like /sys/... + # Need to be careful to avoid doing it on a code block + if d[0] not in [" ", "\t"]: + matches = self.re_xref_node.findall(d) + for m in matches: + # Finding ABI here is more complex due to wildcards + xref = self.what_refs.get(m) + if xref: + new = self.re_escape.sub(r"\\\1", m) + d = re.sub(fr"\b{m}\b", f":ref:`{new} <{xref}>`", d) + + new_desc += d + "\n" + + print(new_desc + "\n") + + def print_data(self, enable_lineno, output_in_txt, show_file=False): + """Print ABI at stdout""" + + part = None + for key, v in sorted(self.data.items(), + key=lambda x: (x[1].get("type", ""), + x[1].get("what"))): + + wtype = v.get("type", "Var") + file_ref = v.get("file") + names = v.get("what", [""]) + + if not show_file and wtype == "File": + continue + + if enable_lineno: + ln = v.get("line_no", 1) + print(f".. LINENO {file_ref[0][0]}#{ln}\n") + + if wtype != "File": + cur_part = names[0] + if cur_part.find("/") >= 0: + match = self.re_what.match(cur_part) + if match: + symbol = match.group(1).rstrip("/") + cur_part = "Symbols under " + symbol + + if cur_part and cur_part != part: + part = cur_part + print(f"{part}\n{"-" * len(part)}\n") + + print(f".. _{key}:\n") + + max_len = 0 + for i in range(0, len(names)): # pylint: disable=C0200 + names[i] = "**" + self.re_escape.sub(r"\\\1", names[i]) + "**" + + max_len = max(max_len, len(names[i])) + + print("+-" + "-" * max_len + "-+") + for name in names: + print(f"| {name}" + " " * (max_len - len(name)) + " |") + print("+-" + "-" * max_len + "-+") + print() + + for ref in file_ref: + if wtype == "File": + print(f".. _{ref[1]}:\n") + else: + base = os.path.basename(ref[0]) + print(f"Defined on file :ref:`{base} <{ref[1]}>`\n") + + if wtype == "File": + print(f"{names[0]}\n{"-" * len(names[0])}\n") + + desc = v.get("description") + if not desc and wtype != "File": + print(f"DESCRIPTION MISSING for {names[0]}\n") + + if desc: + if output_in_txt: + self.print_desc_txt(desc) + else: + self.print_desc_rst(desc) + + symbols = v.get("symbols") + if symbols: + print("Has the following ABI:\n") + + for w, label in symbols: + # Escape special chars from content + content = self.re_escape.sub(r"\\\1", w) + + print(f"- :ref:`{content} <{label}>`\n") + + users = v.get("users") + if users and users.strip(" \t\n"): + print(f"Users:\n\t{users.strip("\n").replace('\n', '\n\t')}\n") + + def check_issues(self): + """Warn about duplicated ABI entries""" + + for what, v in self.what_symbols.items(): + files = v.get("file") + if not files: + # Should never happen if the parser works properly + self.log.warning("%s doesn't have a file associated", what) + continue + + if len(files) == 1: + continue + + f = [] + for fname, lines in sorted(files.items()): + if not lines: + f.append(f"{fname}") + elif len(lines) == 1: + f.append(f"{fname}:{lines[0]}") + else: + f.append(f"{fname} lines {", ".join(str(x) for x in lines)}") + + self.log.warning("%s is defined %d times: %s", what, len(f), "; ".join(f)) diff --git a/scripts/lib/abi/helpers.py b/scripts/lib/abi/helpers.py new file mode 100644 index 000000000000..84a253ed5058 --- /dev/null +++ b/scripts/lib/abi/helpers.py @@ -0,0 +1,28 @@ +#!/usr/bin/env python3 +# Copyright(c) 2025: Mauro Carvalho Chehab <mchehab@kernel.org>. +# pylint: disable=R0903 +# SPDX-License-Identifier: GPL-2.0 + +""" +Helper classes for ABI parser +""" + +ABI_DIR = "Documentation/ABI/" + + +class AbiDebug: + """Debug levels""" + + WHAT_PARSING = 1 + WHAT_OPEN = 2 + DUMP_ABI_STRUCTS = 4 + + +DEBUG_HELP = """ +Print debug information according with the level(s), +which is given by the following bitmask: + +1 - enable debug parsing logic +2 - enable debug messages on file open +4 - enable debug for ABI parse data +""" -- cgit v1.2.3 From 6b48bea16848dd7c771411db3dcc01b3bc4dd4c2 Mon Sep 17 00:00:00 2001 From: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Date: Mon, 10 Feb 2025 11:18:00 +0100 Subject: scripts/get_abi.py: add support for symbol search Add support for searching symbols from Documentation/ABI using regular expressions to match the symbols' names. Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Link: https://lore.kernel.org/r/21b2c48657dde112d5417dcd7e0aa7cd383b9a0a.1739182025.git.mchehab+huawei@kernel.org --- scripts/get_abi.py | 24 ++++++++++++++++++++ scripts/lib/abi/abi_parser.py | 52 +++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 76 insertions(+) (limited to 'scripts') diff --git a/scripts/get_abi.py b/scripts/get_abi.py index bb17c54feeff..30439f21fdd0 100755 --- a/scripts/get_abi.py +++ b/scripts/get_abi.py @@ -85,6 +85,29 @@ class AbiValidate: parser.check_issues() +class AbiSearch: + """Initialize an argparse subparser for ABI search""" + + def __init__(self, subparsers): + """Initialize argparse subparsers""" + + parser = subparsers.add_parser("search", + formatter_class=argparse.ArgumentDefaultsHelpFormatter, + description="Search ABI using a regular expression") + + parser.add_argument("expression", + help="Case-insensitive search pattern for the ABI symbol") + + parser.set_defaults(func=self.run) + + def run(self, args): + """Run subparser""" + + parser = AbiParser(args.dir, debug=args.debug) + parser.parse_abi() + parser.search_symbols(args.expression) + + def main(): """Main program""" @@ -97,6 +120,7 @@ def main(): AbiRest(subparsers) AbiValidate(subparsers) + AbiSearch(subparsers) args = parser.parse_args() diff --git a/scripts/lib/abi/abi_parser.py b/scripts/lib/abi/abi_parser.py index b3fa70eee412..bea7f1a76165 100644 --- a/scripts/lib/abi/abi_parser.py +++ b/scripts/lib/abi/abi_parser.py @@ -510,3 +510,55 @@ class AbiParser: f.append(f"{fname} lines {", ".join(str(x) for x in lines)}") self.log.warning("%s is defined %d times: %s", what, len(f), "; ".join(f)) + + def search_symbols(self, expr): + """ Searches for ABI symbols """ + + regex = re.compile(expr, re.I) + + found_keys = 0 + for t in sorted(self.data.items(), key=lambda x: [0]): + v = t[1] + + wtype = v.get("type", "") + if wtype == "File": + continue + + for what in v.get("what", [""]): + if regex.search(what): + found_keys += 1 + + kernelversion = v.get("kernelversion", "").strip(" \t\n") + date = v.get("date", "").strip(" \t\n") + contact = v.get("contact", "").strip(" \t\n") + users = v.get("users", "").strip(" \t\n") + desc = v.get("description", "").strip(" \t\n") + + files = [] + for f in v.get("file", ()): + files.append(f[0]) + + what = str(found_keys) + ". " + what + title_tag = "-" * len(what) + + print(f"\n{what}\n{title_tag}\n") + + if kernelversion: + print(f"Kernel version:\t\t{kernelversion}") + + if date: + print(f"Date:\t\t\t{date}") + + if contact: + print(f"Contact:\t\t{contact}") + + if users: + print(f"Users:\t\t\t{users}") + + print(f"Defined on file{'s'[:len(files) ^ 1]}:\t{", ".join(files)}") + + if desc: + print(f"\n{desc.strip("\n")}\n") + + if not found_keys: + print(f"Regular expression /{expr}/ not found.") -- cgit v1.2.3 From c67c3fbdd917884e38a366c38717c9f769075c15 Mon Sep 17 00:00:00 2001 From: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Date: Mon, 10 Feb 2025 11:18:02 +0100 Subject: scripts/lib/abi/abi_parser.py: optimize parse_abi() function Instead of using glob, use a recursive function to parse all files. Such change reduces the total excecution time by 15% with my SSD disks. Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Link: https://lore.kernel.org/r/190dd358897017ed82c56f1e263192215ffbae43.1739182025.git.mchehab+huawei@kernel.org --- scripts/lib/abi/abi_parser.py | 49 ++++++++++++++++++++++++++++++------------- 1 file changed, 34 insertions(+), 15 deletions(-) (limited to 'scripts') diff --git a/scripts/lib/abi/abi_parser.py b/scripts/lib/abi/abi_parser.py index bea7f1a76165..6052a8aec443 100644 --- a/scripts/lib/abi/abi_parser.py +++ b/scripts/lib/abi/abi_parser.py @@ -12,7 +12,6 @@ import logging import os import re -from glob import glob from pprint import pformat from random import randrange, seed @@ -46,7 +45,11 @@ class AbiParser: self.file_refs = {} self.what_refs = {} + # Ignore files that contain such suffixes + self.ignore_suffixes = (".rej", ".org", ".orig", ".bak", "~") + # Regular expressions used on parser + self.re_abi_dir = re.compile(r"(.*)" + ABI_DIR) self.re_tag = re.compile(r"(\S+)(:\s*)(.*)", re.I) self.re_valid = re.compile(self.TAGS) self.re_start_spc = re.compile(r"(\s*)(\S.*)") @@ -322,26 +325,42 @@ class AbiParser: for w in fdata.what: self.add_symbol(what=w, fname=fname, xref=fdata.key) - def parse_abi(self): - """Parse documentation ABI""" + def _parse_abi(self, root=None): + """Internal function to parse documentation ABI recursively""" - ignore_suffixes = ("rej", "org", "orig", "bak", "~") - re_abi = re.compile(r".*" + ABI_DIR) + if not root: + root = self.directory - for fname in glob(os.path.join(self.directory, "**"), recursive=True): - if os.path.isdir(fname): - continue + with os.scandir(root) as obj: + for entry in obj: + name = os.path.join(root, entry.name) - basename = os.path.basename(fname) + if entry.is_dir(): + self.parse_abi(name) + continue - if basename == "README": - continue - if basename.startswith(".") or basename.endswith(ignore_suffixes): - continue + if not entry.is_file(): + continue + + basename = os.path.basename(name) - path = re_abi.sub("", os.path.dirname(fname)) + if basename == "README": + continue + + if basename.startswith("."): + continue + + if basename.endswith(self.ignore_suffixes): + continue + + path = self.re_abi_dir.sub("", os.path.dirname(name)) + + self.parse_file(name, path, basename) + + def parse_abi(self, root=None): + """Parse documentation ABI""" - self.parse_file(fname, path, basename) + self._parse_abi(root) if self.debug & AbiDebug.DUMP_ABI_STRUCTS: self.log.debug(pformat(self.data)) -- cgit v1.2.3 From 9bec7870c64c00983773cfddab8d6a037f7767f3 Mon Sep 17 00:00:00 2001 From: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Date: Mon, 10 Feb 2025 11:18:03 +0100 Subject: scripts/lib/abi/abi_parser.py: use an interactor for ReST output Instead of printing all results line per line, use an interactor to return each variable as a separate message. This won't change much when using it via command line, but it will help Sphinx integration by providing an interactor that could be used there to handle ABI symbol by symbol. Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Link: https://lore.kernel.org/r/e3c94b8cdfd5e955aa19a703921f364a89089634.1739182025.git.mchehab+huawei@kernel.org --- scripts/get_abi.py | 3 ++- scripts/lib/abi/abi_parser.py | 48 +++++++++++++++++++++++-------------------- 2 files changed, 28 insertions(+), 23 deletions(-) (limited to 'scripts') diff --git a/scripts/get_abi.py b/scripts/get_abi.py index 30439f21fdd0..93b973bc07ed 100755 --- a/scripts/get_abi.py +++ b/scripts/get_abi.py @@ -62,8 +62,9 @@ class AbiRest: parser = AbiParser(args.dir, debug=args.debug) parser.parse_abi() parser.check_issues() - parser.print_data(args.enable_lineno, args.raw, not args.no_file) + for msg in parser.doc(args.enable_lineno, args.raw, not args.no_file): + print(msg) class AbiValidate: """Initialize an argparse subparser for ABI validation""" diff --git a/scripts/lib/abi/abi_parser.py b/scripts/lib/abi/abi_parser.py index 6052a8aec443..960e27161c26 100644 --- a/scripts/lib/abi/abi_parser.py +++ b/scripts/lib/abi/abi_parser.py @@ -336,7 +336,7 @@ class AbiParser: name = os.path.join(root, entry.name) if entry.is_dir(): - self.parse_abi(name) + self._parse_abi(name) continue if not entry.is_file(): @@ -365,14 +365,14 @@ class AbiParser: if self.debug & AbiDebug.DUMP_ABI_STRUCTS: self.log.debug(pformat(self.data)) - def print_desc_txt(self, desc): + def desc_txt(self, desc): """Print description as found inside ABI files""" desc = desc.strip(" \t\n") - print(desc + "\n") + return desc + "\n\n" - def print_desc_rst(self, desc): + def desc_rst(self, desc): """Enrich ReST output by creating cross-references""" # Remove title markups from the description @@ -425,9 +425,9 @@ class AbiParser: new_desc += d + "\n" - print(new_desc + "\n") + return new_desc + "\n\n" - def print_data(self, enable_lineno, output_in_txt, show_file=False): + def doc(self, enable_lineno, output_in_txt, show_file=False): """Print ABI at stdout""" part = None @@ -442,9 +442,11 @@ class AbiParser: if not show_file and wtype == "File": continue + msg = "" + if enable_lineno: ln = v.get("line_no", 1) - print(f".. LINENO {file_ref[0][0]}#{ln}\n") + msg += f".. LINENO {file_ref[0][0]}#{ln}\n\n" if wtype != "File": cur_part = names[0] @@ -456,9 +458,9 @@ class AbiParser: if cur_part and cur_part != part: part = cur_part - print(f"{part}\n{"-" * len(part)}\n") + msg += f"{part}\n{"-" * len(part)}\n\n" - print(f".. _{key}:\n") + msg += f".. _{key}:\n\n" max_len = 0 for i in range(0, len(names)): # pylint: disable=C0200 @@ -466,45 +468,47 @@ class AbiParser: max_len = max(max_len, len(names[i])) - print("+-" + "-" * max_len + "-+") + msg += "+-" + "-" * max_len + "-+\n" for name in names: - print(f"| {name}" + " " * (max_len - len(name)) + " |") - print("+-" + "-" * max_len + "-+") - print() + msg += f"| {name}" + " " * (max_len - len(name)) + " |\n" + msg += "+-" + "-" * max_len + "-+\n" + msg += "\n" for ref in file_ref: if wtype == "File": - print(f".. _{ref[1]}:\n") + msg += f".. _{ref[1]}:\n\n" else: base = os.path.basename(ref[0]) - print(f"Defined on file :ref:`{base} <{ref[1]}>`\n") + msg += f"Defined on file :ref:`{base} <{ref[1]}>`\n\n" if wtype == "File": - print(f"{names[0]}\n{"-" * len(names[0])}\n") + msg += f"{names[0]}\n{"-" * len(names[0])}\n\n" desc = v.get("description") if not desc and wtype != "File": - print(f"DESCRIPTION MISSING for {names[0]}\n") + msg += f"DESCRIPTION MISSING for {names[0]}\n\n" if desc: if output_in_txt: - self.print_desc_txt(desc) + msg += self.desc_txt(desc) else: - self.print_desc_rst(desc) + msg += self.desc_rst(desc) symbols = v.get("symbols") if symbols: - print("Has the following ABI:\n") + msg += "Has the following ABI:\n\n" for w, label in symbols: # Escape special chars from content content = self.re_escape.sub(r"\\\1", w) - print(f"- :ref:`{content} <{label}>`\n") + msg += f"- :ref:`{content} <{label}>`\n\n" users = v.get("users") if users and users.strip(" \t\n"): - print(f"Users:\n\t{users.strip("\n").replace('\n', '\n\t')}\n") + msg += f"Users:\n\t{users.strip("\n").replace('\n', '\n\t')}\n\n" + + yield msg def check_issues(self): """Warn about duplicated ABI entries""" -- cgit v1.2.3 From ee34f8300c8940758dc69f80107d9f5873c08f17 Mon Sep 17 00:00:00 2001 From: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Date: Mon, 10 Feb 2025 11:18:04 +0100 Subject: docs: sphinx/kernel_abi: use AbiParser directly Instead of running get_abi.py script, import AbiParser class and handle messages directly there using an interactor. This shold save some memory, as there's no need to exec python inside the Sphinx python extension. Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Link: https://lore.kernel.org/r/8dbc244dcda97112c1b694e2512a5d600e62873b.1739182025.git.mchehab+huawei@kernel.org --- Documentation/sphinx/kernel_abi.py | 29 ++++++++++++++++------------- scripts/lib/abi/abi_parser.py | 2 +- 2 files changed, 17 insertions(+), 14 deletions(-) (limited to 'scripts') diff --git a/Documentation/sphinx/kernel_abi.py b/Documentation/sphinx/kernel_abi.py index f314b888d3de..f7b22abebcf4 100644 --- a/Documentation/sphinx/kernel_abi.py +++ b/Documentation/sphinx/kernel_abi.py @@ -34,7 +34,6 @@ u""" import os import re -import subprocess import sys from docutils import nodes @@ -43,6 +42,11 @@ from docutils.parsers.rst import directives, Directive from sphinx.util.docutils import switch_source_input from sphinx.util import logging +srctree = os.path.abspath(os.environ["srctree"]) +sys.path.insert(0, os.path.join(srctree, "scripts/lib/abi")) + +from abi_parser import AbiParser + __version__ = "1.0" @@ -66,7 +70,7 @@ class KernelCmd(Directive): logger = logging.getLogger('kernel_abi') option_spec = { - "debug" : directives.flag, + "debug": directives.flag, } def run(self): @@ -74,20 +78,19 @@ class KernelCmd(Directive): if not doc.settings.file_insertion_enabled: raise self.warning("docutils: file insertion disabled") - srctree = os.path.abspath(os.environ["srctree"]) + path = os.path.join(srctree, "Documentation", self.arguments[0]) + parser = AbiParser(path, logger=self.logger) + parser.parse_abi() + parser.check_issues() - args = [ - os.path.join(srctree, 'scripts/get_abi.py'), - '-D', os.path.join(srctree, 'Documentation', self.arguments[0]), - 'rest', - '--enable-lineno', - ] + msg = "" + for m in parser.doc(enable_lineno=True, show_file=True): + msg += m - lines = subprocess.check_output(args, cwd=os.path.dirname(doc.current_source)).decode('utf-8') - nodeList = self.nestedParse(lines, self.arguments[0]) - return nodeList + node = self.nested_parse(msg, self.arguments[0]) + return node - def nestedParse(self, lines, fname): + def nested_parse(self, lines, fname): env = self.state.document.settings.env content = ViewList() node = nodes.section() diff --git a/scripts/lib/abi/abi_parser.py b/scripts/lib/abi/abi_parser.py index 960e27161c26..57c125fd40a5 100644 --- a/scripts/lib/abi/abi_parser.py +++ b/scripts/lib/abi/abi_parser.py @@ -427,7 +427,7 @@ class AbiParser: return new_desc + "\n\n" - def doc(self, enable_lineno, output_in_txt, show_file=False): + def doc(self, enable_lineno, output_in_txt=False, show_file=False): """Print ABI at stdout""" part = None -- cgit v1.2.3 From aea5e52dce74f679b91c66caad91d587d5504f6c Mon Sep 17 00:00:00 2001 From: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Date: Mon, 10 Feb 2025 11:18:05 +0100 Subject: docs: sphinx/kernel_abi: reduce buffer usage for ABI messages Instead of producing a big message with all ABI contents and then parse as a whole, simplify the code by handling each ABI symbol in separate. As an additional benefit, there's no need to place file/line nubers inlined at the data and use a regex to convert them. Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Link: https://lore.kernel.org/r/15be22955e3c6df49d7256c8fd24f62b397ad0ff.1739182025.git.mchehab+huawei@kernel.org --- Documentation/sphinx/kernel_abi.py | 82 +++++++++++++++++++------------------- scripts/get_abi.py | 7 +++- scripts/lib/abi/abi_parser.py | 10 ++--- 3 files changed, 51 insertions(+), 48 deletions(-) (limited to 'scripts') diff --git a/Documentation/sphinx/kernel_abi.py b/Documentation/sphinx/kernel_abi.py index f7b22abebcf4..742ebd35454f 100644 --- a/Documentation/sphinx/kernel_abi.py +++ b/Documentation/sphinx/kernel_abi.py @@ -68,6 +68,7 @@ class KernelCmd(Directive): has_content = False final_argument_whitespace = True logger = logging.getLogger('kernel_abi') + parser = None option_spec = { "debug": directives.flag, @@ -79,59 +80,60 @@ class KernelCmd(Directive): raise self.warning("docutils: file insertion disabled") path = os.path.join(srctree, "Documentation", self.arguments[0]) - parser = AbiParser(path, logger=self.logger) - parser.parse_abi() - parser.check_issues() + self.parser = AbiParser(path, logger=self.logger) + self.parser.parse_abi() + self.parser.check_issues() - msg = "" - for m in parser.doc(enable_lineno=True, show_file=True): - msg += m - - node = self.nested_parse(msg, self.arguments[0]) + node = self.nested_parse(None, self.arguments[0]) return node - def nested_parse(self, lines, fname): + def nested_parse(self, data, fname): env = self.state.document.settings.env content = ViewList() node = nodes.section() - if "debug" in self.options: - code_block = "\n\n.. code-block:: rst\n :linenos:\n" - for line in lines.split("\n"): - code_block += "\n " + line - lines = code_block + "\n\n" - - line_regex = re.compile(r"^\.\. LINENO (\S+)\#([0-9]+)$") - ln = 0 - n = 0 - f = fname - - for line in lines.split("\n"): - n = n + 1 - match = line_regex.search(line) - if match: - new_f = match.group(1) - - # Sphinx parser is lazy: it stops parsing contents in the - # middle, if it is too big. So, handle it per input file - if new_f != f and content: - self.do_parse(content, node) - content = ViewList() + if data is not None: + # Handles the .rst file + for line in data.split("\n"): + content.append(line, fname, 0) + + self.do_parse(content, node) + else: + # Handles the ABI parser content, symbol by symbol + + old_f = fname + n = 0 + for msg, f, ln in self.parser.doc(): + msg_list = msg.split("\n") + if "debug" in self.options: + lines = [ + "", "", ".. code-block:: rst", + " :linenos:", "" + ] + for m in msg_list: + lines.append(" " + m) + else: + lines = msg_list + + for line in lines: + # sphinx counts lines from 0 + content.append(line, f, ln - 1) + n += 1 + + if f != old_f: # Add the file to Sphinx build dependencies env.note_dependency(os.path.abspath(f)) - f = new_f + old_f = f - # sphinx counts lines from 0 - ln = int(match.group(2)) - 1 - else: - content.append(line, f, ln) - - self.logger.info("%s: parsed %i lines" % (fname, n)) + # Sphinx doesn't like to parse big messages. So, let's + # add content symbol by symbol + if content: + self.do_parse(content, node) + content = ViewList() - if content: - self.do_parse(content, node) + self.logger.info("%s: parsed %i lines" % (fname, n)) return node.children diff --git a/scripts/get_abi.py b/scripts/get_abi.py index 93b973bc07ed..19f78d6aa407 100755 --- a/scripts/get_abi.py +++ b/scripts/get_abi.py @@ -63,8 +63,11 @@ class AbiRest: parser.parse_abi() parser.check_issues() - for msg in parser.doc(args.enable_lineno, args.raw, not args.no_file): - print(msg) + for t in parser.doc(args.raw, not args.no_file): + if args.enable_lineno: + print (f".. LINENO {t[1]}#{t[2]}\n\n") + + print(t[0]) class AbiValidate: """Initialize an argparse subparser for ABI validation""" diff --git a/scripts/lib/abi/abi_parser.py b/scripts/lib/abi/abi_parser.py index 57c125fd40a5..1db6c54fc65a 100644 --- a/scripts/lib/abi/abi_parser.py +++ b/scripts/lib/abi/abi_parser.py @@ -427,7 +427,7 @@ class AbiParser: return new_desc + "\n\n" - def doc(self, enable_lineno, output_in_txt=False, show_file=False): + def doc(self, output_in_txt=False, show_file=True): """Print ABI at stdout""" part = None @@ -444,10 +444,6 @@ class AbiParser: msg = "" - if enable_lineno: - ln = v.get("line_no", 1) - msg += f".. LINENO {file_ref[0][0]}#{ln}\n\n" - if wtype != "File": cur_part = names[0] if cur_part.find("/") >= 0: @@ -508,7 +504,9 @@ class AbiParser: if users and users.strip(" \t\n"): msg += f"Users:\n\t{users.strip("\n").replace('\n', '\n\t')}\n\n" - yield msg + ln = v.get("line_no", 1) + + yield (msg, file_ref[0][0], ln) def check_issues(self): """Warn about duplicated ABI entries""" -- cgit v1.2.3 From 2a21d80dfb4135b4766d8ff3231a3ea1c19bcc83 Mon Sep 17 00:00:00 2001 From: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Date: Mon, 10 Feb 2025 11:18:07 +0100 Subject: scripts/get_abi.pl: Add filtering capabilities to rest output This way, Sphinx ABI extension can parse symbols only once, while keep displaying results in separate files. Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Link: https://lore.kernel.org/r/41e108e816e46434aa596e5c0d25d227cb9f0fe5.1739182025.git.mchehab+huawei@kernel.org --- scripts/lib/abi/abi_parser.py | 21 +++++++++++++++------ 1 file changed, 15 insertions(+), 6 deletions(-) (limited to 'scripts') diff --git a/scripts/lib/abi/abi_parser.py b/scripts/lib/abi/abi_parser.py index 1db6c54fc65a..b20d5c9d920e 100644 --- a/scripts/lib/abi/abi_parser.py +++ b/scripts/lib/abi/abi_parser.py @@ -160,6 +160,7 @@ class AbiParser: self.data[fdata.key] = { "what": [content], "file": [fdata.file_ref], + "path": fdata.ftype, "line_no": fdata.ln, } @@ -182,8 +183,6 @@ class AbiParser: if new_what: fdata.label = "" - self.data[fdata.key]["type"] = fdata.ftype - if "description" in self.data[fdata.key]: self.data[fdata.key]["description"] += "\n\n" @@ -299,6 +298,7 @@ class AbiParser: fdata.nametag = {} fdata.nametag["what"] = [f"File {path}/{basename}"] fdata.nametag["type"] = "File" + fdata.nametag["path"] = fdata.ftype fdata.nametag["file"] = [fdata.file_ref] fdata.nametag["line_no"] = 1 fdata.nametag["description"] = "" @@ -427,7 +427,8 @@ class AbiParser: return new_desc + "\n\n" - def doc(self, output_in_txt=False, show_file=True): + def doc(self, output_in_txt=False, show_symbols=True, show_file=True, + filter_path=None): """Print ABI at stdout""" part = None @@ -435,12 +436,20 @@ class AbiParser: key=lambda x: (x[1].get("type", ""), x[1].get("what"))): - wtype = v.get("type", "Var") + wtype = v.get("type", "Symbol") file_ref = v.get("file") names = v.get("what", [""]) - if not show_file and wtype == "File": - continue + if wtype == "File": + if not show_file: + continue + else: + if not show_symbols: + continue + + if filter_path: + if v.get("path") != filter_path: + continue msg = "" -- cgit v1.2.3 From 98a4324a8b7bbe433483c90524026be0ccc9ffa8 Mon Sep 17 00:00:00 2001 From: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Date: Mon, 10 Feb 2025 11:18:08 +0100 Subject: scripts/get_abi.pl: add support to parse ABI README file The Documentation/ABI/README file is currently outside the documentation tree. Yet, it may still provide some useful information. Add it to the documentation parsing. As a plus, this avoids a warning when detecting missing cross-references. Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Link: https://lore.kernel.org/r/f1285dedfe4d0eb0f0af34f6a68bee6fde36dd7d.1739182025.git.mchehab+huawei@kernel.org --- scripts/lib/abi/abi_parser.py | 25 ++++++++++++++++++++----- 1 file changed, 20 insertions(+), 5 deletions(-) (limited to 'scripts') diff --git a/scripts/lib/abi/abi_parser.py b/scripts/lib/abi/abi_parser.py index b20d5c9d920e..6fac461d794c 100644 --- a/scripts/lib/abi/abi_parser.py +++ b/scripts/lib/abi/abi_parser.py @@ -263,6 +263,16 @@ class AbiParser: if content: self.warn(fdata, "Unexpected content", line) + def parse_readme(self, nametag, fname): + """Parse ABI README file""" + + with open(fname, "r", encoding="utf8", errors="backslashreplace") as fp: + nametag["description"] = "```\n" + for line in fp: + nametag["description"] += " " + line + + nametag["description"] += "```\n" + def parse_file(self, fname, path, basename): """Parse a single file""" @@ -309,6 +319,10 @@ class AbiParser: if self.debug & AbiDebug.WHAT_OPEN: self.log.debug("Opening file %s", fname) + if basename == "README": + self.parse_readme(fdata.nametag, fname) + return + with open(fname, "r", encoding="utf8", errors="backslashreplace") as fp: for line in fp: fdata.ln += 1 @@ -344,9 +358,6 @@ class AbiParser: basename = os.path.basename(name) - if basename == "README": - continue - if basename.startswith("."): continue @@ -448,8 +459,12 @@ class AbiParser: continue if filter_path: - if v.get("path") != filter_path: - continue + if filter_path == "README": + if not names[0].endswith("README"): + continue + else: + if v.get("path") != filter_path: + continue msg = "" -- cgit v1.2.3 From 5d7871d77f6d62406b3d459a58810c1ddb8904c2 Mon Sep 17 00:00:00 2001 From: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Date: Mon, 10 Feb 2025 11:18:09 +0100 Subject: docs: sphinx/kernel_abi: parse ABI files only once Right now, the logic parses ABI files on 4 steps, one for each directory. While this is fine in principle, by doing that, not all symbol cross-references will be created. Change the logic to do the parsing only once in order to get a global dictionary to be used when creating ABI cross-references. Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Link: https://lore.kernel.org/r/5205c53838b6ea25f4cdd4cc1e3d17c0141e75a6.1739182025.git.mchehab+huawei@kernel.org --- Documentation/admin-guide/abi-obsolete.rst | 2 +- Documentation/admin-guide/abi-removed.rst | 2 +- Documentation/admin-guide/abi-stable.rst | 2 +- Documentation/admin-guide/abi-testing.rst | 2 +- Documentation/sphinx/kernel_abi.py | 115 ++++++++++++++++------------- scripts/lib/abi/abi_parser.py | 22 +++--- 6 files changed, 81 insertions(+), 64 deletions(-) (limited to 'scripts') diff --git a/Documentation/admin-guide/abi-obsolete.rst b/Documentation/admin-guide/abi-obsolete.rst index 6d4d9ab7b8c3..bdef91d2cea4 100644 --- a/Documentation/admin-guide/abi-obsolete.rst +++ b/Documentation/admin-guide/abi-obsolete.rst @@ -9,4 +9,4 @@ marked to be removed at some later point in time. The description of the interface will document the reason why it is obsolete and when it can be expected to be removed. -.. kernel-abi:: ABI/obsolete +.. kernel-abi:: obsolete diff --git a/Documentation/admin-guide/abi-removed.rst b/Documentation/admin-guide/abi-removed.rst index 9fc78af6f077..bea0608b8442 100644 --- a/Documentation/admin-guide/abi-removed.rst +++ b/Documentation/admin-guide/abi-removed.rst @@ -3,4 +3,4 @@ ABI removed symbols =================== -.. kernel-abi:: ABI/removed +.. kernel-abi:: removed diff --git a/Documentation/admin-guide/abi-stable.rst b/Documentation/admin-guide/abi-stable.rst index c47c2a295865..33637c0d4fd5 100644 --- a/Documentation/admin-guide/abi-stable.rst +++ b/Documentation/admin-guide/abi-stable.rst @@ -12,4 +12,4 @@ for at least 2 years. Most interfaces (like syscalls) are expected to never change and always be available. -.. kernel-abi:: ABI/stable +.. kernel-abi:: stable diff --git a/Documentation/admin-guide/abi-testing.rst b/Documentation/admin-guide/abi-testing.rst index 40b31985e587..55054985a8ff 100644 --- a/Documentation/admin-guide/abi-testing.rst +++ b/Documentation/admin-guide/abi-testing.rst @@ -18,4 +18,4 @@ Programs that use these interfaces are strongly encouraged to add their name to the description of these interfaces, so that the kernel developers can easily notify them if any changes occur. -.. kernel-abi:: ABI/testing +.. kernel-abi:: testing diff --git a/Documentation/sphinx/kernel_abi.py b/Documentation/sphinx/kernel_abi.py index 0a4057183208..964f586de171 100644 --- a/Documentation/sphinx/kernel_abi.py +++ b/Documentation/sphinx/kernel_abi.py @@ -49,6 +49,13 @@ from abi_parser import AbiParser __version__ = "1.0" +logger = logging.getLogger('kernel_abi') +path = os.path.join(srctree, "Documentation/ABI") + +# Parse ABI symbols only once +kernel_abi = AbiParser(path, logger=logger) +kernel_abi.parse_abi() +kernel_abi.check_issues() def setup(app): @@ -64,14 +71,15 @@ class KernelCmd(Directive): u"""KernelABI (``kernel-abi``) directive""" required_arguments = 1 - optional_arguments = 2 + optional_arguments = 3 has_content = False final_argument_whitespace = True - logger = logging.getLogger('kernel_abi') parser = None option_spec = { "debug": directives.flag, + "no-symbols": directives.flag, + "no-files": directives.flag, } def run(self): @@ -79,62 +87,67 @@ class KernelCmd(Directive): if not doc.settings.file_insertion_enabled: raise self.warning("docutils: file insertion disabled") - path = os.path.join(srctree, "Documentation", self.arguments[0]) - self.parser = AbiParser(path, logger=self.logger) - self.parser.parse_abi() - self.parser.check_issues() - - node = self.nested_parse(None, self.arguments[0]) - return node - - def nested_parse(self, data, fname): env = self.state.document.settings.env content = ViewList() node = nodes.section() - if data is not None: - # Handles the .rst file - for line in data.split("\n"): - content.append(line, fname, 0) + abi_type = self.arguments[0] - self.do_parse(content, node) + if "no-symbols" in self.options: + show_symbols = False + else: + show_symbols = True + if "no-files" in self.options: + show_file = False + else: + show_file = True + + tab_width = self.options.get('tab-width', + self.state.document.settings.tab_width) + + old_f = None + n = 0 + n_sym = 0 + for msg, f, ln in kernel_abi.doc(show_file=show_file, + show_symbols=show_symbols, + filter_path=abi_type): + n_sym += 1 + msg_list = statemachine.string2lines(msg, tab_width, + convert_whitespace=True) + if "debug" in self.options: + lines = [ + "", "", ".. code-block:: rst", + " :linenos:", "" + ] + for m in msg_list: + lines.append(" " + m) + else: + lines = msg_list + + for line in lines: + # sphinx counts lines from 0 + content.append(line, f, ln - 1) + n += 1 + + if f != old_f: + # Add the file to Sphinx build dependencies + env.note_dependency(os.path.abspath(f)) + + old_f = f + + # Sphinx doesn't like to parse big messages. So, let's + # add content symbol by symbol + if content: + self.do_parse(content, node) + content = ViewList() + + if show_symbols and not show_file: + logger.verbose("%s ABI: %i symbols (%i ReST lines)" % (abi_type, n_sym, n)) + elif not show_symbols and show_file: + logger.verbose("%s ABI: %i files (%i ReST lines)" % (abi_type, n_sym, n)) else: - # Handles the ABI parser content, symbol by symbol - - old_f = fname - n = 0 - for msg, f, ln in self.parser.doc(): - msg_list = statemachine.string2lines(msg, tab_width, - convert_whitespace=True) - if "debug" in self.options: - lines = [ - "", "", ".. code-block:: rst", - " :linenos:", "" - ] - for m in msg_list: - lines.append(" " + m) - else: - lines = msg_list - - for line in lines: - # sphinx counts lines from 0 - content.append(line, f, ln - 1) - n += 1 - - if f != old_f: - # Add the file to Sphinx build dependencies - env.note_dependency(os.path.abspath(f)) - - old_f = f - - # Sphinx doesn't like to parse big messages. So, let's - # add content symbol by symbol - if content: - self.do_parse(content, node) - content = ViewList() - - self.logger.info("%s: parsed %i lines" % (fname, n)) + logger.verbose("%s ABI: %i data (%i ReST lines)" % (abi_type, n_sym, n)) return node.children diff --git a/scripts/lib/abi/abi_parser.py b/scripts/lib/abi/abi_parser.py index 6fac461d794c..87d1b9e14bb3 100644 --- a/scripts/lib/abi/abi_parser.py +++ b/scripts/lib/abi/abi_parser.py @@ -266,12 +266,20 @@ class AbiParser: def parse_readme(self, nametag, fname): """Parse ABI README file""" + nametag["what"] = ["ABI file contents"] + nametag["path"] = "README" with open(fname, "r", encoding="utf8", errors="backslashreplace") as fp: - nametag["description"] = "```\n" for line in fp: - nametag["description"] += " " + line + match = self.re_tag.match(line) + if match: + new = match.group(1).lower() + + match = self.re_valid.search(new) + if match: + nametag["description"] += "\n:" + line + continue - nametag["description"] += "```\n" + nametag["description"] += line def parse_file(self, fname, path, basename): """Parse a single file""" @@ -459,12 +467,8 @@ class AbiParser: continue if filter_path: - if filter_path == "README": - if not names[0].endswith("README"): - continue - else: - if v.get("path") != filter_path: - continue + if v.get("path") != filter_path: + continue msg = "" -- cgit v1.2.3 From c940816968da6ef9a9462b7c070cc333d609a16c Mon Sep 17 00:00:00 2001 From: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Date: Mon, 10 Feb 2025 11:18:11 +0100 Subject: docs: sphinx/automarkup: add cross-references for ABI Now that all ABI files are handled together, we can add a feature at automarkup for it to generate cross-references for ABI symbols. The cross-reference logic can produce references for all existing files, except for README (as this is not parsed). For symbols, they need to be an exact match of what it is described at the docs, which is not always true due to wildcards. If symbols at /sys /proc and /config are identical, a cross-reference will be used. Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Link: https://lore.kernel.org/r/0b97a51b68b1c20127ad4a6a55658557fe0848d0.1739182025.git.mchehab+huawei@kernel.org --- Documentation/sphinx/automarkup.py | 45 ++++++++++++++++++++++++++++++++++++++ scripts/lib/abi/abi_parser.py | 11 ++++++++++ 2 files changed, 56 insertions(+) (limited to 'scripts') diff --git a/Documentation/sphinx/automarkup.py b/Documentation/sphinx/automarkup.py index a413f8dd5115..7d91c39b4ca6 100644 --- a/Documentation/sphinx/automarkup.py +++ b/Documentation/sphinx/automarkup.py @@ -11,6 +11,8 @@ from sphinx.errors import NoUri import re from itertools import chain +from kernel_abi import kernel_abi + # # Python 2 lacks re.ASCII... # @@ -48,6 +50,8 @@ RE_typedef = re.compile(r'\b(typedef)\s+([a-zA-Z_]\w+)', flags=ascii_p3) # an optional extension # RE_doc = re.compile(r'(\bDocumentation/)?((\.\./)*[\w\-/]+)\.(rst|txt)') +RE_abi_file = re.compile(r'(\bDocumentation/ABI/[\w\-/]+)') +RE_abi_symbol = re.compile(r'(\b/(sys|config|proc)/[\w\-/]+)') RE_namespace = re.compile(r'^\s*..\s*c:namespace::\s*(\S+)\s*$') @@ -84,10 +88,14 @@ def markup_refs(docname, app, node): # Associate each regex with the function that will markup its matches # markup_func_sphinx2 = {RE_doc: markup_doc_ref, + RE_abi_file: markup_abi_ref, + RE_abi_symbol: markup_abi_ref, RE_function: markup_c_ref, RE_generic_type: markup_c_ref} markup_func_sphinx3 = {RE_doc: markup_doc_ref, + RE_abi_file: markup_abi_ref, + RE_abi_symbol: markup_abi_ref, RE_function: markup_func_ref_sphinx3, RE_struct: markup_c_ref, RE_union: markup_c_ref, @@ -270,6 +278,43 @@ def markup_doc_ref(docname, app, match): else: return nodes.Text(match.group(0)) +# +# Try to replace a documentation reference of the form Documentation/ABI/... +# with a cross reference to that page +# +def markup_abi_ref(docname, app, match): + stddom = app.env.domains['std'] + # + # Go through the dance of getting an xref out of the std domain + # + fname = match.group(1) + target = kernel_abi.xref(fname) + + # Kernel ABI doesn't describe such file or symbol + if not target: + return nodes.Text(match.group(0)) + + pxref = addnodes.pending_xref('', refdomain = 'std', reftype = 'ref', + reftarget = target, modname = None, + classname = None, refexplicit = False) + + # + # XXX The Latex builder will throw NoUri exceptions here, + # work around that by ignoring them. + # + try: + xref = stddom.resolve_xref(app.env, docname, app.builder, 'ref', + target, pxref, None) + except NoUri: + xref = None + # + # Return the xref if we got it; otherwise just return the plain text. + # + if xref: + return xref + else: + return nodes.Text(match.group(0)) + def get_c_namespace(app, docname): source = app.env.doc2path(docname) with open(source) as f: diff --git a/scripts/lib/abi/abi_parser.py b/scripts/lib/abi/abi_parser.py index 87d1b9e14bb3..3b1ab4c0bdd7 100644 --- a/scripts/lib/abi/abi_parser.py +++ b/scripts/lib/abi/abi_parser.py @@ -391,6 +391,17 @@ class AbiParser: return desc + "\n\n" + def xref(self, fname): + """ + Converts a Documentation/ABI + basename into a ReST cross-reference + """ + + xref = self.file_refs.get(fname) + if not xref: + return None + else: + return xref + def desc_rst(self, desc): """Enrich ReST output by creating cross-references""" -- cgit v1.2.3 From dc525a7650d70668c4d54cf03b4cba793b72cb5a Mon Sep 17 00:00:00 2001 From: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Date: Mon, 10 Feb 2025 11:18:13 +0100 Subject: scripts/lib/abi/abi_parser.py: Rename title name for ABI files This makes them look better when generating cross-references. Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Link: https://lore.kernel.org/r/e44574cb2796861d6acbce839068ed3ef385d16c.1739182025.git.mchehab+huawei@kernel.org --- scripts/lib/abi/abi_parser.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'scripts') diff --git a/scripts/lib/abi/abi_parser.py b/scripts/lib/abi/abi_parser.py index 3b1ab4c0bdd7..0c3837e52afa 100644 --- a/scripts/lib/abi/abi_parser.py +++ b/scripts/lib/abi/abi_parser.py @@ -314,7 +314,7 @@ class AbiParser: fdata.ftype = path.split("/")[0] fdata.nametag = {} - fdata.nametag["what"] = [f"File {path}/{basename}"] + fdata.nametag["what"] = [f"ABI file {path}/{basename}"] fdata.nametag["type"] = "File" fdata.nametag["path"] = fdata.ftype fdata.nametag["file"] = [fdata.file_ref] -- cgit v1.2.3 From 6649b4217089c5d17dc210946baf9c9537c7fb5d Mon Sep 17 00:00:00 2001 From: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Date: Mon, 10 Feb 2025 11:18:14 +0100 Subject: scripts/lib/abi/abi_parser.py: make it backward-compatible with Python 3.6 Despite being introduced on Python 3.6, the original implementation was too limited: it doesn't accept anything but the argument. Even on python 3.10.12, support was still limited, as more complex operations cause SyntaxError: Exception occurred: File ".../linux/Documentation/sphinx/kernel_abi.py", line 48, in <module> from get_abi import AbiParser File ".../linux/scripts/lib/abi/abi_parser.py", line 525 msg += f"{part}\n{"-" * len(part)}\n\n" ^ SyntaxError: f-string: expecting '}' Replace f-strings by normal string concatenation when it doesn't work on Python 3.6. Reported-by: Akira Yokosawa <akiyks@gmail.com> Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Link: https://lore.kernel.org/r/41d2f85df134a46db46fed73a0f9697a3d2ae9ba.1739182025.git.mchehab+huawei@kernel.org --- scripts/lib/abi/abi_parser.py | 16 ++++++++++------ 1 file changed, 10 insertions(+), 6 deletions(-) (limited to 'scripts') diff --git a/scripts/lib/abi/abi_parser.py b/scripts/lib/abi/abi_parser.py index 0c3837e52afa..f08de6d3bf7c 100644 --- a/scripts/lib/abi/abi_parser.py +++ b/scripts/lib/abi/abi_parser.py @@ -493,7 +493,7 @@ class AbiParser: if cur_part and cur_part != part: part = cur_part - msg += f"{part}\n{"-" * len(part)}\n\n" + msg += part + "\n"+ "-" * len(part) +"\n\n" msg += f".. _{key}:\n\n" @@ -517,7 +517,7 @@ class AbiParser: msg += f"Defined on file :ref:`{base} <{ref[1]}>`\n\n" if wtype == "File": - msg += f"{names[0]}\n{"-" * len(names[0])}\n\n" + msg += names[0] +"\n" + "-" * len(names[0]) +"\n\n" desc = v.get("description") if not desc and wtype != "File": @@ -541,7 +541,8 @@ class AbiParser: users = v.get("users") if users and users.strip(" \t\n"): - msg += f"Users:\n\t{users.strip("\n").replace('\n', '\n\t')}\n\n" + users = users.strip("\n").replace('\n', '\n\t') + msg += f"Users:\n\t{users}\n\n" ln = v.get("line_no", 1) @@ -567,7 +568,9 @@ class AbiParser: elif len(lines) == 1: f.append(f"{fname}:{lines[0]}") else: - f.append(f"{fname} lines {", ".join(str(x) for x in lines)}") + m = fname + "lines " + m += ", ".join(str(x) for x in lines) + f.append(m) self.log.warning("%s is defined %d times: %s", what, len(f), "; ".join(f)) @@ -615,10 +618,11 @@ class AbiParser: if users: print(f"Users:\t\t\t{users}") - print(f"Defined on file{'s'[:len(files) ^ 1]}:\t{", ".join(files)}") + print("Defined on file(s):\t" + ", ".join(files)) if desc: - print(f"\n{desc.strip("\n")}\n") + desc = desc.strip("\n") + print(f"\n{desc}\n") if not found_keys: print(f"Regular expression /{expr}/ not found.") -- cgit v1.2.3 From 0d5fd96880d9135a4b35fb5523896b21b13dde78 Mon Sep 17 00:00:00 2001 From: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Date: Mon, 10 Feb 2025 11:18:15 +0100 Subject: scripts/get_abi.py: add support for undefined ABIs The undefined logic is complex and has lots of magic on it. Implement it, using the same algorithm we have at get_abi.pl. Yet, some tweaks to optimize performance and to make the code simpler were added here: - at the perl version, the tree graph had loops, so we had to use BFS to traverse it. On this version, the graph is a tree, so, it simplifies the what group for sysfs aliases; - the logic which splits regular expressions into subgroups was re-written to make it faster; - it may optionally use multiple processes to search for symbol matches; - it has some additional debug levels. Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Link: https://lore.kernel.org/r/1529c255845d117696d5af57d8dc05554663afdf.1739182025.git.mchehab+huawei@kernel.org --- scripts/get_abi.py | 68 +++++++ scripts/lib/abi/abi_regex.py | 234 +++++++++++++++++++++++ scripts/lib/abi/helpers.py | 16 +- scripts/lib/abi/system_symbols.py | 378 ++++++++++++++++++++++++++++++++++++++ 4 files changed, 693 insertions(+), 3 deletions(-) create mode 100644 scripts/lib/abi/abi_regex.py create mode 100644 scripts/lib/abi/system_symbols.py (limited to 'scripts') diff --git a/scripts/get_abi.py b/scripts/get_abi.py index 19f78d6aa407..7ce4748a46d2 100755 --- a/scripts/get_abi.py +++ b/scripts/get_abi.py @@ -20,7 +20,9 @@ SRC_DIR = os.path.dirname(os.path.realpath(__file__)) sys.path.insert(0, os.path.join(SRC_DIR, LIB_DIR)) from abi_parser import AbiParser # pylint: disable=C0413 +from abi_regex import AbiRegex # pylint: disable=C0413 from helpers import ABI_DIR, DEBUG_HELP # pylint: disable=C0413 +from system_symbols import SystemSymbols # pylint: disable=C0413 # Command line classes @@ -111,6 +113,71 @@ class AbiSearch: parser.parse_abi() parser.search_symbols(args.expression) +UNDEFINED_DESC=""" +Check undefined ABIs on local machine. + +Read sysfs devnodes and check if the devnodes there are defined inside +ABI documentation. + +The search logic tries to minimize the number of regular expressions to +search per each symbol. + +By default, it runs on a single CPU, as Python support for CPU threads +is still experimental, and multi-process runs on Python is very slow. + +On experimental tests, if the number of ABI symbols to search per devnode +is contained on a limit of ~150 regular expressions, using a single CPU +is a lot faster than using multiple processes. However, if the number of +regular expressions to check is at the order of ~30000, using multiple +CPUs speeds up the check. +""" + +class AbiUndefined: + """ + Initialize an argparse subparser for logic to check undefined ABI at + the current machine's sysfs + """ + + def __init__(self, subparsers): + """Initialize argparse subparsers""" + + parser = subparsers.add_parser("undefined", + formatter_class=argparse.RawTextHelpFormatter, + description=UNDEFINED_DESC) + + parser.add_argument("-S", "--sysfs-dir", default="/sys", + help="directory where sysfs is mounted") + parser.add_argument("-s", "--search-string", + help="search string regular expression to limit symbol search") + parser.add_argument("-H", "--show-hints", action="store_true", + help="Hints about definitions for missing ABI symbols.") + parser.add_argument("-j", "--jobs", "--max-workers", type=int, default=1, + help="If bigger than one, enables multiprocessing.") + parser.add_argument("-c", "--max-chunk-size", type=int, default=50, + help="Maximum number of chunk size") + parser.add_argument("-f", "--found", action="store_true", + help="Also show found items. " + "Helpful to debug the parser."), + parser.add_argument("-d", "--dry-run", action="store_true", + help="Don't actually search for undefined. " + "Helpful to debug the parser."), + + parser.set_defaults(func=self.run) + + def run(self, args): + """Run subparser""" + + abi = AbiRegex(args.dir, debug=args.debug, + search_string=args.search_string) + + abi_symbols = SystemSymbols(abi=abi, hints=args.show_hints, + sysfs=args.sysfs_dir) + + abi_symbols.check_undefined_symbols(dry_run=args.dry_run, + found=args.found, + max_workers=args.jobs, + chunk_size=args.max_chunk_size) + def main(): """Main program""" @@ -125,6 +192,7 @@ def main(): AbiRest(subparsers) AbiValidate(subparsers) AbiSearch(subparsers) + AbiUndefined(subparsers) args = parser.parse_args() diff --git a/scripts/lib/abi/abi_regex.py b/scripts/lib/abi/abi_regex.py new file mode 100644 index 000000000000..8a57846cbc69 --- /dev/null +++ b/scripts/lib/abi/abi_regex.py @@ -0,0 +1,234 @@ +#!/usr/bin/env python3 +# xxpylint: disable=R0903 +# Copyright(c) 2025: Mauro Carvalho Chehab <mchehab@kernel.org>. +# SPDX-License-Identifier: GPL-2.0 + +""" +Convert ABI what into regular expressions +""" + +import re +import sys + +from pprint import pformat + +from abi_parser import AbiParser +from helpers import AbiDebug + +class AbiRegex(AbiParser): + """Extends AbiParser to search ABI nodes with regular expressions""" + + # Escape only ASCII visible characters + escape_symbols = r"([\x21-\x29\x2b-\x2d\x3a-\x40\x5c\x60\x7b-\x7e])" + leave_others = "others" + + # Tuples with regular expressions to be compiled and replacement data + re_whats = [ + # Drop escape characters that might exist + (re.compile("\\\\"), ""), + + # Temporarily escape dot characters + (re.compile(r"\."), "\xf6"), + + # Temporarily change [0-9]+ type of patterns + (re.compile(r"\[0\-9\]\+"), "\xff"), + + # Temporarily change [\d+-\d+] type of patterns + (re.compile(r"\[0\-\d+\]"), "\xff"), + (re.compile(r"\[0:\d+\]"), "\xff"), + (re.compile(r"\[(\d+)\]"), "\xf4\\\\d+\xf5"), + + # Temporarily change [0-9] type of patterns + (re.compile(r"\[(\d)\-(\d)\]"), "\xf4\1-\2\xf5"), + + # Handle multiple option patterns + (re.compile(r"[\{\<\[]([\w_]+)(?:[,|]+([\w_]+)){1,}[\}\>\]]"), r"(\1|\2)"), + + # Handle wildcards + (re.compile(r"([^\/])\*"), "\\1\\\\w\xf7"), + (re.compile(r"/\*/"), "/.*/"), + (re.compile(r"/\xf6\xf6\xf6"), "/.*"), + (re.compile(r"\<[^\>]+\>"), "\\\\w\xf7"), + (re.compile(r"\{[^\}]+\}"), "\\\\w\xf7"), + (re.compile(r"\[[^\]]+\]"), "\\\\w\xf7"), + + (re.compile(r"XX+"), "\\\\w\xf7"), + (re.compile(r"([^A-Z])[XYZ]([^A-Z])"), "\\1\\\\w\xf7\\2"), + (re.compile(r"([^A-Z])[XYZ]$"), "\\1\\\\w\xf7"), + (re.compile(r"_[AB]_"), "_\\\\w\xf7_"), + + # Recover [0-9] type of patterns + (re.compile(r"\xf4"), "["), + (re.compile(r"\xf5"), "]"), + + # Remove duplicated spaces + (re.compile(r"\s+"), r" "), + + # Special case: drop comparison as in: + # What: foo = <something> + # (this happens on a few IIO definitions) + (re.compile(r"\s*\=.*$"), ""), + + # Escape all other symbols + (re.compile(escape_symbols), r"\\\1"), + (re.compile(r"\\\\"), r"\\"), + (re.compile(r"\\([\[\]\(\)\|])"), r"\1"), + (re.compile(r"(\d+)\\(-\d+)"), r"\1\2"), + + (re.compile(r"\xff"), r"\\d+"), + + # Special case: IIO ABI which a parenthesis. + (re.compile(r"sqrt(.*)"), r"sqrt(.*)"), + + # Simplify regexes with multiple .* + (re.compile(r"(?:\.\*){2,}"), ""), + + # Recover dot characters + (re.compile(r"\xf6"), "\\."), + # Recover plus characters + (re.compile(r"\xf7"), "+"), + ] + re_has_num = re.compile(r"\\d") + + # Symbol name after escape_chars that are considered a devnode basename + re_symbol_name = re.compile(r"(\w|\\[\.\-\:])+$") + + # List of popular group names to be skipped to minimize regex group size + # Use AbiDebug.SUBGROUP_SIZE to detect those + skip_names = set(["devices", "hwmon"]) + + def regex_append(self, what, new): + """ + Get a search group for a subset of regular expressions. + + As ABI may have thousands of symbols, using a for to search all + regular expressions is at least O(n^2). When there are wildcards, + the complexity increases substantially, eventually becoming exponential. + + To avoid spending too much time on them, use a logic to split + them into groups. The smaller the group, the better, as it would + mean that searches will be confined to a small number of regular + expressions. + + The conversion to a regex subset is tricky, as we need something + that can be easily obtained from the sysfs symbol and from the + regular expression. So, we need to discard nodes that have + wildcards. + + If it can't obtain a subgroup, place the regular expression inside + a special group (self.leave_others). + """ + + search_group = None + + for search_group in reversed(new.split("/")): + if not search_group or search_group in self.skip_names: + continue + if self.re_symbol_name.match(search_group): + break + + if not search_group: + search_group = self.leave_others + + if self.debug & AbiDebug.SUBGROUP_MAP: + self.log.debug("%s: mapped as %s", what, search_group) + + try: + if search_group not in self.regex_group: + self.regex_group[search_group] = [] + + self.regex_group[search_group].append(re.compile(new)) + if self.search_string: + if what.find(self.search_string) >= 0: + print(f"What: {what}") + except re.PatternError: + self.log.warning("Ignoring '%s' as it produced an invalid regex:\n" + " '%s'", what, new) + + def get_regexes(self, what): + """ + Given an ABI devnode, return a list of all regular expressions that + may match it, based on the sub-groups created by regex_append() + """ + + re_list = [] + + patches = what.split("/") + patches.reverse() + patches.append(self.leave_others) + + for search_group in patches: + if search_group in self.regex_group: + re_list += self.regex_group[search_group] + + return re_list + + def __init__(self, *args, **kwargs): + """ + Override init method to get verbose argument + """ + + self.regex_group = None + self.search_string = None + self.re_string = None + + if "search_string" in kwargs: + self.search_string = kwargs.get("search_string") + del kwargs["search_string"] + + if self.search_string: + + try: + self.re_string = re.compile(self.search_string) + except re.PatternError as e: + msg = f"{self.search_string} is not a valid regular expression" + raise ValueError(msg) from e + + super().__init__(*args, **kwargs) + + def parse_abi(self, *args, **kwargs): + + super().parse_abi(*args, **kwargs) + + self.regex_group = {} + + print("Converting ABI What fields into regexes...", file=sys.stderr) + + for t in sorted(self.data.items(), key=lambda x: x[0]): + v = t[1] + if v.get("type") == "File": + continue + + v["regex"] = [] + + for what in v.get("what", []): + if not what.startswith("/sys"): + continue + + new = what + for r, s in self.re_whats: + try: + new = r.sub(s, new) + except re.PatternError as e: + # Help debugging troubles with new regexes + raise re.PatternError(f"{e}\nwhile re.sub('{r.pattern}', {s}, str)") from e + + v["regex"].append(new) + + if self.debug & AbiDebug.REGEX: + self.log.debug("%-90s <== %s", new, what) + + # Store regex into a subgroup to speedup searches + self.regex_append(what, new) + + if self.debug & AbiDebug.SUBGROUP_DICT: + self.log.debug("%s", pformat(self.regex_group)) + + if self.debug & AbiDebug.SUBGROUP_SIZE: + biggestd_keys = sorted(self.regex_group.keys(), + key= lambda k: len(self.regex_group[k]), + reverse=True) + + print("Top regex subgroups:", file=sys.stderr) + for k in biggestd_keys[:10]: + print(f"{k} has {len(self.regex_group[k])} elements", file=sys.stderr) diff --git a/scripts/lib/abi/helpers.py b/scripts/lib/abi/helpers.py index 84a253ed5058..639b23e4ca33 100644 --- a/scripts/lib/abi/helpers.py +++ b/scripts/lib/abi/helpers.py @@ -16,13 +16,23 @@ class AbiDebug: WHAT_PARSING = 1 WHAT_OPEN = 2 DUMP_ABI_STRUCTS = 4 + UNDEFINED = 8 + REGEX = 16 + SUBGROUP_MAP = 32 + SUBGROUP_DICT = 64 + SUBGROUP_SIZE = 128 + GRAPH = 256 DEBUG_HELP = """ -Print debug information according with the level(s), -which is given by the following bitmask: - 1 - enable debug parsing logic 2 - enable debug messages on file open 4 - enable debug for ABI parse data +8 - enable extra debug information to identify troubles + with ABI symbols found at the local machine that + weren't found on ABI documentation (used only for + undefined subcommand) +16 - enable debug for what to regex conversion +32 - enable debug for symbol regex subgroups +64 - enable debug for sysfs graph tree variable """ diff --git a/scripts/lib/abi/system_symbols.py b/scripts/lib/abi/system_symbols.py new file mode 100644 index 000000000000..f15c94a6e33c --- /dev/null +++ b/scripts/lib/abi/system_symbols.py @@ -0,0 +1,378 @@ +#!/usr/bin/env python3 +# pylint: disable=R0902,R0912,R0914,R0915,R1702 +# Copyright(c) 2025: Mauro Carvalho Chehab <mchehab@kernel.org>. +# SPDX-License-Identifier: GPL-2.0 + +""" +Parse ABI documentation and produce results from it. +""" + +import os +import re +import sys + +from concurrent import futures +from datetime import datetime +from random import shuffle + +from helpers import AbiDebug + +class SystemSymbols: + """Stores arguments for the class and initialize class vars""" + + def graph_add_file(self, path, link=None): + """ + add a file path to the sysfs graph stored at self.root + """ + + if path in self.files: + return + + name = "" + ref = self.root + for edge in path.split("/"): + name += edge + "/" + if edge not in ref: + ref[edge] = {"__name": [name.rstrip("/")]} + + ref = ref[edge] + + if link and link not in ref["__name"]: + ref["__name"].append(link.rstrip("/")) + + self.files.add(path) + + def print_graph(self, root_prefix="", root=None, level=0): + """Prints a reference tree graph using UTF-8 characters""" + + if not root: + root = self.root + level = 0 + + # Prevent endless traverse + if level > 5: + return + + if level > 0: + prefix = "├──" + last_prefix = "└──" + else: + prefix = "" + last_prefix = "" + + items = list(root.items()) + + names = root.get("__name", []) + for k, edge in items: + if k == "__name": + continue + + if not k: + k = "/" + + if len(names) > 1: + k += " links: " + ",".join(names[1:]) + + if edge == items[-1][1]: + print(root_prefix + last_prefix + k) + p = root_prefix + if level > 0: + p += " " + self.print_graph(p, edge, level + 1) + else: + print(root_prefix + prefix + k) + p = root_prefix + "│ " + self.print_graph(p, edge, level + 1) + + def _walk(self, root): + """ + Walk through sysfs to get all devnodes that aren't ignored. + + By default, uses /sys as sysfs mounting point. If another + directory is used, it replaces them to /sys at the patches. + """ + + with os.scandir(root) as obj: + for entry in obj: + path = os.path.join(root, entry.name) + if self.sysfs: + p = path.replace(self.sysfs, "/sys", count=1) + else: + p = path + + if self.re_ignore.search(p): + return + + # Handle link first to avoid directory recursion + if entry.is_symlink(): + real = os.path.realpath(path) + if not self.sysfs: + self.aliases[path] = real + else: + real = real.replace(self.sysfs, "/sys", count=1) + + # Add absfile location to graph if it doesn't exist + if not self.re_ignore.search(real): + # Add link to the graph + self.graph_add_file(real, p) + + elif entry.is_file(): + self.graph_add_file(p) + + elif entry.is_dir(): + self._walk(path) + + def __init__(self, abi, sysfs="/sys", hints=False): + """ + Initialize internal variables and get a list of all files inside + sysfs that can currently be parsed. + + Please notice that there are several entries on sysfs that aren't + documented as ABI. Ignore those. + + The real paths will be stored under self.files. Aliases will be + stored in separate, as self.aliases. + """ + + self.abi = abi + self.log = abi.log + + if sysfs != "/sys": + self.sysfs = sysfs.rstrip("/") + else: + self.sysfs = None + + self.hints = hints + + self.root = {} + self.aliases = {} + self.files = set() + + dont_walk = [ + # Those require root access and aren't documented at ABI + f"^{sysfs}/kernel/debug", + f"^{sysfs}/kernel/tracing", + f"^{sysfs}/fs/pstore", + f"^{sysfs}/fs/bpf", + f"^{sysfs}/fs/fuse", + + # This is not documented at ABI + f"^{sysfs}/module", + + f"^{sysfs}/fs/cgroup", # this is big and has zero docs under ABI + f"^{sysfs}/firmware", # documented elsewhere: ACPI, DT bindings + "sections|notes", # aren't actually part of ABI + + # kernel-parameters.txt - not easy to parse + "parameters", + ] + + self.re_ignore = re.compile("|".join(dont_walk)) + + print(f"Reading {sysfs} directory contents...", file=sys.stderr) + self._walk(sysfs) + + def check_file(self, refs, found): + """Check missing ABI symbols for a given sysfs file""" + + res_list = [] + + try: + for names in refs: + fname = names[0] + + res = { + "found": False, + "fname": fname, + "msg": "", + } + res_list.append(res) + + re_what = self.abi.get_regexes(fname) + if not re_what: + self.abi.log.warning(f"missing rules for {fname}") + continue + + for name in names: + for r in re_what: + if self.abi.debug & AbiDebug.UNDEFINED: + self.log.debug("check if %s matches '%s'", name, r.pattern) + if r.match(name): + res["found"] = True + if found: + res["msg"] += f" {fname}: regex:\n\t" + continue + + if self.hints and not res["found"]: + res["msg"] += f" {fname} not found. Tested regexes:\n" + for r in re_what: + res["msg"] += " " + r.pattern + "\n" + + except KeyboardInterrupt: + pass + + return res_list + + def _ref_interactor(self, root): + """Recursive function to interact over the sysfs tree""" + + for k, v in root.items(): + if isinstance(v, dict): + yield from self._ref_interactor(v) + + if root == self.root or k == "__name": + continue + + if self.abi.re_string: + fname = v["__name"][0] + if self.abi.re_string.search(fname): + yield v + else: + yield v + + + def get_fileref(self, all_refs, chunk_size): + """Interactor to group refs into chunks""" + + n = 0 + refs = [] + + for ref in all_refs: + refs.append(ref) + + n += 1 + if n >= chunk_size: + yield refs + n = 0 + refs = [] + + yield refs + + def check_undefined_symbols(self, max_workers=None, chunk_size=50, + found=None, dry_run=None): + """Seach ABI for sysfs symbols missing documentation""" + + self.abi.parse_abi() + + if self.abi.debug & AbiDebug.GRAPH: + self.print_graph() + + all_refs = [] + for ref in self._ref_interactor(self.root): + all_refs.append(ref["__name"]) + + if dry_run: + print("Would check", file=sys.stderr) + for ref in all_refs: + print(", ".join(ref)) + + return + + print("Starting to search symbols (it may take several minutes):", + file=sys.stderr) + start = datetime.now() + old_elapsed = None + + # Python doesn't support multithreading due to limitations on its + # global lock (GIL). While Python 3.13 finally made GIL optional, + # there are still issues related to it. Also, we want to have + # backward compatibility with older versions of Python. + # + # So, use instead multiprocess. However, Python is very slow passing + # data from/to multiple processes. Also, it may consume lots of memory + # if the data to be shared is not small. So, we need to group workload + # in chunks that are big enough to generate performance gains while + # not being so big that would cause out-of-memory. + + num_refs = len(all_refs) + print(f"Number of references to parse: {num_refs}", file=sys.stderr) + + if not max_workers: + max_workers = os.cpu_count() + elif max_workers > os.cpu_count(): + max_workers = os.cpu_count() + + max_workers = max(max_workers, 1) + + max_chunk_size = int((num_refs + max_workers - 1) / max_workers) + chunk_size = min(chunk_size, max_chunk_size) + chunk_size = max(1, chunk_size) + + if max_workers > 1: + executor = futures.ProcessPoolExecutor + + # Place references in a random order. This may help improving + # performance, by mixing complex/simple expressions when creating + # chunks + shuffle(all_refs) + else: + # Python has a high overhead with processes. When there's just + # one worker, it is faster to not create a new process. + # Yet, User still deserves to have a progress print. So, use + # python's "thread", which is actually a single process, using + # an internal schedule to switch between tasks. No performance + # gains for non-IO tasks, but still it can be quickly interrupted + # from time to time to display progress. + executor = futures.ThreadPoolExecutor + + not_found = [] + f_list = [] + with executor(max_workers=max_workers) as exe: + for refs in self.get_fileref(all_refs, chunk_size): + if refs: + try: + f_list.append(exe.submit(self.check_file, refs, found)) + + except KeyboardInterrupt: + return + + total = len(f_list) + + if not total: + if self.abi.re_string: + print(f"No ABI symbol matches {self.abi.search_string}") + else: + self.abi.log.warning("No ABI symbols found") + return + + print(f"{len(f_list):6d} jobs queued on {max_workers} workers", + file=sys.stderr) + + while f_list: + try: + t = futures.wait(f_list, timeout=1, + return_when=futures.FIRST_COMPLETED) + + done = t[0] + + for fut in done: + res_list = fut.result() + + for res in res_list: + if not res["found"]: + not_found.append(res["fname"]) + if res["msg"]: + print(res["msg"]) + + f_list.remove(fut) + except KeyboardInterrupt: + return + + except RuntimeError as e: + self.abi.log.warning(f"Future: {e}") + break + + if sys.stderr.isatty(): + elapsed = str(datetime.now() - start).split(".", maxsplit=1)[0] + if len(f_list) < total: + elapsed += f" ({total - len(f_list)}/{total} jobs completed). " + if elapsed != old_elapsed: + print(elapsed + "\r", end="", flush=True, + file=sys.stderr) + old_elapsed = elapsed + + elapsed = str(datetime.now() - start).split(".", maxsplit=1)[0] + print(elapsed, file=sys.stderr) + + for f in sorted(not_found): + print(f"{f} not found.") -- cgit v1.2.3 From 1c7e66bc5d20ac7779130e146d70066b3af4711c Mon Sep 17 00:00:00 2001 From: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Date: Mon, 10 Feb 2025 11:18:16 +0100 Subject: scripts/get_abi.pl: drop now obsoleted script As all functionalities of it were migrated to get_abi.py, drop the now obsoleted script. Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Link: https://lore.kernel.org/r/698ec258b36b63ccde5f7da1af9c97cf8df51050.1739182025.git.mchehab+huawei@kernel.org --- scripts/get_abi.pl | 1103 ---------------------------------------------------- 1 file changed, 1103 deletions(-) delete mode 100755 scripts/get_abi.pl (limited to 'scripts') diff --git a/scripts/get_abi.pl b/scripts/get_abi.pl deleted file mode 100755 index de1c0354b50c..000000000000 --- a/scripts/get_abi.pl +++ /dev/null @@ -1,1103 +0,0 @@ -#!/usr/bin/env perl -# SPDX-License-Identifier: GPL-2.0 - -BEGIN { $Pod::Usage::Formatter = 'Pod::Text::Termcap'; } - -use strict; -use warnings; -use utf8; -use Pod::Usage qw(pod2usage); -use Getopt::Long; -use File::Find; -use IO::Handle; -use Fcntl ':mode'; -use Cwd 'abs_path'; -use Data::Dumper; - -my $help = 0; -my $hint = 0; -my $man = 0; -my $debug = 0; -my $enable_lineno = 0; -my $show_warnings = 1; -my $prefix="Documentation/ABI"; -my $sysfs_prefix="/sys"; -my $search_string; - -# Debug options -my $dbg_what_parsing = 1; -my $dbg_what_open = 2; -my $dbg_dump_abi_structs = 4; -my $dbg_undefined = 8; - -$Data::Dumper::Indent = 1; -$Data::Dumper::Terse = 1; - -# -# If true, assumes that the description is formatted with ReST -# -my $description_is_rst = 1; - -GetOptions( - "debug=i" => \$debug, - "enable-lineno" => \$enable_lineno, - "rst-source!" => \$description_is_rst, - "dir=s" => \$prefix, - 'help|?' => \$help, - "show-hints" => \$hint, - "search-string=s" => \$search_string, - man => \$man -) or pod2usage(2); - -pod2usage(1) if $help; -pod2usage(-exitstatus => 0, -noperldoc, -verbose => 2) if $man; - -pod2usage(2) if (scalar @ARGV < 1 || @ARGV > 2); - -my ($cmd, $arg) = @ARGV; - -pod2usage(2) if ($cmd ne "search" && $cmd ne "rest" && $cmd ne "validate" && $cmd ne "undefined"); -pod2usage(2) if ($cmd eq "search" && !$arg); - -require Data::Dumper if ($debug & $dbg_dump_abi_structs); - -my %data; -my %symbols; - -# -# Displays an error message, printing file name and line -# -sub parse_error($$$$) { - my ($file, $ln, $msg, $data) = @_; - - return if (!$show_warnings); - - $data =~ s/\s+$/\n/; - - print STDERR "Warning: file $file#$ln:\n\t$msg"; - - if ($data ne "") { - print STDERR ". Line\n\t\t$data"; - } else { - print STDERR "\n"; - } -} - -# -# Parse an ABI file, storing its contents at %data -# -sub parse_abi { - my $file = $File::Find::name; - - my $mode = (stat($file))[2]; - return if ($mode & S_IFDIR); - return if ($file =~ m,/README,); - return if ($file =~ m,/\.,); - return if ($file =~ m,\.(rej|org|orig|bak)$,); - - my $name = $file; - $name =~ s,.*/,,; - - my $fn = $file; - $fn =~ s,.*Documentation/ABI/,,; - - my $nametag = "File $fn"; - $data{$nametag}->{what} = "File $name"; - $data{$nametag}->{type} = "File"; - $data{$nametag}->{file} = $name; - $data{$nametag}->{filepath} = $file; - $data{$nametag}->{is_file} = 1; - $data{$nametag}->{line_no} = 1; - - my $type = $file; - $type =~ s,.*/(.*)/.*,$1,; - - my $what; - my $new_what; - my $tag = ""; - my $ln; - my $xrefs; - my $space; - my @labels; - my $label = ""; - - print STDERR "Opening $file\n" if ($debug & $dbg_what_open); - open IN, $file; - while(<IN>) { - $ln++; - if (m/^(\S+)(:\s*)(.*)/i) { - my $new_tag = lc($1); - my $sep = $2; - my $content = $3; - - if (!($new_tag =~ m/(what|where|date|kernelversion|contact|description|users)/)) { - if ($tag eq "description") { - # New "tag" is actually part of - # description. Don't consider it a tag - $new_tag = ""; - } elsif ($tag ne "") { - parse_error($file, $ln, "tag '$tag' is invalid", $_); - } - } - - # Invalid, but it is a common mistake - if ($new_tag eq "where") { - parse_error($file, $ln, "tag 'Where' is invalid. Should be 'What:' instead", ""); - $new_tag = "what"; - } - - if ($new_tag =~ m/what/) { - $space = ""; - $content =~ s/[,.;]$//; - - push @{$symbols{$content}->{file}}, " $file:" . ($ln - 1); - - if ($tag =~ m/what/) { - $what .= "\xac" . $content; - } else { - if ($what) { - parse_error($file, $ln, "What '$what' doesn't have a description", "") if (!$data{$what}->{description}); - - foreach my $w(split /\xac/, $what) { - $symbols{$w}->{xref} = $what; - }; - } - - $what = $content; - $label = $content; - $new_what = 1; - } - push @labels, [($content, $label)]; - $tag = $new_tag; - - push @{$data{$nametag}->{symbols}}, $content if ($data{$nametag}->{what}); - next; - } - - if ($tag ne "" && $new_tag) { - $tag = $new_tag; - - if ($new_what) { - @{$data{$what}->{label_list}} = @labels if ($data{$nametag}->{what}); - @labels = (); - $label = ""; - $new_what = 0; - - $data{$what}->{type} = $type; - if (!defined($data{$what}->{file})) { - $data{$what}->{file} = $name; - $data{$what}->{filepath} = $file; - } else { - $data{$what}->{description} .= "\n\n" if (defined($data{$what}->{description})); - if ($name ne $data{$what}->{file}) { - $data{$what}->{file} .= " " . $name; - $data{$what}->{filepath} .= " " . $file; - } - } - print STDERR "\twhat: $what\n" if ($debug & $dbg_what_parsing); - $data{$what}->{line_no} = $ln; - } else { - $data{$what}->{line_no} = $ln if (!defined($data{$what}->{line_no})); - } - - if (!$what) { - parse_error($file, $ln, "'What:' should come first:", $_); - next; - } - if ($new_tag eq "description") { - $sep =~ s,:, ,; - $content = ' ' x length($new_tag) . $sep . $content; - while ($content =~ s/\t+/' ' x (length($&) * 8 - length($`) % 8)/e) {} - if ($content =~ m/^(\s*)(\S.*)$/) { - # Preserve initial spaces for the first line - $space = $1; - $content = "$2\n"; - $data{$what}->{$tag} .= $content; - } else { - undef($space); - } - - } else { - $data{$what}->{$tag} = $content; - } - next; - } - } - - # Store any contents before tags at the database - if (!$tag && $data{$nametag}->{what}) { - $data{$nametag}->{description} .= $_; - next; - } - - if ($tag eq "description") { - my $content = $_; - while ($content =~ s/\t+/' ' x (length($&) * 8 - length($`) % 8)/e) {} - if (m/^\s*\n/) { - $data{$what}->{$tag} .= "\n"; - next; - } - - if (!defined($space)) { - # Preserve initial spaces for the first line - if ($content =~ m/^(\s*)(\S.*)$/) { - $space = $1; - $content = "$2\n"; - } - } else { - $space = "" if (!($content =~ s/^($space)//)); - } - $data{$what}->{$tag} .= $content; - - next; - } - if (m/^\s*(.*)/) { - $data{$what}->{$tag} .= "\n$1"; - $data{$what}->{$tag} =~ s/\n+$//; - next; - } - - # Everything else is error - parse_error($file, $ln, "Unexpected content", $_); - } - $data{$nametag}->{description} =~ s/^\n+// if ($data{$nametag}->{description}); - if ($what) { - parse_error($file, $ln, "What '$what' doesn't have a description", "") if (!$data{$what}->{description}); - - foreach my $w(split /\xac/,$what) { - $symbols{$w}->{xref} = $what; - }; - } - close IN; -} - -sub create_labels { - my %labels; - - foreach my $what (keys %data) { - next if ($data{$what}->{file} eq "File"); - - foreach my $p (@{$data{$what}->{label_list}}) { - my ($content, $label) = @{$p}; - $label = "abi_" . $label . " "; - $label =~ tr/A-Z/a-z/; - - # Convert special chars to "_" - $label =~s/([\x00-\x2f\x3a-\x40\x5b-\x60\x7b-\xff])/_/g; - $label =~ s,_+,_,g; - $label =~ s,_$,,; - - # Avoid duplicated labels - while (defined($labels{$label})) { - my @chars = ("A".."Z", "a".."z"); - $label .= $chars[rand @chars]; - } - $labels{$label} = 1; - - $data{$what}->{label} = $label; - - # only one label is enough - last; - } - } -} - -# -# Outputs the book on ReST format -# - -# \b doesn't work well with paths. So, we need to define something else: -# Boundaries are punct characters, spaces and end-of-line -my $start = qr {(^|\s|\() }x; -my $bondary = qr { ([,.:;\)\s]|\z) }x; -my $xref_match = qr { $start(\/(sys|config|proc|dev|kvd)\/[^,.:;\)\s]+)$bondary }x; -my $symbols = qr { ([\x01-\x08\x0e-\x1f\x21-\x2f\x3a-\x40\x7b-\xff]) }x; - -sub output_rest { - create_labels(); - - my $part = ""; - - foreach my $what (sort { - ($data{$a}->{type} eq "File") cmp ($data{$b}->{type} eq "File") || - $a cmp $b - } keys %data) { - my $type = $data{$what}->{type}; - - my @file = split / /, $data{$what}->{file}; - my @filepath = split / /, $data{$what}->{filepath}; - - if ($enable_lineno) { - printf ".. LINENO %s%s#%s\n\n", - $prefix, $file[0], - $data{$what}->{line_no}; - } - - my $w = $what; - - if ($type ne "File") { - my $cur_part = $what; - if ($what =~ '/') { - if ($what =~ m#^(\/?(?:[\w\-]+\/?){1,2})#) { - $cur_part = "Symbols under $1"; - $cur_part =~ s,/$,,; - } - } - - if ($cur_part ne "" && $part ne $cur_part) { - $part = $cur_part; - my $bar = $part; - $bar =~ s/./-/g; - print "$part\n$bar\n\n"; - } - - printf ".. _%s:\n\n", $data{$what}->{label}; - - my @names = split /\xac/,$w; - my $len = 0; - - foreach my $name (@names) { - $name =~ s/$symbols/\\$1/g; - $name = "**$name**"; - $len = length($name) if (length($name) > $len); - } - - print "+-" . "-" x $len . "-+\n"; - foreach my $name (@names) { - printf "| %s", $name . " " x ($len - length($name)) . " |\n"; - print "+-" . "-" x $len . "-+\n"; - } - - print "\n"; - } - - for (my $i = 0; $i < scalar(@filepath); $i++) { - my $path = $filepath[$i]; - my $f = $file[$i]; - - $path =~ s,.*/(.*/.*),$1,;; - $path =~ s,[/\-],_,g;; - my $fileref = "abi_file_".$path; - - if ($type eq "File") { - print ".. _$fileref:\n\n"; - } else { - print "Defined on file :ref:`$f <$fileref>`\n\n"; - } - } - - if ($type eq "File") { - my $bar = $w; - $bar =~ s/./-/g; - print "$w\n$bar\n\n"; - } - - my $desc = ""; - $desc = $data{$what}->{description} if (defined($data{$what}->{description})); - $desc =~ s/\s+$/\n/; - - if (!($desc =~ /^\s*$/)) { - if ($description_is_rst) { - # Remove title markups from the description - # Having titles inside ABI files will only work if extra - # care would be taken in order to strictly follow the same - # level order for each markup. - $desc =~ s/\n[\-\*\=\^\~]+\n/\n\n/g; - - # Enrich text by creating cross-references - - my $new_desc = ""; - my $init_indent = -1; - my $literal_indent = -1; - - open(my $fh, "+<", \$desc); - while (my $d = <$fh>) { - my $indent = $d =~ m/^(\s+)/; - my $spaces = length($indent); - $init_indent = $indent if ($init_indent < 0); - if ($literal_indent >= 0) { - if ($spaces > $literal_indent) { - $new_desc .= $d; - next; - } else { - $literal_indent = -1; - } - } else { - if ($d =~ /()::$/ && !($d =~ /^\s*\.\./)) { - $literal_indent = $spaces; - } - } - - $d =~ s,Documentation/(?!devicetree)(\S+)\.rst,:doc:`/$1`,g; - - my @matches = $d =~ m,Documentation/ABI/([\w\/\-]+),g; - foreach my $f (@matches) { - my $xref = $f; - my $path = $f; - $path =~ s,.*/(.*/.*),$1,;; - $path =~ s,[/\-],_,g;; - $xref .= " <abi_file_" . $path . ">"; - $d =~ s,\bDocumentation/ABI/$f\b,:ref:`$xref`,g; - } - - # Seek for cross reference symbols like /sys/... - @matches = $d =~ m/$xref_match/g; - - foreach my $s (@matches) { - next if (!($s =~ m,/,)); - if (defined($data{$s}) && defined($data{$s}->{label})) { - my $xref = $s; - - $xref =~ s/$symbols/\\$1/g; - $xref = ":ref:`$xref <" . $data{$s}->{label} . ">`"; - - $d =~ s,$start$s$bondary,$1$xref$2,g; - } - } - $new_desc .= $d; - } - close $fh; - - - print "$new_desc\n\n"; - } else { - $desc =~ s/^\s+//; - - # Remove title markups from the description, as they won't work - $desc =~ s/\n[\-\*\=\^\~]+\n/\n\n/g; - - if ($desc =~ m/\:\n/ || $desc =~ m/\n[\t ]+/ || $desc =~ m/[\x00-\x08\x0b-\x1f\x7b-\xff]/) { - # put everything inside a code block - $desc =~ s/\n/\n /g; - - print "::\n\n"; - print " $desc\n\n"; - } else { - # Escape any special chars from description - $desc =~s/([\x00-\x08\x0b-\x1f\x21-\x2a\x2d\x2f\x3c-\x40\x5c\x5e-\x60\x7b-\xff])/\\$1/g; - print "$desc\n\n"; - } - } - } else { - print "DESCRIPTION MISSING for $what\n\n" if (!$data{$what}->{is_file}); - } - - if ($data{$what}->{symbols}) { - printf "Has the following ABI:\n\n"; - - foreach my $content(@{$data{$what}->{symbols}}) { - my $label = $data{$symbols{$content}->{xref}}->{label}; - - # Escape special chars from content - $content =~s/([\x00-\x1f\x21-\x2f\x3a-\x40\x7b-\xff])/\\$1/g; - - print "- :ref:`$content <$label>`\n\n"; - } - } - - if (defined($data{$what}->{users})) { - my $users = $data{$what}->{users}; - - $users =~ s/\n/\n\t/g; - printf "Users:\n\t%s\n\n", $users if ($users ne ""); - } - - } -} - -# -# Searches for ABI symbols -# -sub search_symbols { - foreach my $what (sort keys %data) { - next if (!($what =~ m/($arg)/)); - - my $type = $data{$what}->{type}; - next if ($type eq "File"); - - my $file = $data{$what}->{filepath}; - - $what =~ s/\xac/, /g; - my $bar = $what; - $bar =~ s/./-/g; - - print "\n$what\n$bar\n\n"; - - my $kernelversion = $data{$what}->{kernelversion} if (defined($data{$what}->{kernelversion})); - my $contact = $data{$what}->{contact} if (defined($data{$what}->{contact})); - my $users = $data{$what}->{users} if (defined($data{$what}->{users})); - my $date = $data{$what}->{date} if (defined($data{$what}->{date})); - my $desc = $data{$what}->{description} if (defined($data{$what}->{description})); - - $kernelversion =~ s/^\s+// if ($kernelversion); - $contact =~ s/^\s+// if ($contact); - if ($users) { - $users =~ s/^\s+//; - $users =~ s/\n//g; - } - $date =~ s/^\s+// if ($date); - $desc =~ s/^\s+// if ($desc); - - printf "Kernel version:\t\t%s\n", $kernelversion if ($kernelversion); - printf "Date:\t\t\t%s\n", $date if ($date); - printf "Contact:\t\t%s\n", $contact if ($contact); - printf "Users:\t\t\t%s\n", $users if ($users); - print "Defined on file(s):\t$file\n\n"; - print "Description:\n\n$desc"; - } -} - -# Exclude /sys/kernel/debug and /sys/kernel/tracing from the search path -sub dont_parse_special_attributes { - if (($File::Find::dir =~ m,^/sys/kernel,)) { - return grep {!/(debug|tracing)/ } @_; - } - - if (($File::Find::dir =~ m,^/sys/fs,)) { - return grep {!/(pstore|bpf|fuse)/ } @_; - } - - return @_ -} - -my %leaf; -my %aliases; -my @files; -my %root; - -sub graph_add_file { - my $file = shift; - my $type = shift; - - my $dir = $file; - $dir =~ s,^(.*/).*,$1,; - $file =~ s,.*/,,; - - my $name; - my $file_ref = \%root; - foreach my $edge(split "/", $dir) { - $name .= "$edge/"; - if (!defined ${$file_ref}{$edge}) { - ${$file_ref}{$edge} = { }; - } - $file_ref = \%{$$file_ref{$edge}}; - ${$file_ref}{"__name"} = [ $name ]; - } - $name .= "$file"; - ${$file_ref}{$file} = { - "__name" => [ $name ] - }; - - return \%{$$file_ref{$file}}; -} - -sub graph_add_link { - my $file = shift; - my $link = shift; - - # Traverse graph to find the reference - my $file_ref = \%root; - foreach my $edge(split "/", $file) { - $file_ref = \%{$$file_ref{$edge}} || die "Missing node!"; - } - - # do a BFS - - my @queue; - my %seen; - my $st; - - push @queue, $file_ref; - $seen{$start}++; - - while (@queue) { - my $v = shift @queue; - my @child = keys(%{$v}); - - foreach my $c(@child) { - next if $seen{$$v{$c}}; - next if ($c eq "__name"); - - if (!defined($$v{$c}{"__name"})) { - printf STDERR "Error: Couldn't find a non-empty name on a children of $file/.*: "; - print STDERR Dumper(%{$v}); - exit; - } - - # Add new name - my $name = @{$$v{$c}{"__name"}}[0]; - if ($name =~ s#^$file/#$link/#) { - push @{$$v{$c}{"__name"}}, $name; - } - # Add child to the queue and mark as seen - push @queue, $$v{$c}; - $seen{$c}++; - } - } -} - -my $escape_symbols = qr { ([\x01-\x08\x0e-\x1f\x21-\x29\x2b-\x2d\x3a-\x40\x7b-\xfe]) }x; -sub parse_existing_sysfs { - my $file = $File::Find::name; - - my $mode = (lstat($file))[2]; - my $abs_file = abs_path($file); - - my @tmp; - push @tmp, $file; - push @tmp, $abs_file if ($abs_file ne $file); - - foreach my $f(@tmp) { - # Ignore cgroup, as this is big and has zero docs under ABI - return if ($f =~ m#^/sys/fs/cgroup/#); - - # Ignore firmware as it is documented elsewhere - # Either ACPI or under Documentation/devicetree/bindings/ - return if ($f =~ m#^/sys/firmware/#); - - # Ignore some sysfs nodes that aren't actually part of ABI - return if ($f =~ m#/sections|notes/#); - - # Would need to check at - # Documentation/admin-guide/kernel-parameters.txt, but this - # is not easily parseable. - return if ($f =~ m#/parameters/#); - } - - if (S_ISLNK($mode)) { - $aliases{$file} = $abs_file; - return; - } - - return if (S_ISDIR($mode)); - - # Trivial: file is defined exactly the same way at ABI What: - return if (defined($data{$file})); - return if (defined($data{$abs_file})); - - push @files, graph_add_file($abs_file, "file"); -} - -sub get_leave($) -{ - my $what = shift; - my $leave; - - my $l = $what; - my $stop = 1; - - $leave = $l; - $leave =~ s,/$,,; - $leave =~ s,.*/,,; - $leave =~ s/[\(\)]//g; - - # $leave is used to improve search performance at - # check_undefined_symbols, as the algorithm there can seek - # for a small number of "what". It also allows giving a - # hint about a leave with the same name somewhere else. - # However, there are a few occurences where the leave is - # either a wildcard or a number. Just group such cases - # altogether. - if ($leave =~ m/\.\*/ || $leave eq "" || $leave =~ /\\d/) { - $leave = "others"; - } - - return $leave; -} - -my @not_found; - -sub check_file($$) -{ - my $file_ref = shift; - my $names_ref = shift; - my @names = @{$names_ref}; - my $file = $names[0]; - - my $found_string; - - my $leave = get_leave($file); - if (!defined($leaf{$leave})) { - $leave = "others"; - } - my @expr = @{$leaf{$leave}->{expr}}; - die ("\rmissing rules for $leave") if (!defined($leaf{$leave})); - - my $path = $file; - $path =~ s,(.*/).*,$1,; - - if ($search_string) { - return if (!($file =~ m#$search_string#)); - $found_string = 1; - } - - for (my $i = 0; $i < @names; $i++) { - if ($found_string && $hint) { - if (!$i) { - print STDERR "--> $names[$i]\n"; - } else { - print STDERR " $names[$i]\n"; - } - } - foreach my $re (@expr) { - print STDERR "$names[$i] =~ /^$re\$/\n" if ($debug && $dbg_undefined); - if ($names[$i] =~ $re) { - return; - } - } - } - - if ($leave ne "others") { - my @expr = @{$leaf{"others"}->{expr}}; - for (my $i = 0; $i < @names; $i++) { - foreach my $re (@expr) { - print STDERR "$names[$i] =~ /^$re\$/\n" if ($debug && $dbg_undefined); - if ($names[$i] =~ $re) { - return; - } - } - } - } - - push @not_found, $file if (!$search_string || $found_string); - - if ($hint && (!$search_string || $found_string)) { - my $what = $leaf{$leave}->{what}; - $what =~ s/\xac/\n\t/g; - if ($leave ne "others") { - print STDERR "\r more likely regexes:\n\t$what\n"; - } else { - print STDERR "\r tested regexes:\n\t$what\n"; - } - } -} - -sub check_undefined_symbols { - my $num_files = scalar @files; - my $next_i = 0; - my $start_time = times; - - @files = sort @files; - - my $last_time = $start_time; - - # When either debug or hint is enabled, there's no sense showing - # progress, as the progress will be overriden. - if ($hint || ($debug && $dbg_undefined)) { - $next_i = $num_files; - } - - my $is_console; - $is_console = 1 if (-t STDERR); - - for (my $i = 0; $i < $num_files; $i++) { - my $file_ref = $files[$i]; - my @names = @{$$file_ref{"__name"}}; - - check_file($file_ref, \@names); - - my $cur_time = times; - - if ($i == $next_i || $cur_time > $last_time + 1) { - my $percent = $i * 100 / $num_files; - - my $tm = $cur_time - $start_time; - my $time = sprintf "%d:%02d", int($tm), 60 * ($tm - int($tm)); - - printf STDERR "\33[2K\r", if ($is_console); - printf STDERR "%s: processing sysfs files... %i%%: $names[0]", $time, $percent; - printf STDERR "\n", if (!$is_console); - STDERR->flush(); - - $next_i = int (($percent + 1) * $num_files / 100); - $last_time = $cur_time; - } - } - - my $cur_time = times; - my $tm = $cur_time - $start_time; - my $time = sprintf "%d:%02d", int($tm), 60 * ($tm - int($tm)); - - printf STDERR "\33[2K\r", if ($is_console); - printf STDERR "%s: processing sysfs files... done\n", $time; - - foreach my $file (@not_found) { - print "$file not found.\n"; - } -} - -sub undefined_symbols { - print STDERR "Reading $sysfs_prefix directory contents..."; - find({ - wanted =>\&parse_existing_sysfs, - preprocess =>\&dont_parse_special_attributes, - no_chdir => 1 - }, $sysfs_prefix); - print STDERR "done.\n"; - - $leaf{"others"}->{what} = ""; - - print STDERR "Converting ABI What fields into regexes..."; - foreach my $w (sort keys %data) { - foreach my $what (split /\xac/,$w) { - next if (!($what =~ m/^$sysfs_prefix/)); - - # Convert what into regular expressions - - # Escape dot characters - $what =~ s/\./\xf6/g; - - # Temporarily change [0-9]+ type of patterns - $what =~ s/\[0\-9\]\+/\xff/g; - - # Temporarily change [\d+-\d+] type of patterns - $what =~ s/\[0\-\d+\]/\xff/g; - $what =~ s/\[(\d+)\]/\xf4$1\xf5/g; - - # Temporarily change [0-9] type of patterns - $what =~ s/\[(\d)\-(\d)\]/\xf4$1-$2\xf5/g; - - # Handle multiple option patterns - $what =~ s/[\{\<\[]([\w_]+)(?:[,|]+([\w_]+)){1,}[\}\>\]]/($1|$2)/g; - - # Handle wildcards - $what =~ s,\*,.*,g; - $what =~ s,/\xf6..,/.*,g; - $what =~ s/\<[^\>]+\>/.*/g; - $what =~ s/\{[^\}]+\}/.*/g; - $what =~ s/\[[^\]]+\]/.*/g; - - $what =~ s/[XYZ]/.*/g; - - # Recover [0-9] type of patterns - $what =~ s/\xf4/[/g; - $what =~ s/\xf5/]/g; - - # Remove duplicated spaces - $what =~ s/\s+/ /g; - - # Special case: this ABI has a parenthesis on it - $what =~ s/sqrt\(x^2\+y^2\+z^2\)/sqrt\(x^2\+y^2\+z^2\)/; - - # Special case: drop comparition as in: - # What: foo = <something> - # (this happens on a few IIO definitions) - $what =~ s,\s*\=.*$,,; - - # Escape all other symbols - $what =~ s/$escape_symbols/\\$1/g; - $what =~ s/\\\\/\\/g; - $what =~ s/\\([\[\]\(\)\|])/$1/g; - $what =~ s/(\d+)\\(-\d+)/$1$2/g; - - $what =~ s/\xff/\\d+/g; - - # Special case: IIO ABI which a parenthesis. - $what =~ s/sqrt(.*)/sqrt\(.*\)/; - - # Simplify regexes with multiple .* - $what =~ s#(?:\.\*){2,}##g; -# $what =~ s#\.\*/\.\*#.*#g; - - # Recover dot characters - $what =~ s/\xf6/\./g; - - my $leave = get_leave($what); - - my $added = 0; - foreach my $l (split /\|/, $leave) { - if (defined($leaf{$l})) { - next if ($leaf{$l}->{what} =~ m/\b$what\b/); - $leaf{$l}->{what} .= "\xac" . $what; - $added = 1; - } else { - $leaf{$l}->{what} = $what; - $added = 1; - } - } - if ($search_string && $added) { - print STDERR "What: $what\n" if ($what =~ m#$search_string#); - } - - } - } - # Compile regexes - foreach my $l (sort keys %leaf) { - my @expr; - foreach my $w(sort split /\xac/, $leaf{$l}->{what}) { - push @expr, qr /^$w$/; - } - $leaf{$l}->{expr} = \@expr; - } - - # Take links into account - foreach my $link (sort keys %aliases) { - my $abs_file = $aliases{$link}; - graph_add_link($abs_file, $link); - } - print STDERR "done.\n"; - - check_undefined_symbols; -} - -# Ensure that the prefix will always end with a slash -# While this is not needed for find, it makes the patch nicer -# with --enable-lineno -$prefix =~ s,/?$,/,; - -if ($cmd eq "undefined" || $cmd eq "search") { - $show_warnings = 0; -} -# -# Parses all ABI files located at $prefix dir -# -find({wanted =>\&parse_abi, no_chdir => 1}, $prefix); - -print STDERR Data::Dumper->Dump([\%data], [qw(*data)]) if ($debug & $dbg_dump_abi_structs); - -# -# Handles the command -# -if ($cmd eq "undefined") { - undefined_symbols; -} elsif ($cmd eq "search") { - search_symbols; -} else { - if ($cmd eq "rest") { - output_rest; - } - - # Warn about duplicated ABI entries - foreach my $what(sort keys %symbols) { - my @files = @{$symbols{$what}->{file}}; - - next if (scalar(@files) == 1); - - printf STDERR "Warning: $what is defined %d times: @files\n", - scalar(@files); - } -} - -__END__ - -=head1 NAME - -get_abi.pl - parse the Linux ABI files and produce a ReST book. - -=head1 SYNOPSIS - -B<get_abi.pl> [--debug <level>] [--enable-lineno] [--man] [--help] - [--(no-)rst-source] [--dir=<dir>] [--show-hints] - [--search-string <regex>] - <COMMAND> [<ARGUMENT>] - -Where B<COMMAND> can be: - -=over 8 - -B<search> I<SEARCH_REGEX> - search for I<SEARCH_REGEX> inside ABI - -B<rest> - output the ABI in ReST markup language - -B<validate> - validate the ABI contents - -B<undefined> - existing symbols at the system that aren't - defined at Documentation/ABI - -=back - -=head1 OPTIONS - -=over 8 - -=item B<--dir> - -Changes the location of the ABI search. By default, it uses -the Documentation/ABI directory. - -=item B<--rst-source> and B<--no-rst-source> - -The input file may be using ReST syntax or not. Those two options allow -selecting between a rst-compliant source ABI (B<--rst-source>), or a -plain text that may be violating ReST spec, so it requres some escaping -logic (B<--no-rst-source>). - -=item B<--enable-lineno> - -Enable output of .. LINENO lines. - -=item B<--debug> I<debug level> - -Print debug information according with the level, which is given by the -following bitmask: - - - 1: Debug parsing What entries from ABI files; - - 2: Shows what files are opened from ABI files; - - 4: Dump the structs used to store the contents of the ABI files. - -=item B<--show-hints> - -Show hints about possible definitions for the missing ABI symbols. -Used only when B<undefined>. - -=item B<--search-string> I<regex string> - -Show only occurences that match a search string. -Used only when B<undefined>. - -=item B<--help> - -Prints a brief help message and exits. - -=item B<--man> - -Prints the manual page and exits. - -=back - -=head1 DESCRIPTION - -Parse the Linux ABI files from ABI DIR (usually located at Documentation/ABI), -allowing to search for ABI symbols or to produce a ReST book containing -the Linux ABI documentation. - -=head1 EXAMPLES - -Search for all stable symbols with the word "usb": - -=over 8 - -$ scripts/get_abi.pl search usb --dir Documentation/ABI/stable - -=back - -Search for all symbols that match the regex expression "usb.*cap": - -=over 8 - -$ scripts/get_abi.pl search usb.*cap - -=back - -Output all obsoleted symbols in ReST format - -=over 8 - -$ scripts/get_abi.pl rest --dir Documentation/ABI/obsolete - -=back - -=head1 BUGS - -Report bugs to Mauro Carvalho Chehab <mchehab+huawei@kernel.org> - -=head1 COPYRIGHT - -Copyright (c) 2016-2021 by Mauro Carvalho Chehab <mchehab+huawei@kernel.org>. - -License GPLv2: GNU GPL version 2 <http://gnu.org/licenses/gpl.html>. - -This is free software: you are free to change and redistribute it. -There is NO WARRANTY, to the extent permitted by law. - -=cut -- cgit v1.2.3 From c4a16820d90199409c9bf01c4f794e1e9e8d8fd8 Mon Sep 17 00:00:00 2001 From: Christian Brauner <brauner@kernel.org> Date: Tue, 28 Jan 2025 11:33:41 +0100 Subject: fs: add open_tree_attr() Add open_tree_attr() which allow to atomically create a detached mount tree and set mount options on it. If OPEN_TREE_CLONE is used this will allow the creation of a detached mount with a new set of mount options without it ever being exposed to userspace without that set of mount options applied. Link: https://lore.kernel.org/r/20250128-work-mnt_idmap-update-v2-v1-3-c25feb0d2eb3@kernel.org Reviewed-by: "Seth Forshee (DigitalOcean)" <sforshee@kernel.org> Signed-off-by: Christian Brauner <brauner@kernel.org> --- arch/alpha/kernel/syscalls/syscall.tbl | 1 + arch/arm/tools/syscall.tbl | 1 + arch/arm64/tools/syscall_32.tbl | 1 + arch/m68k/kernel/syscalls/syscall.tbl | 1 + arch/microblaze/kernel/syscalls/syscall.tbl | 1 + arch/mips/kernel/syscalls/syscall_n32.tbl | 1 + arch/mips/kernel/syscalls/syscall_n64.tbl | 1 + arch/mips/kernel/syscalls/syscall_o32.tbl | 1 + arch/parisc/kernel/syscalls/syscall.tbl | 1 + arch/powerpc/kernel/syscalls/syscall.tbl | 1 + arch/s390/kernel/syscalls/syscall.tbl | 1 + arch/sh/kernel/syscalls/syscall.tbl | 1 + arch/sparc/kernel/syscalls/syscall.tbl | 1 + arch/x86/entry/syscalls/syscall_32.tbl | 1 + arch/x86/entry/syscalls/syscall_64.tbl | 1 + arch/xtensa/kernel/syscalls/syscall.tbl | 1 + fs/namespace.c | 39 +++++++++++++++++++++++++++++ include/linux/syscalls.h | 4 +++ include/uapi/asm-generic/unistd.h | 4 ++- scripts/syscall.tbl | 1 + 20 files changed, 63 insertions(+), 1 deletion(-) (limited to 'scripts') diff --git a/arch/alpha/kernel/syscalls/syscall.tbl b/arch/alpha/kernel/syscalls/syscall.tbl index c59d53d6d3f3..2dd6340de6b4 100644 --- a/arch/alpha/kernel/syscalls/syscall.tbl +++ b/arch/alpha/kernel/syscalls/syscall.tbl @@ -506,3 +506,4 @@ 574 common getxattrat sys_getxattrat 575 common listxattrat sys_listxattrat 576 common removexattrat sys_removexattrat +577 common open_tree_attr sys_open_tree_attr diff --git a/arch/arm/tools/syscall.tbl b/arch/arm/tools/syscall.tbl index 49eeb2ad8dbd..27c1d5ebcd91 100644 --- a/arch/arm/tools/syscall.tbl +++ b/arch/arm/tools/syscall.tbl @@ -481,3 +481,4 @@ 464 common getxattrat sys_getxattrat 465 common listxattrat sys_listxattrat 466 common removexattrat sys_removexattrat +467 common open_tree_attr sys_open_tree_attr diff --git a/arch/arm64/tools/syscall_32.tbl b/arch/arm64/tools/syscall_32.tbl index 69a829912a05..0765b3a8d6d6 100644 --- a/arch/arm64/tools/syscall_32.tbl +++ b/arch/arm64/tools/syscall_32.tbl @@ -478,3 +478,4 @@ 464 common getxattrat sys_getxattrat 465 common listxattrat sys_listxattrat 466 common removexattrat sys_removexattrat +467 common open_tree_attr sys_open_tree_attr diff --git a/arch/m68k/kernel/syscalls/syscall.tbl b/arch/m68k/kernel/syscalls/syscall.tbl index f5ed71f1910d..9fe47112c586 100644 --- a/arch/m68k/kernel/syscalls/syscall.tbl +++ b/arch/m68k/kernel/syscalls/syscall.tbl @@ -466,3 +466,4 @@ 464 common getxattrat sys_getxattrat 465 common listxattrat sys_listxattrat 466 common removexattrat sys_removexattrat +467 common open_tree_attr sys_open_tree_attr diff --git a/arch/microblaze/kernel/syscalls/syscall.tbl b/arch/microblaze/kernel/syscalls/syscall.tbl index 680f568b77f2..7b6e97828e55 100644 --- a/arch/microblaze/kernel/syscalls/syscall.tbl +++ b/arch/microblaze/kernel/syscalls/syscall.tbl @@ -472,3 +472,4 @@ 464 common getxattrat sys_getxattrat 465 common listxattrat sys_listxattrat 466 common removexattrat sys_removexattrat +467 common open_tree_attr sys_open_tree_attr diff --git a/arch/mips/kernel/syscalls/syscall_n32.tbl b/arch/mips/kernel/syscalls/syscall_n32.tbl index 0b9b7e25b69a..aa70e371bb54 100644 --- a/arch/mips/kernel/syscalls/syscall_n32.tbl +++ b/arch/mips/kernel/syscalls/syscall_n32.tbl @@ -405,3 +405,4 @@ 464 n32 getxattrat sys_getxattrat 465 n32 listxattrat sys_listxattrat 466 n32 removexattrat sys_removexattrat +467 n32 open_tree_attr sys_open_tree_attr diff --git a/arch/mips/kernel/syscalls/syscall_n64.tbl b/arch/mips/kernel/syscalls/syscall_n64.tbl index c844cd5cda62..1e8c44c7b614 100644 --- a/arch/mips/kernel/syscalls/syscall_n64.tbl +++ b/arch/mips/kernel/syscalls/syscall_n64.tbl @@ -381,3 +381,4 @@ 464 n64 getxattrat sys_getxattrat 465 n64 listxattrat sys_listxattrat 466 n64 removexattrat sys_removexattrat +467 n64 open_tree_attr sys_open_tree_attr diff --git a/arch/mips/kernel/syscalls/syscall_o32.tbl b/arch/mips/kernel/syscalls/syscall_o32.tbl index 349b8aad1159..114a5a1a6230 100644 --- a/arch/mips/kernel/syscalls/syscall_o32.tbl +++ b/arch/mips/kernel/syscalls/syscall_o32.tbl @@ -454,3 +454,4 @@ 464 o32 getxattrat sys_getxattrat 465 o32 listxattrat sys_listxattrat 466 o32 removexattrat sys_removexattrat +467 o32 open_tree_attr sys_open_tree_attr diff --git a/arch/parisc/kernel/syscalls/syscall.tbl b/arch/parisc/kernel/syscalls/syscall.tbl index d9fc94c86965..94df3cb957e9 100644 --- a/arch/parisc/kernel/syscalls/syscall.tbl +++ b/arch/parisc/kernel/syscalls/syscall.tbl @@ -465,3 +465,4 @@ 464 common getxattrat sys_getxattrat 465 common listxattrat sys_listxattrat 466 common removexattrat sys_removexattrat +467 common open_tree_attr sys_open_tree_attr diff --git a/arch/powerpc/kernel/syscalls/syscall.tbl b/arch/powerpc/kernel/syscalls/syscall.tbl index d8b4ab78bef0..9a084bdb8926 100644 --- a/arch/powerpc/kernel/syscalls/syscall.tbl +++ b/arch/powerpc/kernel/syscalls/syscall.tbl @@ -557,3 +557,4 @@ 464 common getxattrat sys_getxattrat 465 common listxattrat sys_listxattrat 466 common removexattrat sys_removexattrat +467 common open_tree_attr sys_open_tree_attr diff --git a/arch/s390/kernel/syscalls/syscall.tbl b/arch/s390/kernel/syscalls/syscall.tbl index e9115b4d8b63..a4569b96ef06 100644 --- a/arch/s390/kernel/syscalls/syscall.tbl +++ b/arch/s390/kernel/syscalls/syscall.tbl @@ -469,3 +469,4 @@ 464 common getxattrat sys_getxattrat sys_getxattrat 465 common listxattrat sys_listxattrat sys_listxattrat 466 common removexattrat sys_removexattrat sys_removexattrat +467 common open_tree_attr sys_open_tree_attr sys_open_tree_attr diff --git a/arch/sh/kernel/syscalls/syscall.tbl b/arch/sh/kernel/syscalls/syscall.tbl index c8cad33bf250..52a7652fcff6 100644 --- a/arch/sh/kernel/syscalls/syscall.tbl +++ b/arch/sh/kernel/syscalls/syscall.tbl @@ -470,3 +470,4 @@ 464 common getxattrat sys_getxattrat 465 common listxattrat sys_listxattrat 466 common removexattrat sys_removexattrat +467 common open_tree_attr sys_open_tree_attr diff --git a/arch/sparc/kernel/syscalls/syscall.tbl b/arch/sparc/kernel/syscalls/syscall.tbl index 727f99d333b3..83e45eb6c095 100644 --- a/arch/sparc/kernel/syscalls/syscall.tbl +++ b/arch/sparc/kernel/syscalls/syscall.tbl @@ -512,3 +512,4 @@ 464 common getxattrat sys_getxattrat 465 common listxattrat sys_listxattrat 466 common removexattrat sys_removexattrat +467 common open_tree_attr sys_open_tree_attr diff --git a/arch/x86/entry/syscalls/syscall_32.tbl b/arch/x86/entry/syscalls/syscall_32.tbl index 4d0fb2fba7e2..3f0ec87d5db4 100644 --- a/arch/x86/entry/syscalls/syscall_32.tbl +++ b/arch/x86/entry/syscalls/syscall_32.tbl @@ -472,3 +472,4 @@ 464 i386 getxattrat sys_getxattrat 465 i386 listxattrat sys_listxattrat 466 i386 removexattrat sys_removexattrat +467 i386 open_tree_attr sys_open_tree_attr diff --git a/arch/x86/entry/syscalls/syscall_64.tbl b/arch/x86/entry/syscalls/syscall_64.tbl index 5eb708bff1c7..cfb5ca41e30d 100644 --- a/arch/x86/entry/syscalls/syscall_64.tbl +++ b/arch/x86/entry/syscalls/syscall_64.tbl @@ -390,6 +390,7 @@ 464 common getxattrat sys_getxattrat 465 common listxattrat sys_listxattrat 466 common removexattrat sys_removexattrat +467 common open_tree_attr sys_open_tree_attr # # Due to a historical design error, certain syscalls are numbered differently diff --git a/arch/xtensa/kernel/syscalls/syscall.tbl b/arch/xtensa/kernel/syscalls/syscall.tbl index 37effc1b134e..f657a77314f8 100644 --- a/arch/xtensa/kernel/syscalls/syscall.tbl +++ b/arch/xtensa/kernel/syscalls/syscall.tbl @@ -437,3 +437,4 @@ 464 common getxattrat sys_getxattrat 465 common listxattrat sys_listxattrat 466 common removexattrat sys_removexattrat +467 common open_tree_attr sys_open_tree_attr diff --git a/fs/namespace.c b/fs/namespace.c index d2ef1d69839b..ac4ad746c770 100644 --- a/fs/namespace.c +++ b/fs/namespace.c @@ -4995,6 +4995,45 @@ SYSCALL_DEFINE5(mount_setattr, int, dfd, const char __user *, path, return err; } +SYSCALL_DEFINE5(open_tree_attr, int, dfd, const char __user *, filename, + unsigned, flags, struct mount_attr __user *, uattr, + size_t, usize) +{ + struct file __free(fput) *file = NULL; + int fd; + + if (!uattr && usize) + return -EINVAL; + + file = vfs_open_tree(dfd, filename, flags); + if (IS_ERR(file)) + return PTR_ERR(file); + + if (uattr) { + int ret; + struct mount_kattr kattr = { + .recurse = !!(flags & AT_RECURSIVE), + }; + + ret = copy_mount_setattr(uattr, usize, &kattr); + if (ret) + return ret; + + ret = do_mount_setattr(&file->f_path, &kattr); + if (ret) + return ret; + + finish_mount_kattr(&kattr); + } + + fd = get_unused_fd_flags(flags & O_CLOEXEC); + if (fd < 0) + return fd; + + fd_install(fd, no_free_ptr(file)); + return fd; +} + int show_path(struct seq_file *m, struct dentry *root) { if (root->d_sb->s_op->show_path) diff --git a/include/linux/syscalls.h b/include/linux/syscalls.h index c6333204d451..079ea1d09d85 100644 --- a/include/linux/syscalls.h +++ b/include/linux/syscalls.h @@ -951,6 +951,10 @@ asmlinkage long sys_statx(int dfd, const char __user *path, unsigned flags, asmlinkage long sys_rseq(struct rseq __user *rseq, uint32_t rseq_len, int flags, uint32_t sig); asmlinkage long sys_open_tree(int dfd, const char __user *path, unsigned flags); +asmlinkage long sys_open_tree_attr(int dfd, const char __user *path, + unsigned flags, + struct mount_attr __user *uattr, + size_t usize); asmlinkage long sys_move_mount(int from_dfd, const char __user *from_path, int to_dfd, const char __user *to_path, unsigned int ms_flags); diff --git a/include/uapi/asm-generic/unistd.h b/include/uapi/asm-generic/unistd.h index 88dc393c2bca..2892a45023af 100644 --- a/include/uapi/asm-generic/unistd.h +++ b/include/uapi/asm-generic/unistd.h @@ -849,9 +849,11 @@ __SYSCALL(__NR_getxattrat, sys_getxattrat) __SYSCALL(__NR_listxattrat, sys_listxattrat) #define __NR_removexattrat 466 __SYSCALL(__NR_removexattrat, sys_removexattrat) +#define __NR_open_tree_attr 467 +__SYSCALL(__NR_open_tree_attr, sys_open_tree_attr) #undef __NR_syscalls -#define __NR_syscalls 467 +#define __NR_syscalls 468 /* * 32 bit systems traditionally used different diff --git a/scripts/syscall.tbl b/scripts/syscall.tbl index ebbdb3c42e9f..580b4e246aec 100644 --- a/scripts/syscall.tbl +++ b/scripts/syscall.tbl @@ -407,3 +407,4 @@ 464 common getxattrat sys_getxattrat 465 common listxattrat sys_listxattrat 466 common removexattrat sys_removexattrat +467 common open_tree_attr sys_open_tree_attr -- cgit v1.2.3 From ba561b485709b6160e56d1fe32a2717fffb332cc Mon Sep 17 00:00:00 2001 From: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Date: Wed, 12 Feb 2025 07:02:52 +0100 Subject: scripts/kernel-doc: remove an obscure logic from kernel-doc Kernel-doc has an obscure logic that uses an external file to map files via a .tmp_filelist.txt file stored at the current directory. The rationale for such code predates git time, as it was added on Kernel v2.4.5.5, with the following description: # 26/05/2001 - Support for separate source and object trees. # Return error code. # Keith Owens <kaos@ocs.com.au> from commit 396a6123577d ("v2.4.5.4 -> v2.4.5.5") at the historic tree: https://git.kernel.org/pub/scm/linux/kernel/git/history/history.git/ Support for separate source and object trees is now done on a different way via make O=<object>. There's no logic to create such file, so it sounds to me that this is just dead code. So, drop it. Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Link: https://lore.kernel.org/r/fd3b28dec36ba1668325d6770d4c4754414337fc.1739340170.git.mchehab+huawei@kernel.org Signed-off-by: Jonathan Corbet <corbet@lwn.net> --- scripts/kernel-doc | 19 +------------------ 1 file changed, 1 insertion(+), 18 deletions(-) (limited to 'scripts') diff --git a/scripts/kernel-doc b/scripts/kernel-doc index e57c5e989a0a..70da9a3369c6 100755 --- a/scripts/kernel-doc +++ b/scripts/kernel-doc @@ -179,7 +179,7 @@ my ($function, %function_table, %parametertypes, $declaration_purpose); my %nosymbol_table = (); my $declaration_start_line; my ($type, $declaration_name, $return_type); -my ($newsection, $newcontents, $prototype, $brcount, %source_map); +my ($newsection, $newcontents, $prototype, $brcount); if (defined($ENV{'KBUILD_VERBOSE'}) && $ENV{'KBUILD_VERBOSE'} =~ '1') { $verbose = 1; @@ -2005,10 +2005,6 @@ sub map_filename($) { $file = $orig_file; } - if (defined($source_map{$file})) { - $file = $source_map{$file}; - } - return $file; } @@ -2403,19 +2399,6 @@ for (my $k = 0; $k < @highlights; $k++) { $dohighlight .= "\$contents =~ s:$pattern:$result:gs;\n"; } -# Read the file that maps relative names to absolute names for -# separate source and object directories and for shadow trees. -if (open(SOURCE_MAP, "<.tmp_filelist.txt")) { - my ($relname, $absname); - while(<SOURCE_MAP>) { - chop(); - ($relname, $absname) = (split())[0..1]; - $relname =~ s:^/+::; - $source_map{$relname} = $absname; - } - close(SOURCE_MAP); -} - if ($output_selection == OUTPUT_EXPORTED || $output_selection == OUTPUT_INTERNAL) { -- cgit v1.2.3 From de61d6515baece4610e401d9d7c18cac6bd77198 Mon Sep 17 00:00:00 2001 From: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Date: Tue, 11 Feb 2025 06:57:57 +0100 Subject: docs: ABI: move README contents to the top The ABI documentation looks a little bit better if it starts with the contents of the README is placed at the beginning. Move it to the beginning of the ABI chapter. While here, improve the README text and change the title that will be shown at the html/pdf output to be coherent with both ABI file contents and with the generated documentation output. Suggested-by: Jonathan Corbet <corbet@lwn.net> Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Link: https://lore.kernel.org/r/20250211055809.1898623-1-mchehab+huawei@kernel.org Signed-off-by: Jonathan Corbet <corbet@lwn.net> --- Documentation/ABI/README | 3 ++- Documentation/admin-guide/abi-readme-file.rst | 6 ------ Documentation/admin-guide/abi.rst | 3 ++- scripts/lib/abi/abi_parser.py | 2 +- 4 files changed, 5 insertions(+), 9 deletions(-) delete mode 100644 Documentation/admin-guide/abi-readme-file.rst (limited to 'scripts') diff --git a/Documentation/ABI/README b/Documentation/ABI/README index 8bac9cb09a6d..ef0e6d11e919 100644 --- a/Documentation/ABI/README +++ b/Documentation/ABI/README @@ -1,4 +1,5 @@ -This directory attempts to document the ABI between the Linux kernel and +This part of the documentation inside Documentation/ABI directory +attempts to document the ABI between the Linux kernel and userspace, and the relative stability of these interfaces. Due to the everchanging nature of Linux, and the differing maturity levels, these interfaces should be used by userspace programs in different ways. diff --git a/Documentation/admin-guide/abi-readme-file.rst b/Documentation/admin-guide/abi-readme-file.rst deleted file mode 100644 index 6172e4ccbda2..000000000000 --- a/Documentation/admin-guide/abi-readme-file.rst +++ /dev/null @@ -1,6 +0,0 @@ -.. SPDX-License-Identifier: GPL-2.0 - -ABI README -========== - -.. kernel-abi:: README diff --git a/Documentation/admin-guide/abi.rst b/Documentation/admin-guide/abi.rst index 15a2dcb1388c..c6039359e585 100644 --- a/Documentation/admin-guide/abi.rst +++ b/Documentation/admin-guide/abi.rst @@ -4,6 +4,8 @@ Linux ABI description ===================== +.. kernel-abi:: README + ABI symbols ----------- @@ -21,7 +23,6 @@ ABI files .. toctree:: :maxdepth: 2 - abi-readme-file abi-stable-files abi-testing-files abi-obsolete-files diff --git a/scripts/lib/abi/abi_parser.py b/scripts/lib/abi/abi_parser.py index f08de6d3bf7c..66a738013ce1 100644 --- a/scripts/lib/abi/abi_parser.py +++ b/scripts/lib/abi/abi_parser.py @@ -266,7 +266,7 @@ class AbiParser: def parse_readme(self, nametag, fname): """Parse ABI README file""" - nametag["what"] = ["ABI file contents"] + nametag["what"] = ["Introduction"] nametag["path"] = "README" with open(fname, "r", encoding="utf8", errors="backslashreplace") as fp: for line in fp: -- cgit v1.2.3 From 089e06c3f113a8641a6cf502a34284a7c0ca1630 Mon Sep 17 00:00:00 2001 From: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Date: Tue, 11 Feb 2025 07:19:04 +0100 Subject: scripts/kernel-doc: drop Sphinx version check As the current minimal supported Sphinx version is 3.4.3, drop support for older versions. Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Reviewed-by: Kees Cook <kees@kernel.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Link: https://lore.kernel.org/r/0d002e7550476a68547ee53ad06cfd8fdcaf7c3a.1739254187.git.mchehab+huawei@kernel.org --- Documentation/sphinx/cdomain.py | 3 - Documentation/sphinx/kerneldoc.py | 5 -- scripts/kernel-doc | 129 +++++--------------------------------- 3 files changed, 16 insertions(+), 121 deletions(-) (limited to 'scripts') diff --git a/Documentation/sphinx/cdomain.py b/Documentation/sphinx/cdomain.py index 6596fd00663f..e8ea80d4324c 100644 --- a/Documentation/sphinx/cdomain.py +++ b/Documentation/sphinx/cdomain.py @@ -45,9 +45,6 @@ import re __version__ = '1.1' -# Get Sphinx version -major, minor, patch = sphinx.version_info[:3] - # Namespace to be prepended to the full name namespace = None diff --git a/Documentation/sphinx/kerneldoc.py b/Documentation/sphinx/kerneldoc.py index be5b8fbf373f..39ddae6ae7dd 100644 --- a/Documentation/sphinx/kerneldoc.py +++ b/Documentation/sphinx/kerneldoc.py @@ -62,11 +62,6 @@ class KernelDocDirective(Directive): env = self.state.document.settings.env cmd = [env.config.kerneldoc_bin, '-rst', '-enable-lineno'] - # Pass the version string to kernel-doc, as it needs to use a different - # dialect, depending what the C domain supports for each specific - # Sphinx versions - cmd += ['-sphinx-version', sphinx.__version__] - filename = env.config.kerneldoc_srctree + '/' + self.arguments[0] export_file_patterns = [] diff --git a/scripts/kernel-doc b/scripts/kernel-doc index 70da9a3369c6..2c77b914d017 100755 --- a/scripts/kernel-doc +++ b/scripts/kernel-doc @@ -26,7 +26,7 @@ kernel-doc - Print formatted kernel documentation to stdout kernel-doc [-h] [-v] [-Werror] [-Wall] [-Wreturn] [-Wshort-desc[ription]] [-Wcontents-before-sections] [ -man | - -rst [-sphinx-version VERSION] [-enable-lineno] | + -rst [-enable-lineno] | -none ] [ @@ -130,7 +130,6 @@ if ($#ARGV == -1) { } my $kernelversion; -my ($sphinx_major, $sphinx_minor, $sphinx_patch); my $dohighlight = ""; @@ -347,23 +346,6 @@ while ($ARGV[0] =~ m/^--?(.*)/) { $enable_lineno = 1; } elsif ($cmd eq 'show-not-found') { $show_not_found = 1; # A no-op but don't fail - } elsif ($cmd eq "sphinx-version") { - my $ver_string = shift @ARGV; - if ($ver_string =~ m/^(\d+)(\.\d+)?(\.\d+)?/) { - $sphinx_major = $1; - if (defined($2)) { - $sphinx_minor = substr($2,1); - } else { - $sphinx_minor = 0; - } - if (defined($3)) { - $sphinx_patch = substr($3,1) - } else { - $sphinx_patch = 0; - } - } else { - die "Sphinx version should either major.minor or major.minor.patch format\n"; - } } else { # Unknown argument pod2usage( @@ -387,8 +369,6 @@ while ($ARGV[0] =~ m/^--?(.*)/) { # continue execution near EOF; -# The C domain dialect changed on Sphinx 3. So, we need to check the -# version in order to produce the right tags. sub findprog($) { foreach(split(/:/, $ENV{PATH})) { @@ -396,42 +376,6 @@ sub findprog($) } } -sub get_sphinx_version() -{ - my $ver; - - my $cmd = "sphinx-build"; - if (!findprog($cmd)) { - my $cmd = "sphinx-build3"; - if (!findprog($cmd)) { - $sphinx_major = 1; - $sphinx_minor = 2; - $sphinx_patch = 0; - printf STDERR "Warning: Sphinx version not found. Using default (Sphinx version %d.%d.%d)\n", - $sphinx_major, $sphinx_minor, $sphinx_patch; - return; - } - } - - open IN, "$cmd --version 2>&1 |"; - while (<IN>) { - if (m/^\s*sphinx-build\s+([\d]+)\.([\d\.]+)(\+\/[\da-f]+)?$/) { - $sphinx_major = $1; - $sphinx_minor = $2; - $sphinx_patch = $3; - last; - } - # Sphinx 1.2.x uses a different format - if (m/^\s*Sphinx.*\s+([\d]+)\.([\d\.]+)$/) { - $sphinx_major = $1; - $sphinx_minor = $2; - $sphinx_patch = $3; - last; - } - } - close IN; -} - # get kernel version from env sub get_kernel_version() { my $version = 'unknown kernel version'; @@ -859,9 +803,10 @@ sub output_function_rst(%) { $signature .= ")"; } - if ($sphinx_major < 3) { + if ($args{'typedef'} || $args{'functiontype'} eq "") { + print ".. c:macro:: ". $args{'function'} . "\n\n"; + if ($args{'typedef'}) { - print ".. c:type:: ". $args{'function'} . "\n\n"; print_lineno($declaration_start_line); print " **Typedef**: "; $lineprefix = ""; @@ -869,25 +814,10 @@ sub output_function_rst(%) { print "\n\n**Syntax**\n\n"; print " ``$signature``\n\n"; } else { - print ".. c:function:: $signature\n\n"; + print "``$signature``\n\n"; } } else { - if ($args{'typedef'} || $args{'functiontype'} eq "") { - print ".. c:macro:: ". $args{'function'} . "\n\n"; - - if ($args{'typedef'}) { - print_lineno($declaration_start_line); - print " **Typedef**: "; - $lineprefix = ""; - output_highlight_rst($args{'purpose'}); - print "\n\n**Syntax**\n\n"; - print " ``$signature``\n\n"; - } else { - print "``$signature``\n\n"; - } - } else { - print ".. c:function:: $signature\n\n"; - } + print ".. c:function:: $signature\n\n"; } if (!$args{'typedef'}) { @@ -955,13 +885,9 @@ sub output_enum_rst(%) { my $count; my $outer; - if ($sphinx_major < 3) { - my $name = "enum " . $args{'enum'}; - print "\n\n.. c:type:: " . $name . "\n\n"; - } else { - my $name = $args{'enum'}; - print "\n\n.. c:enum:: " . $name . "\n\n"; - } + my $name = $args{'enum'}; + print "\n\n.. c:enum:: " . $name . "\n\n"; + print_lineno($declaration_start_line); $lineprefix = " "; output_highlight_rst($args{'purpose'}); @@ -992,11 +918,8 @@ sub output_typedef_rst(%) { my $oldprefix = $lineprefix; my $name; - if ($sphinx_major < 3) { - $name = "typedef " . $args{'typedef'}; - } else { - $name = $args{'typedef'}; - } + $name = $args{'typedef'}; + print "\n\n.. c:type:: " . $name . "\n\n"; print_lineno($declaration_start_line); $lineprefix = " "; @@ -1012,17 +935,13 @@ sub output_struct_rst(%) { my ($parameter); my $oldprefix = $lineprefix; - if ($sphinx_major < 3) { - my $name = $args{'type'} . " " . $args{'struct'}; - print "\n\n.. c:type:: " . $name . "\n\n"; + my $name = $args{'struct'}; + if ($args{'type'} eq 'union') { + print "\n\n.. c:union:: " . $name . "\n\n"; } else { - my $name = $args{'struct'}; - if ($args{'type'} eq 'union') { - print "\n\n.. c:union:: " . $name . "\n\n"; - } else { - print "\n\n.. c:struct:: " . $name . "\n\n"; - } + print "\n\n.. c:struct:: " . $name . "\n\n"; } + print_lineno($declaration_start_line); $lineprefix = " "; output_highlight_rst($args{'purpose'}); @@ -2383,11 +2302,6 @@ sub process_file($) { close IN_FILE; } - -if ($output_mode eq "rst") { - get_sphinx_version() if (!$sphinx_major); -} - $kernelversion = get_kernel_version(); # generate a sequence of code that will splice in highlighting information @@ -2454,17 +2368,6 @@ Do not output documentation, only warnings. =head3 reStructuredText only -=over 8 - -=item -sphinx-version VERSION - -Use the ReST C domain dialect compatible with a specific Sphinx Version. - -If not specified, kernel-doc will auto-detect using the sphinx-build version -found on PATH. - -=back - =head2 Output selection (mutually exclusive): =over 8 -- cgit v1.2.3 From 140332b6ed727a4ec2e5722a1ccda28b52d45771 Mon Sep 17 00:00:00 2001 From: Masahiro Yamada <masahiroy@kernel.org> Date: Thu, 13 Feb 2025 15:26:44 +0900 Subject: kbuild: fix linux-headers package build when $(CC) cannot link userspace Since commit 5f73e7d0386d ("kbuild: refactor cross-compiling linux-headers package"), the linux-headers Debian package fails to build when $(CC) cannot build userspace applications, for example, when using toolchains installed by the 0day bot. The host programs in the linux-headers package should be rebuilt using the disto's cross-compiler, ${DEB_HOST_GNU_TYPE}-gcc instead of $(CC). Hence, the variable 'CC' must be expanded in this shell script instead of in the top-level Makefile. Commit f354fc88a72a ("kbuild: install-extmod-build: add missing quotation marks for CC variable") was not a correct fix because CC="ccache gcc" should be unrelated when rebuilding userspace tools. Fixes: 5f73e7d0386d ("kbuild: refactor cross-compiling linux-headers package") Reported-by: Jeff Johnson <jeff.johnson@oss.qualcomm.com> Closes: https://lore.kernel.org/linux-kbuild/CAK7LNARb3xO3ptBWOMpwKcyf3=zkfhMey5H2KnB1dOmUwM79dA@mail.gmail.com/T/#t Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Tested-by: Jeff Johnson <jeff.johnson@oss.qualcomm.com> --- scripts/package/install-extmod-build | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) (limited to 'scripts') diff --git a/scripts/package/install-extmod-build b/scripts/package/install-extmod-build index b724626ea0ca..2966473b4660 100755 --- a/scripts/package/install-extmod-build +++ b/scripts/package/install-extmod-build @@ -62,8 +62,8 @@ if [ "${CC}" != "${HOSTCC}" ]; then # # Clear VPATH and srcroot because the source files reside in the output # directory. - # shellcheck disable=SC2016 # $(MAKE), $(CC), and $(build) will be expanded by Make - "${MAKE}" run-command KBUILD_RUN_COMMAND='+$(MAKE) HOSTCC="$(CC)" VPATH= srcroot=. $(build)='"${destdir}"/scripts + # shellcheck disable=SC2016 # $(MAKE) and $(build) will be expanded by Make + "${MAKE}" run-command KBUILD_RUN_COMMAND='+$(MAKE) HOSTCC='"${CC}"' VPATH= srcroot=. $(build)='"${destdir}"/scripts rm -f "${destdir}/scripts/Kbuild" fi -- cgit v1.2.3 From b28fb1f2ef45eeef1cd2c23149b50d184d545a3e Mon Sep 17 00:00:00 2001 From: Uwe Kleine-König <u.kleine-koenig@baylibre.com> Date: Thu, 13 Feb 2025 17:04:29 +0100 Subject: modpost: Fix a few typos in a comment MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Namely: s/becasue/because/ and s/wiht/with/ plus an added article. Signed-off-by: Uwe Kleine-König <u.kleine-koenig@baylibre.com> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> --- scripts/mod/modpost.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) (limited to 'scripts') diff --git a/scripts/mod/modpost.c b/scripts/mod/modpost.c index 36b28987a2f0..c35d22607978 100644 --- a/scripts/mod/modpost.c +++ b/scripts/mod/modpost.c @@ -190,8 +190,8 @@ static struct module *new_module(const char *name, size_t namelen) /* * Set mod->is_gpl_compatible to true by default. If MODULE_LICENSE() - * is missing, do not check the use for EXPORT_SYMBOL_GPL() becasue - * modpost will exit wiht error anyway. + * is missing, do not check the use for EXPORT_SYMBOL_GPL() because + * modpost will exit with an error anyway. */ mod->is_gpl_compatible = true; -- cgit v1.2.3 From a3e8fe814ad15c16735cdf394454a8bd96eb4d56 Mon Sep 17 00:00:00 2001 From: Brian Gerst <brgerst@gmail.com> Date: Thu, 23 Jan 2025 14:07:33 -0500 Subject: x86/build: Raise the minimum GCC version to 8.1 Stack protector support on 64-bit currently requires that the percpu section is linked at absolute address 0, because older compilers fixed the location of the canary value relative to the GS segment base. GCC 8.1 introduced options to change where the canary value is located, allowing it to be configured as a standard per-CPU variable. This has already been done for 32-bit. Doing the same for 64-bit will enable removing the code needed to support zero-based percpu. Signed-off-by: Brian Gerst <brgerst@gmail.com> Signed-off-by: Ingo Molnar <mingo@kernel.org> Reviewed-by: Ard Biesheuvel <ardb@kernel.org> Cc: Linus Torvalds <torvalds@linux-foundation.org> Link: https://lore.kernel.org/r/20250123190747.745588-2-brgerst@gmail.com --- scripts/min-tool-version.sh | 2 ++ 1 file changed, 2 insertions(+) (limited to 'scripts') diff --git a/scripts/min-tool-version.sh b/scripts/min-tool-version.sh index 91c91201212c..06c4e410ecab 100755 --- a/scripts/min-tool-version.sh +++ b/scripts/min-tool-version.sh @@ -19,6 +19,8 @@ binutils) gcc) if [ "$ARCH" = parisc64 ]; then echo 12.0.0 + elif [ "$SRCARCH" = x86 ]; then + echo 8.1.0 else echo 5.1.0 fi -- cgit v1.2.3 From 0ee2689b9374d6fd5f43b703713a532278654749 Mon Sep 17 00:00:00 2001 From: Brian Gerst <brgerst@gmail.com> Date: Thu, 23 Jan 2025 14:07:34 -0500 Subject: x86/stackprotector: Remove stack protector test scripts With GCC 8.1 now the minimum supported compiler for x86, these scripts are no longer needed. Signed-off-by: Brian Gerst <brgerst@gmail.com> Signed-off-by: Ingo Molnar <mingo@kernel.org> Reviewed-by: Ard Biesheuvel <ardb@kernel.org> Reviewed-by: Uros Bizjak <ubizjak@gmail.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Link: https://lore.kernel.org/r/20250123190747.745588-3-brgerst@gmail.com --- arch/x86/Kconfig | 11 +---------- scripts/gcc-x86_32-has-stack-protector.sh | 8 -------- scripts/gcc-x86_64-has-stack-protector.sh | 4 ---- 3 files changed, 1 insertion(+), 22 deletions(-) delete mode 100755 scripts/gcc-x86_32-has-stack-protector.sh delete mode 100755 scripts/gcc-x86_64-has-stack-protector.sh (limited to 'scripts') diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index be2c311f5118..6595b35dd52d 100644 --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -285,7 +285,7 @@ config X86 select HAVE_FUNCTION_ARG_ACCESS_API select HAVE_SETUP_PER_CPU_AREA select HAVE_SOFTIRQ_ON_OWN_STACK - select HAVE_STACKPROTECTOR if CC_HAS_SANE_STACKPROTECTOR + select HAVE_STACKPROTECTOR select HAVE_STACK_VALIDATION if HAVE_OBJTOOL select HAVE_STATIC_CALL select HAVE_STATIC_CALL_INLINE if HAVE_OBJTOOL @@ -426,15 +426,6 @@ config PGTABLE_LEVELS default 3 if X86_PAE default 2 -config CC_HAS_SANE_STACKPROTECTOR - bool - default $(success,$(srctree)/scripts/gcc-x86_64-has-stack-protector.sh $(CC) $(CLANG_FLAGS)) if 64BIT - default $(success,$(srctree)/scripts/gcc-x86_32-has-stack-protector.sh $(CC) $(CLANG_FLAGS)) - help - We have to make sure stack protector is unconditionally disabled if - the compiler produces broken code or if it does not let us control - the segment on 32-bit kernels. - menu "Processor type and features" config SMP diff --git a/scripts/gcc-x86_32-has-stack-protector.sh b/scripts/gcc-x86_32-has-stack-protector.sh deleted file mode 100755 index 9459ca4f0f11..000000000000 --- a/scripts/gcc-x86_32-has-stack-protector.sh +++ /dev/null @@ -1,8 +0,0 @@ -#!/bin/sh -# SPDX-License-Identifier: GPL-2.0 - -# This requires GCC 8.1 or better. Specifically, we require -# -mstack-protector-guard-reg, added by -# https://gcc.gnu.org/bugzilla/show_bug.cgi?id=81708 - -echo "int foo(void) { char X[200]; return 3; }" | $* -S -x c -m32 -O0 -fstack-protector -mstack-protector-guard-reg=fs -mstack-protector-guard-symbol=__stack_chk_guard - -o - 2> /dev/null | grep -q "%fs" diff --git a/scripts/gcc-x86_64-has-stack-protector.sh b/scripts/gcc-x86_64-has-stack-protector.sh deleted file mode 100755 index f680bb01aeeb..000000000000 --- a/scripts/gcc-x86_64-has-stack-protector.sh +++ /dev/null @@ -1,4 +0,0 @@ -#!/bin/sh -# SPDX-License-Identifier: GPL-2.0 - -echo "int foo(void) { char X[200]; return 3; }" | $* -S -x c -m64 -O0 -mcmodel=kernel -fno-PIE -fstack-protector - -o - 2> /dev/null | grep -q "%gs" -- cgit v1.2.3 From 01157ddc58dc2fe428ec17dd5a18cc13f134639f Mon Sep 17 00:00:00 2001 From: Brian Gerst <brgerst@gmail.com> Date: Thu, 23 Jan 2025 14:07:47 -0500 Subject: kallsyms: Remove KALLSYMS_ABSOLUTE_PERCPU x86-64 was the only user. Signed-off-by: Brian Gerst <brgerst@gmail.com> Signed-off-by: Ingo Molnar <mingo@kernel.org> Reviewed-by: Ard Biesheuvel <ardb@kernel.org> Cc: Linus Torvalds <torvalds@linux-foundation.org> Link: https://lore.kernel.org/r/20250123190747.745588-16-brgerst@gmail.com --- init/Kconfig | 5 ---- kernel/kallsyms.c | 12 ++------- scripts/kallsyms.c | 72 +++++++++---------------------------------------- scripts/link-vmlinux.sh | 4 --- 4 files changed, 14 insertions(+), 79 deletions(-) (limited to 'scripts') diff --git a/init/Kconfig b/init/Kconfig index b5d9c0fa69f6..a0ea04c17784 100644 --- a/init/Kconfig +++ b/init/Kconfig @@ -1869,11 +1869,6 @@ config KALLSYMS_ALL Say N unless you really need all symbols, or kernel live patching. -config KALLSYMS_ABSOLUTE_PERCPU - bool - depends on KALLSYMS - default n - # end of the "standard kernel features (expert users)" menu config ARCH_HAS_MEMBARRIER_CALLBACKS diff --git a/kernel/kallsyms.c b/kernel/kallsyms.c index a9a0ca605d4a..4198f30aac3c 100644 --- a/kernel/kallsyms.c +++ b/kernel/kallsyms.c @@ -148,16 +148,8 @@ static unsigned int get_symbol_offset(unsigned long pos) unsigned long kallsyms_sym_address(int idx) { - /* values are unsigned offsets if --absolute-percpu is not in effect */ - if (!IS_ENABLED(CONFIG_KALLSYMS_ABSOLUTE_PERCPU)) - return kallsyms_relative_base + (u32)kallsyms_offsets[idx]; - - /* ...otherwise, positive offsets are absolute values */ - if (kallsyms_offsets[idx] >= 0) - return kallsyms_offsets[idx]; - - /* ...and negative offsets are relative to kallsyms_relative_base - 1 */ - return kallsyms_relative_base - 1 - kallsyms_offsets[idx]; + /* values are unsigned offsets */ + return kallsyms_relative_base + (u32)kallsyms_offsets[idx]; } static unsigned int get_symbol_seq(int index) diff --git a/scripts/kallsyms.c b/scripts/kallsyms.c index 03852da3d249..4b0234e4b12f 100644 --- a/scripts/kallsyms.c +++ b/scripts/kallsyms.c @@ -5,7 +5,7 @@ * This software may be used and distributed according to the terms * of the GNU General Public License, incorporated herein by reference. * - * Usage: kallsyms [--all-symbols] [--absolute-percpu] in.map > out.S + * Usage: kallsyms [--all-symbols] in.map > out.S * * Table compression uses all the unused char codes on the symbols and * maps these to the most used substrings (tokens). For instance, it might @@ -37,7 +37,6 @@ struct sym_entry { unsigned long long addr; unsigned int len; unsigned int seq; - bool percpu_absolute; unsigned char sym[]; }; @@ -55,14 +54,9 @@ static struct addr_range text_ranges[] = { #define text_range_text (&text_ranges[0]) #define text_range_inittext (&text_ranges[1]) -static struct addr_range percpu_range = { - "__per_cpu_start", "__per_cpu_end", -1ULL, 0 -}; - static struct sym_entry **table; static unsigned int table_size, table_cnt; static int all_symbols; -static int absolute_percpu; static int token_profit[0x10000]; @@ -73,7 +67,7 @@ static unsigned char best_table_len[256]; static void usage(void) { - fprintf(stderr, "Usage: kallsyms [--all-symbols] [--absolute-percpu] in.map > out.S\n"); + fprintf(stderr, "Usage: kallsyms [--all-symbols] in.map > out.S\n"); exit(1); } @@ -164,7 +158,6 @@ static struct sym_entry *read_symbol(FILE *in, char **buf, size_t *buf_len) return NULL; check_symbol_range(name, addr, text_ranges, ARRAY_SIZE(text_ranges)); - check_symbol_range(name, addr, &percpu_range, 1); /* include the type field in the symbol name, so that it gets * compressed together */ @@ -175,7 +168,6 @@ static struct sym_entry *read_symbol(FILE *in, char **buf, size_t *buf_len) sym->len = len; sym->sym[0] = type; strcpy(sym_name(sym), name); - sym->percpu_absolute = false; return sym; } @@ -319,11 +311,6 @@ static int expand_symbol(const unsigned char *data, int len, char *result) return total; } -static bool symbol_absolute(const struct sym_entry *s) -{ - return s->percpu_absolute; -} - static int compare_names(const void *a, const void *b) { int ret; @@ -455,22 +442,11 @@ static void write_src(void) */ long long offset; - bool overflow; - - if (!absolute_percpu) { - offset = table[i]->addr - relative_base; - overflow = offset < 0 || offset > UINT_MAX; - } else if (symbol_absolute(table[i])) { - offset = table[i]->addr; - overflow = offset < 0 || offset > INT_MAX; - } else { - offset = relative_base - table[i]->addr - 1; - overflow = offset < INT_MIN || offset >= 0; - } - if (overflow) { + + offset = table[i]->addr - relative_base; + if (offset < 0 || offset > UINT_MAX) { fprintf(stderr, "kallsyms failure: " - "%s symbol value %#llx out of range in relative mode\n", - symbol_absolute(table[i]) ? "absolute" : "relative", + "relative symbol value %#llx out of range\n", table[i]->addr); exit(EXIT_FAILURE); } @@ -725,36 +701,15 @@ static void sort_symbols(void) qsort(table, table_cnt, sizeof(table[0]), compare_symbols); } -static void make_percpus_absolute(void) -{ - unsigned int i; - - for (i = 0; i < table_cnt; i++) - if (symbol_in_range(table[i], &percpu_range, 1)) { - /* - * Keep the 'A' override for percpu symbols to - * ensure consistent behavior compared to older - * versions of this tool. - */ - table[i]->sym[0] = 'A'; - table[i]->percpu_absolute = true; - } -} - /* find the minimum non-absolute symbol address */ static void record_relative_base(void) { - unsigned int i; - - for (i = 0; i < table_cnt; i++) - if (!symbol_absolute(table[i])) { - /* - * The table is sorted by address. - * Take the first non-absolute symbol value. - */ - relative_base = table[i]->addr; - return; - } + /* + * The table is sorted by address. + * Take the first symbol value. + */ + if (table_cnt) + relative_base = table[0]->addr; } int main(int argc, char **argv) @@ -762,7 +717,6 @@ int main(int argc, char **argv) while (1) { static const struct option long_options[] = { {"all-symbols", no_argument, &all_symbols, 1}, - {"absolute-percpu", no_argument, &absolute_percpu, 1}, {}, }; @@ -779,8 +733,6 @@ int main(int argc, char **argv) read_map(argv[optind]); shrink_table(); - if (absolute_percpu) - make_percpus_absolute(); sort_symbols(); record_relative_base(); optimize_token_table(); diff --git a/scripts/link-vmlinux.sh b/scripts/link-vmlinux.sh index 56a077d204cf..67e66333bd2a 100755 --- a/scripts/link-vmlinux.sh +++ b/scripts/link-vmlinux.sh @@ -144,10 +144,6 @@ kallsyms() kallsymopt="${kallsymopt} --all-symbols" fi - if is_enabled CONFIG_KALLSYMS_ABSOLUTE_PERCPU; then - kallsymopt="${kallsymopt} --absolute-percpu" - fi - info KSYMS "${2}.S" scripts/kallsyms ${kallsymopt} "${1}" > "${2}.S" -- cgit v1.2.3 From b3d09d06e052e1d754645acea4e4d1e96f81c934 Mon Sep 17 00:00:00 2001 From: Steven Rostedt <rostedt@goodmis.org> Date: Tue, 18 Feb 2025 14:59:19 -0500 Subject: arm64: scripts/sorttable: Implement sorting mcount_loc at boot for arm64 The mcount_loc section holds the addresses of the functions that get patched by ftrace when enabling function callbacks. It can contain tens of thousands of entries. These addresses must be sorted. If they are not sorted at compile time, they are sorted at boot. Sorting at boot does take some time and does have a small impact on boot performance. x86 and arm32 have the addresses in the mcount_loc section of the ELF file. But for arm64, the section just contains zeros. The .rela.dyn Elf_Rela section holds the addresses and they get patched at boot during the relocation phase. In order to sort these addresses, the Elf_Rela needs to be updated instead of the location in the binary that holds the mcount_loc section. Have the sorttable code, allocate an array to hold the functions, load the addresses from the Elf_Rela entries, sort them, then put them back in order into the Elf_rela entries so that they will be sorted at boot up without having to sort them during boot up. Cc: bpf <bpf@vger.kernel.org> Cc: Masami Hiramatsu <mhiramat@kernel.org> Cc: Mark Rutland <mark.rutland@arm.com> Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Masahiro Yamada <masahiroy@kernel.org> Cc: Nathan Chancellor <nathan@kernel.org> Cc: Nicolas Schier <nicolas@fjasle.eu> Cc: Zheng Yejian <zhengyejian1@huawei.com> Cc: Martin Kelly <martin.kelly@crowdstrike.com> Cc: Christophe Leroy <christophe.leroy@csgroup.eu> Cc: Josh Poimboeuf <jpoimboe@redhat.com> Cc: Heiko Carstens <hca@linux.ibm.com> Cc: Will Deacon <will@kernel.org> Cc: Vasily Gorbik <gor@linux.ibm.com> Cc: Alexander Gordeev <agordeev@linux.ibm.com> Link: https://lore.kernel.org/20250218200022.373319428@goodmis.org Acked-by: Catalin Marinas <catalin.marinas@arm.com> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org> --- arch/arm64/Kconfig | 1 + scripts/sorttable.c | 185 +++++++++++++++++++++++++++++++++++++++++++++++++++- 2 files changed, 183 insertions(+), 3 deletions(-) (limited to 'scripts') diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index 940343beb3d4..4521ecefc031 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -217,6 +217,7 @@ config ARM64 if DYNAMIC_FTRACE_WITH_ARGS select HAVE_SAMPLE_FTRACE_DIRECT select HAVE_SAMPLE_FTRACE_DIRECT_MULTI + select HAVE_BUILDTIME_MCOUNT_SORT select HAVE_EFFICIENT_UNALIGNED_ACCESS select HAVE_GUP_FAST select HAVE_FTRACE_GRAPH_FUNC diff --git a/scripts/sorttable.c b/scripts/sorttable.c index 9f41575afd7a..4a34c275123e 100644 --- a/scripts/sorttable.c +++ b/scripts/sorttable.c @@ -28,6 +28,7 @@ #include <fcntl.h> #include <stdio.h> #include <stdlib.h> +#include <stdbool.h> #include <string.h> #include <unistd.h> #include <errno.h> @@ -79,10 +80,16 @@ typedef union { Elf64_Sym e64; } Elf_Sym; +typedef union { + Elf32_Rela e32; + Elf64_Rela e64; +} Elf_Rela; + static uint32_t (*r)(const uint32_t *); static uint16_t (*r2)(const uint16_t *); static uint64_t (*r8)(const uint64_t *); static void (*w)(uint32_t, uint32_t *); +static void (*w8)(uint64_t, uint64_t *); typedef void (*table_sort_t)(char *, int); static struct elf_funcs { @@ -102,6 +109,10 @@ static struct elf_funcs { uint32_t (*sym_name)(Elf_Sym *sym); uint64_t (*sym_value)(Elf_Sym *sym); uint16_t (*sym_shndx)(Elf_Sym *sym); + uint64_t (*rela_offset)(Elf_Rela *rela); + uint64_t (*rela_info)(Elf_Rela *rela); + uint64_t (*rela_addend)(Elf_Rela *rela); + void (*rela_write_addend)(Elf_Rela *rela, uint64_t val); } e; static uint64_t ehdr64_shoff(Elf_Ehdr *ehdr) @@ -262,6 +273,38 @@ SYM_ADDR(value) SYM_WORD(name) SYM_HALF(shndx) +#define __maybe_unused __attribute__((__unused__)) + +#define RELA_ADDR(fn_name) \ +static uint64_t rela64_##fn_name(Elf_Rela *rela) \ +{ \ + return r8((uint64_t *)&rela->e64.r_##fn_name); \ +} \ + \ +static uint64_t rela32_##fn_name(Elf_Rela *rela) \ +{ \ + return r((uint32_t *)&rela->e32.r_##fn_name); \ +} \ + \ +static uint64_t __maybe_unused rela_##fn_name(Elf_Rela *rela) \ +{ \ + return e.rela_##fn_name(rela); \ +} + +RELA_ADDR(offset) +RELA_ADDR(info) +RELA_ADDR(addend) + +static void rela64_write_addend(Elf_Rela *rela, uint64_t val) +{ + w8(val, (uint64_t *)&rela->e64.r_addend); +} + +static void rela32_write_addend(Elf_Rela *rela, uint64_t val) +{ + w(val, (uint32_t *)&rela->e32.r_addend); +} + /* * Get the whole file as a programming convenience in order to avoid * malloc+lseek+read+free of many pieces. If successful, then mmap @@ -341,6 +384,16 @@ static void wle(uint32_t val, uint32_t *x) put_unaligned_le32(val, x); } +static void w8be(uint64_t val, uint64_t *x) +{ + put_unaligned_be64(val, x); +} + +static void w8le(uint64_t val, uint64_t *x) +{ + put_unaligned_le64(val, x); +} + /* * Move reserved section indices SHN_LORESERVE..SHN_HIRESERVE out of * the way to -256..-1, to avoid conflicting with real section @@ -398,13 +451,12 @@ static inline void *get_index(void *start, int entsize, int index) static int extable_ent_size; static int long_size; +#define ERRSTR_MAXSZ 256 #ifdef UNWINDER_ORC_ENABLED /* ORC unwinder only support X86_64 */ #include <asm/orc_types.h> -#define ERRSTR_MAXSZ 256 - static char g_err[ERRSTR_MAXSZ]; static int *g_orc_ip_table; static struct orc_entry *g_orc_table; @@ -499,7 +551,19 @@ static void *sort_orctable(void *arg) #endif #ifdef MCOUNT_SORT_ENABLED + +/* Only used for sorting mcount table */ +static void rela_write_addend(Elf_Rela *rela, uint64_t val) +{ + e.rela_write_addend(rela, val); +} + static pthread_t mcount_sort_thread; +static bool sort_reloc; + +static long rela_type; + +static char m_err[ERRSTR_MAXSZ]; struct elf_mcount_loc { Elf_Ehdr *ehdr; @@ -508,6 +572,103 @@ struct elf_mcount_loc { uint64_t stop_mcount_loc; }; +/* Sort the relocations not the address itself */ +static void *sort_relocs(Elf_Ehdr *ehdr, uint64_t start_loc, uint64_t size) +{ + Elf_Shdr *shdr_start; + Elf_Rela *rel; + unsigned int shnum; + unsigned int count; + int shentsize; + void *vals; + void *ptr; + + shdr_start = (Elf_Shdr *)((char *)ehdr + ehdr_shoff(ehdr)); + shentsize = ehdr_shentsize(ehdr); + + vals = malloc(long_size * size); + if (!vals) { + snprintf(m_err, ERRSTR_MAXSZ, "Failed to allocate sort array"); + pthread_exit(m_err); + return NULL; + } + + ptr = vals; + + shnum = ehdr_shnum(ehdr); + if (shnum == SHN_UNDEF) + shnum = shdr_size(shdr_start); + + for (int i = 0; i < shnum; i++) { + Elf_Shdr *shdr = get_index(shdr_start, shentsize, i); + void *end; + + if (shdr_type(shdr) != SHT_RELA) + continue; + + rel = (void *)ehdr + shdr_offset(shdr); + end = (void *)rel + shdr_size(shdr); + + for (; (void *)rel < end; rel = (void *)rel + shdr_entsize(shdr)) { + uint64_t offset = rela_offset(rel); + + if (offset >= start_loc && offset < start_loc + size) { + if (ptr + long_size > vals + size) { + free(vals); + snprintf(m_err, ERRSTR_MAXSZ, + "Too many relocations"); + pthread_exit(m_err); + return NULL; + } + + /* Make sure this has the correct type */ + if (rela_info(rel) != rela_type) { + free(vals); + snprintf(m_err, ERRSTR_MAXSZ, + "rela has type %lx but expected %lx\n", + (long)rela_info(rel), rela_type); + pthread_exit(m_err); + return NULL; + } + + if (long_size == 4) + *(uint32_t *)ptr = rela_addend(rel); + else + *(uint64_t *)ptr = rela_addend(rel); + ptr += long_size; + } + } + } + count = ptr - vals; + qsort(vals, count / long_size, long_size, compare_extable); + + ptr = vals; + for (int i = 0; i < shnum; i++) { + Elf_Shdr *shdr = get_index(shdr_start, shentsize, i); + void *end; + + if (shdr_type(shdr) != SHT_RELA) + continue; + + rel = (void *)ehdr + shdr_offset(shdr); + end = (void *)rel + shdr_size(shdr); + + for (; (void *)rel < end; rel = (void *)rel + shdr_entsize(shdr)) { + uint64_t offset = rela_offset(rel); + + if (offset >= start_loc && offset < start_loc + size) { + if (long_size == 4) + rela_write_addend(rel, *(uint32_t *)ptr); + else + rela_write_addend(rel, *(uint64_t *)ptr); + ptr += long_size; + } + } + } + free(vals); + return NULL; +} + /* Sort the addresses stored between __start_mcount_loc to __stop_mcount_loc in vmlinux */ static void *sort_mcount_loc(void *arg) { @@ -517,6 +678,9 @@ static void *sort_mcount_loc(void *arg) uint64_t count = emloc->stop_mcount_loc - emloc->start_mcount_loc; unsigned char *start_loc = (void *)emloc->ehdr + offset; + if (sort_reloc) + return sort_relocs(emloc->ehdr, emloc->start_mcount_loc, count); + qsort(start_loc, count/long_size, long_size, compare_extable); return NULL; } @@ -866,12 +1030,14 @@ static int do_file(char const *const fname, void *addr) r2 = r2le; r8 = r8le; w = wle; + w8 = w8le; break; case ELFDATA2MSB: r = rbe; r2 = r2be; r8 = r8be; w = wbe; + w8 = w8be; break; default: fprintf(stderr, "unrecognized ELF data encoding %d: %s\n", @@ -887,8 +1053,13 @@ static int do_file(char const *const fname, void *addr) } switch (r2(&ehdr->e32.e_machine)) { - case EM_386: case EM_AARCH64: +#ifdef MCOUNT_SORT_ENABLED + sort_reloc = true; + rela_type = 0x403; +#endif + /* fallthrough */ + case EM_386: case EM_LOONGARCH: case EM_RISCV: case EM_S390: @@ -932,6 +1103,10 @@ static int do_file(char const *const fname, void *addr) .sym_name = sym32_name, .sym_value = sym32_value, .sym_shndx = sym32_shndx, + .rela_offset = rela32_offset, + .rela_info = rela32_info, + .rela_addend = rela32_addend, + .rela_write_addend = rela32_write_addend, }; e = efuncs; @@ -965,6 +1140,10 @@ static int do_file(char const *const fname, void *addr) .sym_name = sym64_name, .sym_value = sym64_value, .sym_shndx = sym64_shndx, + .rela_offset = rela64_offset, + .rela_info = rela64_info, + .rela_addend = rela64_addend, + .rela_write_addend = rela64_write_addend, }; e = efuncs; -- cgit v1.2.3 From a0265659322540d656727b9e132edfb6f06b6c1a Mon Sep 17 00:00:00 2001 From: Steven Rostedt <rostedt@goodmis.org> Date: Tue, 18 Feb 2025 14:59:20 -0500 Subject: scripts/sorttable: Have mcount rela sort use direct values The mcount_loc sorting for when the values are stored in the Elf_Rela entries uses the compare_extable() function to do the compares in the qsort(). That function does handle byte swapping if the machine being compiled for is a different endian than the host machine. But the sort_relocs() function sorts an array that pulled in the values from the Elf_Rela section and has already done the swapping. Create two new compare functions that will sort the direct values. One will sort 32 bit values and the other will sort the 64 bit value. One of these will be assigned to a compare_values function pointer and that will be used for sorting the Elf_Rela mcount values. Cc: bpf <bpf@vger.kernel.org> Cc: Masami Hiramatsu <mhiramat@kernel.org> Cc: Mark Rutland <mark.rutland@arm.com> Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Masahiro Yamada <masahiroy@kernel.org> Cc: Nathan Chancellor <nathan@kernel.org> Cc: Nicolas Schier <nicolas@fjasle.eu> Cc: Zheng Yejian <zhengyejian1@huawei.com> Cc: Martin Kelly <martin.kelly@crowdstrike.com> Cc: Christophe Leroy <christophe.leroy@csgroup.eu> Cc: Josh Poimboeuf <jpoimboe@redhat.com> Cc: Heiko Carstens <hca@linux.ibm.com> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Will Deacon <will@kernel.org> Cc: Vasily Gorbik <gor@linux.ibm.com> Cc: Alexander Gordeev <agordeev@linux.ibm.com> Link: https://lore.kernel.org/20250218200022.538888594@goodmis.org Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org> --- scripts/sorttable.c | 26 +++++++++++++++++++++++++- 1 file changed, 25 insertions(+), 1 deletion(-) (limited to 'scripts') diff --git a/scripts/sorttable.c b/scripts/sorttable.c index 4a34c275123e..f62a91d8af0a 100644 --- a/scripts/sorttable.c +++ b/scripts/sorttable.c @@ -552,6 +552,28 @@ static void *sort_orctable(void *arg) #ifdef MCOUNT_SORT_ENABLED +static int compare_values_64(const void *a, const void *b) +{ + uint64_t av = *(uint64_t *)a; + uint64_t bv = *(uint64_t *)b; + + if (av < bv) + return -1; + return av > bv; +} + +static int compare_values_32(const void *a, const void *b) +{ + uint32_t av = *(uint32_t *)a; + uint32_t bv = *(uint32_t *)b; + + if (av < bv) + return -1; + return av > bv; +} + +static int (*compare_values)(const void *a, const void *b); + /* Only used for sorting mcount table */ static void rela_write_addend(Elf_Rela *rela, uint64_t val) { @@ -583,6 +605,8 @@ static void *sort_relocs(Elf_Ehdr *ehdr, uint64_t start_loc, uint64_t size) void *vals; void *ptr; + compare_values = long_size == 4 ? compare_values_32 : compare_values_64; + shdr_start = (Elf_Shdr *)((char *)ehdr + ehdr_shoff(ehdr)); shentsize = ehdr_shentsize(ehdr); @@ -640,7 +664,7 @@ static void *sort_relocs(Elf_Ehdr *ehdr, uint64_t start_loc, uint64_t size) } } count = ptr - vals; - qsort(vals, count / long_size, long_size, compare_extable); + qsort(vals, count / long_size, long_size, compare_values); ptr = vals; for (int i = 0; i < shnum; i++) { -- cgit v1.2.3 From 5fb964f5ba53afda0e2b6dbc00b8205461ffe04a Mon Sep 17 00:00:00 2001 From: Steven Rostedt <rostedt@goodmis.org> Date: Tue, 18 Feb 2025 14:59:21 -0500 Subject: scripts/sorttable: Always use an array for the mcount_loc sorting The sorting of the mcount_loc section is done directly to the section for x86 and arm32 but it uses a separate array for arm64 as arm64 has the values for the mcount_loc stored in the rela sections of the vmlinux ELF file. In order to use the same code to remove weak functions, always use a separate array to do the sorting. This requires splitting up the filling of the array into one function and the placing the contents of the array back into the rela sections or into the mcount_loc section into a separate file. Cc: bpf <bpf@vger.kernel.org> Cc: Masami Hiramatsu <mhiramat@kernel.org> Cc: Mark Rutland <mark.rutland@arm.com> Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Masahiro Yamada <masahiroy@kernel.org> Cc: Nathan Chancellor <nathan@kernel.org> Cc: Nicolas Schier <nicolas@fjasle.eu> Cc: Zheng Yejian <zhengyejian1@huawei.com> Cc: Martin Kelly <martin.kelly@crowdstrike.com> Cc: Christophe Leroy <christophe.leroy@csgroup.eu> Cc: Josh Poimboeuf <jpoimboe@redhat.com> Cc: Heiko Carstens <hca@linux.ibm.com> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Will Deacon <will@kernel.org> Cc: Vasily Gorbik <gor@linux.ibm.com> Cc: Alexander Gordeev <agordeev@linux.ibm.com> Link: https://lore.kernel.org/20250218200022.710676551@goodmis.org Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org> --- scripts/sorttable.c | 122 ++++++++++++++++++++++++++++++++++++++-------------- 1 file changed, 90 insertions(+), 32 deletions(-) (limited to 'scripts') diff --git a/scripts/sorttable.c b/scripts/sorttable.c index f62a91d8af0a..ec02a2852efb 100644 --- a/scripts/sorttable.c +++ b/scripts/sorttable.c @@ -594,31 +594,19 @@ struct elf_mcount_loc { uint64_t stop_mcount_loc; }; -/* Sort the relocations not the address itself */ -static void *sort_relocs(Elf_Ehdr *ehdr, uint64_t start_loc, uint64_t size) +/* Fill the array with the content of the relocs */ +static int fill_relocs(void *ptr, uint64_t size, Elf_Ehdr *ehdr, uint64_t start_loc) { Elf_Shdr *shdr_start; Elf_Rela *rel; unsigned int shnum; - unsigned int count; + unsigned int count = 0; int shentsize; - void *vals; - void *ptr; - - compare_values = long_size == 4 ? compare_values_32 : compare_values_64; + void *array_end = ptr + size; shdr_start = (Elf_Shdr *)((char *)ehdr + ehdr_shoff(ehdr)); shentsize = ehdr_shentsize(ehdr); - vals = malloc(long_size * size); - if (!vals) { - snprintf(m_err, ERRSTR_MAXSZ, "Failed to allocate sort array"); - pthread_exit(m_err); - return NULL; - } - - ptr = vals; - shnum = ehdr_shnum(ehdr); if (shnum == SHN_UNDEF) shnum = shdr_size(shdr_start); @@ -637,22 +625,18 @@ static void *sort_relocs(Elf_Ehdr *ehdr, uint64_t start_loc, uint64_t size) uint64_t offset = rela_offset(rel); if (offset >= start_loc && offset < start_loc + size) { - if (ptr + long_size > vals + size) { - free(vals); + if (ptr + long_size > array_end) { snprintf(m_err, ERRSTR_MAXSZ, "Too many relocations"); - pthread_exit(m_err); - return NULL; + return -1; } /* Make sure this has the correct type */ if (rela_info(rel) != rela_type) { - free(vals); snprintf(m_err, ERRSTR_MAXSZ, "rela has type %lx but expected %lx\n", (long)rela_info(rel), rela_type); - pthread_exit(m_err); - return NULL; + return -1; } if (long_size == 4) @@ -660,13 +644,28 @@ static void *sort_relocs(Elf_Ehdr *ehdr, uint64_t start_loc, uint64_t size) else *(uint64_t *)ptr = rela_addend(rel); ptr += long_size; + count++; } } } - count = ptr - vals; - qsort(vals, count / long_size, long_size, compare_values); + return count; +} + +/* Put the sorted vals back into the relocation elements */ +static void replace_relocs(void *ptr, uint64_t size, Elf_Ehdr *ehdr, uint64_t start_loc) +{ + Elf_Shdr *shdr_start; + Elf_Rela *rel; + unsigned int shnum; + int shentsize; + + shdr_start = (Elf_Shdr *)((char *)ehdr + ehdr_shoff(ehdr)); + shentsize = ehdr_shentsize(ehdr); + + shnum = ehdr_shnum(ehdr); + if (shnum == SHN_UNDEF) + shnum = shdr_size(shdr_start); - ptr = vals; for (int i = 0; i < shnum; i++) { Elf_Shdr *shdr = get_index(shdr_start, shentsize, i); void *end; @@ -689,8 +688,32 @@ static void *sort_relocs(Elf_Ehdr *ehdr, uint64_t start_loc, uint64_t size) } } } - free(vals); - return NULL; +} + +static int fill_addrs(void *ptr, uint64_t size, void *addrs) +{ + void *end = ptr + size; + int count = 0; + + for (; ptr < end; ptr += long_size, addrs += long_size, count++) { + if (long_size == 4) + *(uint32_t *)ptr = r(addrs); + else + *(uint64_t *)ptr = r8(addrs); + } + return count; +} + +static void replace_addrs(void *ptr, uint64_t size, void *addrs) +{ + void *end = ptr + size; + + for (; ptr < end; ptr += long_size, addrs += long_size) { + if (long_size == 4) + w(*(uint32_t *)ptr, addrs); + else + w8(*(uint64_t *)ptr, addrs); + } } /* Sort the addresses stored between __start_mcount_loc to __stop_mcount_loc in vmlinux */ @@ -699,14 +722,49 @@ static void *sort_mcount_loc(void *arg) struct elf_mcount_loc *emloc = (struct elf_mcount_loc *)arg; uint64_t offset = emloc->start_mcount_loc - shdr_addr(emloc->init_data_sec) + shdr_offset(emloc->init_data_sec); - uint64_t count = emloc->stop_mcount_loc - emloc->start_mcount_loc; + uint64_t size = emloc->stop_mcount_loc - emloc->start_mcount_loc; unsigned char *start_loc = (void *)emloc->ehdr + offset; + Elf_Ehdr *ehdr = emloc->ehdr; + void *e_msg = NULL; + void *vals; + int count; + + vals = malloc(long_size * size); + if (!vals) { + snprintf(m_err, ERRSTR_MAXSZ, "Failed to allocate sort array"); + pthread_exit(m_err); + } if (sort_reloc) - return sort_relocs(emloc->ehdr, emloc->start_mcount_loc, count); + count = fill_relocs(vals, size, ehdr, emloc->start_mcount_loc); + else + count = fill_addrs(vals, size, start_loc); + + if (count < 0) { + e_msg = m_err; + goto out; + } + + if (count != size / long_size) { + snprintf(m_err, ERRSTR_MAXSZ, "Expected %u mcount elements but found %u\n", + (int)(size / long_size), count); + e_msg = m_err; + goto out; + } + + compare_values = long_size == 4 ? compare_values_32 : compare_values_64; + + qsort(vals, count, long_size, compare_values); + + if (sort_reloc) + replace_relocs(vals, size, ehdr, emloc->start_mcount_loc); + else + replace_addrs(vals, size, start_loc); + +out: + free(vals); - qsort(start_loc, count/long_size, long_size, compare_extable); - return NULL; + pthread_exit(e_msg); } /* Get the address of __start_mcount_loc and __stop_mcount_loc in System.map */ -- cgit v1.2.3 From ef378c3b8233855497a414b9d67bf22592c928a4 Mon Sep 17 00:00:00 2001 From: Steven Rostedt <rostedt@goodmis.org> Date: Tue, 18 Feb 2025 14:59:22 -0500 Subject: scripts/sorttable: Zero out weak functions in mcount_loc table When a function is annotated as "weak" and is overridden, the code is not removed. If it is traced, the fentry/mcount location in the weak function will be referenced by the "__mcount_loc" section. This will then be added to the available_filter_functions list. Since only the address of the functions are listed, to find the name to show, a search of kallsyms is used. Since kallsyms will return the function by simply finding the function that the address is after but before the next function, an address of a weak function will show up as the function before it. This is because kallsyms does not save names of weak functions. This has caused issues in the past, as now the traced weak function will be listed in available_filter_functions with the name of the function before it. At best, this will cause the previous function's name to be listed twice. At worse, if the previous function was marked notrace, it will now show up as a function that can be traced. Note that it only shows up that it can be traced but will not be if enabled, which causes confusion. https://lore.kernel.org/all/20220412094923.0abe90955e5db486b7bca279@kernel.org/ The commit b39181f7c6907 ("ftrace: Add FTRACE_MCOUNT_MAX_OFFSET to avoid adding weak function") was a workaround to this by checking the function address before printing its name. If the address was too far from the function given by the name then instead of printing the name it would print: __ftrace_invalid_address___<invalid-offset> The real issue is that these invalid addresses are listed in the ftrace table look up which available_filter_functions is derived from. A place holder must be listed in that file because set_ftrace_filter may take a series of indexes into that file instead of names to be able to do O(1) lookups to enable filtering (many tools use this method). Even if kallsyms saved the size of the function, it does not remove the need of having these place holders. The real solution is to not add a weak function into the ftrace table in the first place. To solve this, the sorttable.c code that sorts the mcount regions during the build is modified to take a "nm -S vmlinux" input, sort it, and any function listed in the mcount_loc section that is not within a boundary of the function list given by nm is considered a weak function and is zeroed out. Note, this does not mean they will remain zero when booting as KASLR will still shift those addresses. To handle this, the entries in the mcount_loc section will be ignored if they are zero or match the kaslr_offset() value. Before: ~# grep __ftrace_invalid_address___ /sys/kernel/tracing/available_filter_functions | wc -l 551 After: ~# grep __ftrace_invalid_address___ /sys/kernel/tracing/available_filter_functions | wc -l 0 Cc: bpf <bpf@vger.kernel.org> Cc: Masami Hiramatsu <mhiramat@kernel.org> Cc: Mark Rutland <mark.rutland@arm.com> Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Masahiro Yamada <masahiroy@kernel.org> Cc: Nathan Chancellor <nathan@kernel.org> Cc: Nicolas Schier <nicolas@fjasle.eu> Cc: Zheng Yejian <zhengyejian1@huawei.com> Cc: Martin Kelly <martin.kelly@crowdstrike.com> Cc: Christophe Leroy <christophe.leroy@csgroup.eu> Cc: Josh Poimboeuf <jpoimboe@redhat.com> Cc: Heiko Carstens <hca@linux.ibm.com> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Will Deacon <will@kernel.org> Cc: Vasily Gorbik <gor@linux.ibm.com> Cc: Alexander Gordeev <agordeev@linux.ibm.com> Link: https://lore.kernel.org/20250218200022.883095980@goodmis.org Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org> --- kernel/trace/ftrace.c | 6 ++- scripts/link-vmlinux.sh | 4 +- scripts/sorttable.c | 128 +++++++++++++++++++++++++++++++++++++++++++++++- 3 files changed, 134 insertions(+), 4 deletions(-) (limited to 'scripts') diff --git a/kernel/trace/ftrace.c b/kernel/trace/ftrace.c index 728ecda6e8d4..e3f89924f603 100644 --- a/kernel/trace/ftrace.c +++ b/kernel/trace/ftrace.c @@ -7004,6 +7004,7 @@ static int ftrace_process_locs(struct module *mod, unsigned long count; unsigned long *p; unsigned long addr; + unsigned long kaslr; unsigned long flags = 0; /* Shut up gcc */ int ret = -ENOMEM; @@ -7052,6 +7053,9 @@ static int ftrace_process_locs(struct module *mod, ftrace_pages->next = start_pg; } + /* For zeroed locations that were shifted for core kernel */ + kaslr = !mod ? kaslr_offset() : 0; + p = start; pg = start_pg; while (p < end) { @@ -7063,7 +7067,7 @@ static int ftrace_process_locs(struct module *mod, * object files to satisfy alignments. * Skip any NULL pointers. */ - if (!addr) { + if (!addr || addr == kaslr) { skipped++; continue; } diff --git a/scripts/link-vmlinux.sh b/scripts/link-vmlinux.sh index 56a077d204cf..59b07fe6fd00 100755 --- a/scripts/link-vmlinux.sh +++ b/scripts/link-vmlinux.sh @@ -177,12 +177,14 @@ mksysmap() sorttable() { - ${objtree}/scripts/sorttable ${1} + ${NM} -S ${1} > .tmp_vmlinux.nm-sort + ${objtree}/scripts/sorttable -s .tmp_vmlinux.nm-sort ${1} } cleanup() { rm -f .btf.* + rm -f .tmp_vmlinux.nm-sort rm -f System.map rm -f vmlinux rm -f vmlinux.map diff --git a/scripts/sorttable.c b/scripts/sorttable.c index ec02a2852efb..23c7e0e6c024 100644 --- a/scripts/sorttable.c +++ b/scripts/sorttable.c @@ -580,6 +580,98 @@ static void rela_write_addend(Elf_Rela *rela, uint64_t val) e.rela_write_addend(rela, val); } +struct func_info { + uint64_t addr; + uint64_t size; +}; + +/* List of functions created by: nm -S vmlinux */ +static struct func_info *function_list; +static int function_list_size; + +/* Allocate functions in 1k blocks */ +#define FUNC_BLK_SIZE 1024 +#define FUNC_BLK_MASK (FUNC_BLK_SIZE - 1) + +static int add_field(uint64_t addr, uint64_t size) +{ + struct func_info *fi; + int fsize = function_list_size; + + if (!(fsize & FUNC_BLK_MASK)) { + fsize += FUNC_BLK_SIZE; + fi = realloc(function_list, fsize * sizeof(struct func_info)); + if (!fi) + return -1; + function_list = fi; + } + fi = &function_list[function_list_size++]; + fi->addr = addr; + fi->size = size; + return 0; +} + +/* Only return match if the address lies inside the function size */ +static int cmp_func_addr(const void *K, const void *A) +{ + uint64_t key = *(const uint64_t *)K; + const struct func_info *a = A; + + if (key < a->addr) + return -1; + return key >= a->addr + a->size; +} + +/* Find the function in function list that is bounded by the function size */ +static int find_func(uint64_t key) +{ + return bsearch(&key, function_list, function_list_size, + sizeof(struct func_info), cmp_func_addr) != NULL; +} + +static int cmp_funcs(const void *A, const void *B) +{ + const struct func_info *a = A; + const struct func_info *b = B; + + if (a->addr < b->addr) + return -1; + return a->addr > b->addr; +} + +static int parse_symbols(const char *fname) +{ + FILE *fp; + char addr_str[20]; /* Only need 17, but round up to next int size */ + char size_str[20]; + char type; + + fp = fopen(fname, "r"); + if (!fp) { + perror(fname); + return -1; + } + + while (fscanf(fp, "%16s %16s %c %*s\n", addr_str, size_str, &type) == 3) { + uint64_t addr; + uint64_t size; + + /* Only care about functions */ + if (type != 't' && type != 'T' && type != 'W') + continue; + + addr = strtoull(addr_str, NULL, 16); + size = strtoull(size_str, NULL, 16); + if (add_field(addr, size) < 0) + return -1; + } + fclose(fp); + + qsort(function_list, function_list_size, sizeof(struct func_info), cmp_funcs); + + return 0; +} + static pthread_t mcount_sort_thread; static bool sort_reloc; @@ -752,6 +844,21 @@ static void *sort_mcount_loc(void *arg) goto out; } + /* zero out any locations not found by function list */ + if (function_list_size) { + for (void *ptr = vals; ptr < vals + size; ptr += long_size) { + uint64_t key; + + key = long_size == 4 ? r((uint32_t *)ptr) : r8((uint64_t *)ptr); + if (!find_func(key)) { + if (long_size == 4) + *(uint32_t *)ptr = 0; + else + *(uint64_t *)ptr = 0; + } + } + } + compare_values = long_size == 4 ? compare_values_32 : compare_values_64; qsort(vals, count, long_size, compare_values); @@ -801,6 +908,8 @@ static void get_mcount_loc(struct elf_mcount_loc *emloc, Elf_Shdr *symtab_sec, return; } } +#else /* MCOUNT_SORT_ENABLED */ +static inline int parse_symbols(const char *fname) { return 0; } #endif static int do_sort(Elf_Ehdr *ehdr, @@ -1256,14 +1365,29 @@ int main(int argc, char *argv[]) int i, n_error = 0; /* gcc-4.3.0 false positive complaint */ size_t size = 0; void *addr = NULL; + int c; + + while ((c = getopt(argc, argv, "s:")) >= 0) { + switch (c) { + case 's': + if (parse_symbols(optarg) < 0) { + fprintf(stderr, "Could not parse %s\n", optarg); + return -1; + } + break; + default: + fprintf(stderr, "usage: sorttable [-s nm-file] vmlinux...\n"); + return 0; + } + } - if (argc < 2) { + if ((argc - optind) < 1) { fprintf(stderr, "usage: sorttable vmlinux...\n"); return 0; } /* Process each file in turn, allowing deep failure. */ - for (i = 1; i < argc; i++) { + for (i = optind; i < argc; i++) { addr = mmap_file(argv[i], &size); if (!addr) { ++n_error; -- cgit v1.2.3 From 1a09cd9b7bc76e9e1d753c77584079d302241c23 Mon Sep 17 00:00:00 2001 From: Costa Shulyupin <costa.shul@redhat.com> Date: Fri, 31 Jan 2025 17:54:30 +0200 Subject: scripts/tags.sh: tag SYM_*START*() assembler symbols The `startup_64` symbol and many other assembler symbols are not tagged. Add a generic rule to tag assembler symbols defined with macros like SYM_*START*(symbol). Signed-off-by: Costa Shulyupin <costa.shul@redhat.com> Link: https://lore.kernel.org/r/20250131155439.2025038-1-costa.shul@redhat.com Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> --- scripts/tags.sh | 1 + 1 file changed, 1 insertion(+) (limited to 'scripts') diff --git a/scripts/tags.sh b/scripts/tags.sh index 45eaf35f5bff..98680e9cd7be 100755 --- a/scripts/tags.sh +++ b/scripts/tags.sh @@ -146,6 +146,7 @@ dogtags() # a ^[^#] is prepended by setup_regex unless an anchor is already present regex_asm=( '/^\(ENTRY\|_GLOBAL\)([[:space:]]*\([[:alnum:]_\\]*\)).*/\2/' + '/^SYM_[[:alnum:]_]*START[[:alnum:]_]*([[:space:]]*\([[:alnum:]_\\]*\)).*/\1/' ) regex_c=( '/^SYSCALL_DEFINE[0-9]([[:space:]]*\([[:alnum:]_]*\).*/sys_\1/' -- cgit v1.2.3 From 7861640aac52bbbb3dc2cd40fb93dfb3b3d0f43c Mon Sep 17 00:00:00 2001 From: Nathan Chancellor <nathan@kernel.org> Date: Thu, 20 Feb 2025 13:08:12 -0700 Subject: x86/build: Raise the minimum LLVM version to 15.0.0 In a similar vein as to this pending commit in the x86/asm tree: a3e8fe814ad1 ("x86/build: Raise the minimum GCC version to 8.1") ... bump the minimum supported version of LLVM for building x86 kernels to 15.0.0, as that is the first version that has support for '-mstack-protector-guard-symbol', which is used unconditionally after: 80d47defddc0 ("x86/stackprotector/64: Convert to normal per-CPU variable"): Older Clang versions will fail the build with: clang-14: error: unknown argument: '-mstack-protector-guard-symbol=__ref_stack_chk_guard' Fixes: 80d47defddc0 ("x86/stackprotector/64: Convert to normal per-CPU variable") Signed-off-by: Nathan Chancellor <nathan@kernel.org> Signed-off-by: Ingo Molnar <mingo@kernel.org> Cc: Linus Torvalds <torvalds@linux-foundation.org> Reviewed-by: Ard Biesheuvel <ardb@kernel.org> Reviewed-by: Brian Gerst <brgerst@gmail.com> Link: https://lore.kernel.org/r/20250220-x86-bump-min-llvm-for-stackp-v1-1-ecb3c906e790@kernel.org --- scripts/min-tool-version.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'scripts') diff --git a/scripts/min-tool-version.sh b/scripts/min-tool-version.sh index 06c4e410ecab..787868183b84 100755 --- a/scripts/min-tool-version.sh +++ b/scripts/min-tool-version.sh @@ -26,7 +26,7 @@ gcc) fi ;; llvm) - if [ "$SRCARCH" = s390 ]; then + if [ "$SRCARCH" = s390 -o "$SRCARCH" = x86 ]; then echo 15.0.0 elif [ "$SRCARCH" = loongarch ]; then echo 18.0.0 -- cgit v1.2.3 From 1ffe30efd2f2e58c36b754c42c2b61906078a4cf Mon Sep 17 00:00:00 2001 From: Pu Lehui <pulehui@huawei.com> Date: Wed, 19 Feb 2025 06:31:13 +0000 Subject: kbuild, bpf: Correct pahole version that supports distilled base btf feature pahole commit [0] of supporting distilled base btf feature released on pahole v1.28 rather than v1.26. So let's correct this. Signed-off-by: Pu Lehui <pulehui@huawei.com> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Link: https://git.kernel.org/pub/scm/devel/pahole/pahole.git/commit/?id=c7b1f6a29ba1 [0] Link: https://lore.kernel.org/bpf/20250219063113.706600-1-pulehui@huaweicloud.com --- scripts/Makefile.btf | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'scripts') diff --git a/scripts/Makefile.btf b/scripts/Makefile.btf index c3cbeb13de50..fbaaec2187e5 100644 --- a/scripts/Makefile.btf +++ b/scripts/Makefile.btf @@ -24,7 +24,7 @@ else pahole-flags-$(call test-ge, $(pahole-ver), 126) = -j$(JOBS) --btf_features=encode_force,var,float,enum64,decl_tag,type_tag,optimized_func,consistent_func,decl_tag_kfuncs ifneq ($(KBUILD_EXTMOD),) -module-pahole-flags-$(call test-ge, $(pahole-ver), 126) += --btf_features=distilled_base +module-pahole-flags-$(call test-ge, $(pahole-ver), 128) += --btf_features=distilled_base endif endif -- cgit v1.2.3 From 46514b3c2c17c67cefe84b0c1a59e0aaf6093131 Mon Sep 17 00:00:00 2001 From: Steven Rostedt <rostedt@goodmis.org> Date: Tue, 25 Feb 2025 13:20:07 -0500 Subject: scripts/sorttable: Use normal sort if theres no relocs in the mcount section When ARM 64 is compiled with gcc, the mcount_loc section will be filled with zeros and the addresses will be located in the Elf_Rela sections. To sort the mcount_loc section, the addresses from the Elf_Rela need to be placed into an array and that is sorted. But when ARM 64 is compiled with clang, it does it the same way as other architectures and leaves the addresses as is in the mcount_loc section. To handle both cases, ARM 64 will first try to sort the Elf_Rela section, and if it doesn't find any functions, it will then fall back to the sorting of the addresses in the mcount_loc section itself. Cc: Masami Hiramatsu <mhiramat@kernel.org> Cc: Mark Rutland <mark.rutland@arm.com> Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Masahiro Yamada <masahiroy@kernel.org> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Will Deacon <will@kernel.org> Cc: Mark Brown <broonie@kernel.org> Link: https://lore.kernel.org/20250225182054.648398403@goodmis.org Fixes: b3d09d06e052 ("arm64: scripts/sorttable: Implement sorting mcount_loc at boot for arm64") Reported-by: "Arnd Bergmann" <arnd@arndb.de> Tested-by: Nathan Chancellor <nathan@kernel.org> Closes: https://lore.kernel.org/all/893cd8f1-8585-4d25-bf0f-4197bf872465@app.fastmail.com/ Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org> --- scripts/sorttable.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) (limited to 'scripts') diff --git a/scripts/sorttable.c b/scripts/sorttable.c index 23c7e0e6c024..07ad8116bc8d 100644 --- a/scripts/sorttable.c +++ b/scripts/sorttable.c @@ -827,9 +827,14 @@ static void *sort_mcount_loc(void *arg) pthread_exit(m_err); } - if (sort_reloc) + if (sort_reloc) { count = fill_relocs(vals, size, ehdr, emloc->start_mcount_loc); - else + /* gcc may use relocs to save the addresses, but clang does not. */ + if (!count) { + count = fill_addrs(vals, size, start_loc); + sort_reloc = 0; + } + } else count = fill_addrs(vals, size, start_loc); if (count < 0) { -- cgit v1.2.3 From dc208c69c033d3caba0509da1ae065d2b5ff165f Mon Sep 17 00:00:00 2001 From: Steven Rostedt <rostedt@goodmis.org> Date: Tue, 25 Feb 2025 13:20:08 -0500 Subject: scripts/sorttable: Allow matches to functions before function entry ARM 64 uses -fpatchable-function-entry=4,2 which adds padding before the function and the addresses in the mcount_loc point there instead of the function entry that is returned by nm. In order to find a function from nm to make sure it's not an unused weak function, the entries in the mcount_loc section needs to match the entries from nm. Since it can be an instruction before the entry, add a before_func variable that ARM 64 can set to 8, and if the mcount_loc entry is within 8 bytes of the nm function entry, then it will be considered a match. Cc: Masami Hiramatsu <mhiramat@kernel.org> Cc: Mark Rutland <mark.rutland@arm.com> Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Masahiro Yamada <masahiroy@kernel.org> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Will Deacon <will@kernel.org> Cc: "Arnd Bergmann" <arnd@arndb.de> Cc: Mark Brown <broonie@kernel.org> Link: https://lore.kernel.org/20250225182054.815536219@goodmis.org Fixes: ef378c3b82338 ("scripts/sorttable: Zero out weak functions in mcount_loc table") Tested-by: Nathan Chancellor <nathan@kernel.org> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org> --- scripts/sorttable.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) (limited to 'scripts') diff --git a/scripts/sorttable.c b/scripts/sorttable.c index 07ad8116bc8d..7b4b3714b1af 100644 --- a/scripts/sorttable.c +++ b/scripts/sorttable.c @@ -611,13 +611,16 @@ static int add_field(uint64_t addr, uint64_t size) return 0; } +/* Used for when mcount/fentry is before the function entry */ +static int before_func; + /* Only return match if the address lies inside the function size */ static int cmp_func_addr(const void *K, const void *A) { uint64_t key = *(const uint64_t *)K; const struct func_info *a = A; - if (key < a->addr) + if (key + before_func < a->addr) return -1; return key >= a->addr + a->size; } @@ -1253,6 +1256,8 @@ static int do_file(char const *const fname, void *addr) #ifdef MCOUNT_SORT_ENABLED sort_reloc = true; rela_type = 0x403; + /* arm64 uses patchable function entry placing before function */ + before_func = 8; #endif /* fallthrough */ case EM_386: -- cgit v1.2.3 From 5ace19bd8395e8a98ff0bca0fd20ae3fac3e1d6f Mon Sep 17 00:00:00 2001 From: Gal Pressman <gal@nvidia.com> Date: Wed, 26 Feb 2025 11:39:00 +0200 Subject: coccinelle: Add missing (GE)NL_SET_ERR_MSG_* to strings ending with newline test Add missing (GE)NL_SET_ERR_MSG_*() variants to the list of macros checked for strings ending with a newline. Reviewed-by: Tariq Toukan <tariqt@nvidia.com> Signed-off-by: Gal Pressman <gal@nvidia.com> Reviewed-by: Michal Swiatkowski <michal.swiatkowski@linux.intel.com> Link: https://patch.msgid.link/20250226093904.6632-2-gal@nvidia.com Signed-off-by: Jakub Kicinski <kuba@kernel.org> --- scripts/coccinelle/misc/newline_in_nl_msg.cocci | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-) (limited to 'scripts') diff --git a/scripts/coccinelle/misc/newline_in_nl_msg.cocci b/scripts/coccinelle/misc/newline_in_nl_msg.cocci index 9baffe55d917..2814f6b205b9 100644 --- a/scripts/coccinelle/misc/newline_in_nl_msg.cocci +++ b/scripts/coccinelle/misc/newline_in_nl_msg.cocci @@ -1,7 +1,6 @@ // SPDX-License-Identifier: GPL-2.0-only /// -/// Catch strings ending in newline with GENL_SET_ERR_MSG, NL_SET_ERR_MSG, -/// NL_SET_ERR_MSG_MOD. +/// Catch strings ending in newline with (GE)NL_SET_ERR_MSG*. /// // Confidence: Very High // Copyright: (C) 2020 Intel Corporation @@ -17,7 +16,11 @@ expression e; constant m; position p; @@ - \(GENL_SET_ERR_MSG\|NL_SET_ERR_MSG\|NL_SET_ERR_MSG_MOD\)(e,m@p) + \(GENL_SET_ERR_MSG\|GENL_SET_ERR_MSG_FMT\|NL_SET_ERR_MSG\|NL_SET_ERR_MSG_MOD\| + NL_SET_ERR_MSG_FMT\|NL_SET_ERR_MSG_FMT_MOD\|NL_SET_ERR_MSG_WEAK\| + NL_SET_ERR_MSG_WEAK_MOD\|NL_SET_ERR_MSG_ATTR_POL\| + NL_SET_ERR_MSG_ATTR_POL_FMT\|NL_SET_ERR_MSG_ATTR\| + NL_SET_ERR_MSG_ATTR_FMT\)(e,m@p,...) @script:python@ m << r.m; @@ -32,7 +35,7 @@ expression r.e; constant r.m; position r.p; @@ - fname(e,m@p) + fname(e,m@p,...) //---------------------------------------------------------- // For context mode @@ -43,7 +46,7 @@ identifier r1.fname; expression r.e; constant r.m; @@ -* fname(e,m) +* fname(e,m,...) //---------------------------------------------------------- // For org mode -- cgit v1.2.3 From b9609ecba35ecf7fa85d9bcd519242009174e6a2 Mon Sep 17 00:00:00 2001 From: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Date: Mon, 24 Feb 2025 10:08:11 +0100 Subject: scripts/kernel-doc: don't add not needed new lines This helps comparing kernel-doc output with the new .py version of it. Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Link: https://lore.kernel.org/r/6b036ef7d746f26d7d0044626b04d1f0880a2188.1740387599.git.mchehab+huawei@kernel.org Signed-off-by: Jonathan Corbet <corbet@lwn.net> --- scripts/kernel-doc | 4 ++++ 1 file changed, 4 insertions(+) (limited to 'scripts') diff --git a/scripts/kernel-doc b/scripts/kernel-doc index 2c77b914d017..d59552e1a31d 100755 --- a/scripts/kernel-doc +++ b/scripts/kernel-doc @@ -760,6 +760,10 @@ sub output_highlight_rst { if ($block) { $output .= highlight_block($block); } + + $output =~ s/^\n+//g; + $output =~ s/\n+$//g; + foreach $line (split "\n", $output) { print $lineprefix . $line . "\n"; } -- cgit v1.2.3 From 19b100b0116d703b9529f7bbbf797428de51816a Mon Sep 17 00:00:00 2001 From: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Date: Mon, 24 Feb 2025 10:08:12 +0100 Subject: scripts/kernel-doc: drop dead code for Wcontents_before_sections There is a warning about contents before sections, which doesn't work, since in_doc_sect variable is always true at the point it is checked. Drop the dead code. Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Link: https://lore.kernel.org/r/174a15607fd057c736dc9123c53d0835ce20e68b.1740387599.git.mchehab+huawei@kernel.org Signed-off-by: Jonathan Corbet <corbet@lwn.net> --- scripts/kernel-doc | 11 ----------- 1 file changed, 11 deletions(-) (limited to 'scripts') diff --git a/scripts/kernel-doc b/scripts/kernel-doc index d59552e1a31d..af6cf408b96d 100755 --- a/scripts/kernel-doc +++ b/scripts/kernel-doc @@ -137,7 +137,6 @@ my $verbose = 0; my $Werror = 0; my $Wreturn = 0; my $Wshort_desc = 0; -my $Wcontents_before_sections = 0; my $output_mode = "rst"; my $output_preformatted = 0; my $no_doc_sections = 0; @@ -223,7 +222,6 @@ use constant { STATE_INLINE => 7, # gathering doc outside main block }; my $state; -my $in_doc_sect; my $leading_space; # Inline documentation state @@ -332,12 +330,9 @@ while ($ARGV[0] =~ m/^--?(.*)/) { $Wreturn = 1; } elsif ($cmd eq "Wshort-desc" or $cmd eq "Wshort-description") { $Wshort_desc = 1; - } elsif ($cmd eq "Wcontents-before-sections") { - $Wcontents_before_sections = 1; } elsif ($cmd eq "Wall") { $Wreturn = 1; $Wshort_desc = 1; - $Wcontents_before_sections = 1; } elsif (($cmd eq "h") || ($cmd eq "help")) { pod2usage(-exitval => 0, -verbose => 2); } elsif ($cmd eq 'no-doc-sections') { @@ -1963,7 +1958,6 @@ sub process_export_file($) { sub process_normal() { if (/$doc_start/o) { $state = STATE_NAME; # next line is always the function name - $in_doc_sect = 0; $declaration_start_line = $. + 1; } } @@ -2068,7 +2062,6 @@ sub process_body($$) { } if (/$doc_sect/i) { # case insensitive for supported section names - $in_doc_sect = 1; $newsection = $1; $newcontents = $2; @@ -2085,14 +2078,10 @@ sub process_body($$) { } if (($contents ne "") && ($contents ne "\n")) { - if (!$in_doc_sect && $Wcontents_before_sections) { - emit_warning("${file}:$.", "contents before sections\n"); - } dump_section($file, $section, $contents); $section = $section_default; } - $in_doc_sect = 1; $state = STATE_BODY; $contents = $newcontents; $new_start_line = $.; -- cgit v1.2.3 From a3aac126ca3a71b6612a817ef24db325618fd902 Mon Sep 17 00:00:00 2001 From: Kees Cook <kees@kernel.org> Date: Tue, 4 Mar 2025 08:21:29 -0800 Subject: kbuild: clang: Support building UM with SUBARCH=i386 The UM builds distinguish i386 from x86_64 via SUBARCH, but we don't support building i386 directly with Clang. To make SUBARCH work for i386 UM, we need to explicitly test for it. This lets me run i386 KUnit tests with Clang: $ ./tools/testing/kunit/kunit.py run \ --make_options LLVM=1 \ --make_options SUBARCH=i386 ... Fixes: c7500c1b53bf ("um: Allow builds with Clang") Reviewed-by: Nathan Chancellor <nathan@kernel.org> Link: https://lore.kernel.org/r/20250304162124.it.785-kees@kernel.org Tested-by: David Gow <davidgow@google.com> Signed-off-by: Kees Cook <kees@kernel.org> --- scripts/Makefile.clang | 2 ++ 1 file changed, 2 insertions(+) (limited to 'scripts') diff --git a/scripts/Makefile.clang b/scripts/Makefile.clang index 2435efae67f5..b67636b28c35 100644 --- a/scripts/Makefile.clang +++ b/scripts/Makefile.clang @@ -12,6 +12,8 @@ CLANG_TARGET_FLAGS_riscv := riscv64-linux-gnu CLANG_TARGET_FLAGS_s390 := s390x-linux-gnu CLANG_TARGET_FLAGS_sparc := sparc64-linux-gnu CLANG_TARGET_FLAGS_x86 := x86_64-linux-gnu +# This is only for i386 UM builds, which need the 32-bit target not -m32 +CLANG_TARGET_FLAGS_i386 := i386-linux-gnu CLANG_TARGET_FLAGS_um := $(CLANG_TARGET_FLAGS_$(SUBARCH)) CLANG_TARGET_FLAGS := $(CLANG_TARGET_FLAGS_$(SRCARCH)) -- cgit v1.2.3 From a1e4cc0155ad577adc3a2c563fc5eec625945ce7 Mon Sep 17 00:00:00 2001 From: Brian Gerst <brgerst@gmail.com> Date: Mon, 3 Mar 2025 11:52:44 -0500 Subject: x86/percpu: Move current_task to percpu hot section No functional change. Signed-off-by: Brian Gerst <brgerst@gmail.com> Signed-off-by: Ingo Molnar <mingo@kernel.org> Acked-by: Uros Bizjak <ubizjak@gmail.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Peter Zijlstra <peterz@infradead.org> Link: https://lore.kernel.org/r/20250303165246.2175811-10-brgerst@gmail.com --- arch/x86/include/asm/current.h | 17 ++++++----------- arch/x86/include/asm/percpu.h | 2 +- arch/x86/kernel/asm-offsets.c | 1 - arch/x86/kernel/cpu/common.c | 8 +++----- arch/x86/kernel/head_64.S | 4 ++-- arch/x86/kernel/process_32.c | 2 +- arch/x86/kernel/process_64.c | 2 +- arch/x86/kernel/smpboot.c | 2 +- arch/x86/kernel/vmlinux.lds.S | 2 +- scripts/gdb/linux/cpus.py | 2 +- 10 files changed, 17 insertions(+), 25 deletions(-) (limited to 'scripts') diff --git a/arch/x86/include/asm/current.h b/arch/x86/include/asm/current.h index 3d1b123c2ee3..dea7d8b854f0 100644 --- a/arch/x86/include/asm/current.h +++ b/arch/x86/include/asm/current.h @@ -12,22 +12,17 @@ struct task_struct; -struct pcpu_hot { - struct task_struct *current_task; -}; - -DECLARE_PER_CPU_CACHE_HOT(struct pcpu_hot, pcpu_hot); - -/* const-qualified alias to pcpu_hot, aliased by linker. */ -DECLARE_PER_CPU_CACHE_HOT(const struct pcpu_hot __percpu_seg_override, - const_pcpu_hot); +DECLARE_PER_CPU_CACHE_HOT(struct task_struct *, current_task); +/* const-qualified alias provided by the linker. */ +DECLARE_PER_CPU_CACHE_HOT(struct task_struct * const __percpu_seg_override, + const_current_task); static __always_inline struct task_struct *get_current(void) { if (IS_ENABLED(CONFIG_USE_X86_SEG_SUPPORT)) - return this_cpu_read_const(const_pcpu_hot.current_task); + return this_cpu_read_const(const_current_task); - return this_cpu_read_stable(pcpu_hot.current_task); + return this_cpu_read_stable(current_task); } #define current get_current() diff --git a/arch/x86/include/asm/percpu.h b/arch/x86/include/asm/percpu.h index 41517a7f7f1c..8cc9548205b0 100644 --- a/arch/x86/include/asm/percpu.h +++ b/arch/x86/include/asm/percpu.h @@ -551,7 +551,7 @@ do { \ * it is accessed while this_cpu_read_stable() allows the value to be cached. * this_cpu_read_stable() is more efficient and can be used if its value * is guaranteed to be valid across CPUs. The current users include - * pcpu_hot.current_task and cpu_current_top_of_stack, both of which are + * current_task and cpu_current_top_of_stack, both of which are * actually per-thread variables implemented as per-CPU variables and * thus stable for the duration of the respective task. */ diff --git a/arch/x86/kernel/asm-offsets.c b/arch/x86/kernel/asm-offsets.c index 54ace808defd..ad4ea6fb3b6c 100644 --- a/arch/x86/kernel/asm-offsets.c +++ b/arch/x86/kernel/asm-offsets.c @@ -107,7 +107,6 @@ static void __used common(void) OFFSET(TSS_sp0, tss_struct, x86_tss.sp0); OFFSET(TSS_sp1, tss_struct, x86_tss.sp1); OFFSET(TSS_sp2, tss_struct, x86_tss.sp2); - OFFSET(X86_current_task, pcpu_hot, current_task); #if IS_ENABLED(CONFIG_CRYPTO_ARIA_AESNI_AVX_X86_64) /* Offset for fields in aria_ctx */ BLANK(); diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c index 51653e01a716..3f45fdd48bdb 100644 --- a/arch/x86/kernel/cpu/common.c +++ b/arch/x86/kernel/cpu/common.c @@ -2064,11 +2064,9 @@ static __init int setup_setcpuid(char *arg) } __setup("setcpuid=", setup_setcpuid); -DEFINE_PER_CPU_CACHE_HOT(struct pcpu_hot, pcpu_hot) = { - .current_task = &init_task, -}; -EXPORT_PER_CPU_SYMBOL(pcpu_hot); -EXPORT_PER_CPU_SYMBOL(const_pcpu_hot); +DEFINE_PER_CPU_CACHE_HOT(struct task_struct *, current_task) = &init_task; +EXPORT_PER_CPU_SYMBOL(current_task); +EXPORT_PER_CPU_SYMBOL(const_current_task); DEFINE_PER_CPU_CACHE_HOT(int, __preempt_count) = INIT_PREEMPT_COUNT; EXPORT_PER_CPU_SYMBOL(__preempt_count); diff --git a/arch/x86/kernel/head_64.S b/arch/x86/kernel/head_64.S index 2843b0a56198..fefe2a25cf02 100644 --- a/arch/x86/kernel/head_64.S +++ b/arch/x86/kernel/head_64.S @@ -322,7 +322,7 @@ SYM_INNER_LABEL(common_startup_64, SYM_L_LOCAL) * * RDX contains the per-cpu offset */ - movq pcpu_hot + X86_current_task(%rdx), %rax + movq current_task(%rdx), %rax movq TASK_threadsp(%rax), %rsp /* @@ -433,7 +433,7 @@ SYM_CODE_START(soft_restart_cpu) UNWIND_HINT_END_OF_STACK /* Find the idle task stack */ - movq PER_CPU_VAR(pcpu_hot + X86_current_task), %rcx + movq PER_CPU_VAR(current_task), %rcx movq TASK_threadsp(%rcx), %rsp jmp .Ljump_to_C_code diff --git a/arch/x86/kernel/process_32.c b/arch/x86/kernel/process_32.c index 8ec44acb863b..4636ef359973 100644 --- a/arch/x86/kernel/process_32.c +++ b/arch/x86/kernel/process_32.c @@ -206,7 +206,7 @@ __switch_to(struct task_struct *prev_p, struct task_struct *next_p) if (prev->gs | next->gs) loadsegment(gs, next->gs); - raw_cpu_write(pcpu_hot.current_task, next_p); + raw_cpu_write(current_task, next_p); switch_fpu_finish(next_p); diff --git a/arch/x86/kernel/process_64.c b/arch/x86/kernel/process_64.c index d8f4bcef8ee4..7196ca7048be 100644 --- a/arch/x86/kernel/process_64.c +++ b/arch/x86/kernel/process_64.c @@ -668,7 +668,7 @@ __switch_to(struct task_struct *prev_p, struct task_struct *next_p) /* * Switch the PDA and FPU contexts. */ - raw_cpu_write(pcpu_hot.current_task, next_p); + raw_cpu_write(current_task, next_p); raw_cpu_write(cpu_current_top_of_stack, task_top_of_stack(next_p)); switch_fpu_finish(next_p); diff --git a/arch/x86/kernel/smpboot.c b/arch/x86/kernel/smpboot.c index c3a26e60e3c4..42ca131bebbc 100644 --- a/arch/x86/kernel/smpboot.c +++ b/arch/x86/kernel/smpboot.c @@ -822,7 +822,7 @@ int common_cpu_up(unsigned int cpu, struct task_struct *idle) /* Just in case we booted with a single CPU. */ alternatives_enable_smp(); - per_cpu(pcpu_hot.current_task, cpu) = idle; + per_cpu(current_task, cpu) = idle; cpu_init_stack_canary(cpu, idle); /* Initialize the interrupt stack(s) */ diff --git a/arch/x86/kernel/vmlinux.lds.S b/arch/x86/kernel/vmlinux.lds.S index 475f6717f27a..31f9102b107f 100644 --- a/arch/x86/kernel/vmlinux.lds.S +++ b/arch/x86/kernel/vmlinux.lds.S @@ -43,7 +43,7 @@ ENTRY(phys_startup_64) #endif jiffies = jiffies_64; -const_pcpu_hot = pcpu_hot; +const_current_task = current_task; const_cpu_current_top_of_stack = cpu_current_top_of_stack; #if defined(CONFIG_X86_64) diff --git a/scripts/gdb/linux/cpus.py b/scripts/gdb/linux/cpus.py index 13eb8b3901b8..8f7c4fb78c2c 100644 --- a/scripts/gdb/linux/cpus.py +++ b/scripts/gdb/linux/cpus.py @@ -164,7 +164,7 @@ def get_current_task(cpu): var_ptr = gdb.parse_and_eval("(struct task_struct *)cpu_tasks[0].task") return var_ptr.dereference() else: - var_ptr = gdb.parse_and_eval("&pcpu_hot.current_task") + var_ptr = gdb.parse_and_eval("¤t_task") return per_cpu(var_ptr, cpu).dereference() elif utils.is_target_arch("aarch64"): current_task_addr = gdb.parse_and_eval("(unsigned long)$SP_EL0") -- cgit v1.2.3 From 374908a15af4cd60862ebc51a6e012ace2212c76 Mon Sep 17 00:00:00 2001 From: Miguel Ojeda <ojeda@kernel.org> Date: Mon, 3 Mar 2025 18:10:30 +0100 Subject: rust: remove leftover mentions of the `alloc` crate In commit 392e34b6bc22 ("kbuild: rust: remove the `alloc` crate and `GlobalAlloc`") we stopped using the upstream `alloc` crate. Thus remove a few leftover mentions treewide. Cc: stable@vger.kernel.org # Also to 6.12.y after the `alloc` backport lands Fixes: 392e34b6bc22 ("kbuild: rust: remove the `alloc` crate and `GlobalAlloc`") Reviewed-by: Danilo Krummrich <dakr@kernel.org> Reviewed-by: Andreas Hindborg <a.hindborg@kernel.org> Link: https://lore.kernel.org/r/20250303171030.1081134-1-ojeda@kernel.org Signed-off-by: Miguel Ojeda <ojeda@kernel.org> --- Documentation/rust/quick-start.rst | 2 +- rust/kernel/lib.rs | 2 +- scripts/rustdoc_test_gen.rs | 4 ++-- 3 files changed, 4 insertions(+), 4 deletions(-) (limited to 'scripts') diff --git a/Documentation/rust/quick-start.rst b/Documentation/rust/quick-start.rst index 4aa50e5fcb8c..6d2607870ba4 100644 --- a/Documentation/rust/quick-start.rst +++ b/Documentation/rust/quick-start.rst @@ -145,7 +145,7 @@ Rust standard library source **************************** The Rust standard library source is required because the build system will -cross-compile ``core`` and ``alloc``. +cross-compile ``core``. If ``rustup`` is being used, run:: diff --git a/rust/kernel/lib.rs b/rust/kernel/lib.rs index 398242f92a96..7697c60b2d1a 100644 --- a/rust/kernel/lib.rs +++ b/rust/kernel/lib.rs @@ -6,7 +6,7 @@ //! usage by Rust code in the kernel and is shared by all of them. //! //! In other words, all the rest of the Rust code in the kernel (e.g. kernel -//! modules written in Rust) depends on [`core`], [`alloc`] and this crate. +//! modules written in Rust) depends on [`core`] and this crate. //! //! If you need a kernel C API that is not ported or wrapped yet here, then //! do so first instead of bypassing this crate. diff --git a/scripts/rustdoc_test_gen.rs b/scripts/rustdoc_test_gen.rs index 5ebd42ae4a3f..76aaa8329413 100644 --- a/scripts/rustdoc_test_gen.rs +++ b/scripts/rustdoc_test_gen.rs @@ -15,8 +15,8 @@ //! - Test code should be able to define functions and call them, without having to carry //! the context. //! -//! - Later on, we may want to be able to test non-kernel code (e.g. `core`, `alloc` or -//! third-party crates) which likely use the standard library `assert*!` macros. +//! - Later on, we may want to be able to test non-kernel code (e.g. `core` or third-party +//! crates) which likely use the standard library `assert*!` macros. //! //! For this reason, instead of the passed context, `kunit_get_current_test()` is used instead //! (i.e. `current->kunit_test`). -- cgit v1.2.3 From b5e3956535466187657563b754ba0f1da8626c7f Mon Sep 17 00:00:00 2001 From: Inochi Amaoto <inochiama@gmail.com> Date: Thu, 6 Mar 2025 14:39:51 +0800 Subject: kbuild: install-extmod-build: Fix build when specifying KBUILD_OUTPUT Since commit 5f73e7d0386d ("kbuild: refactor cross-compiling linux-headers package"), the linux-headers pacman package fails to build when "O=" is set. The build system complains: /mnt/chroot/linux/scripts/Makefile.build:41: mnt/chroots/linux-mainline/pacman/linux-upstream/pkg/linux-upstream-headers/usr//lib/modules/6.14.0-rc3-00350-g771dba31fffc/build/scripts/Makefile: No such file or directory This is because the "srcroot" variable is set to "." and the "build" variable is set to the absolute path. This makes the "src" variables point to wrong directory. Change the "build" variable to a relative path to "." to fix build. Fixes: 5f73e7d0386d ("kbuild: refactor cross-compiling linux-headers package") Signed-off-by: Inochi Amaoto <inochiama@gmail.com> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> --- scripts/package/install-extmod-build | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'scripts') diff --git a/scripts/package/install-extmod-build b/scripts/package/install-extmod-build index 2966473b4660..b96538787f3d 100755 --- a/scripts/package/install-extmod-build +++ b/scripts/package/install-extmod-build @@ -63,7 +63,7 @@ if [ "${CC}" != "${HOSTCC}" ]; then # Clear VPATH and srcroot because the source files reside in the output # directory. # shellcheck disable=SC2016 # $(MAKE) and $(build) will be expanded by Make - "${MAKE}" run-command KBUILD_RUN_COMMAND='+$(MAKE) HOSTCC='"${CC}"' VPATH= srcroot=. $(build)='"${destdir}"/scripts + "${MAKE}" run-command KBUILD_RUN_COMMAND='+$(MAKE) HOSTCC='"${CC}"' VPATH= srcroot=. $(build)='"$(realpath --relative-base=. "${destdir}")"/scripts rm -f "${destdir}/scripts/Kbuild" fi -- cgit v1.2.3 From 6ae0042f4d3f331e841495eb0a3d51598e593ec2 Mon Sep 17 00:00:00 2001 From: Tim Schumacher <tim.schumacher1@huawei.com> Date: Fri, 7 Mar 2025 10:56:43 +0100 Subject: selinux: Chain up tool resolving errors in install_policy.sh Subshell evaluations are not exempt from errexit, so if a command is not available, `which` will fail and exit the script as a whole. This causes the helpful error messages to not be printed if they are tacked on using a `$?` comparison. Resolve the issue by using chains of logical operators, which are not subject to the effects of errexit. Fixes: e37c1877ba5b1 ("scripts/selinux: modernize mdp") Signed-off-by: Tim Schumacher <tim.schumacher1@huawei.com> Signed-off-by: Paul Moore <paul@paul-moore.com> --- scripts/selinux/install_policy.sh | 15 ++++++--------- 1 file changed, 6 insertions(+), 9 deletions(-) (limited to 'scripts') diff --git a/scripts/selinux/install_policy.sh b/scripts/selinux/install_policy.sh index 24086793b0d8..db40237e60ce 100755 --- a/scripts/selinux/install_policy.sh +++ b/scripts/selinux/install_policy.sh @@ -6,27 +6,24 @@ if [ `id -u` -ne 0 ]; then exit 1 fi -SF=`which setfiles` -if [ $? -eq 1 ]; then +SF=`which setfiles` || { echo "Could not find setfiles" echo "Do you have policycoreutils installed?" exit 1 -fi +} -CP=`which checkpolicy` -if [ $? -eq 1 ]; then +CP=`which checkpolicy` || { echo "Could not find checkpolicy" echo "Do you have checkpolicy installed?" exit 1 -fi +} VERS=`$CP -V | awk '{print $1}'` -ENABLED=`which selinuxenabled` -if [ $? -eq 1 ]; then +ENABLED=`which selinuxenabled` || { echo "Could not find selinuxenabled" echo "Do you have libselinux-utils installed?" exit 1 -fi +} if selinuxenabled; then echo "SELinux is already enabled" -- cgit v1.2.3 From ed2b548f1017586c44f50654ef9febb42d491f31 Mon Sep 17 00:00:00 2001 From: Kees Cook <kees@kernel.org> Date: Thu, 6 Mar 2025 20:19:09 -0800 Subject: ubsan/overflow: Rework integer overflow sanitizer option to turn on everything Since we're going to approach integer overflow mitigation a type at a time, we need to enable all of the associated sanitizers, and then opt into types one at a time. Rename the existing "signed wrap" sanitizer to just the entire topic area: "integer wrap". Enable the implicit integer truncation sanitizers, with required callbacks and tests. Notably, this requires features (currently) only available in Clang, so we can depend on the cc-option tests to determine availability instead of doing version tests. Link: https://lore.kernel.org/r/20250307041914.937329-1-kees@kernel.org Signed-off-by: Kees Cook <kees@kernel.org> --- include/linux/compiler_types.h | 2 +- kernel/configs/hardening.config | 2 +- lib/Kconfig.ubsan | 23 +++++++++++------------ lib/test_ubsan.c | 18 ++++++++++++++---- lib/ubsan.c | 28 ++++++++++++++++++++++++++-- lib/ubsan.h | 8 ++++++++ scripts/Makefile.lib | 4 ++-- scripts/Makefile.ubsan | 8 ++++++-- 8 files changed, 69 insertions(+), 24 deletions(-) (limited to 'scripts') diff --git a/include/linux/compiler_types.h b/include/linux/compiler_types.h index f59393464ea7..4ad3e900bc3d 100644 --- a/include/linux/compiler_types.h +++ b/include/linux/compiler_types.h @@ -360,7 +360,7 @@ struct ftrace_likely_data { #endif /* Do not trap wrapping arithmetic within an annotated function. */ -#ifdef CONFIG_UBSAN_SIGNED_WRAP +#ifdef CONFIG_UBSAN_INTEGER_WRAP # define __signed_wrap __attribute__((no_sanitize("signed-integer-overflow"))) #else # define __signed_wrap diff --git a/kernel/configs/hardening.config b/kernel/configs/hardening.config index 3fabb8f55ef6..dd7c32fb5ac1 100644 --- a/kernel/configs/hardening.config +++ b/kernel/configs/hardening.config @@ -46,7 +46,7 @@ CONFIG_UBSAN_BOUNDS=y # CONFIG_UBSAN_SHIFT is not set # CONFIG_UBSAN_DIV_ZERO is not set # CONFIG_UBSAN_UNREACHABLE is not set -# CONFIG_UBSAN_SIGNED_WRAP is not set +# CONFIG_UBSAN_INTEGER_WRAP is not set # CONFIG_UBSAN_BOOL is not set # CONFIG_UBSAN_ENUM is not set # CONFIG_UBSAN_ALIGNMENT is not set diff --git a/lib/Kconfig.ubsan b/lib/Kconfig.ubsan index 1d4aa7a83b3a..63e5622010e0 100644 --- a/lib/Kconfig.ubsan +++ b/lib/Kconfig.ubsan @@ -116,21 +116,20 @@ config UBSAN_UNREACHABLE This option enables -fsanitize=unreachable which checks for control flow reaching an expected-to-be-unreachable position. -config UBSAN_SIGNED_WRAP - bool "Perform checking for signed arithmetic wrap-around" +config UBSAN_INTEGER_WRAP + bool "Perform checking for integer arithmetic wrap-around" default UBSAN depends on !COMPILE_TEST - # The no_sanitize attribute was introduced in GCC with version 8. - depends on !CC_IS_GCC || GCC_VERSION >= 80000 depends on $(cc-option,-fsanitize=signed-integer-overflow) - help - This option enables -fsanitize=signed-integer-overflow which checks - for wrap-around of any arithmetic operations with signed integers. - This currently performs nearly no instrumentation due to the - kernel's use of -fno-strict-overflow which converts all would-be - arithmetic undefined behavior into wrap-around arithmetic. Future - sanitizer versions will allow for wrap-around checking (rather than - exclusively undefined behavior). + depends on $(cc-option,-fsanitize=unsigned-integer-overflow) + depends on $(cc-option,-fsanitize=implicit-signed-integer-truncation) + depends on $(cc-option,-fsanitize=implicit-unsigned-integer-truncation) + help + This option enables all of the sanitizers involved in integer overflow + (wrap-around) mitigation: signed-integer-overflow, unsigned-integer-overflow, + implicit-signed-integer-truncation, and implicit-unsigned-integer-truncation. + This is currently limited only to the size_t type while testing and + compiler development continues. config UBSAN_BOOL bool "Perform checking for non-boolean values used as boolean" diff --git a/lib/test_ubsan.c b/lib/test_ubsan.c index 5d7b10e98610..8772e5edaa4f 100644 --- a/lib/test_ubsan.c +++ b/lib/test_ubsan.c @@ -15,7 +15,7 @@ static void test_ubsan_add_overflow(void) { volatile int val = INT_MAX; - UBSAN_TEST(CONFIG_UBSAN_SIGNED_WRAP); + UBSAN_TEST(CONFIG_UBSAN_INTEGER_WRAP); val += 2; } @@ -24,7 +24,7 @@ static void test_ubsan_sub_overflow(void) volatile int val = INT_MIN; volatile int val2 = 2; - UBSAN_TEST(CONFIG_UBSAN_SIGNED_WRAP); + UBSAN_TEST(CONFIG_UBSAN_INTEGER_WRAP); val -= val2; } @@ -32,7 +32,7 @@ static void test_ubsan_mul_overflow(void) { volatile int val = INT_MAX / 2; - UBSAN_TEST(CONFIG_UBSAN_SIGNED_WRAP); + UBSAN_TEST(CONFIG_UBSAN_INTEGER_WRAP); val *= 3; } @@ -40,7 +40,7 @@ static void test_ubsan_negate_overflow(void) { volatile int val = INT_MIN; - UBSAN_TEST(CONFIG_UBSAN_SIGNED_WRAP); + UBSAN_TEST(CONFIG_UBSAN_INTEGER_WRAP); val = -val; } @@ -53,6 +53,15 @@ static void test_ubsan_divrem_overflow(void) val /= val2; } +static void test_ubsan_truncate_signed(void) +{ + volatile long val = LONG_MAX; + volatile int val2 = 0; + + UBSAN_TEST(CONFIG_UBSAN_INTEGER_WRAP); + val2 = val; +} + static void test_ubsan_shift_out_of_bounds(void) { volatile int neg = -1, wrap = 4; @@ -127,6 +136,7 @@ static const test_ubsan_fp test_ubsan_array[] = { test_ubsan_sub_overflow, test_ubsan_mul_overflow, test_ubsan_negate_overflow, + test_ubsan_truncate_signed, test_ubsan_shift_out_of_bounds, test_ubsan_out_of_bounds, test_ubsan_load_invalid_value, diff --git a/lib/ubsan.c b/lib/ubsan.c index a1c983d148f1..cdc1d31c3821 100644 --- a/lib/ubsan.c +++ b/lib/ubsan.c @@ -44,7 +44,7 @@ const char *report_ubsan_failure(struct pt_regs *regs, u32 check_type) case ubsan_shift_out_of_bounds: return "UBSAN: shift out of bounds"; #endif -#if defined(CONFIG_UBSAN_DIV_ZERO) || defined(CONFIG_UBSAN_SIGNED_WRAP) +#if defined(CONFIG_UBSAN_DIV_ZERO) || defined(CONFIG_UBSAN_INTEGER_WRAP) /* * SanitizerKind::IntegerDivideByZero and * SanitizerKind::SignedIntegerOverflow emit @@ -79,7 +79,7 @@ const char *report_ubsan_failure(struct pt_regs *regs, u32 check_type) case ubsan_type_mismatch: return "UBSAN: type mismatch"; #endif -#ifdef CONFIG_UBSAN_SIGNED_WRAP +#ifdef CONFIG_UBSAN_INTEGER_WRAP /* * SanitizerKind::SignedIntegerOverflow emits * SanitizerHandler::AddOverflow, SanitizerHandler::SubOverflow, @@ -303,6 +303,30 @@ void __ubsan_handle_negate_overflow(void *_data, void *old_val) } EXPORT_SYMBOL(__ubsan_handle_negate_overflow); +void __ubsan_handle_implicit_conversion(void *_data, void *from_val, void *to_val) +{ + struct implicit_conversion_data *data = _data; + char from_val_str[VALUE_LENGTH]; + char to_val_str[VALUE_LENGTH]; + + if (suppress_report(&data->location)) + return; + + val_to_string(from_val_str, sizeof(from_val_str), data->from_type, from_val); + val_to_string(to_val_str, sizeof(to_val_str), data->to_type, to_val); + + ubsan_prologue(&data->location, "implicit-conversion"); + + pr_err("cannot represent %s value %s during %s %s, truncated to %s\n", + data->from_type->type_name, + from_val_str, + type_check_kinds[data->type_check_kind], + data->to_type->type_name, + to_val_str); + + ubsan_epilogue(); +} +EXPORT_SYMBOL(__ubsan_handle_implicit_conversion); void __ubsan_handle_divrem_overflow(void *_data, void *lhs, void *rhs) { diff --git a/lib/ubsan.h b/lib/ubsan.h index 07e37d4429b4..b37e22374e77 100644 --- a/lib/ubsan.h +++ b/lib/ubsan.h @@ -62,6 +62,13 @@ struct overflow_data { struct type_descriptor *type; }; +struct implicit_conversion_data { + struct source_location location; + struct type_descriptor *from_type; + struct type_descriptor *to_type; + unsigned char type_check_kind; +}; + struct type_mismatch_data { struct source_location location; struct type_descriptor *type; @@ -142,6 +149,7 @@ void ubsan_linkage __ubsan_handle_sub_overflow(void *data, void *lhs, void *rhs) void ubsan_linkage __ubsan_handle_mul_overflow(void *data, void *lhs, void *rhs); void ubsan_linkage __ubsan_handle_negate_overflow(void *_data, void *old_val); void ubsan_linkage __ubsan_handle_divrem_overflow(void *_data, void *lhs, void *rhs); +void ubsan_linkage __ubsan_handle_implicit_conversion(void *_data, void *lhs, void *rhs); void ubsan_linkage __ubsan_handle_type_mismatch(struct type_mismatch_data *data, void *ptr); void ubsan_linkage __ubsan_handle_type_mismatch_v1(void *_data, void *ptr); void ubsan_linkage __ubsan_handle_out_of_bounds(void *_data, void *index); diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib index cad20f0e66ee..981d14ef9db2 100644 --- a/scripts/Makefile.lib +++ b/scripts/Makefile.lib @@ -166,8 +166,8 @@ _c_flags += $(if $(patsubst n%,, \ $(UBSAN_SANITIZE_$(target-stem).o)$(UBSAN_SANITIZE)$(is-kernel-object)), \ $(CFLAGS_UBSAN)) _c_flags += $(if $(patsubst n%,, \ - $(UBSAN_SIGNED_WRAP_$(target-stem).o)$(UBSAN_SANITIZE_$(target-stem).o)$(UBSAN_SIGNED_WRAP)$(UBSAN_SANITIZE)$(is-kernel-object)), \ - $(CFLAGS_UBSAN_SIGNED_WRAP)) + $(UBSAN_INTEGER_WRAP_$(target-stem).o)$(UBSAN_SANITIZE_$(target-stem).o)$(UBSAN_INTEGER_WRAP)$(UBSAN_SANITIZE)$(is-kernel-object)), \ + $(CFLAGS_UBSAN_INTEGER_WRAP)) endif ifeq ($(CONFIG_KCOV),y) diff --git a/scripts/Makefile.ubsan b/scripts/Makefile.ubsan index b2d3b273b802..4fad9afed24c 100644 --- a/scripts/Makefile.ubsan +++ b/scripts/Makefile.ubsan @@ -14,5 +14,9 @@ ubsan-cflags-$(CONFIG_UBSAN_TRAP) += $(call cc-option,-fsanitize-trap=undefined export CFLAGS_UBSAN := $(ubsan-cflags-y) -ubsan-signed-wrap-cflags-$(CONFIG_UBSAN_SIGNED_WRAP) += -fsanitize=signed-integer-overflow -export CFLAGS_UBSAN_SIGNED_WRAP := $(ubsan-signed-wrap-cflags-y) +ubsan-integer-wrap-cflags-$(CONFIG_UBSAN_INTEGER_WRAP) += \ + -fsanitize=signed-integer-overflow \ + -fsanitize=unsigned-integer-overflow \ + -fsanitize=implicit-signed-integer-truncation \ + -fsanitize=implicit-unsigned-integer-truncation +export CFLAGS_UBSAN_INTEGER_WRAP := $(ubsan-integer-wrap-cflags-y) -- cgit v1.2.3 From 272a767063a6856cd1e18bb951d2be4f047b9858 Mon Sep 17 00:00:00 2001 From: Kees Cook <kees@kernel.org> Date: Thu, 6 Mar 2025 20:19:10 -0800 Subject: ubsan/overflow: Enable pattern exclusions To make integer wrap-around mitigation actually useful, the associated sanitizers must not instrument cases where the wrap-around is explicitly defined (e.g. "-2UL"), being tested for (e.g. "if (a + b < a)"), or where it has no impact on code flow (e.g. "while (var--)"). Enable pattern exclusions for the integer wrap sanitizers. Reviewed-by: Justin Stitt <justinstitt@google.com> Link: https://lore.kernel.org/r/20250307041914.937329-2-kees@kernel.org Signed-off-by: Kees Cook <kees@kernel.org> --- lib/Kconfig.ubsan | 1 + scripts/Makefile.ubsan | 1 + 2 files changed, 2 insertions(+) (limited to 'scripts') diff --git a/lib/Kconfig.ubsan b/lib/Kconfig.ubsan index 63e5622010e0..888c2e72c586 100644 --- a/lib/Kconfig.ubsan +++ b/lib/Kconfig.ubsan @@ -120,6 +120,7 @@ config UBSAN_INTEGER_WRAP bool "Perform checking for integer arithmetic wrap-around" default UBSAN depends on !COMPILE_TEST + depends on $(cc-option,-fsanitize-undefined-ignore-overflow-pattern=all) depends on $(cc-option,-fsanitize=signed-integer-overflow) depends on $(cc-option,-fsanitize=unsigned-integer-overflow) depends on $(cc-option,-fsanitize=implicit-signed-integer-truncation) diff --git a/scripts/Makefile.ubsan b/scripts/Makefile.ubsan index 4fad9afed24c..233379c193a7 100644 --- a/scripts/Makefile.ubsan +++ b/scripts/Makefile.ubsan @@ -15,6 +15,7 @@ ubsan-cflags-$(CONFIG_UBSAN_TRAP) += $(call cc-option,-fsanitize-trap=undefined export CFLAGS_UBSAN := $(ubsan-cflags-y) ubsan-integer-wrap-cflags-$(CONFIG_UBSAN_INTEGER_WRAP) += \ + -fsanitize-undefined-ignore-overflow-pattern=all \ -fsanitize=signed-integer-overflow \ -fsanitize=unsigned-integer-overflow \ -fsanitize=implicit-signed-integer-truncation \ -- cgit v1.2.3 From 47f4af43e7c0cf702d6a6321542f0c0d9c4216e3 Mon Sep 17 00:00:00 2001 From: Kees Cook <kees@kernel.org> Date: Thu, 6 Mar 2025 20:19:11 -0800 Subject: ubsan/overflow: Enable ignorelist parsing and add type filter Limit integer wrap-around mitigation to only the "size_t" type (for now). Notably this covers all special functions/builtins that return "size_t", like sizeof(). This remains an experimental feature and is likely to be replaced with type annotations. Reviewed-by: Justin Stitt <justinstitt@google.com> Link: https://lore.kernel.org/r/20250307041914.937329-3-kees@kernel.org Signed-off-by: Kees Cook <kees@kernel.org> --- lib/Kconfig.ubsan | 1 + scripts/Makefile.ubsan | 3 ++- scripts/integer-wrap-ignore.scl | 3 +++ 3 files changed, 6 insertions(+), 1 deletion(-) create mode 100644 scripts/integer-wrap-ignore.scl (limited to 'scripts') diff --git a/lib/Kconfig.ubsan b/lib/Kconfig.ubsan index 888c2e72c586..4216b3a4ff21 100644 --- a/lib/Kconfig.ubsan +++ b/lib/Kconfig.ubsan @@ -125,6 +125,7 @@ config UBSAN_INTEGER_WRAP depends on $(cc-option,-fsanitize=unsigned-integer-overflow) depends on $(cc-option,-fsanitize=implicit-signed-integer-truncation) depends on $(cc-option,-fsanitize=implicit-unsigned-integer-truncation) + depends on $(cc-option,-fsanitize-ignorelist=/dev/null) help This option enables all of the sanitizers involved in integer overflow (wrap-around) mitigation: signed-integer-overflow, unsigned-integer-overflow, diff --git a/scripts/Makefile.ubsan b/scripts/Makefile.ubsan index 233379c193a7..9e35198edbf0 100644 --- a/scripts/Makefile.ubsan +++ b/scripts/Makefile.ubsan @@ -19,5 +19,6 @@ ubsan-integer-wrap-cflags-$(CONFIG_UBSAN_INTEGER_WRAP) += \ -fsanitize=signed-integer-overflow \ -fsanitize=unsigned-integer-overflow \ -fsanitize=implicit-signed-integer-truncation \ - -fsanitize=implicit-unsigned-integer-truncation + -fsanitize=implicit-unsigned-integer-truncation \ + -fsanitize-ignorelist=$(srctree)/scripts/integer-wrap-ignore.scl export CFLAGS_UBSAN_INTEGER_WRAP := $(ubsan-integer-wrap-cflags-y) diff --git a/scripts/integer-wrap-ignore.scl b/scripts/integer-wrap-ignore.scl new file mode 100644 index 000000000000..431c3053a4a2 --- /dev/null +++ b/scripts/integer-wrap-ignore.scl @@ -0,0 +1,3 @@ +[{unsigned-integer-overflow,signed-integer-overflow,implicit-signed-integer-truncation,implicit-unsigned-integer-truncation}] +type:* +type:size_t=sanitize -- cgit v1.2.3 From fbefae55991f688f5f1615af30fe64823076b072 Mon Sep 17 00:00:00 2001 From: Guillaume Gomez <guillaume1.gomez@gmail.com> Date: Fri, 28 Feb 2025 18:05:31 +0100 Subject: scripts: rust: mention file name in error messages Improve two error messages in the script by mentioning the doctest file path from which the doctest was generated from. This will allow, in case the conversion fails, to get directly the file name triggering the issue, making the bug fixing process faster. Signed-off-by: Guillaume Gomez <guillaume1.gomez@gmail.com> Link: https://lore.kernel.org/r/20250228170530.950268-2-guillaume1.gomez@gmail.com [ Reworded and removed an unneeded added parameter comma. - Miguel ] Signed-off-by: Miguel Ojeda <ojeda@kernel.org> --- scripts/rustdoc_test_gen.rs | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) (limited to 'scripts') diff --git a/scripts/rustdoc_test_gen.rs b/scripts/rustdoc_test_gen.rs index 5ebd42ae4a3f..036635fb1621 100644 --- a/scripts/rustdoc_test_gen.rs +++ b/scripts/rustdoc_test_gen.rs @@ -87,8 +87,8 @@ fn find_real_path<'a>(srctree: &Path, valid_paths: &'a mut Vec<PathBuf>, file: & assert!( valid_paths.len() > 0, - "No path candidates found. This is likely a bug in the build system, or some files went \ - away while compiling." + "No path candidates found for `{file}`. This is likely a bug in the build system, or some \ + files went away while compiling." ); if valid_paths.len() > 1 { @@ -97,8 +97,8 @@ fn find_real_path<'a>(srctree: &Path, valid_paths: &'a mut Vec<PathBuf>, file: & eprintln!(" {path:?}"); } panic!( - "Several path candidates found, please resolve the ambiguity by renaming a file or \ - folder." + "Several path candidates found for `{file}`, please resolve the ambiguity by renaming \ + a file or folder." ); } -- cgit v1.2.3 From bbe2610bc5ada51418a4191e799cfb4577302a31 Mon Sep 17 00:00:00 2001 From: Eric Biggers <ebiggers@google.com> Date: Sun, 16 Feb 2025 14:55:27 -0800 Subject: riscv/crc: add "template" for Zbc optimized CRC functions MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Add a "template" crc-clmul-template.h that can generate RISC-V Zbc optimized CRC functions. Each generated CRC function is parameterized by CRC length and bit order, and it accepts a pointer to the constants struct required for the specific CRC polynomial desired. Update gen-crc-consts.py to support generating the needed constants structs. This makes it possible to easily wire up a Zbc optimized implementation of almost any CRC. The design generally follows what I did for x86, but it is simplified by using RISC-V's scalar carryless multiplication Zbc, which has no equivalent on x86. RISC-V's clmulr instruction is also helpful. A potential switch to Zvbc (or support for Zvbc alongside Zbc) is left for future work. For long messages Zvbc should be fastest, but it would need to be shown to be worthwhile over just using Zbc which is significantly more convenient to use, especially in the kernel context. Compared to the existing Zbc-optimized CRC32 code and the earlier proposed Zbc-optimized CRC-T10DIF code (https://lore.kernel.org/r/20250211071101.181652-1-zhihang.shao.iscas@gmail.com), this submission deduplicates the code among CRC variants and is significantly more optimized. It uses "folding" to take better advantage of instruction-level parallelism (to a more limited extent than x86 for now, but it could be extended to more), it reworks the Barrett reduction to eliminate unnecessary instructions, and it documents all the math used and makes all the constants reproducible. Tested-by: Björn Töpel <bjorn@rivosinc.com> Acked-by: Alexandre Ghiti <alexghiti@rivosinc.com> Link: https://lore.kernel.org/r/20250216225530.306980-2-ebiggers@kernel.org Signed-off-by: Eric Biggers <ebiggers@google.com> --- arch/riscv/lib/crc-clmul-template.h | 265 ++++++++++++++++++++++++++++++++++++ scripts/gen-crc-consts.py | 55 +++++++- 2 files changed, 319 insertions(+), 1 deletion(-) create mode 100644 arch/riscv/lib/crc-clmul-template.h (limited to 'scripts') diff --git a/arch/riscv/lib/crc-clmul-template.h b/arch/riscv/lib/crc-clmul-template.h new file mode 100644 index 000000000000..77187e7f1762 --- /dev/null +++ b/arch/riscv/lib/crc-clmul-template.h @@ -0,0 +1,265 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ +/* Copyright 2025 Google LLC */ + +/* + * This file is a "template" that generates a CRC function optimized using the + * RISC-V Zbc (scalar carryless multiplication) extension. The includer of this + * file must define the following parameters to specify the type of CRC: + * + * crc_t: the data type of the CRC, e.g. u32 for a 32-bit CRC + * LSB_CRC: 0 for a msb (most-significant-bit) first CRC, i.e. natural + * mapping between bits and polynomial coefficients + * 1 for a lsb (least-significant-bit) first CRC, i.e. reflected + * mapping between bits and polynomial coefficients + */ + +#include <asm/byteorder.h> +#include <linux/minmax.h> + +#define CRC_BITS (8 * sizeof(crc_t)) /* a.k.a. 'n' */ + +static inline unsigned long clmul(unsigned long a, unsigned long b) +{ + unsigned long res; + + asm(".option push\n" + ".option arch,+zbc\n" + "clmul %0, %1, %2\n" + ".option pop\n" + : "=r" (res) : "r" (a), "r" (b)); + return res; +} + +static inline unsigned long clmulh(unsigned long a, unsigned long b) +{ + unsigned long res; + + asm(".option push\n" + ".option arch,+zbc\n" + "clmulh %0, %1, %2\n" + ".option pop\n" + : "=r" (res) : "r" (a), "r" (b)); + return res; +} + +static inline unsigned long clmulr(unsigned long a, unsigned long b) +{ + unsigned long res; + + asm(".option push\n" + ".option arch,+zbc\n" + "clmulr %0, %1, %2\n" + ".option pop\n" + : "=r" (res) : "r" (a), "r" (b)); + return res; +} + +/* + * crc_load_long() loads one "unsigned long" of aligned data bytes, producing a + * polynomial whose bit order matches the CRC's bit order. + */ +#ifdef CONFIG_64BIT +# if LSB_CRC +# define crc_load_long(x) le64_to_cpup(x) +# else +# define crc_load_long(x) be64_to_cpup(x) +# endif +#else +# if LSB_CRC +# define crc_load_long(x) le32_to_cpup(x) +# else +# define crc_load_long(x) be32_to_cpup(x) +# endif +#endif + +/* XOR @crc into the end of @msgpoly that represents the high-order terms. */ +static inline unsigned long +crc_clmul_prep(crc_t crc, unsigned long msgpoly) +{ +#if LSB_CRC + return msgpoly ^ crc; +#else + return msgpoly ^ ((unsigned long)crc << (BITS_PER_LONG - CRC_BITS)); +#endif +} + +/* + * Multiply the long-sized @msgpoly by x^n (a.k.a. x^CRC_BITS) and reduce it + * modulo the generator polynomial G. This gives the CRC of @msgpoly. + */ +static inline crc_t +crc_clmul_long(unsigned long msgpoly, const struct crc_clmul_consts *consts) +{ + unsigned long tmp; + + /* + * First step of Barrett reduction with integrated multiplication by + * x^n: calculate floor((msgpoly * x^n) / G). This is the value by + * which G needs to be multiplied to cancel out the x^n and higher terms + * of msgpoly * x^n. Do it using the following formula: + * + * msb-first: + * floor((msgpoly * floor(x^(BITS_PER_LONG-1+n) / G)) / x^(BITS_PER_LONG-1)) + * lsb-first: + * floor((msgpoly * floor(x^(BITS_PER_LONG-1+n) / G) * x) / x^BITS_PER_LONG) + * + * barrett_reduction_const_1 contains floor(x^(BITS_PER_LONG-1+n) / G), + * which fits a long exactly. Using any lower power of x there would + * not carry enough precision through the calculation, while using any + * higher power of x would require extra instructions to handle a wider + * multiplication. In the msb-first case, using this power of x results + * in needing a floored division by x^(BITS_PER_LONG-1), which matches + * what clmulr produces. In the lsb-first case, a factor of x gets + * implicitly introduced by each carryless multiplication (shown as + * '* x' above), and the floored division instead needs to be by + * x^BITS_PER_LONG which matches what clmul produces. + */ +#if LSB_CRC + tmp = clmul(msgpoly, consts->barrett_reduction_const_1); +#else + tmp = clmulr(msgpoly, consts->barrett_reduction_const_1); +#endif + + /* + * Second step of Barrett reduction: + * + * crc := (msgpoly * x^n) + (G * floor((msgpoly * x^n) / G)) + * + * This reduces (msgpoly * x^n) modulo G by adding the appropriate + * multiple of G to it. The result uses only the x^0..x^(n-1) terms. + * HOWEVER, since the unreduced value (msgpoly * x^n) is zero in those + * terms in the first place, it is more efficient to do the equivalent: + * + * crc := ((G - x^n) * floor((msgpoly * x^n) / G)) mod x^n + * + * In the lsb-first case further modify it to the following which avoids + * a shift, as the crc ends up in the physically low n bits from clmulr: + * + * product := ((G - x^n) * x^(BITS_PER_LONG - n)) * floor((msgpoly * x^n) / G) * x + * crc := floor(product / x^(BITS_PER_LONG + 1 - n)) mod x^n + * + * barrett_reduction_const_2 contains the constant multiplier (G - x^n) + * or (G - x^n) * x^(BITS_PER_LONG - n) from the formulas above. The + * cast of the result to crc_t is essential, as it applies the mod x^n! + */ +#if LSB_CRC + return clmulr(tmp, consts->barrett_reduction_const_2); +#else + return clmul(tmp, consts->barrett_reduction_const_2); +#endif +} + +/* Update @crc with the data from @msgpoly. */ +static inline crc_t +crc_clmul_update_long(crc_t crc, unsigned long msgpoly, + const struct crc_clmul_consts *consts) +{ + return crc_clmul_long(crc_clmul_prep(crc, msgpoly), consts); +} + +/* Update @crc with 1 <= @len < sizeof(unsigned long) bytes of data. */ +static inline crc_t +crc_clmul_update_partial(crc_t crc, const u8 *p, size_t len, + const struct crc_clmul_consts *consts) +{ + unsigned long msgpoly; + size_t i; + +#if LSB_CRC + msgpoly = (unsigned long)p[0] << (BITS_PER_LONG - 8); + for (i = 1; i < len; i++) + msgpoly = (msgpoly >> 8) ^ ((unsigned long)p[i] << (BITS_PER_LONG - 8)); +#else + msgpoly = p[0]; + for (i = 1; i < len; i++) + msgpoly = (msgpoly << 8) ^ p[i]; +#endif + + if (len >= sizeof(crc_t)) { + #if LSB_CRC + msgpoly ^= (unsigned long)crc << (BITS_PER_LONG - 8*len); + #else + msgpoly ^= (unsigned long)crc << (8*len - CRC_BITS); + #endif + return crc_clmul_long(msgpoly, consts); + } +#if LSB_CRC + msgpoly ^= (unsigned long)crc << (BITS_PER_LONG - 8*len); + return crc_clmul_long(msgpoly, consts) ^ (crc >> (8*len)); +#else + msgpoly ^= crc >> (CRC_BITS - 8*len); + return crc_clmul_long(msgpoly, consts) ^ (crc << (8*len)); +#endif +} + +static inline crc_t +crc_clmul(crc_t crc, const void *p, size_t len, + const struct crc_clmul_consts *consts) +{ + size_t align; + + /* This implementation assumes that the CRC fits in an unsigned long. */ + BUILD_BUG_ON(sizeof(crc_t) > sizeof(unsigned long)); + + /* If the buffer is not long-aligned, align it. */ + align = (unsigned long)p % sizeof(unsigned long); + if (align && len) { + align = min(sizeof(unsigned long) - align, len); + crc = crc_clmul_update_partial(crc, p, align, consts); + p += align; + len -= align; + } + + if (len >= 4 * sizeof(unsigned long)) { + unsigned long m0, m1; + + m0 = crc_clmul_prep(crc, crc_load_long(p)); + m1 = crc_load_long(p + sizeof(unsigned long)); + p += 2 * sizeof(unsigned long); + len -= 2 * sizeof(unsigned long); + /* + * Main loop. Each iteration starts with a message polynomial + * (x^BITS_PER_LONG)*m0 + m1, then logically extends it by two + * more longs of data to form x^(3*BITS_PER_LONG)*m0 + + * x^(2*BITS_PER_LONG)*m1 + x^BITS_PER_LONG*m2 + m3, then + * "folds" that back into a congruent (modulo G) value that uses + * just m0 and m1 again. This is done by multiplying m0 by the + * precomputed constant (x^(3*BITS_PER_LONG) mod G) and m1 by + * the precomputed constant (x^(2*BITS_PER_LONG) mod G), then + * adding the results to m2 and m3 as appropriate. Each such + * multiplication produces a result twice the length of a long, + * which in RISC-V is two instructions clmul and clmulh. + * + * This could be changed to fold across more than 2 longs at a + * time if there is a CPU that can take advantage of it. + */ + do { + unsigned long p0, p1, p2, p3; + + p0 = clmulh(m0, consts->fold_across_2_longs_const_hi); + p1 = clmul(m0, consts->fold_across_2_longs_const_hi); + p2 = clmulh(m1, consts->fold_across_2_longs_const_lo); + p3 = clmul(m1, consts->fold_across_2_longs_const_lo); + m0 = (LSB_CRC ? p1 ^ p3 : p0 ^ p2) ^ crc_load_long(p); + m1 = (LSB_CRC ? p0 ^ p2 : p1 ^ p3) ^ + crc_load_long(p + sizeof(unsigned long)); + + p += 2 * sizeof(unsigned long); + len -= 2 * sizeof(unsigned long); + } while (len >= 2 * sizeof(unsigned long)); + + crc = crc_clmul_long(m0, consts); + crc = crc_clmul_update_long(crc, m1, consts); + } + + while (len >= sizeof(unsigned long)) { + crc = crc_clmul_update_long(crc, crc_load_long(p), consts); + p += sizeof(unsigned long); + len -= sizeof(unsigned long); + } + + if (len) + crc = crc_clmul_update_partial(crc, p, len, consts); + + return crc; +} diff --git a/scripts/gen-crc-consts.py b/scripts/gen-crc-consts.py index aa678a50897d..f9b44fc3a03f 100755 --- a/scripts/gen-crc-consts.py +++ b/scripts/gen-crc-consts.py @@ -105,6 +105,57 @@ def gen_slicebyN_tables(variants, n): print(f'\t{s}') print('};') +def print_riscv_const(v, bits_per_long, name, val, desc): + print(f'\t.{name} = {fmt_poly(v, val, bits_per_long)}, /* {desc} */') + +def do_gen_riscv_clmul_consts(v, bits_per_long): + (G, n, lsb) = (v.G, v.bits, v.lsb) + + pow_of_x = 3 * bits_per_long - (1 if lsb else 0) + print_riscv_const(v, bits_per_long, 'fold_across_2_longs_const_hi', + reduce(1 << pow_of_x, G), f'x^{pow_of_x} mod G') + pow_of_x = 2 * bits_per_long - (1 if lsb else 0) + print_riscv_const(v, bits_per_long, 'fold_across_2_longs_const_lo', + reduce(1 << pow_of_x, G), f'x^{pow_of_x} mod G') + + pow_of_x = bits_per_long - 1 + n + print_riscv_const(v, bits_per_long, 'barrett_reduction_const_1', + div(1 << pow_of_x, G), f'floor(x^{pow_of_x} / G)') + + val = G - (1 << n) + desc = f'G - x^{n}' + if lsb: + val <<= bits_per_long - n + desc = f'({desc}) * x^{bits_per_long - n}' + print_riscv_const(v, bits_per_long, 'barrett_reduction_const_2', val, desc) + +def gen_riscv_clmul_consts(variants): + print('') + print('struct crc_clmul_consts {'); + print('\tunsigned long fold_across_2_longs_const_hi;'); + print('\tunsigned long fold_across_2_longs_const_lo;'); + print('\tunsigned long barrett_reduction_const_1;'); + print('\tunsigned long barrett_reduction_const_2;'); + print('};'); + for v in variants: + print(''); + if v.bits > 32: + print_header(v, 'Constants') + print('#ifdef CONFIG_64BIT') + print(f'static const struct crc_clmul_consts {v.name}_consts __maybe_unused = {{') + do_gen_riscv_clmul_consts(v, 64) + print('};') + print('#endif') + else: + print_header(v, 'Constants') + print(f'static const struct crc_clmul_consts {v.name}_consts __maybe_unused = {{') + print('#ifdef CONFIG_64BIT') + do_gen_riscv_clmul_consts(v, 64) + print('#else') + do_gen_riscv_clmul_consts(v, 32) + print('#endif') + print('};') + # Generate constants for carryless multiplication based CRC computation. def gen_x86_pclmul_consts(variants): # These are the distances, in bits, to generate folding constants for. @@ -213,7 +264,7 @@ def parse_crc_variants(vars_string): if len(sys.argv) != 3: sys.stderr.write(f'Usage: {sys.argv[0]} CONSTS_TYPE[,CONSTS_TYPE]... CRC_VARIANT[,CRC_VARIANT]...\n') - sys.stderr.write(' CONSTS_TYPE can be sliceby[1-8] or x86_pclmul\n') + sys.stderr.write(' CONSTS_TYPE can be sliceby[1-8], riscv_clmul, or x86_pclmul\n') sys.stderr.write(' CRC_VARIANT is crc${num_bits}_${bit_order}_${generator_poly_as_hex}\n') sys.stderr.write(' E.g. crc16_msb_0x8bb7 or crc32_lsb_0xedb88320\n') sys.stderr.write(' Polynomial must use the given bit_order and exclude x^{num_bits}\n') @@ -232,6 +283,8 @@ variants = parse_crc_variants(sys.argv[2]) for consts_type in consts_types: if consts_type.startswith('sliceby'): gen_slicebyN_tables(variants, int(consts_type.removeprefix('sliceby'))) + elif consts_type == 'riscv_clmul': + gen_riscv_clmul_consts(variants) elif consts_type == 'x86_pclmul': gen_x86_pclmul_consts(variants) else: -- cgit v1.2.3 From 2e0f91aba507a3cb59f7a12fc3ea2b7d4d6675b7 Mon Sep 17 00:00:00 2001 From: Tamir Duberstein <tamird@gmail.com> Date: Mon, 10 Feb 2025 12:03:24 -0500 Subject: scripts: generate_rust_analyzer: add missing macros deps The macros crate has depended on std and proc_macro since its introduction in commit 1fbde52bde73 ("rust: add `macros` crate"). These dependencies were omitted from commit 8c4555ccc55c ("scripts: add `generate_rust_analyzer.py`") resulting in missing go-to-definition and autocomplete, and false-positive warnings emitted from rust-analyzer such as: [{ "resource": "/Users/tamird/src/linux/rust/macros/module.rs", "owner": "_generated_diagnostic_collection_name_#1", "code": { "value": "non_snake_case", "target": { "$mid": 1, "path": "/rustc/", "scheme": "https", "authority": "doc.rust-lang.org", "query": "search=non_snake_case" } }, "severity": 4, "message": "Variable `None` should have snake_case name, e.g. `none`", "source": "rust-analyzer", "startLineNumber": 123, "startColumn": 17, "endLineNumber": 123, "endColumn": 21 }] Add the missing dependencies to improve the developer experience. [ Fiona had a different approach (thanks!) at: https://lore.kernel.org/rust-for-linux/20241205115438.234221-1-me@kloenk.dev/ But Tamir and Fiona agreed to this one. - Miguel ] Fixes: 8c4555ccc55c ("scripts: add `generate_rust_analyzer.py`") Reviewed-by: Fiona Behrens <me@kloenk.dev> Diagnosed-by: Chayim Refael Friedman <chayimfr@gmail.com> Link: https://github.com/rust-lang/rust-analyzer/issues/17759#issuecomment-2646328275 Signed-off-by: Tamir Duberstein <tamird@gmail.com> Tested-by: Andreas Hindborg <a.hindborg@kernel.org> Link: https://lore.kernel.org/r/20250210-rust-analyzer-macros-core-dep-v3-1-45eb4836f218@gmail.com [ Removed `return`. Changed tag name. Added Link. Slightly reworded. - Miguel ] Signed-off-by: Miguel Ojeda <ojeda@kernel.org> --- scripts/generate_rust_analyzer.py | 30 +++++++++++++++++++++--------- 1 file changed, 21 insertions(+), 9 deletions(-) (limited to 'scripts') diff --git a/scripts/generate_rust_analyzer.py b/scripts/generate_rust_analyzer.py index aa8ea1a4dbe5..b40679a90843 100755 --- a/scripts/generate_rust_analyzer.py +++ b/scripts/generate_rust_analyzer.py @@ -57,14 +57,26 @@ def generate_crates(srctree, objtree, sysroot_src, external_src, cfgs): crates_indexes[display_name] = len(crates) crates.append(crate) - # First, the ones in `rust/` since they are a bit special. - append_crate( - "core", - sysroot_src / "core" / "src" / "lib.rs", - [], - cfg=crates_cfgs.get("core", []), - is_workspace_member=False, - ) + def append_sysroot_crate( + display_name, + deps, + cfg=[], + ): + append_crate( + display_name, + sysroot_src / display_name / "src" / "lib.rs", + deps, + cfg, + is_workspace_member=False, + ) + + # NB: sysroot crates reexport items from one another so setting up our transitive dependencies + # here is important for ensuring that rust-analyzer can resolve symbols. The sources of truth + # for this dependency graph are `(sysroot_src / crate / "Cargo.toml" for crate in crates)`. + append_sysroot_crate("core", [], cfg=crates_cfgs.get("core", [])) + append_sysroot_crate("alloc", ["core"]) + append_sysroot_crate("std", ["alloc", "core"]) + append_sysroot_crate("proc_macro", ["core", "std"]) append_crate( "compiler_builtins", @@ -75,7 +87,7 @@ def generate_crates(srctree, objtree, sysroot_src, external_src, cfgs): append_crate( "macros", srctree / "rust" / "macros" / "lib.rs", - [], + ["std", "proc_macro"], is_proc_macro=True, ) -- cgit v1.2.3 From d1f928052439cad028438a8b8b34c1f01bc06068 Mon Sep 17 00:00:00 2001 From: Tamir Duberstein <tamird@gmail.com> Date: Mon, 10 Feb 2025 13:04:16 -0500 Subject: scripts: generate_rust_analyzer: add missing include_dirs Commit 8c4555ccc55c ("scripts: add `generate_rust_analyzer.py`") specified OBJTREE for the bindings crate, and `source.include_dirs` for the kernel crate, likely in an attempt to support out-of-source builds for those crates where the generated files reside in `objtree` rather than `srctree`. This was insufficient because both bits of configuration are required for each crate; the result is that rust-analyzer is unable to resolve generated files for either crate in an out-of-source build. [ Originally we were not using `OBJTREE` in the `kernel` crate, but we did pass the variable anyway, so conceptually it could have been there since then. Regarding `include_dirs`, it started in `kernel` before being in mainline because we included the bindings directly there (i.e. there was no `bindings` crate). However, when that crate got created, we moved the `OBJTREE` there but not the `include_dirs`. Nowadays, though, we happen to need the `include_dirs` also in the `kernel` crate for `generated_arch_static_branch_asm.rs` which was not there back then -- Tamir confirms it is indeed required for that reason. - Miguel ] Add the missing bits to improve the developer experience. Fixes: 8c4555ccc55c ("scripts: add `generate_rust_analyzer.py`") Signed-off-by: Tamir Duberstein <tamird@gmail.com> Tested-by: Andreas Hindborg <a.hindborg@kernel.org> Link: https://lore.kernel.org/r/20250210-rust-analyzer-bindings-include-v2-1-23dff845edc3@gmail.com [ Slightly reworded title. - Miguel ] Signed-off-by: Miguel Ojeda <ojeda@kernel.org> --- scripts/generate_rust_analyzer.py | 40 +++++++++++++++++++-------------------- 1 file changed, 20 insertions(+), 20 deletions(-) (limited to 'scripts') diff --git a/scripts/generate_rust_analyzer.py b/scripts/generate_rust_analyzer.py index b40679a90843..f2d6787e9c0c 100755 --- a/scripts/generate_rust_analyzer.py +++ b/scripts/generate_rust_analyzer.py @@ -97,27 +97,27 @@ def generate_crates(srctree, objtree, sysroot_src, external_src, cfgs): ["core", "compiler_builtins"], ) - append_crate( - "bindings", - srctree / "rust"/ "bindings" / "lib.rs", - ["core"], - cfg=cfg, - ) - crates[-1]["env"]["OBJTREE"] = str(objtree.resolve(True)) + def append_crate_with_generated( + display_name, + deps, + ): + append_crate( + display_name, + srctree / "rust"/ display_name / "lib.rs", + deps, + cfg=cfg, + ) + crates[-1]["env"]["OBJTREE"] = str(objtree.resolve(True)) + crates[-1]["source"] = { + "include_dirs": [ + str(srctree / "rust" / display_name), + str(objtree / "rust") + ], + "exclude_dirs": [], + } - append_crate( - "kernel", - srctree / "rust" / "kernel" / "lib.rs", - ["core", "macros", "build_error", "bindings"], - cfg=cfg, - ) - crates[-1]["source"] = { - "include_dirs": [ - str(srctree / "rust" / "kernel"), - str(objtree / "rust") - ], - "exclude_dirs": [], - } + append_crate_with_generated("bindings", ["core"]) + append_crate_with_generated("kernel", ["core", "macros", "build_error", "bindings"]) def is_root_crate(build_file, target): try: -- cgit v1.2.3 From a1eb95d6b5f4cf5cc7b081e85e374d1dd98a213b Mon Sep 17 00:00:00 2001 From: Tamir Duberstein <tamird@gmail.com> Date: Mon, 10 Feb 2025 13:04:17 -0500 Subject: scripts: generate_rust_analyzer: add uapi crate Commit 4e1746656839 ("rust: uapi: Add UAPI crate") did not update rust-analyzer to include the new crate. Add the missing definition to improve the developer experience. Fixes: 4e1746656839 ("rust: uapi: Add UAPI crate") Signed-off-by: Tamir Duberstein <tamird@gmail.com> Tested-by: Andreas Hindborg <a.hindborg@kernel.org> Link: https://lore.kernel.org/r/20250210-rust-analyzer-bindings-include-v2-2-23dff845edc3@gmail.com [ Slightly reworded title. - Miguel ] Signed-off-by: Miguel Ojeda <ojeda@kernel.org> --- scripts/generate_rust_analyzer.py | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) (limited to 'scripts') diff --git a/scripts/generate_rust_analyzer.py b/scripts/generate_rust_analyzer.py index f2d6787e9c0c..adae71544cbd 100755 --- a/scripts/generate_rust_analyzer.py +++ b/scripts/generate_rust_analyzer.py @@ -117,7 +117,8 @@ def generate_crates(srctree, objtree, sysroot_src, external_src, cfgs): } append_crate_with_generated("bindings", ["core"]) - append_crate_with_generated("kernel", ["core", "macros", "build_error", "bindings"]) + append_crate_with_generated("uapi", ["core"]) + append_crate_with_generated("kernel", ["core", "macros", "build_error", "bindings", "uapi"]) def is_root_crate(build_file, target): try: -- cgit v1.2.3 From 78da89c6398335c3de7c09e319d8e82126f18126 Mon Sep 17 00:00:00 2001 From: Heiko Carstens <hca@linux.ibm.com> Date: Wed, 12 Mar 2025 16:52:19 +0100 Subject: scripts: get_feat.pl: substitute s390x with s390 Both get_feat.pl and list-arch.sh use uname -m to get the machine hardware name to figure out the current architecture if no architecture is specified with a command line option. This doesn't work for s390, since for 64 bit kernels the hardware name is s390x, while the architecture name within the kernel, as well as in all feature files is s390. Therefore substitute s390x with s390 similar to what is already done for x86_64 and i386. Signed-off-by: Heiko Carstens <hca@linux.ibm.com> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Link: https://lore.kernel.org/r/20250312155219.3597768-1-hca@linux.ibm.com --- Documentation/features/list-arch.sh | 2 +- scripts/get_feat.pl | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) (limited to 'scripts') diff --git a/Documentation/features/list-arch.sh b/Documentation/features/list-arch.sh index e73aa35848f0..ac8ff7f6f859 100755 --- a/Documentation/features/list-arch.sh +++ b/Documentation/features/list-arch.sh @@ -6,6 +6,6 @@ # (If no arguments are given then it will print the host architecture's status.) # -ARCH=${1:-$(uname -m | sed 's/x86_64/x86/' | sed 's/i386/x86/')} +ARCH=${1:-$(uname -m | sed 's/x86_64/x86/' | sed 's/i386/x86/' | sed 's/s390x/s390/')} $(dirname $0)/../../scripts/get_feat.pl list --arch $ARCH diff --git a/scripts/get_feat.pl b/scripts/get_feat.pl index 5c5397eeb237..40fb28c8424e 100755 --- a/scripts/get_feat.pl +++ b/scripts/get_feat.pl @@ -512,13 +512,13 @@ print STDERR Data::Dumper->Dump([\%data], [qw(*data)]) if ($debug); # Handles the command # if ($cmd eq "current") { - $arch = qx(uname -m | sed 's/x86_64/x86/' | sed 's/i386/x86/'); + $arch = qx(uname -m | sed 's/x86_64/x86/' | sed 's/i386/x86/' | sed 's/s390x/s390/'); $arch =~s/\s+$//; } if ($cmd eq "ls" or $cmd eq "list") { if (!$arch) { - $arch = qx(uname -m | sed 's/x86_64/x86/' | sed 's/i386/x86/'); + $arch = qx(uname -m | sed 's/x86_64/x86/' | sed 's/i386/x86/' | sed 's/s390x/s390/'); $arch =~s/\s+$//; } -- cgit v1.2.3 From 10e9510a6d238a8e6c994b81748b00b9c696c48b Mon Sep 17 00:00:00 2001 From: Sami Tolvanen <samitolvanen@google.com> Date: Mon, 3 Feb 2025 21:26:32 +0000 Subject: gendwarfksyms: Add a separate pass to resolve FQNs Using dwarf_getscopes_die to resolve fully-qualified names turns out to be rather slow, and also results in duplicate scopes being processed, which doesn't help. Simply adding an extra pass to resolve names for all DIEs before processing exports is noticeably faster. For the object files with the most exports in a defconfig+Rust build, the performance improvement is consistently >50%: rust/bindings.o: 1038 exports before: 9.5980 +- 0.0183 seconds time elapsed ( +- 0.19% ) after: 4.3116 +- 0.0287 seconds time elapsed ( +- 0.67% ) rust/core.o: 424 exports before: 5.3584 +- 0.0204 seconds time elapsed ( +- 0.38% ) after: 0.05348 +- 0.00129 seconds time elapsed ( +- 2.42% ) ^ Not a mistake. net/core/dev.o: 190 exports before: 9.0507 +- 0.0297 seconds time elapsed ( +- 0.33% ) after: 3.2882 +- 0.0165 seconds time elapsed ( +- 0.50% ) rust/kernel.o: 129 exports before: 6.8571 +- 0.0317 seconds time elapsed ( +- 0.46% ) after: 2.9096 +- 0.0316 seconds time elapsed ( +- 1.09% ) net/core/skbuff.o: 120 exports before: 5.4805 +- 0.0291 seconds time elapsed ( +- 0.53% ) after: 2.0339 +- 0.0231 seconds time elapsed ( +- 1.14% ) drivers/gpu/drm/display/drm_dp_helper.o: 101 exports before: 1.7877 +- 0.0187 seconds time elapsed ( +- 1.05% ) after: 0.69245 +- 0.00994 seconds time elapsed ( +- 1.44% ) net/core/sock.o: 97 exports before: 5.8327 +- 0.0653 seconds time elapsed ( +- 1.12% ) after: 2.0784 +- 0.0291 seconds time elapsed ( +- 1.40% ) drivers/net/phy/phy_device.o: 95 exports before: 3.0671 +- 0.0371 seconds time elapsed ( +- 1.21% ) after: 1.2127 +- 0.0207 seconds time elapsed ( +- 1.70% ) drivers/pci/pci.o: 93 exports before: 1.1130 +- 0.0113 seconds time elapsed ( +- 1.01% ) after: 0.4848 +- 0.0127 seconds time elapsed ( +- 2.63% ) kernel/sched/core.o: 83 exports before: 3.5092 +- 0.0223 seconds time elapsed ( +- 0.64% ) after: 1.1231 +- 0.0145 seconds time elapsed ( +- 1.29% ) Overall, a defconfig+DWARF5 build with gendwarfksyms and Rust is 14.8% faster with this patch applied on my test system. Without Rust, there's still a 10.4% improvement in build time when gendwarfksyms is used. Note that symbol versions are unchanged with this patch. Suggested-by: Giuliano Procida <gprocida@google.com> Signed-off-by: Sami Tolvanen <samitolvanen@google.com> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> --- scripts/gendwarfksyms/die.c | 2 +- scripts/gendwarfksyms/dwarf.c | 152 ++++++++++++++++++---------------- scripts/gendwarfksyms/gendwarfksyms.h | 2 + scripts/gendwarfksyms/types.c | 2 +- 4 files changed, 86 insertions(+), 72 deletions(-) (limited to 'scripts') diff --git a/scripts/gendwarfksyms/die.c b/scripts/gendwarfksyms/die.c index 66bd4c9bc952..6183bbbe7b54 100644 --- a/scripts/gendwarfksyms/die.c +++ b/scripts/gendwarfksyms/die.c @@ -6,7 +6,7 @@ #include <string.h> #include "gendwarfksyms.h" -#define DIE_HASH_BITS 15 +#define DIE_HASH_BITS 16 /* {die->addr, state} -> struct die * */ static HASHTABLE_DEFINE(die_map, 1 << DIE_HASH_BITS); diff --git a/scripts/gendwarfksyms/dwarf.c b/scripts/gendwarfksyms/dwarf.c index 534d9aa7c114..eed247d8abfc 100644 --- a/scripts/gendwarfksyms/dwarf.c +++ b/scripts/gendwarfksyms/dwarf.c @@ -3,6 +3,7 @@ * Copyright (C) 2024 Google LLC */ +#define _GNU_SOURCE #include <assert.h> #include <inttypes.h> #include <stdarg.h> @@ -193,79 +194,17 @@ static void process_fmt(struct die *cache, const char *fmt, ...) va_end(args); } -#define MAX_FQN_SIZE 64 - -/* Get a fully qualified name from DWARF scopes */ -static char *get_fqn(Dwarf_Die *die) +static void update_fqn(struct die *cache, Dwarf_Die *die) { - const char *list[MAX_FQN_SIZE]; - Dwarf_Die *scopes = NULL; - bool has_name = false; - char *fqn = NULL; - char *p; - int count = 0; - int len = 0; - int res; - int i; - - res = checkp(dwarf_getscopes_die(die, &scopes)); - if (!res) { - list[count] = get_name_attr(die); - - if (!list[count]) - return NULL; - - len += strlen(list[count]); - count++; - - goto done; - } - - for (i = res - 1; i >= 0 && count < MAX_FQN_SIZE; i--) { - if (dwarf_tag(&scopes[i]) == DW_TAG_compile_unit) - continue; - - list[count] = get_name_attr(&scopes[i]); - - if (list[count]) { - has_name = true; - } else { - list[count] = "<anonymous>"; - has_name = false; - } + struct die *fqn; - len += strlen(list[count]); - count++; - - if (i > 0) { - list[count++] = "::"; - len += 2; - } + if (!cache->fqn) { + if (!__die_map_get((uintptr_t)die->addr, DIE_FQN, &fqn) && + *fqn->fqn) + cache->fqn = xstrdup(fqn->fqn); + else + cache->fqn = ""; } - - free(scopes); - - if (count == MAX_FQN_SIZE) - warn("increase MAX_FQN_SIZE: reached the maximum"); - - /* Consider the DIE unnamed if the last scope doesn't have a name */ - if (!has_name) - return NULL; -done: - fqn = xmalloc(len + 1); - *fqn = '\0'; - - p = fqn; - for (i = 0; i < count; i++) - p = stpcpy(p, list[i]); - - return fqn; -} - -static void update_fqn(struct die *cache, Dwarf_Die *die) -{ - if (!cache->fqn) - cache->fqn = get_fqn(die) ?: ""; } static void process_fqn(struct die *cache, Dwarf_Die *die) @@ -1148,8 +1087,81 @@ static void process_symbol_ptr(struct symbol *sym, void *arg) cache_free(&state.expansion_cache); } +static int resolve_fqns(struct state *parent, struct die *unused, + Dwarf_Die *die) +{ + struct state state; + struct die *cache; + const char *name; + bool use_prefix; + char *prefix = NULL; + char *fqn = ""; + int tag; + + if (!__die_map_get((uintptr_t)die->addr, DIE_FQN, &cache)) + return 0; + + tag = dwarf_tag(die); + + /* + * Only namespaces and structures need to pass a prefix to the next + * scope. + */ + use_prefix = tag == DW_TAG_namespace || tag == DW_TAG_class_type || + tag == DW_TAG_structure_type; + + state.expand.current_fqn = NULL; + name = get_name_attr(die); + + if (parent && parent->expand.current_fqn && (use_prefix || name)) { + /* + * The fqn for the current DIE, and if needed, a prefix for the + * next scope. + */ + if (asprintf(&prefix, "%s::%s", parent->expand.current_fqn, + name ? name : "<anonymous>") < 0) + error("asprintf failed"); + + if (use_prefix) + state.expand.current_fqn = prefix; + + /* + * Use fqn only if the DIE has a name. Otherwise fqn will + * remain empty. + */ + if (name) { + fqn = prefix; + /* prefix will be freed by die_map. */ + prefix = NULL; + } + } else if (name) { + /* No prefix from the previous scope. Use only the name. */ + fqn = xstrdup(name); + + if (use_prefix) + state.expand.current_fqn = fqn; + } + + /* If the DIE has a non-empty name, cache it. */ + if (*fqn) { + cache = die_map_get(die, DIE_FQN); + /* Move ownership of fqn to die_map. */ + cache->fqn = fqn; + cache->state = DIE_FQN; + } + + check(process_die_container(&state, NULL, die, resolve_fqns, + match_all)); + + free(prefix); + return 0; +} + void process_cu(Dwarf_Die *cudie) { + check(process_die_container(NULL, NULL, cudie, resolve_fqns, + match_all)); + check(process_die_container(NULL, NULL, cudie, process_exported_symbols, match_all)); diff --git a/scripts/gendwarfksyms/gendwarfksyms.h b/scripts/gendwarfksyms/gendwarfksyms.h index 197a1a8123c6..2feec168bf73 100644 --- a/scripts/gendwarfksyms/gendwarfksyms.h +++ b/scripts/gendwarfksyms/gendwarfksyms.h @@ -139,6 +139,7 @@ void symbol_free(void); enum die_state { DIE_INCOMPLETE, + DIE_FQN, DIE_UNEXPANDED, DIE_COMPLETE, DIE_SYMBOL, @@ -170,6 +171,7 @@ static inline const char *die_state_name(enum die_state state) { switch (state) { CASE_CONST_TO_STR(DIE_INCOMPLETE) + CASE_CONST_TO_STR(DIE_FQN) CASE_CONST_TO_STR(DIE_UNEXPANDED) CASE_CONST_TO_STR(DIE_COMPLETE) CASE_CONST_TO_STR(DIE_SYMBOL) diff --git a/scripts/gendwarfksyms/types.c b/scripts/gendwarfksyms/types.c index 6c03265f4d10..6f37289104ff 100644 --- a/scripts/gendwarfksyms/types.c +++ b/scripts/gendwarfksyms/types.c @@ -248,7 +248,7 @@ static char *get_type_name(struct die *cache) warn("found incomplete cache entry: %p", cache); return NULL; } - if (cache->state == DIE_SYMBOL) + if (cache->state == DIE_SYMBOL || cache->state == DIE_FQN) return NULL; if (!cache->fqn || !*cache->fqn) return NULL; -- cgit v1.2.3 From e966ad0edd0056c7491b8f23992c11734ab61ddf Mon Sep 17 00:00:00 2001 From: Masahiro Yamada <masahiroy@kernel.org> Date: Thu, 6 Feb 2025 01:39:38 +0900 Subject: kbuild: remove EXTRA_*FLAGS support Commit f77bf01425b1 ("kbuild: introduce ccflags-y, asflags-y and ldflags-y") deprecated these in 2007. The migration should have been completed by now. Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Reviewed-by: Kees Cook <kees@kernel.org> Reviewed-by: Nathan Chancellor <nathan@kernel.org> --- Documentation/dev-tools/checkpatch.rst | 18 ------------------ Documentation/kbuild/makefiles.rst | 3 --- scripts/Makefile.build | 4 ---- scripts/Makefile.lib | 5 ----- scripts/checkpatch.pl | 14 -------------- 5 files changed, 44 deletions(-) (limited to 'scripts') diff --git a/Documentation/dev-tools/checkpatch.rst b/Documentation/dev-tools/checkpatch.rst index abb3ff682076..76bd0ddb0041 100644 --- a/Documentation/dev-tools/checkpatch.rst +++ b/Documentation/dev-tools/checkpatch.rst @@ -342,24 +342,6 @@ API usage See: https://www.kernel.org/doc/html/latest/RCU/whatisRCU.html#full-list-of-rcu-apis - **DEPRECATED_VARIABLE** - EXTRA_{A,C,CPP,LD}FLAGS are deprecated and should be replaced by the new - flags added via commit f77bf01425b1 ("kbuild: introduce ccflags-y, - asflags-y and ldflags-y"). - - The following conversion scheme maybe used:: - - EXTRA_AFLAGS -> asflags-y - EXTRA_CFLAGS -> ccflags-y - EXTRA_CPPFLAGS -> cppflags-y - EXTRA_LDFLAGS -> ldflags-y - - See: - - 1. https://lore.kernel.org/lkml/20070930191054.GA15876@uranus.ravnborg.org/ - 2. https://lore.kernel.org/lkml/1313384834-24433-12-git-send-email-lacombar@gmail.com/ - 3. https://www.kernel.org/doc/html/latest/kbuild/makefiles.html#compilation-flags - **DEVICE_ATTR_FUNCTIONS** The function names used in DEVICE_ATTR is unusual. Typically, the store and show functions are used with <attr>_store and diff --git a/Documentation/kbuild/makefiles.rst b/Documentation/kbuild/makefiles.rst index d36519f194dc..25e04e47faff 100644 --- a/Documentation/kbuild/makefiles.rst +++ b/Documentation/kbuild/makefiles.rst @@ -318,9 +318,6 @@ ccflags-y, asflags-y and ldflags-y These three flags apply only to the kbuild makefile in which they are assigned. They are used for all the normal cc, as and ld invocations happening during a recursive build. - Note: Flags with the same behaviour were previously named: - EXTRA_CFLAGS, EXTRA_AFLAGS and EXTRA_LDFLAGS. - They are still supported but their usage is deprecated. ccflags-y specifies options for compiling with $(CC). diff --git a/scripts/Makefile.build b/scripts/Makefile.build index 993708d11874..a59650ba140b 100644 --- a/scripts/Makefile.build +++ b/scripts/Makefile.build @@ -20,10 +20,6 @@ always-m := targets := subdir-y := subdir-m := -EXTRA_AFLAGS := -EXTRA_CFLAGS := -EXTRA_CPPFLAGS := -EXTRA_LDFLAGS := asflags-y := ccflags-y := rustflags-y := diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib index cad20f0e66ee..47f56ef71937 100644 --- a/scripts/Makefile.lib +++ b/scripts/Makefile.lib @@ -1,9 +1,4 @@ # SPDX-License-Identifier: GPL-2.0 -# Backward compatibility -asflags-y += $(EXTRA_AFLAGS) -ccflags-y += $(EXTRA_CFLAGS) -cppflags-y += $(EXTRA_CPPFLAGS) -ldflags-y += $(EXTRA_LDFLAGS) # flags that take effect in current and sub directories KBUILD_AFLAGS += $(subdir-asflags-y) diff --git a/scripts/checkpatch.pl b/scripts/checkpatch.pl index 7b28ad331742..8f70bedc18be 100755 --- a/scripts/checkpatch.pl +++ b/scripts/checkpatch.pl @@ -3689,20 +3689,6 @@ sub process { } } - if (($realfile =~ /Makefile.*/ || $realfile =~ /Kbuild.*/) && - ($line =~ /\+(EXTRA_[A-Z]+FLAGS).*/)) { - my $flag = $1; - my $replacement = { - 'EXTRA_AFLAGS' => 'asflags-y', - 'EXTRA_CFLAGS' => 'ccflags-y', - 'EXTRA_CPPFLAGS' => 'cppflags-y', - 'EXTRA_LDFLAGS' => 'ldflags-y', - }; - - WARN("DEPRECATED_VARIABLE", - "Use of $flag is deprecated, please use \`$replacement->{$flag} instead.\n" . $herecurr) if ($replacement->{$flag}); - } - # check for DT compatible documentation if (defined $root && (($realfile =~ /\.dtsi?$/ && $line =~ /^\+\s*compatible\s*=\s*\"/) || -- cgit v1.2.3 From 90efe2b9119ff8a83a67eb5323e52311a1a49de9 Mon Sep 17 00:00:00 2001 From: Masahiro Yamada <masahiroy@kernel.org> Date: Thu, 6 Feb 2025 02:18:02 +0900 Subject: gen_compile_commands.py: remove code for '\#' replacement Since commit 9564a8cf422d ("Kbuild: fix # escaping in .cmd files for future Make"), '#' in the build command is replaced with $(pound) rather than '\#'. Calling .replace(r'\#', '#') is only necessary when this tool is used to parse .*.cmd files generated by Linux 4.16 or earlier, which is unlikely to happen. Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Reviewed-by: Nathan Chancellor <nathan@kernel.org> --- scripts/clang-tools/gen_compile_commands.py | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) (limited to 'scripts') diff --git a/scripts/clang-tools/gen_compile_commands.py b/scripts/clang-tools/gen_compile_commands.py index e4fb686dfaa9..96e6e46ad1a7 100755 --- a/scripts/clang-tools/gen_compile_commands.py +++ b/scripts/clang-tools/gen_compile_commands.py @@ -167,10 +167,10 @@ def process_line(root_directory, command_prefix, file_path): root_directory or file_directory. """ # The .cmd files are intended to be included directly by Make, so they - # escape the pound sign '#', either as '\#' or '$(pound)' (depending on the - # kernel version). The compile_commands.json file is not interepreted - # by Make, so this code replaces the escaped version with '#'. - prefix = command_prefix.replace(r'\#', '#').replace('$(pound)', '#') + # escape the pound sign '#' as '$(pound)'. The compile_commands.json file + # is not interepreted by Make, so this code replaces the escaped version + # with '#'. + prefix = command_prefix.replace('$(pound)', '#') # Return the canonical path, eliminating any symbolic links encountered in the path. abs_path = os.path.realpath(os.path.join(root_directory, file_path)) -- cgit v1.2.3 From 6c3fb0bb4d4f1173f32cc26d077b151d72226a2f Mon Sep 17 00:00:00 2001 From: Masahiro Yamada <masahiroy@kernel.org> Date: Thu, 6 Feb 2025 02:45:28 +0900 Subject: genksyms: factor out APP for the ST_NORMAL state For the ST_NORMAL state, APP is called regardless of the token type. Factor it out. Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> --- scripts/genksyms/lex.l | 6 +----- 1 file changed, 1 insertion(+), 5 deletions(-) (limited to 'scripts') diff --git a/scripts/genksyms/lex.l b/scripts/genksyms/lex.l index 22aeb57649d9..f81033af1528 100644 --- a/scripts/genksyms/lex.l +++ b/scripts/genksyms/lex.l @@ -176,10 +176,10 @@ repeat: switch (lexstate) { case ST_NORMAL: + APP; switch (token) { case IDENT: - APP; { int r = is_reserved_word(yytext, yyleng); if (r >= 0) @@ -224,13 +224,11 @@ repeat: break; case '[': - APP; lexstate = ST_BRACKET; count = 1; goto repeat; case '{': - APP; if (dont_want_brace_phrase) break; lexstate = ST_BRACE; @@ -238,12 +236,10 @@ repeat: goto repeat; case '=': case ':': - APP; lexstate = ST_EXPRESSION; break; default: - APP; break; } break; -- cgit v1.2.3 From 226ac19c217f24f0927d0a73cf9ee613971a188d Mon Sep 17 00:00:00 2001 From: Masahiro Yamada <masahiroy@kernel.org> Date: Sat, 8 Feb 2025 03:41:55 +0900 Subject: kconfig: do not clear SYMBOL_VALID when reading include/config/auto.conf When conf_read_simple() is called with S_DEF_AUTO, it is meant to read previous symbol values from include/config/auto.conf to determine which include/config/* files should be touched. This process should not modify the current symbol status in any way. However, conf_touch_deps() currently invalidates all symbol values and recalculates them, which is totally unneeded. Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> --- scripts/kconfig/confdata.c | 19 ++++++++++++------- 1 file changed, 12 insertions(+), 7 deletions(-) (limited to 'scripts') diff --git a/scripts/kconfig/confdata.c b/scripts/kconfig/confdata.c index 3b55e7a4131d..ac95661a1c9d 100644 --- a/scripts/kconfig/confdata.c +++ b/scripts/kconfig/confdata.c @@ -385,7 +385,7 @@ load: def_flags = SYMBOL_DEF << def; for_all_symbols(sym) { - sym->flags &= ~(def_flags|SYMBOL_VALID); + sym->flags &= ~def_flags; switch (sym->type) { case S_INT: case S_HEX: @@ -398,7 +398,11 @@ load: } } - expr_invalidate_all(); + if (def == S_DEF_USER) { + for_all_symbols(sym) + sym->flags &= ~SYMBOL_VALID; + expr_invalidate_all(); + } while (getline_stripped(&line, &line_asize, in) != -1) { struct menu *choice; @@ -464,6 +468,9 @@ load: if (conf_set_sym_val(sym, def, def_flags, val)) continue; + if (def != S_DEF_USER) + continue; + /* * If this is a choice member, give it the highest priority. * If conflicting CONFIG options are given from an input file, @@ -967,10 +974,8 @@ static int conf_touch_deps(void) depfile_path[depfile_prefix_len] = 0; conf_read_simple(name, S_DEF_AUTO); - sym_calc_value(modules_sym); for_all_symbols(sym) { - sym_calc_value(sym); if (sym_is_choice(sym)) continue; if (sym->flags & SYMBOL_WRITE) { @@ -1084,12 +1089,12 @@ int conf_write_autoconf(int overwrite) if (ret) return -1; - if (conf_touch_deps()) - return 1; - for_all_symbols(sym) sym_calc_value(sym); + if (conf_touch_deps()) + return 1; + ret = __conf_write_autoconf(conf_get_autoheader_name(), print_symbol_for_c, &comment_style_c); -- cgit v1.2.3 From ab5bc764bdc270d1c55aaf9d238e0986ff301355 Mon Sep 17 00:00:00 2001 From: Masahiro Yamada <masahiroy@kernel.org> Date: Sat, 8 Feb 2025 03:42:48 +0900 Subject: kconfig: remove unnecessary cast in sym_get_string() The explicit casting from (char *) to (const char *) is unneeded. Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> --- scripts/kconfig/symbol.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'scripts') diff --git a/scripts/kconfig/symbol.c b/scripts/kconfig/symbol.c index 7beb59dec5a0..d57f8cbba291 100644 --- a/scripts/kconfig/symbol.c +++ b/scripts/kconfig/symbol.c @@ -879,7 +879,7 @@ const char *sym_get_string_value(struct symbol *sym) default: ; } - return (const char *)sym->curr.val; + return sym->curr.val; } bool sym_is_changeable(const struct symbol *sym) -- cgit v1.2.3 From 59d60d26a58be75e9e3b813104c2dfd3b58eb662 Mon Sep 17 00:00:00 2001 From: Masahiro Yamada <masahiroy@kernel.org> Date: Sat, 8 Feb 2025 02:50:13 +0900 Subject: modpost: introduce get_basename() helper The logic to retrieve the basename appears multiple times. Factor out the common pattern into a helper function. I copied kbasename() from include/linux/string.h and renamed it to get_basename(). Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Reviewed-by: Nicolas Schier <nicolas@fjasle.eu> --- scripts/mod/modpost.c | 41 ++++++++++++++++++----------------------- scripts/mod/modpost.h | 1 + scripts/mod/sumversion.c | 13 ++++--------- 3 files changed, 23 insertions(+), 32 deletions(-) (limited to 'scripts') diff --git a/scripts/mod/modpost.c b/scripts/mod/modpost.c index c35d22607978..7f4c72eca72c 100644 --- a/scripts/mod/modpost.c +++ b/scripts/mod/modpost.c @@ -98,6 +98,18 @@ static inline bool strends(const char *str, const char *postfix) return strcmp(str + strlen(str) - strlen(postfix), postfix) == 0; } +/** + * get_basename - return the last part of a pathname. + * + * @path: path to extract the filename from. + */ +const char *get_basename(const char *path) +{ + const char *tail = strrchr(path, '/'); + + return tail ? tail + 1 : path; +} + char *read_text_file(const char *filename) { struct stat st; @@ -1461,14 +1473,8 @@ static void extract_crcs_for_object(const char *object, struct module *mod) const char *base; int dirlen, ret; - base = strrchr(object, '/'); - if (base) { - base++; - dirlen = base - object; - } else { - dirlen = 0; - base = object; - } + base = get_basename(object); + dirlen = base - object; ret = snprintf(cmd_file, sizeof(cmd_file), "%.*s.%s.cmd", dirlen, object, base); @@ -1703,11 +1709,7 @@ static void check_exports(struct module *mod) s->crc_valid = exp->crc_valid; s->crc = exp->crc; - basename = strrchr(mod->name, '/'); - if (basename) - basename++; - else - basename = mod->name; + basename = get_basename(mod->name); if (!contains_namespace(&mod->imported_namespaces, exp->namespace)) { modpost_log(!allow_missing_ns_imports, @@ -1765,11 +1767,8 @@ static void check_modname_len(struct module *mod) { const char *mod_name; - mod_name = strrchr(mod->name, '/'); - if (mod_name == NULL) - mod_name = mod->name; - else - mod_name++; + mod_name = get_basename(mod->name); + if (strlen(mod_name) >= MODULE_NAME_LEN) error("module name is too long [%s.ko]\n", mod->name); } @@ -1946,11 +1945,7 @@ static void add_depends(struct buffer *b, struct module *mod) continue; s->module->seen = true; - p = strrchr(s->module->name, '/'); - if (p) - p++; - else - p = s->module->name; + p = get_basename(s->module->name); buf_printf(b, "%s%s", first ? "" : ",", p); first = 0; } diff --git a/scripts/mod/modpost.h b/scripts/mod/modpost.h index 59366f456b76..9133e4c3803f 100644 --- a/scripts/mod/modpost.h +++ b/scripts/mod/modpost.h @@ -216,6 +216,7 @@ void get_src_version(const char *modname, char sum[], unsigned sumlen); /* from modpost.c */ extern bool target_is_big_endian; extern bool host_is_big_endian; +const char *get_basename(const char *path); char *read_text_file(const char *filename); char *get_line(char **stringp); void *sym_get_data(const struct elf_info *info, const Elf_Sym *sym); diff --git a/scripts/mod/sumversion.c b/scripts/mod/sumversion.c index 6de9af17599d..e79fc40d852f 100644 --- a/scripts/mod/sumversion.c +++ b/scripts/mod/sumversion.c @@ -309,15 +309,10 @@ static int parse_source_files(const char *objfile, struct md4_ctx *md) cmd = xmalloc(strlen(objfile) + sizeof("..cmd")); - base = strrchr(objfile, '/'); - if (base) { - base++; - dirlen = base - objfile; - sprintf(cmd, "%.*s.%s.cmd", dirlen, objfile, base); - } else { - dirlen = 0; - sprintf(cmd, ".%s.cmd", objfile); - } + base = get_basename(objfile); + dirlen = base - objfile; + sprintf(cmd, "%.*s.%s.cmd", dirlen, objfile, base); + dir = xmalloc(dirlen + 1); strncpy(dir, objfile, dirlen); dir[dirlen] = '\0'; -- cgit v1.2.3 From 144fced6852ba11c90560c996e986b4563f089af Mon Sep 17 00:00:00 2001 From: Masahiro Yamada <masahiroy@kernel.org> Date: Sat, 8 Feb 2025 02:50:55 +0900 Subject: modpost: use strstarts() to clean up parse_source_files() No functional changes are intended. Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Reviewed-by: Nicolas Schier <nicolas@fjasle.eu> --- scripts/mod/sumversion.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) (limited to 'scripts') diff --git a/scripts/mod/sumversion.c b/scripts/mod/sumversion.c index e79fc40d852f..3dd28b4d0099 100644 --- a/scripts/mod/sumversion.c +++ b/scripts/mod/sumversion.c @@ -330,7 +330,7 @@ static int parse_source_files(const char *objfile, struct md4_ctx *md) line++; p = line; - if (strncmp(line, "source_", sizeof("source_")-1) == 0) { + if (strstarts(line, "source_")) { p = strrchr(line, ' '); if (!p) { warn("malformed line: %s\n", line); @@ -344,7 +344,7 @@ static int parse_source_files(const char *objfile, struct md4_ctx *md) } continue; } - if (strncmp(line, "deps_", sizeof("deps_")-1) == 0) { + if (strstarts(line, "deps_")) { check_files = 1; continue; } -- cgit v1.2.3 From ac954145e1ee3f72033161cbe4ac0b16b5354ae7 Mon Sep 17 00:00:00 2001 From: Miguel Ojeda <ojeda@kernel.org> Date: Mon, 10 Feb 2025 17:42:45 +0100 Subject: kbuild: rust: add rustc-min-version support function Introduce `rustc-min-version` support function that mimics `{gcc,clang}-min-version` ones, following commit 88b61e3bff93 ("Makefile.compiler: replace cc-ifversion with compiler-specific macros"). In addition, use it in the first use case we have in the kernel (which was done independently to minimize the changes needed for the fix). Signed-off-by: Miguel Ojeda <ojeda@kernel.org> Reviewed-by: Fiona Behrens <me@Kloenk.dev> Reviewed-by: Nicolas Schier <n.schier@avm.de> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> --- Documentation/kbuild/makefiles.rst | 14 ++++++++++++++ arch/arm64/Makefile | 2 +- scripts/Makefile.compiler | 4 ++++ 3 files changed, 19 insertions(+), 1 deletion(-) (limited to 'scripts') diff --git a/Documentation/kbuild/makefiles.rst b/Documentation/kbuild/makefiles.rst index 25e04e47faff..3b9a8bc671e2 100644 --- a/Documentation/kbuild/makefiles.rst +++ b/Documentation/kbuild/makefiles.rst @@ -667,6 +667,20 @@ cc-cross-prefix endif endif +$(RUSTC) support functions +-------------------------- + +rustc-min-version + rustc-min-version tests if the value of $(CONFIG_RUSTC_VERSION) is greater + than or equal to the provided value and evaluates to y if so. + + Example:: + + rustflags-$(call rustc-min-version, 108500) := -Cfoo + + In this example, rustflags-y will be assigned the value -Cfoo if + $(CONFIG_RUSTC_VERSION) is >= 1.85.0. + $(LD) support functions ----------------------- diff --git a/arch/arm64/Makefile b/arch/arm64/Makefile index 2b25d671365f..1d5dfcd1c13e 100644 --- a/arch/arm64/Makefile +++ b/arch/arm64/Makefile @@ -48,7 +48,7 @@ KBUILD_CFLAGS += $(CC_FLAGS_NO_FPU) \ KBUILD_CFLAGS += $(call cc-disable-warning, psabi) KBUILD_AFLAGS += $(compat_vdso) -ifeq ($(call test-ge, $(CONFIG_RUSTC_VERSION), 108500),y) +ifeq ($(call rustc-min-version, 108500),y) KBUILD_RUSTFLAGS += --target=aarch64-unknown-none-softfloat else KBUILD_RUSTFLAGS += --target=aarch64-unknown-none -Ctarget-feature="-neon" diff --git a/scripts/Makefile.compiler b/scripts/Makefile.compiler index 8c1029687e2e..8956587b8547 100644 --- a/scripts/Makefile.compiler +++ b/scripts/Makefile.compiler @@ -67,6 +67,10 @@ gcc-min-version = $(call test-ge, $(CONFIG_GCC_VERSION), $1) # Usage: cflags-$(call clang-min-version, 110000) += -foo clang-min-version = $(call test-ge, $(CONFIG_CLANG_VERSION), $1) +# rustc-min-version +# Usage: rustc-$(call rustc-min-version, 108500) += -Cfoo +rustc-min-version = $(call test-ge, $(CONFIG_RUSTC_VERSION), $1) + # ld-option # Usage: KBUILD_LDFLAGS += $(call ld-option, -X, -Y) ld-option = $(call try-run, $(LD) $(KBUILD_LDFLAGS) $(1) -v,$(1),$(2),$(3)) -- cgit v1.2.3 From 9d702bb1d3c03bb78d4fd2b3424169e3ef4cd402 Mon Sep 17 00:00:00 2001 From: Uday Shankar <ushankar@purestorage.com> Date: Mon, 10 Feb 2025 18:11:54 -0700 Subject: scripts: make python shebangs specific about desired version The RPM packaging tools like to make sure that all packaged python scripts have version-unambiguous shebangs. Be more specific about the desired python version in a couple of places to avoid having to disable these checks in make rpm-pkg. Signed-off-by: Uday Shankar <ushankar@purestorage.com> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> --- scripts/show_delta | 2 +- scripts/tracing/draw_functrace.py | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) (limited to 'scripts') diff --git a/scripts/show_delta b/scripts/show_delta index 291ad65e3089..3755b6c6e557 100755 --- a/scripts/show_delta +++ b/scripts/show_delta @@ -1,4 +1,4 @@ -#!/usr/bin/env python +#!/usr/bin/env python3 # SPDX-License-Identifier: GPL-2.0-only # # show_deltas: Read list of printk messages instrumented with diff --git a/scripts/tracing/draw_functrace.py b/scripts/tracing/draw_functrace.py index 42fa87300941..97594b65f8ce 100755 --- a/scripts/tracing/draw_functrace.py +++ b/scripts/tracing/draw_functrace.py @@ -1,4 +1,4 @@ -#!/usr/bin/env python +#!/usr/bin/env python3 # SPDX-License-Identifier: GPL-2.0-only """ -- cgit v1.2.3 From c15253494fd98cd76250c9faaebbc8b45f7d0072 Mon Sep 17 00:00:00 2001 From: Masahiro Yamada <masahiroy@kernel.org> Date: Sun, 16 Feb 2025 01:15:52 +0900 Subject: kbuild: move -fzero-init-padding-bits=all to the top-level Makefile The -fzero-init-padding-bits=all option is not a warning flag, so defining it in scripts/Makefile.extrawarn is inconsistent. Move it to the top-level Makefile for consistency. Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Reviewed-by: Nathan Chancellor <nathan@kernel.org> Reviewed-by: Kees Cook <kees@kernel.org> --- Makefile | 3 +++ scripts/Makefile.extrawarn | 3 --- 2 files changed, 3 insertions(+), 3 deletions(-) (limited to 'scripts') diff --git a/Makefile b/Makefile index 1d6a9ec8a2ac..7cd80ff2d69b 100644 --- a/Makefile +++ b/Makefile @@ -928,6 +928,9 @@ KBUILD_CFLAGS += $(CC_AUTO_VAR_INIT_ZERO_ENABLER) endif endif +# Explicitly clear padding bits during variable initialization +KBUILD_CFLAGS += $(call cc-option,-fzero-init-padding-bits=all) + # While VLAs have been removed, GCC produces unreachable stack probes # for the randomize_kstack_offset feature. Disable it for all compilers. KBUILD_CFLAGS += $(call cc-option, -fno-stack-clash-protection) diff --git a/scripts/Makefile.extrawarn b/scripts/Makefile.extrawarn index dc081cf46d21..d75897559d18 100644 --- a/scripts/Makefile.extrawarn +++ b/scripts/Makefile.extrawarn @@ -82,9 +82,6 @@ KBUILD_CFLAGS += $(call cc-option,-Werror=designated-init) # Warn if there is an enum types mismatch KBUILD_CFLAGS += $(call cc-option,-Wenum-conversion) -# Explicitly clear padding bits during variable initialization -KBUILD_CFLAGS += $(call cc-option,-fzero-init-padding-bits=all) - KBUILD_CFLAGS += -Wextra KBUILD_CFLAGS += -Wunused -- cgit v1.2.3 From 1195306ee359ced2cb9d7a192e973872571cb93a Mon Sep 17 00:00:00 2001 From: WangYuli <wangyuli@uniontech.com> Date: Tue, 25 Feb 2025 02:26:19 +0800 Subject: kbuild: deb-pkg: add debarch for ARCH=loongarch64 Fix follow warning when 'make ARCH=loongarch64 bindeb-pkg': ** ** ** WARNING ** ** ** Your architecture doesn't have its equivalent Debian userspace architecture defined! Falling back to the current host architecture (loong64). Please add support for loongarch64 to ./scripts/package/mkdebian ... Reported-by: Shiwei Liu <liushiwei@anheng.com.cn> Signed-off-by: WangYuli <wangyuli@uniontech.com> Reviewed-by: Nathan Chancellor <nathan@kernel.org> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> --- scripts/package/mkdebian | 2 ++ 1 file changed, 2 insertions(+) (limited to 'scripts') diff --git a/scripts/package/mkdebian b/scripts/package/mkdebian index b6dd98ca860b..0178000197fe 100755 --- a/scripts/package/mkdebian +++ b/scripts/package/mkdebian @@ -77,6 +77,8 @@ set_debarch() { debarch=i386 fi ;; + loongarch64) + debarch=loong64 ;; esac if [ -z "$debarch" ]; then debarch=$(dpkg-architecture -qDEB_HOST_ARCH) -- cgit v1.2.3 From 82c09de2d4c472ab1b973e6e033671020691e637 Mon Sep 17 00:00:00 2001 From: Xi Ruoyao <xry111@xry111.site> Date: Wed, 26 Feb 2025 21:30:14 +0800 Subject: kbuild: add dependency from vmlinux to sorttable Without this dependency it's really puzzling when we bisect for a "bad" commit in a series of sorttable change: when "git bisect" switches to another commit, "make" just does nothing to vmlinux. Signed-off-by: Xi Ruoyao <xry111@xry111.site> Acked-by: Steven Rostedt (Google) <rostedt@goodmis.org> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> --- scripts/Makefile.vmlinux | 4 ++++ 1 file changed, 4 insertions(+) (limited to 'scripts') diff --git a/scripts/Makefile.vmlinux b/scripts/Makefile.vmlinux index 873caaa55313..fb79fd6b2465 100644 --- a/scripts/Makefile.vmlinux +++ b/scripts/Makefile.vmlinux @@ -79,6 +79,10 @@ ifdef CONFIG_DEBUG_INFO_BTF vmlinux: $(RESOLVE_BTFIDS) endif +ifdef CONFIG_BUILDTIME_TABLE_SORT +vmlinux: scripts/sorttable +endif + # module.builtin.ranges # --------------------------------------------------------------------------- ifdef CONFIG_BUILTIN_MODULE_RANGES -- cgit v1.2.3 From f757f6011c92b5a01db742c39149bed9e526478f Mon Sep 17 00:00:00 2001 From: Seyediman Seyedarab <imandevel@gmail.com> Date: Sat, 1 Mar 2025 17:21:37 -0500 Subject: kbuild: fix argument parsing in scripts/config The script previously assumed --file was always the first argument, which caused issues when it appeared later. This patch updates the parsing logic to scan all arguments to find --file, sets the config file correctly, and resets the argument list with the remaining commands. It also fixes --refresh to respect --file by passing KCONFIG_CONFIG=$FN to make oldconfig. Signed-off-by: Seyediman Seyedarab <imandevel@gmail.com> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> --- scripts/config | 26 ++++++++++++++++---------- 1 file changed, 16 insertions(+), 10 deletions(-) (limited to 'scripts') diff --git a/scripts/config b/scripts/config index ff88e2faefd3..ea475c07de28 100755 --- a/scripts/config +++ b/scripts/config @@ -32,6 +32,7 @@ commands: Disable option directly after other option --module-after|-M beforeopt option Turn option into module directly after other option + --refresh Refresh the config using old settings commands can be repeated multiple times @@ -124,16 +125,22 @@ undef_var() { txt_delete "^# $name is not set" "$FN" } -if [ "$1" = "--file" ]; then - FN="$2" - if [ "$FN" = "" ] ; then - usage +FN=.config +CMDS=() +while [[ $# -gt 0 ]]; do + if [ "$1" = "--file" ]; then + if [ "$2" = "" ]; then + usage + fi + FN="$2" + shift 2 + else + CMDS+=("$1") + shift fi - shift 2 -else - FN=.config -fi +done +set -- "${CMDS[@]}" if [ "$1" = "" ] ; then usage fi @@ -217,9 +224,8 @@ while [ "$1" != "" ] ; do set_var "${CONFIG_}$B" "${CONFIG_}$B=m" "${CONFIG_}$A" ;; - # undocumented because it ignores --file (fixme) --refresh) - yes "" | make oldconfig + yes "" | make oldconfig KCONFIG_CONFIG=$FN ;; *) -- cgit v1.2.3 From eb47ee018173f144f10eb38a3f7bd9f17ec6329e Mon Sep 17 00:00:00 2001 From: Masahiro Yamada <masahiroy@kernel.org> Date: Fri, 7 Mar 2025 00:56:22 +0900 Subject: kbuild: add Kbuild bash completion Kernel build commands can sometimes be long, particularly when cross-compiling, making them tedious to type and prone to mistypes. This commit introduces bash completion support for common variables and targets in Kbuild. For installation instructions, please refer to the documentation in Documentation/kbuild/bash-completion.rst. The following examples demonstrate how this saves typing. [Example 1] a long command line for cross-compiling $ make A<TAB> -> completes 'A' to 'ARCH=' $ make ARCH=<TAB> -> displays all supported architectures $ make ARCH=arm64 CR<TAB> -> completes 'CR' to 'CROSS_COMPILE=' $ make ARCH=arm64 CROSS_COMPILE=<TAB> -> displays installed toolchains $ make ARCH=arm64 CROSS_COMPILE=aa<TAB> -> completes 'CROSS_COMPILE=aa' to 'CROSS_COMPILE=aarch64-linux-gnu-' $ make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- def<TAB> -> completes 'def' to 'defconfig' [Example 2] a single build target $ make f<TAB> -> completes 'f' to 'fs/' $ make fs/<TAB> -> displays objects and sub-directories in fs/ $ make fs/xf<TAB> -> completes 'fs/xf' to 'fs/xfs/' $ make fs/xfs/l<TAB> -> completes 'fs/xfs/l' to 'fs/xfs/libxfs/xfs_' $ make fs/xfs/libxfs/xfs_g<TAB> -> completes 'fs/xfs/libxfs/xfs_g' to 'fs/xfs/libxfs/xfs_group.o' This does not aim to provide a complete list of variables and targets, as there are too many. However, it covers variables and targets used in common scenarios, and I hope this is useful enough. Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Reviewed-by: Nicolas Schier <n.schier@avm.de> Tested-by: Nicolas Schier <n.schier@avm.de> --- Documentation/kbuild/bash-completion.rst | 65 +++++ Documentation/kbuild/index.rst | 2 + MAINTAINERS | 1 + scripts/bash-completion/make | 451 +++++++++++++++++++++++++++++++ 4 files changed, 519 insertions(+) create mode 100644 Documentation/kbuild/bash-completion.rst create mode 100644 scripts/bash-completion/make (limited to 'scripts') diff --git a/Documentation/kbuild/bash-completion.rst b/Documentation/kbuild/bash-completion.rst new file mode 100644 index 000000000000..2b52dbcd0933 --- /dev/null +++ b/Documentation/kbuild/bash-completion.rst @@ -0,0 +1,65 @@ +.. SPDX-License-Identifier: GPL-2.0-only + +========================== +Bash completion for Kbuild +========================== + +The kernel build system is written using Makefiles, and Bash completion +for the `make` command is available through the `bash-completion`_ project. + +However, the Makefiles for the kernel build are complex. The generic completion +rules for the `make` command do not provide meaningful suggestions for the +kernel build system, except for the options of the `make` command itself. + +To enhance completion for various variables and targets, the kernel source +includes its own completion script at `scripts/bash-completion/make`. + +This script provides additional completions when working within the kernel tree. +Outside the kernel tree, it defaults to the generic completion rules for the +`make` command. + +Prerequisites +============= + +The script relies on helper functions provided by `bash-completion`_ project. +Please ensure it is installed on your system. On most distributions, you can +install the `bash-completion` package through the standard package manager. + +How to use +========== + +You can source the script directly:: + + $ source scripts/bash-completion/make + +Or, you can copy it into the search path for Bash completion scripts. +For example:: + + $ mkdir -p ~/.local/share/bash-completion/completions + $ cp scripts/bash-completion/make ~/.local/share/bash-completion/completions/ + +Details +======= + +The additional completion for Kbuild is enabled in the following cases: + + - You are in the root directory of the kernel source. + - You are in the top-level build directory created by the O= option + (checked via the `source` symlink pointing to the kernel source). + - The -C make option specifies the kernel source or build directory. + - The -f make option specifies a file in the kernel source or build directory. + +If none of the above are met, it falls back to the generic completion rules. + +The completion supports: + + - Commonly used targets, such as `all`, `menuconfig`, `dtbs`, etc. + - Make (or environment) variables, such as `ARCH`, `LLVM`, etc. + - Single-target builds (`foo/bar/baz.o`) + - Configuration files (`*_defconfig` and `*.config`) + +Some variables offer intelligent behavior. For instance, `CROSS_COMPILE=` +followed by a TAB displays installed toolchains. The list of defconfig files +shown depends on the value of the `ARCH=` variable. + +.. _bash-completion: https://github.com/scop/bash-completion/ diff --git a/Documentation/kbuild/index.rst b/Documentation/kbuild/index.rst index e82af05cd652..3731ab22bfe7 100644 --- a/Documentation/kbuild/index.rst +++ b/Documentation/kbuild/index.rst @@ -23,6 +23,8 @@ Kernel Build System llvm gendwarfksyms + bash-completion + .. only:: subproject and html Indices diff --git a/MAINTAINERS b/MAINTAINERS index ed7aa6867674..e4849f6ebc98 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -12566,6 +12566,7 @@ F: Makefile F: scripts/*vmlinux* F: scripts/Kbuild* F: scripts/Makefile* +F: scripts/bash-completion/ F: scripts/basic/ F: scripts/clang-tools/ F: scripts/dummy-tools/ diff --git a/scripts/bash-completion/make b/scripts/bash-completion/make new file mode 100644 index 000000000000..42e8dcead25a --- /dev/null +++ b/scripts/bash-completion/make @@ -0,0 +1,451 @@ +# SPDX-License-Identifier: GPL-2.0-only +# bash completion for GNU make with kbuild extension -*- shell-script -*- + +# Load the default completion script for make. It is typically located at +# /usr/share/bash-completion/completions/make, but we do not rely on it. +__kbuild_load_default_make_completion() +{ + local -a dirs=("${BASH_COMPLETION_USER_DIR:-${XDG_DATA_HOME:-$HOME/.local/share}/bash-completion}/completions") + local ifs=$IFS IFS=: dir compfile this_dir + + for dir in ${XDG_DATA_DIRS:-/usr/local/share:/usr/share}; do + dirs+=("$dir"/bash-completion/completions) + done + IFS=$ifs + + this_dir="$(realpath "$(dirname "${BASH_SOURCE[0]}")")" + + for dir in "${dirs[@]}"; do + if [[ ! -d ${dir} || ${dir} = "${this_dir}" ]]; then + continue + fi + + for compfile in make make.bash _make; do + compfile=$dir/$compfile + # Avoid trying to source dirs; https://bugzilla.redhat.com/903540 + if [[ -f ${compfile} ]] && . "${compfile}" &>/dev/null; then + + __kbuild_default_make_completion=$( + # shellcheck disable=SC2046 # word splitting is the point here + set -- $(complete -p make) + + while [[ $# -gt 1 && "$1" != -F ]]; do + shift + done + + if [[ "$1" = -F ]]; then + echo "$2" + fi + ) + + return + fi + done + done +} + +__kbuild_load_default_make_completion + +__kbuild_handle_variable() +{ + local var=${1%%=*} + local cur=${cur#"${var}"=} + local srctree=$2 + local keywords=() + + case $var in + ARCH) + # sub-directories under arch/ + keywords+=($(find "${srctree}/arch" -mindepth 1 -maxdepth 1 -type d -printf '%P\n')) + # architectures hard-coded in the top Makefile + keywords+=(i386 x86_64 sparc32 sparc64 parisc64) + ;; + CROSS_COMPILE) + # toolchains with a full path + local cross_compile=() + local c c2 + _filedir + + for c in "${COMPREPLY[@]}"; do + # eval for tilde expansion + # suppress error, as this fails when it contains a space + eval "c2=${c}" 2>/dev/null || continue + if [[ ${c} == *-elfedit && ! -d ${c2} && -x ${c2} ]]; then + cross_compile+=("${c%elfedit}") + fi + done + + # toolchains in the PATH environment + while read -r c; do + if [[ ${c} == *-elfedit ]]; then + keywords+=("${c%elfedit}") + fi + done < <(compgen -c) + + COMPREPLY=() + _filedir -d + + # Add cross_compile directly without passing it to compgen. + # Otherwise, toolchain paths with a tilde do not work. + # e.g.) + # CROSS_COMPILE=~/0day/gcc-14.2.0-nolibc/aarch64-linux/bin/aarch64-linux- + COMPREPLY+=("${cross_compile[@]}") + ;; + LLVM) + # LLVM=1 uses the default 'clang' etc. + keywords+=(1) + + # suffix for a particular version. LLVM=-18 uses 'clang-18' etc. + while read -r c; do + if [[ ${c} == clang-[0-9]* ]]; then + keywords+=("${c#clang}") + fi + done < <(compgen -c) + + # directory path to LLVM toolchains + _filedir -d + ;; + KCONFIG_ALLCONFIG) + # KCONFIG_ALLCONFIG=1 selects the default fragment + keywords+=(1) + # or the path to a fragment file + _filedir + ;; + C | KBUILD_CHECKSRC) + keywords+=(1 2) + ;; + V | KBUILD_VERBOSE) + keywords+=({,1}{,2}) + ;; + W | KBUILD_EXTRA_WARN) + keywords+=({,1}{,2}{,3}{,c}{,e}) + ;; + KBUILD_ABS_SRCTREE | KBUILD_MODPOST_NOFINAL | KBUILD_MODPOST_WARN | \ + CLIPPY | KBUILD_CLIPPY | KCONFIG_NOSILENTUPDATE | \ + KCONFIG_OVERWRITECONFIG | KCONFIG_WARN_UNKNOWN_SYMBOL | \ + KCONFIG_WERROR ) + keywords+=(1) + ;; + INSTALL_MOD_STRIP) + keywords+=(1 --strip-debug --strip-unneeded) + ;; + O | KBUILD_OUTPUT | M | KBUILD_EXTMOD | MO | KBUILD_EXTMOD_OUTPUT | *_PATH) + # variables that take a directory. + _filedir -d + return + ;; + KBUILD_EXTRA_SYMBOL | KBUILD_KCONFIG | KCONFIG_CONFIG) + # variables that take a file. + _filedir + return + esac + + COMPREPLY+=($(compgen -W "${keywords[*]}" -- "${cur}")) +} + +# Check the -C, -f options and 'source' symlink. Return the source tree we are +# working in. +__kbuild_get_srctree() +{ + local words=("$@") + local cwd makef_dir + + # see if a path was specified with -C/--directory + for ((i = 1; i < ${#words[@]}; i++)); do + if [[ ${words[i]} == -@(C|-directory) ]]; then + # eval for tilde expansion. + # suppress error, as this fails when it contains a space + eval "cwd=${words[i + 1]}" 2>/dev/null + break + fi + done + + if [[ -z ${cwd} ]]; then + cwd=. + fi + + # see if a Makefile was specified with -f/--file/--makefile + for ((i = 1; i < ${#words[@]}; i++)); do + if [[ ${words[i]} == -@(f|-?(make)file) ]]; then + # eval for tilde expansion + # suppress error, as this fails when it contains a space + eval "makef_dir=${words[i + 1]%/*}" 2>/dev/null + break + fi + done + + if [ -z "${makef_dir}" ]; then + makef_dir=${cwd} + elif [[ ${makef_dir} != /* ]]; then + makef_dir=${cwd}/${makef_dir} + fi + + # If ${makef_dir} is a build directory created by the O= option, there + # is a symbolic link 'source', which points to the kernel source tree. + if [[ -L ${makef_dir}/source ]]; then + makef_dir=$(readlink "${makef_dir}/source") + fi + + echo "${makef_dir}" +} + +# Get SRCARCH to do a little more clever things +__kbuild_get_srcarch() +{ + local words=("$@") + local arch srcarch uname_m + + # see if ARCH= is explicitly specified + for ((i = 1; i < ${#words[@]}; i++)); do + if [[ ${words[i]} == ARCH=* ]]; then + arch=${words[i]#ARCH=} + break + fi + done + + # If ARCH= is not specified, check the build marchine's architecture + if [[ -z ${arch} ]]; then + uname_m=$(uname -m) + + # shellcheck disable=SC2209 # 'sh' is SuperH, not a shell command + case ${uname_m} in + arm64 | aarch64*) arch=arm64 ;; + arm* | sa110) arch=arm ;; + i?86 | x86_64) arch=x86 ;; + loongarch*) arch=loongarch ;; + mips*) arch=mips ;; + ppc*) arch=powerpc ;; + riscv*) arch=riscv ;; + s390x) arch=s390 ;; + sh[234]*) arch=sh ;; + sun4u) arch=sparc64 ;; + *) arch=${uname_m} ;; + esac + fi + + case ${arch} in + parisc64) srcarch=parisc ;; + sparc32 | sparc64) srcarch=sparc ;; + i386 | x86_64) srcarch=x86 ;; + *) srcarch=${arch} ;; + esac + + echo "$srcarch" +} + +# small Makefile to parse obj-* syntax +__kbuild_tmp_makefile() +{ +cat <<'EOF' +.PHONY: __default +__default: + $(foreach m,$(obj-y) $(obj-m) $(obj-),$(foreach s, -objs -y -m -,$($(m:%.o=%$s))) $(m)) +EOF +echo "include ${1}" +} + +_make_for_kbuild () +{ + # shellcheck disable=SC2034 # these are set by _init_completion + local cur prev words cword split + _init_completion -s || return + + local srctree + srctree=$(__kbuild_get_srctree "${words[@]}") + + # If 'kernel' and 'Documentation' directories are found, we assume this + # is a kernel tree. Otherwise, we fall back to the generic rule provided + # by the bash-completion project. + if [[ ! -d ${srctree}/kernel || ! -d ${srctree}/Documentation ]]; then + if [ -n "${__kbuild_default_make_completion}" ]; then + "${__kbuild_default_make_completion}" "$@" + fi + return + fi + + # make options with a parameter (copied from the bash-completion project) + case ${prev} in + --file | --makefile | --old-file | --assume-old | --what-if | --new-file | \ + --assume-new | -!(-*)[foW]) + _filedir + return + ;; + --include-dir | --directory | -!(-*)[ICm]) + _filedir -d + return + ;; + -!(-*)E) + COMPREPLY=($(compgen -v -- "$cur")) + return + ;; + --eval | -!(-*)[DVx]) + return + ;; + --jobs | -!(-*)j) + COMPREPLY=($(compgen -W "{1..$(($(_ncpus) * 2))}" -- "$cur")) + return + ;; + esac + + local keywords=() + + case ${cur} in + -*) + # make options (copied from the bash-completion project) + local opts + opts="$(_parse_help "$1")" + COMPREPLY=($(compgen -W "${opts:-$(_parse_usage "$1")}" -- "$cur")) + if [[ ${COMPREPLY-} == *= ]]; then + compopt -o nospace + fi + return + ;; + *=*) + __kbuild_handle_variable "${cur}" "${srctree}" + return + ;; + KBUILD_*) + # There are many variables prefixed with 'KBUILD_'. + # Display them only when 'KBUILD_' is entered. + # shellcheck disable=SC2191 # '=' is appended for variables + keywords+=( + KBUILD_{CHECKSRC,EXTMOD,EXTMOD_OUTPUT,OUTPUT,VERBOSE,EXTRA_WARN,CLIPPY}= + KBUILD_BUILD_{USER,HOST,TIMESTAMP}= + KBUILD_MODPOST_{NOFINAL,WARN}= + KBUILD_{ABS_SRCTREE,EXTRA_SYMBOLS,KCONFIG}= + ) + ;; + KCONFIG_*) + # There are many variables prefixed with 'KCONFIG_'. + # Display them only when 'KCONFIG_' is entered. + # shellcheck disable=SC2191 # '=' is appended for variables + keywords+=( + KCONFIG_{CONFIG,ALLCONFIG,NOSILENTUPDATE,OVERWRITECONFIG}= + KCONFIG_{SEED,PROBABILITY}= + KCONFIG_WARN_UNKNOWN_SYMBOL= + KCONFIG_WERROR= + ) + ;; + *) + # By default, hide KBUILD_* and KCONFIG_* variables. + # Instead, display only the prefix parts. + keywords+=(KBUILD_ KCONFIG_) + ;; + esac + + if [[ ${cur} != /* && ${cur} != *//* ]]; then + local dir srcarch kbuild_file tmp + srcarch=$(__kbuild_get_srcarch "${words[@]}") + + # single build + dir=${cur} + while true; do + if [[ ${dir} == */* ]]; then + dir=${dir%/*} + else + dir=. + fi + + # Search for 'Kbuild' or 'Makefile' in the parent + # directories (may not be a direct parent) + if [[ -f ${srctree}/${dir}/Kbuild ]]; then + kbuild_file=${srctree}/${dir}/Kbuild + break + fi + if [[ -f ${srctree}/${dir}/Makefile ]]; then + kbuild_file=${srctree}/${dir}/Makefile + break + fi + + if [[ ${dir} == . ]]; then + break + fi + done + + if [[ -n ${kbuild_file} ]]; then + tmp=($(__kbuild_tmp_makefile "${kbuild_file}" | + SRCARCH=${srcarch} obj=${dir} src=${srctree}/${dir} \ + "${1}" -n -f - 2>/dev/null)) + + # Add $(obj)/ prefix + if [[ ${dir} != . ]]; then + tmp=("${tmp[@]/#/${dir}\/}") + fi + + keywords+=("${tmp[@]}") + fi + + # *_defconfig and *.config files. These might be grouped into + # subdirectories, e.g., arch/powerpc/configs/*/*_defconfig. + if [[ ${cur} == */* ]]; then + dir=${cur%/*} + else + dir=. + fi + + tmp=($(find "${srctree}/arch/${srcarch}/configs/${dir}" \ + "${srctree}/kernel/configs/${dir}" \ + -mindepth 1 -maxdepth 1 -type d -printf '%P/\n' \ + -o -printf '%P\n' 2>/dev/null)) + + if [[ ${dir} != . ]]; then + tmp=("${tmp[@]/#/${dir}\/}") + fi + + keywords+=("${tmp[@]}") + fi + + # shellcheck disable=SC2191 # '=' is appended for variables + keywords+=( + # + # variables (append =) + # + ARCH= + CROSS_COMPILE= + LLVM= + C= M= MO= O= V= W= + INSTALL{,_MOD,_HDR,_DTBS}_PATH= + KERNELRELEASE= + + # + # targets + # + all help + clean mrproper distclean + clang-{tidy,analyzer} compile_commands.json + coccicheck + dtbs{,_check,_install} dt_binding_{check,schemas} + headers{,_install} + vmlinux install + modules{,_prepare,_install,_sign} + vdso_install + tags TAGS cscope gtags + rust{available,fmt,fmtcheck} + kernel{version,release} image_name + kselftest{,-all,-install,-clean,-merge} + + # configuration + {,old,olddef,sync,def,savedef,rand,listnew,helpnew,test,tiny}config + {,build_}{menu,n,g,x}config + local{mod,yes}config + all{no,yes,mod,def}config + {yes2mod,mod2yes,mod2no}config + + # docs + {html,textinfo,info,latex,pdf,epub,xml,linkcheck,refcheck,clean}docs + + # package + {,bin,src}{rpm,deb}-pkg + {pacman,dir,tar}-pkg + tar{,gz,bz2,xz,zst}-pkg + perf-tar{,gz,bz2,xz,zst}-src-pkg + ) + + COMPREPLY=($(compgen -W "${keywords[*]}" -- "${cur}")) + + # Do not append a space for variables, subdirs, "KBUILD_", "KCONFIG_". + if [[ ${COMPREPLY-} == *[=/] || ${COMPREPLY-} =~ ^(KBUILD|KCONFIG)_$ ]]; then + compopt -o nospace + fi + +} && complete -F _make_for_kbuild make -- cgit v1.2.3 From 87bb368d0637c466a8a77433837056f981d01991 Mon Sep 17 00:00:00 2001 From: Kris Van Hees <kris.van.hees@oracle.com> Date: Fri, 7 Mar 2025 11:53:28 -0500 Subject: kbuild: exclude .rodata.(cst|str)* when building ranges The .rodata.(cst|str)* sections are often resized during the final linking and since these sections do not cover actual symbols there is no need to include them in the modules.builtin.ranges data. When these sections were included in processing and resizing occurred, modules were reported with ranges that extended beyond their true end, causing subsequent symbols (in address order) to be associated with the wrong module. Fixes: 5f5e7344322f ("kbuild: generate offset range data for builtin modules") Cc: stable@vger.kernel.org Signed-off-by: Kris Van Hees <kris.van.hees@oracle.com> Reviewed-by: Jack Vogel <jack.vogel@oracle.com> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> --- scripts/generate_builtin_ranges.awk | 5 +++++ 1 file changed, 5 insertions(+) (limited to 'scripts') diff --git a/scripts/generate_builtin_ranges.awk b/scripts/generate_builtin_ranges.awk index b9ec761b3bef..d4bd5c2b998c 100755 --- a/scripts/generate_builtin_ranges.awk +++ b/scripts/generate_builtin_ranges.awk @@ -282,6 +282,11 @@ ARGIND == 2 && !anchor && NF == 2 && $1 ~ /^0x/ && $2 !~ /^0x/ { # section. # ARGIND == 2 && sect && NF == 4 && /^ [^ \*]/ && !($1 in sect_addend) { + # There are a few sections with constant data (without symbols) that + # can get resized during linking, so it is best to ignore them. + if ($1 ~ /^\.rodata\.(cst|str)[0-9]/) + next; + if (!($1 in sect_base)) { sect_base[$1] = base; -- cgit v1.2.3 From ba4d705046fb568bad4aeffeb79db78d4e835a1f Mon Sep 17 00:00:00 2001 From: Masahiro Yamada <masahiroy@kernel.org> Date: Thu, 13 Mar 2025 19:26:03 +0900 Subject: kbuild: do not generate .tmp_vmlinux*.map when CONFIG_VMLINUX_MAP=y Commit 5cc124720461 ("kbuild: add CONFIG_VMLINUX_MAP expert option") mentioned that "the .map file can be rather large (several MB), and that's a waste of space when one isn't interested in these things." If that is the case, generating map files for the intermediate tmp_vmlinux* files is also a waste of space. It is unlikely that anyone would be interested in the .tmp_vmlinux*.map files. This commit stops passing the -Map= option when linking the .tmp_vmlinux* intermediates. I also hard-coded the file name 'vmlinux.map' instead of ${output}.map because a later commit will introduce vmlinux.unstripped but I want to keep the current name of the map file. Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Acked-by: Ard Biesheuvel <ardb@kernel.org> --- scripts/link-vmlinux.sh | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) (limited to 'scripts') diff --git a/scripts/link-vmlinux.sh b/scripts/link-vmlinux.sh index 56a077d204cf..96ae0bb65308 100755 --- a/scripts/link-vmlinux.sh +++ b/scripts/link-vmlinux.sh @@ -97,8 +97,8 @@ vmlinux_link() ldflags="${ldflags} ${wl}--strip-debug" fi - if is_enabled CONFIG_VMLINUX_MAP; then - ldflags="${ldflags} ${wl}-Map=${output}.map" + if [ -n "${generate_map}" ]; then + ldflags="${ldflags} ${wl}-Map=vmlinux.map" fi ${ld} ${ldflags} -o ${output} \ @@ -210,6 +210,7 @@ fi btf_vmlinux_bin_o= kallsymso= strip_debug= +generate_map= if is_enabled CONFIG_KALLSYMS; then true > .tmp_vmlinux0.syms @@ -278,6 +279,10 @@ fi strip_debug= +if is_enabled CONFIG_VMLINUX_MAP; then + generate_map=1 +fi + vmlinux_link vmlinux # fill in BTF IDs -- cgit v1.2.3 From e22bbb8e97846bfb5a6942a2322f0237ff13df0f Mon Sep 17 00:00:00 2001 From: Ard Biesheuvel <ardb@kernel.org> Date: Tue, 11 Mar 2025 12:06:18 +0100 Subject: kbuild: link-vmlinux.sh: Make output file name configurable In order to introduce an intermediate, non-stripped vmlinux build that can be used by other build steps as an input, pass the output file name to link-vmlinux.sh via its command line. Signed-off-by: Ard Biesheuvel <ardb@kernel.org> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> --- scripts/Makefile.vmlinux | 2 +- scripts/link-vmlinux.sh | 15 ++++++++------- 2 files changed, 9 insertions(+), 8 deletions(-) (limited to 'scripts') diff --git a/scripts/Makefile.vmlinux b/scripts/Makefile.vmlinux index fb79fd6b2465..487f0bf716ad 100644 --- a/scripts/Makefile.vmlinux +++ b/scripts/Makefile.vmlinux @@ -69,7 +69,7 @@ ARCH_POSTLINK := $(wildcard $(srctree)/arch/$(SRCARCH)/Makefile.postlink) # Final link of vmlinux with optional arch pass after final link cmd_link_vmlinux = \ - $< "$(LD)" "$(KBUILD_LDFLAGS)" "$(LDFLAGS_vmlinux)"; \ + $< "$(LD)" "$(KBUILD_LDFLAGS)" "$(LDFLAGS_vmlinux)" "$@"; \ $(if $(ARCH_POSTLINK), $(MAKE) -f $(ARCH_POSTLINK) $@, true) targets += vmlinux diff --git a/scripts/link-vmlinux.sh b/scripts/link-vmlinux.sh index 96ae0bb65308..b3d928925598 100755 --- a/scripts/link-vmlinux.sh +++ b/scripts/link-vmlinux.sh @@ -31,6 +31,7 @@ set -e LD="$1" KBUILD_LDFLAGS="$2" LDFLAGS_vmlinux="$3" +VMLINUX="$4" is_enabled() { grep -q "^$1=y" include/config/auto.conf @@ -283,23 +284,23 @@ if is_enabled CONFIG_VMLINUX_MAP; then generate_map=1 fi -vmlinux_link vmlinux +vmlinux_link "${VMLINUX}" # fill in BTF IDs if is_enabled CONFIG_DEBUG_INFO_BTF; then - info BTFIDS vmlinux + info BTFIDS "${VMLINUX}" RESOLVE_BTFIDS_ARGS="" if is_enabled CONFIG_WERROR; then RESOLVE_BTFIDS_ARGS=" --fatal_warnings " fi - ${RESOLVE_BTFIDS} ${RESOLVE_BTFIDS_ARGS} vmlinux + ${RESOLVE_BTFIDS} ${RESOLVE_BTFIDS_ARGS} "${VMLINUX}" fi -mksysmap vmlinux System.map +mksysmap "${VMLINUX}" System.map if is_enabled CONFIG_BUILDTIME_TABLE_SORT; then - info SORTTAB vmlinux - if ! sorttable vmlinux; then + info SORTTAB "${VMLINUX}" + if ! sorttable "${VMLINUX}"; then echo >&2 Failed to sort kernel tables exit 1 fi @@ -315,4 +316,4 @@ if is_enabled CONFIG_KALLSYMS; then fi # For fixdep -echo "vmlinux: $0" > .vmlinux.d +echo "${VMLINUX}: $0" > ".${VMLINUX}.d" -- cgit v1.2.3 From ac4f06789b4f9c17357e81e918879c6e2ffdd075 Mon Sep 17 00:00:00 2001 From: Ard Biesheuvel <ardb@kernel.org> Date: Tue, 11 Mar 2025 12:06:20 +0100 Subject: kbuild: Create intermediate vmlinux build with relocations preserved The imperative paradigm used to build vmlinux, extract some info from it or perform some checks on it, and subsequently modify it again goes against the declarative paradigm that is usually employed for defining make rules. In particular, the Makefile.postlink files that consume their input via an output rule result in some dodgy logic in the decompressor makefiles for RISC-V and x86, given that the vmlinux.relocs input file needed to generate the arch-specific relocation tables may not exist or be out of date, but cannot be constructed using the ordinary Make dependency based rules, because the info needs to be extracted while vmlinux is in its ephemeral, non-stripped form. So instead, for architectures that require the static relocations that are emitted into vmlinux when passing --emit-relocs to the linker, and are subsequently stripped out again, introduce an intermediate vmlinux target called vmlinux.unstripped, and organize the reset of the build logic accordingly: - vmlinux.unstripped is created only once, and not updated again - build rules under arch/*/boot can depend on vmlinux.unstripped without running the risk of the data disappearing or being out of date - the final vmlinux generated by the build is not bloated with static relocations that are never needed again after the build completes. Signed-off-by: Ard Biesheuvel <ardb@kernel.org> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> --- .gitignore | 1 + Makefile | 2 +- arch/mips/Makefile.postlink | 2 +- arch/riscv/Makefile.postlink | 11 +---------- arch/riscv/boot/Makefile | 5 +---- arch/s390/Makefile.postlink | 4 +--- arch/x86/Makefile.postlink | 8 +++----- scripts/Makefile.lib | 3 --- scripts/Makefile.vmlinux | 28 +++++++++++++++++++++------- 9 files changed, 30 insertions(+), 34 deletions(-) (limited to 'scripts') diff --git a/.gitignore b/.gitignore index 5937c74d3dc1..f2f63e47fb88 100644 --- a/.gitignore +++ b/.gitignore @@ -65,6 +65,7 @@ modules.order /vmlinux.32 /vmlinux.map /vmlinux.symvers +/vmlinux.unstripped /vmlinux-gdb.py /vmlinuz /System.map diff --git a/Makefile b/Makefile index 0904b73872cb..22593a774cf6 100644 --- a/Makefile +++ b/Makefile @@ -1569,7 +1569,7 @@ endif # CONFIG_MODULES # Directories & files removed with 'make clean' CLEAN_FILES += vmlinux.symvers modules-only.symvers \ modules.builtin modules.builtin.modinfo modules.nsdeps \ - modules.builtin.ranges vmlinux.o.map \ + modules.builtin.ranges vmlinux.o.map vmlinux.unstripped \ compile_commands.json rust/test \ rust-project.json .vmlinux.objs .vmlinux.export.c \ .builtin-dtbs-list .builtin-dtb.S diff --git a/arch/mips/Makefile.postlink b/arch/mips/Makefile.postlink index 6cfdc149d3bc..ea0add7d56b2 100644 --- a/arch/mips/Makefile.postlink +++ b/arch/mips/Makefile.postlink @@ -22,7 +22,7 @@ quiet_cmd_relocs = RELOCS $@ # `@true` prevents complaint when there is nothing to be done -vmlinux: FORCE +vmlinux vmlinux.unstripped: FORCE @true ifeq ($(CONFIG_CPU_LOONGSON3_WORKAROUNDS),y) $(call if_changed,ls3_llsc) diff --git a/arch/riscv/Makefile.postlink b/arch/riscv/Makefile.postlink index 6b0580949b6a..0e4cf8ad2f14 100644 --- a/arch/riscv/Makefile.postlink +++ b/arch/riscv/Makefile.postlink @@ -10,26 +10,17 @@ __archpost: -include include/config/auto.conf include $(srctree)/scripts/Kbuild.include -include $(srctree)/scripts/Makefile.lib quiet_cmd_relocs_check = CHKREL $@ cmd_relocs_check = \ $(CONFIG_SHELL) $(srctree)/arch/riscv/tools/relocs_check.sh "$(OBJDUMP)" "$(NM)" "$@" -ifdef CONFIG_RELOCATABLE -quiet_cmd_cp_vmlinux_relocs = CPREL vmlinux.relocs -cmd_cp_vmlinux_relocs = cp vmlinux vmlinux.relocs - -endif - # `@true` prevents complaint when there is nothing to be done -vmlinux: FORCE +vmlinux vmlinux.unstripped: FORCE @true ifdef CONFIG_RELOCATABLE $(call if_changed,relocs_check) - $(call if_changed,cp_vmlinux_relocs) - $(call if_changed,strip_relocs) endif clean: diff --git a/arch/riscv/boot/Makefile b/arch/riscv/boot/Makefile index b25d524ce5eb..bfc3d0b75b9b 100644 --- a/arch/riscv/boot/Makefile +++ b/arch/riscv/boot/Makefile @@ -32,10 +32,7 @@ $(obj)/xipImage: vmlinux FORCE endif ifdef CONFIG_RELOCATABLE -vmlinux.relocs: vmlinux - @ (! [ -f vmlinux.relocs ] && echo "vmlinux.relocs can't be found, please remove vmlinux and try again") || true - -$(obj)/Image: vmlinux.relocs FORCE +$(obj)/Image: vmlinux.unstripped FORCE else $(obj)/Image: vmlinux FORCE endif diff --git a/arch/s390/Makefile.postlink b/arch/s390/Makefile.postlink index 1ae5478cd6ac..c2b737500a91 100644 --- a/arch/s390/Makefile.postlink +++ b/arch/s390/Makefile.postlink @@ -11,7 +11,6 @@ __archpost: -include include/config/auto.conf include $(srctree)/scripts/Kbuild.include -include $(srctree)/scripts/Makefile.lib CMD_RELOCS=arch/s390/tools/relocs OUT_RELOCS = arch/s390/boot @@ -20,9 +19,8 @@ quiet_cmd_relocs = RELOCS $(OUT_RELOCS)/relocs.S mkdir -p $(OUT_RELOCS); \ $(CMD_RELOCS) $@ > $(OUT_RELOCS)/relocs.S -vmlinux: FORCE +vmlinux.unstripped: FORCE $(call cmd,relocs) - $(call cmd,strip_relocs) clean: @rm -f $(OUT_RELOCS)/relocs.S diff --git a/arch/x86/Makefile.postlink b/arch/x86/Makefile.postlink index 8b8a68162c94..445fce66630f 100644 --- a/arch/x86/Makefile.postlink +++ b/arch/x86/Makefile.postlink @@ -11,23 +11,21 @@ __archpost: -include include/config/auto.conf include $(srctree)/scripts/Kbuild.include -include $(srctree)/scripts/Makefile.lib CMD_RELOCS = arch/x86/tools/relocs OUT_RELOCS = arch/x86/boot/compressed -quiet_cmd_relocs = RELOCS $(OUT_RELOCS)/$@.relocs +quiet_cmd_relocs = RELOCS $(OUT_RELOCS)/vmlinux.relocs cmd_relocs = \ mkdir -p $(OUT_RELOCS); \ - $(CMD_RELOCS) $@ > $(OUT_RELOCS)/$@.relocs; \ + $(CMD_RELOCS) $@ > $(OUT_RELOCS)/vmlinux.relocs; \ $(CMD_RELOCS) --abs-relocs $@ # `@true` prevents complaint when there is nothing to be done -vmlinux: FORCE +vmlinux vmlinux.unstripped: FORCE @true ifeq ($(CONFIG_X86_NEED_RELOCS),y) $(call cmd,relocs) - $(call cmd,strip_relocs) endif clean: diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib index 47f56ef71937..d1856a70c919 100644 --- a/scripts/Makefile.lib +++ b/scripts/Makefile.lib @@ -371,9 +371,6 @@ quiet_cmd_ar = AR $@ quiet_cmd_objcopy = OBJCOPY $@ cmd_objcopy = $(OBJCOPY) $(OBJCOPYFLAGS) $(OBJCOPYFLAGS_$(@F)) $< $@ -quiet_cmd_strip_relocs = RSTRIP $@ -cmd_strip_relocs = $(OBJCOPY) --remove-section='.rel*' $@ - # Gzip # --------------------------------------------------------------------------- diff --git a/scripts/Makefile.vmlinux b/scripts/Makefile.vmlinux index 487f0bf716ad..b0a6cd5b818c 100644 --- a/scripts/Makefile.vmlinux +++ b/scripts/Makefile.vmlinux @@ -9,6 +9,20 @@ include $(srctree)/scripts/Makefile.lib targets := +ifdef CONFIG_ARCH_VMLINUX_NEEDS_RELOCS +vmlinux-final := vmlinux.unstripped + +quiet_cmd_strip_relocs = RSTRIP $@ + cmd_strip_relocs = $(OBJCOPY) --remove-section='.rel*' $< $@ + +vmlinux: $(vmlinux-final) FORCE + $(call if_changed,strip_relocs) + +targets += vmlinux +else +vmlinux-final := vmlinux +endif + %.o: %.c FORCE $(call if_changed_rule,cc_o_c) @@ -47,7 +61,7 @@ targets += .builtin-dtbs-list ifdef CONFIG_GENERIC_BUILTIN_DTB targets += .builtin-dtbs.S .builtin-dtbs.o -vmlinux: .builtin-dtbs.o +$(vmlinux-final): .builtin-dtbs.o endif # vmlinux @@ -55,11 +69,11 @@ endif ifdef CONFIG_MODULES targets += .vmlinux.export.o -vmlinux: .vmlinux.export.o +$(vmlinux-final): .vmlinux.export.o endif ifdef CONFIG_ARCH_WANTS_PRE_LINK_VMLINUX -vmlinux: arch/$(SRCARCH)/tools/vmlinux.arch.o +$(vmlinux-final): arch/$(SRCARCH)/tools/vmlinux.arch.o arch/$(SRCARCH)/tools/vmlinux.arch.o: vmlinux.o FORCE $(Q)$(MAKE) $(build)=arch/$(SRCARCH)/tools $@ @@ -72,11 +86,11 @@ cmd_link_vmlinux = \ $< "$(LD)" "$(KBUILD_LDFLAGS)" "$(LDFLAGS_vmlinux)" "$@"; \ $(if $(ARCH_POSTLINK), $(MAKE) -f $(ARCH_POSTLINK) $@, true) -targets += vmlinux -vmlinux: scripts/link-vmlinux.sh vmlinux.o $(KBUILD_LDS) FORCE +targets += $(vmlinux-final) +$(vmlinux-final): scripts/link-vmlinux.sh vmlinux.o $(KBUILD_LDS) FORCE +$(call if_changed_dep,link_vmlinux) ifdef CONFIG_DEBUG_INFO_BTF -vmlinux: $(RESOLVE_BTFIDS) +$(vmlinux-final): $(RESOLVE_BTFIDS) endif ifdef CONFIG_BUILDTIME_TABLE_SORT @@ -96,7 +110,7 @@ modules.builtin.ranges: $(srctree)/scripts/generate_builtin_ranges.awk \ modules.builtin vmlinux.map vmlinux.o.map FORCE $(call if_changed,modules_builtin_ranges) -vmlinux.map: vmlinux +vmlinux.map: $(vmlinux-final) @: endif -- cgit v1.2.3 From d7659acca7a390b5830f0b67f3aa4a5f9929ab79 Mon Sep 17 00:00:00 2001 From: Miguel Ojeda <ojeda@kernel.org> Date: Sat, 8 Mar 2025 11:05:01 +0000 Subject: rust: add pin-init crate build infrastructure Add infrastructure for moving the initialization API to its own crate. Covers all make targets such as `rust-analyzer` and `rustdoc`. The tests of pin-init are not added to `rusttest`, as they are already tested in the user-space repository [1]. Link: https://github.com/Rust-for-Linux/pin-init [1] Co-developed-by: Benno Lossin <benno.lossin@proton.me> Signed-off-by: Benno Lossin <benno.lossin@proton.me> Tested-by: Andreas Hindborg <a.hindborg@kernel.org> Link: https://lore.kernel.org/r/20250308110339.2997091-15-benno.lossin@proton.me Signed-off-by: Miguel Ojeda <ojeda@kernel.org> --- rust/Makefile | 75 +++++++++++++++++++++++++++++--------- rust/pin-init/internal/src/_lib.rs | 3 ++ rust/pin-init/internal/src/lib.rs | 4 ++ rust/pin-init/src/_lib.rs | 5 +++ scripts/Makefile.build | 2 +- scripts/generate_rust_analyzer.py | 17 ++++++++- 6 files changed, 86 insertions(+), 20 deletions(-) create mode 100644 rust/pin-init/internal/src/_lib.rs create mode 100644 rust/pin-init/src/_lib.rs (limited to 'scripts') diff --git a/rust/Makefile b/rust/Makefile index ea3849eb78f6..90310f0620eb 100644 --- a/rust/Makefile +++ b/rust/Makefile @@ -12,7 +12,7 @@ obj-$(CONFIG_RUST) += helpers/helpers.o CFLAGS_REMOVE_helpers/helpers.o = -Wmissing-prototypes -Wmissing-declarations always-$(CONFIG_RUST) += bindings/bindings_generated.rs bindings/bindings_helpers_generated.rs -obj-$(CONFIG_RUST) += bindings.o kernel.o +obj-$(CONFIG_RUST) += bindings.o pin_init.o kernel.o always-$(CONFIG_RUST) += exports_helpers_generated.h \ exports_bindings_generated.h exports_kernel_generated.h @@ -41,7 +41,10 @@ ifdef CONFIG_RUST libmacros_name := $(shell MAKEFLAGS= $(RUSTC) --print file-names --crate-name macros --crate-type proc-macro - </dev/null) libmacros_extension := $(patsubst libmacros.%,%,$(libmacros_name)) -always-$(CONFIG_RUST) += $(libmacros_name) +libpin_init_internal_name := $(shell MAKEFLAGS= $(RUSTC) --print file-names --crate-name pin_init_internal --crate-type proc-macro - </dev/null) +libpin_init_internal_extension := $(patsubst libpin_init_internal.%,%,$(libpin_init_internal_name)) + +always-$(CONFIG_RUST) += $(libmacros_name) $(libpin_init_internal_name) # `$(rust_flags)` is passed in case the user added `--sysroot`. rustc_sysroot := $(shell MAKEFLAGS= $(RUSTC) $(rust_flags) --print sysroot) @@ -80,7 +83,7 @@ quiet_cmd_rustdoc = RUSTDOC $(if $(rustdoc_host),H, ) $< # command-like flags to solve the issue. Meanwhile, we use the non-custom case # and then retouch the generated files. rustdoc: rustdoc-core rustdoc-macros rustdoc-compiler_builtins \ - rustdoc-kernel + rustdoc-kernel rustdoc-pin_init $(Q)cp $(srctree)/Documentation/images/logo.svg $(rustdoc_output)/static.files/ $(Q)cp $(srctree)/Documentation/images/COPYING-logo $(rustdoc_output)/static.files/ $(Q)find $(rustdoc_output) -name '*.html' -type f -print0 | xargs -0 sed -Ei \ @@ -110,11 +113,24 @@ rustdoc-compiler_builtins: $(src)/compiler_builtins.rs rustdoc-core FORCE rustdoc-ffi: $(src)/ffi.rs rustdoc-core FORCE +$(call if_changed,rustdoc) -rustdoc-kernel: private rustc_target_flags = --extern ffi \ +rustdoc-pin_init_internal: private rustdoc_host = yes +rustdoc-pin_init_internal: private rustc_target_flags = --cfg kernel \ + --extern proc_macro --crate-type proc-macro +rustdoc-pin_init_internal: $(src)/pin-init/internal/src/_lib.rs FORCE + +$(call if_changed,rustdoc) + +rustdoc-pin_init: private rustdoc_host = yes +rustdoc-pin_init: private rustc_target_flags = --extern pin_init_internal \ + --extern macros --extern alloc --cfg kernel --cfg feature=\"alloc\" +rustdoc-pin_init: $(src)/pin-init/src/_lib.rs rustdoc-pin_init_internal \ + rustdoc-macros FORCE + +$(call if_changed,rustdoc) + +rustdoc-kernel: private rustc_target_flags = --extern ffi --extern pin_init \ --extern build_error --extern macros \ --extern bindings --extern uapi rustdoc-kernel: $(src)/kernel/lib.rs rustdoc-core rustdoc-ffi rustdoc-macros \ - rustdoc-compiler_builtins $(obj)/$(libmacros_name) \ + rustdoc-pin_init rustdoc-compiler_builtins $(obj)/$(libmacros_name) \ $(obj)/bindings.o FORCE +$(call if_changed,rustdoc) @@ -139,12 +155,24 @@ rusttestlib-macros: private rustc_test_library_proc = yes rusttestlib-macros: $(src)/macros/lib.rs FORCE +$(call if_changed,rustc_test_library) +rusttestlib-pin_init_internal: private rustc_target_flags = --cfg kernel \ + --extern proc_macro +rusttestlib-pin_init_internal: private rustc_test_library_proc = yes +rusttestlib-pin_init_internal: $(src)/pin-init/internal/src/_lib.rs FORCE + +$(call if_changed,rustc_test_library) + +rusttestlib-pin_init: private rustc_target_flags = --extern pin_init_internal \ + --extern macros --cfg kernel +rusttestlib-pin_init: $(src)/pin-init/src/_lib.rs rusttestlib-macros \ + rusttestlib-pin_init_internal $(obj)/$(libpin_init_internal_name) FORCE + +$(call if_changed,rustc_test_library) + rusttestlib-kernel: private rustc_target_flags = --extern ffi \ - --extern build_error --extern macros \ + --extern build_error --extern macros --extern pin_init \ --extern bindings --extern uapi -rusttestlib-kernel: $(src)/kernel/lib.rs \ - rusttestlib-bindings rusttestlib-uapi rusttestlib-build_error \ - $(obj)/$(libmacros_name) $(obj)/bindings.o FORCE +rusttestlib-kernel: $(src)/kernel/lib.rs rusttestlib-bindings rusttestlib-uapi \ + rusttestlib-build_error rusttestlib-pin_init $(obj)/$(libmacros_name) \ + $(obj)/bindings.o FORCE +$(call if_changed,rustc_test_library) rusttestlib-bindings: private rustc_target_flags = --extern ffi @@ -172,8 +200,8 @@ quiet_cmd_rustdoc_test_kernel = RUSTDOC TK $< mkdir -p $(objtree)/$(obj)/test/doctests/kernel; \ OBJTREE=$(abspath $(objtree)) \ $(RUSTDOC) --test $(rust_flags) \ - -L$(objtree)/$(obj) --extern ffi --extern kernel \ - --extern build_error --extern macros \ + -L$(objtree)/$(obj) --extern ffi --extern pin_init \ + --extern kernel --extern build_error --extern macros \ --extern bindings --extern uapi \ --no-run --crate-name kernel -Zunstable-options \ --sysroot=/dev/null \ @@ -203,18 +231,18 @@ quiet_cmd_rustc_test = $(RUSTC_OR_CLIPPY_QUIET) T $< rusttest: rusttest-macros rusttest-kernel rusttest-macros: private rustc_target_flags = --extern proc_macro \ - --extern macros --extern kernel + --extern macros --extern kernel --extern pin_init rusttest-macros: private rustdoc_test_target_flags = --crate-type proc-macro rusttest-macros: $(src)/macros/lib.rs \ - rusttestlib-macros rusttestlib-kernel FORCE + rusttestlib-macros rusttestlib-kernel rusttestlib-pin_init FORCE +$(call if_changed,rustc_test) +$(call if_changed,rustdoc_test) -rusttest-kernel: private rustc_target_flags = --extern ffi \ +rusttest-kernel: private rustc_target_flags = --extern ffi --extern pin_init \ --extern build_error --extern macros --extern bindings --extern uapi rusttest-kernel: $(src)/kernel/lib.rs rusttestlib-ffi rusttestlib-kernel \ rusttestlib-build_error rusttestlib-macros rusttestlib-bindings \ - rusttestlib-uapi FORCE + rusttestlib-uapi rusttestlib-pin_init FORCE +$(call if_changed,rustc_test) ifdef CONFIG_CC_IS_CLANG @@ -361,7 +389,7 @@ $(obj)/exports_kernel_generated.h: $(obj)/kernel.o FORCE quiet_cmd_rustc_procmacro = $(RUSTC_OR_CLIPPY_QUIET) P $@ cmd_rustc_procmacro = \ - $(RUSTC_OR_CLIPPY) $(rust_common_flags) \ + $(RUSTC_OR_CLIPPY) $(rust_common_flags) $(rustc_target_flags) \ -Clinker-flavor=gcc -Clinker=$(HOSTCC) \ -Clink-args='$(call escsq,$(KBUILD_PROCMACROLDFLAGS))' \ --emit=dep-info=$(depfile) --emit=link=$@ --extern proc_macro \ @@ -372,6 +400,10 @@ quiet_cmd_rustc_procmacro = $(RUSTC_OR_CLIPPY_QUIET) P $@ $(obj)/$(libmacros_name): $(src)/macros/lib.rs FORCE +$(call if_changed_dep,rustc_procmacro) +$(obj)/$(libpin_init_internal_name): private rustc_target_flags = --cfg kernel +$(obj)/$(libpin_init_internal_name): $(src)/pin-init/internal/src/_lib.rs FORCE + +$(call if_changed_dep,rustc_procmacro) + quiet_cmd_rustc_library = $(if $(skip_clippy),RUSTC,$(RUSTC_OR_CLIPPY_QUIET)) L $@ cmd_rustc_library = \ OBJTREE=$(abspath $(objtree)) \ @@ -451,6 +483,13 @@ $(obj)/compiler_builtins.o: private rustc_objcopy = -w -W '__*' $(obj)/compiler_builtins.o: $(src)/compiler_builtins.rs $(obj)/core.o FORCE +$(call if_changed_rule,rustc_library) +$(obj)/pin_init.o: private skip_gendwarfksyms = 1 +$(obj)/pin_init.o: private rustc_target_flags = --extern pin_init_internal \ + --extern macros --cfg kernel +$(obj)/pin_init.o: $(src)/pin-init/src/_lib.rs $(obj)/compiler_builtins.o \ + $(obj)/$(libpin_init_internal_name) $(obj)/$(libmacros_name) FORCE + +$(call if_changed_rule,rustc_library) + $(obj)/build_error.o: private skip_gendwarfksyms = 1 $(obj)/build_error.o: $(src)/build_error.rs $(obj)/compiler_builtins.o FORCE +$(call if_changed_rule,rustc_library) @@ -473,9 +512,9 @@ $(obj)/uapi.o: $(src)/uapi/lib.rs \ $(obj)/uapi/uapi_generated.rs FORCE +$(call if_changed_rule,rustc_library) -$(obj)/kernel.o: private rustc_target_flags = --extern ffi \ +$(obj)/kernel.o: private rustc_target_flags = --extern ffi --extern pin_init \ --extern build_error --extern macros --extern bindings --extern uapi -$(obj)/kernel.o: $(src)/kernel/lib.rs $(obj)/build_error.o \ +$(obj)/kernel.o: $(src)/kernel/lib.rs $(obj)/build_error.o $(obj)/pin_init.o \ $(obj)/$(libmacros_name) $(obj)/bindings.o $(obj)/uapi.o FORCE +$(call if_changed_rule,rustc_library) diff --git a/rust/pin-init/internal/src/_lib.rs b/rust/pin-init/internal/src/_lib.rs new file mode 100644 index 000000000000..0874cf04e4cb --- /dev/null +++ b/rust/pin-init/internal/src/_lib.rs @@ -0,0 +1,3 @@ +// SPDX-License-Identifier: Apache-2.0 OR MIT + +//! Will be removed in a future commit, only exists to prevent compilation errors. diff --git a/rust/pin-init/internal/src/lib.rs b/rust/pin-init/internal/src/lib.rs index bf66cbee2531..3146da5cc47c 100644 --- a/rust/pin-init/internal/src/lib.rs +++ b/rust/pin-init/internal/src/lib.rs @@ -1,5 +1,9 @@ // SPDX-License-Identifier: Apache-2.0 OR MIT +// When fixdep scans this, it will find this string `CONFIG_RUSTC_VERSION_TEXT` +// and thus add a dependency on `include/config/RUSTC_VERSION_TEXT`, which is +// touched by Kconfig when the version string from the compiler changes. + #[allow(missing_docs)] #[proc_macro_attribute] pub fn pin_data(inner: TokenStream, item: TokenStream) -> TokenStream { diff --git a/rust/pin-init/src/_lib.rs b/rust/pin-init/src/_lib.rs new file mode 100644 index 000000000000..e0918fd8e9e7 --- /dev/null +++ b/rust/pin-init/src/_lib.rs @@ -0,0 +1,5 @@ +// SPDX-License-Identifier: Apache-2.0 OR MIT + +//! Will be removed in a future commit, only exists to prevent compilation errors. + +#![no_std] diff --git a/scripts/Makefile.build b/scripts/Makefile.build index 993708d11874..08b6380933f5 100644 --- a/scripts/Makefile.build +++ b/scripts/Makefile.build @@ -237,7 +237,7 @@ rust_common_cmd = \ -Zallow-features=$(rust_allowed_features) \ -Zcrate-attr=no_std \ -Zcrate-attr='feature($(rust_allowed_features))' \ - -Zunstable-options --extern kernel \ + -Zunstable-options --extern pin_init --extern kernel \ --crate-type rlib -L $(objtree)/rust/ \ --crate-name $(basename $(notdir $@)) \ --sysroot=/dev/null \ diff --git a/scripts/generate_rust_analyzer.py b/scripts/generate_rust_analyzer.py index aa8ea1a4dbe5..a44a4475d11f 100755 --- a/scripts/generate_rust_analyzer.py +++ b/scripts/generate_rust_analyzer.py @@ -93,10 +93,25 @@ def generate_crates(srctree, objtree, sysroot_src, external_src, cfgs): ) crates[-1]["env"]["OBJTREE"] = str(objtree.resolve(True)) + append_crate( + "pin_init_internal", + srctree / "rust" / "pin-init" / "internal" / "src" / "_lib.rs", + [], + cfg=["kernel"], + is_proc_macro=True, + ) + + append_crate( + "pin_init", + srctree / "rust" / "pin-init" / "src" / "_lib.rs", + ["core", "pin_init_internal", "macros"], + cfg=["kernel"], + ) + append_crate( "kernel", srctree / "rust" / "kernel" / "lib.rs", - ["core", "macros", "build_error", "bindings"], + ["core", "macros", "build_error", "bindings", "pin_init"], cfg=cfg, ) crates[-1]["source"] = { -- cgit v1.2.3 From dbd5058ba60c3499b24a7133a4e2e24dba6ea77b Mon Sep 17 00:00:00 2001 From: Benno Lossin <benno.lossin@proton.me> Date: Sat, 8 Mar 2025 11:05:09 +0000 Subject: rust: make pin-init its own crate Rename relative paths inside of the crate to still refer to the same items, also rename paths inside of the kernel crate and adjust the build system to build the crate. [ Remove the `expect` (and thus the `lint_reasons` feature) since the tree now uses `quote!` from `rust/macros/export.rs`. Remove the `TokenStream` import removal, since it is now used as well. In addition, temporarily (i.e. just for this commit) use an `--extern force:alloc` to prevent an unknown `new_uninit` error in the `rustdoc` target. For context, please see a similar case in: https://lore.kernel.org/lkml/20240422090644.525520-1-ojeda@kernel.org/ And adjusted the message above. - Miguel ] Signed-off-by: Benno Lossin <benno.lossin@proton.me> Reviewed-by: Fiona Behrens <me@kloenk.dev> Tested-by: Andreas Hindborg <a.hindborg@kernel.org> Link: https://lore.kernel.org/r/20250308110339.2997091-16-benno.lossin@proton.me Signed-off-by: Miguel Ojeda <ojeda@kernel.org> --- rust/Makefile | 14 ++-- rust/kernel/alloc/kbox.rs | 4 +- rust/kernel/block/mq/tag_set.rs | 5 +- rust/kernel/driver.rs | 6 +- rust/kernel/init.rs | 22 +++--- rust/kernel/lib.rs | 10 +-- rust/kernel/list.rs | 2 +- rust/kernel/prelude.rs | 9 +-- rust/kernel/sync/arc.rs | 7 +- rust/kernel/sync/condvar.rs | 4 +- rust/kernel/sync/lock.rs | 4 +- rust/kernel/sync/lock/mutex.rs | 2 +- rust/kernel/sync/lock/spinlock.rs | 2 +- rust/kernel/types.rs | 10 +-- rust/macros/helpers.rs | 2 - rust/macros/lib.rs | 8 --- rust/macros/module.rs | 2 +- rust/macros/quote.rs | 1 + rust/pin-init/internal/src/_lib.rs | 3 - rust/pin-init/internal/src/helpers.rs | 2 + rust/pin-init/internal/src/lib.rs | 16 +++++ rust/pin-init/internal/src/pin_data.rs | 4 +- rust/pin-init/internal/src/pinned_drop.rs | 4 +- rust/pin-init/internal/src/zeroable.rs | 8 +-- rust/pin-init/src/_lib.rs | 5 -- rust/pin-init/src/lib.rs | 46 ++++++++----- rust/pin-init/src/macros.rs | 111 +++++++++++++++--------------- scripts/generate_rust_analyzer.py | 4 +- 28 files changed, 164 insertions(+), 153 deletions(-) delete mode 100644 rust/pin-init/internal/src/_lib.rs delete mode 100644 rust/pin-init/src/_lib.rs (limited to 'scripts') diff --git a/rust/Makefile b/rust/Makefile index 90310f0620eb..815fbe05ffc8 100644 --- a/rust/Makefile +++ b/rust/Makefile @@ -116,13 +116,13 @@ rustdoc-ffi: $(src)/ffi.rs rustdoc-core FORCE rustdoc-pin_init_internal: private rustdoc_host = yes rustdoc-pin_init_internal: private rustc_target_flags = --cfg kernel \ --extern proc_macro --crate-type proc-macro -rustdoc-pin_init_internal: $(src)/pin-init/internal/src/_lib.rs FORCE +rustdoc-pin_init_internal: $(src)/pin-init/internal/src/lib.rs FORCE +$(call if_changed,rustdoc) rustdoc-pin_init: private rustdoc_host = yes rustdoc-pin_init: private rustc_target_flags = --extern pin_init_internal \ - --extern macros --extern alloc --cfg kernel --cfg feature=\"alloc\" -rustdoc-pin_init: $(src)/pin-init/src/_lib.rs rustdoc-pin_init_internal \ + --extern macros --extern force:alloc --cfg kernel --cfg feature=\"alloc\" +rustdoc-pin_init: $(src)/pin-init/src/lib.rs rustdoc-pin_init_internal \ rustdoc-macros FORCE +$(call if_changed,rustdoc) @@ -158,12 +158,12 @@ rusttestlib-macros: $(src)/macros/lib.rs FORCE rusttestlib-pin_init_internal: private rustc_target_flags = --cfg kernel \ --extern proc_macro rusttestlib-pin_init_internal: private rustc_test_library_proc = yes -rusttestlib-pin_init_internal: $(src)/pin-init/internal/src/_lib.rs FORCE +rusttestlib-pin_init_internal: $(src)/pin-init/internal/src/lib.rs FORCE +$(call if_changed,rustc_test_library) rusttestlib-pin_init: private rustc_target_flags = --extern pin_init_internal \ --extern macros --cfg kernel -rusttestlib-pin_init: $(src)/pin-init/src/_lib.rs rusttestlib-macros \ +rusttestlib-pin_init: $(src)/pin-init/src/lib.rs rusttestlib-macros \ rusttestlib-pin_init_internal $(obj)/$(libpin_init_internal_name) FORCE +$(call if_changed,rustc_test_library) @@ -401,7 +401,7 @@ $(obj)/$(libmacros_name): $(src)/macros/lib.rs FORCE +$(call if_changed_dep,rustc_procmacro) $(obj)/$(libpin_init_internal_name): private rustc_target_flags = --cfg kernel -$(obj)/$(libpin_init_internal_name): $(src)/pin-init/internal/src/_lib.rs FORCE +$(obj)/$(libpin_init_internal_name): $(src)/pin-init/internal/src/lib.rs FORCE +$(call if_changed_dep,rustc_procmacro) quiet_cmd_rustc_library = $(if $(skip_clippy),RUSTC,$(RUSTC_OR_CLIPPY_QUIET)) L $@ @@ -486,7 +486,7 @@ $(obj)/compiler_builtins.o: $(src)/compiler_builtins.rs $(obj)/core.o FORCE $(obj)/pin_init.o: private skip_gendwarfksyms = 1 $(obj)/pin_init.o: private rustc_target_flags = --extern pin_init_internal \ --extern macros --cfg kernel -$(obj)/pin_init.o: $(src)/pin-init/src/_lib.rs $(obj)/compiler_builtins.o \ +$(obj)/pin_init.o: $(src)/pin-init/src/lib.rs $(obj)/compiler_builtins.o \ $(obj)/$(libpin_init_internal_name) $(obj)/$(libmacros_name) FORCE +$(call if_changed_rule,rustc_library) diff --git a/rust/kernel/alloc/kbox.rs b/rust/kernel/alloc/kbox.rs index 07150c038e3f..e6200cd1d06d 100644 --- a/rust/kernel/alloc/kbox.rs +++ b/rust/kernel/alloc/kbox.rs @@ -15,9 +15,9 @@ use core::pin::Pin; use core::ptr::NonNull; use core::result::Result; -use crate::init::{InPlaceWrite, Init, PinInit, ZeroableOption}; -use crate::init_ext::InPlaceInit; +use crate::init::InPlaceInit; use crate::types::ForeignOwnable; +use pin_init::{InPlaceWrite, Init, PinInit, ZeroableOption}; /// The kernel's [`Box`] type -- a heap allocation for a single value of type `T`. /// diff --git a/rust/kernel/block/mq/tag_set.rs b/rust/kernel/block/mq/tag_set.rs index 00ddcc71dfa2..bcf4214ad149 100644 --- a/rust/kernel/block/mq/tag_set.rs +++ b/rust/kernel/block/mq/tag_set.rs @@ -10,12 +10,11 @@ use crate::{ bindings, block::mq::{operations::OperationsVTable, request::RequestDataWrapper, Operations}, error, - prelude::PinInit, - try_pin_init, + prelude::try_pin_init, types::Opaque, }; use core::{convert::TryInto, marker::PhantomData}; -use macros::{pin_data, pinned_drop}; +use pin_init::{pin_data, pinned_drop, PinInit}; /// A wrapper for the C `struct blk_mq_tag_set`. /// diff --git a/rust/kernel/driver.rs b/rust/kernel/driver.rs index 2a16d5e64e6c..ec9166cedfa7 100644 --- a/rust/kernel/driver.rs +++ b/rust/kernel/driver.rs @@ -6,9 +6,9 @@ //! register using the [`Registration`] class. use crate::error::{Error, Result}; -use crate::{device, init::PinInit, of, str::CStr, try_pin_init, types::Opaque, ThisModule}; +use crate::{device, of, str::CStr, try_pin_init, types::Opaque, ThisModule}; use core::pin::Pin; -use macros::{pin_data, pinned_drop}; +use pin_init::{pin_data, pinned_drop, PinInit}; /// The [`RegistrationOps`] trait serves as generic interface for subsystems (e.g., PCI, Platform, /// Amba, etc.) to provide the corresponding subsystem specific implementation to register / @@ -114,7 +114,7 @@ macro_rules! module_driver { impl $crate::InPlaceModule for DriverModule { fn init( module: &'static $crate::ThisModule - ) -> impl $crate::init::PinInit<Self, $crate::error::Error> { + ) -> impl ::pin_init::PinInit<Self, $crate::error::Error> { $crate::try_pin_init!(Self { _driver <- $crate::driver::Registration::new( <Self as $crate::ModuleMetadata>::NAME, diff --git a/rust/kernel/init.rs b/rust/kernel/init.rs index d8eb6d7873b7..32d6e4167650 100644 --- a/rust/kernel/init.rs +++ b/rust/kernel/init.rs @@ -23,7 +23,7 @@ //! //! [`Opaque<T>`]: crate::types::Opaque //! [`Opaque::ffi_init`]: crate::types::Opaque::ffi_init -//! [`pin_init!`]: crate::pin_init +//! [`pin_init!`]: pin_init::pin_init //! //! # Examples //! @@ -137,8 +137,8 @@ use crate::{ alloc::{AllocError, Flags}, error::{self, Error}, - init::{init_from_closure, pin_init_from_closure, Init, PinInit}, }; +use pin_init::{init_from_closure, pin_init_from_closure, Init, PinInit}; /// Smart pointer that can initialize memory in-place. pub trait InPlaceInit<T>: Sized { @@ -205,7 +205,8 @@ pub trait InPlaceInit<T>: Sized { /// # Examples /// /// ```rust -/// use kernel::{init::zeroed, error::Error}; +/// use kernel::error::Error; +/// use pin_init::zeroed; /// struct BigBuf { /// big: KBox<[u8; 1024 * 1024 * 1024]>, /// small: [u8; 1024 * 1024], @@ -222,7 +223,7 @@ pub trait InPlaceInit<T>: Sized { /// ``` /// /// [`Infallible`]: core::convert::Infallible -/// [`init!`]: crate::init! +/// [`init!`]: pin_init::init /// [`try_pin_init!`]: crate::try_pin_init! /// [`Error`]: crate::error::Error #[macro_export] @@ -230,14 +231,14 @@ macro_rules! try_init { ($(&$this:ident in)? $t:ident $(::<$($generics:ty),* $(,)?>)? { $($fields:tt)* }) => { - $crate::_try_init!($(&$this in)? $t $(::<$($generics),* $(,)?>)? { + ::pin_init::try_init!($(&$this in)? $t $(::<$($generics),* $(,)?>)? { $($fields)* }? $crate::error::Error) }; ($(&$this:ident in)? $t:ident $(::<$($generics:ty),* $(,)?>)? { $($fields:tt)* }? $err:ty) => { - $crate::_try_init!($(&$this in)? $t $(::<$($generics),* $(,)?>)? { + ::pin_init::try_init!($(&$this in)? $t $(::<$($generics),* $(,)?>)? { $($fields)* }? $err) }; @@ -262,7 +263,8 @@ macro_rules! try_init { /// /// ```rust /// # #![feature(new_uninit)] -/// use kernel::{init::zeroed, error::Error}; +/// use kernel::error::Error; +/// use pin_init::zeroed; /// #[pin_data] /// struct BigBuf { /// big: KBox<[u8; 1024 * 1024 * 1024]>, @@ -282,21 +284,21 @@ macro_rules! try_init { /// ``` /// /// [`Infallible`]: core::convert::Infallible -/// [`pin_init!`]: crate::pin_init +/// [`pin_init!`]: pin_init::pin_init /// [`Error`]: crate::error::Error #[macro_export] macro_rules! try_pin_init { ($(&$this:ident in)? $t:ident $(::<$($generics:ty),* $(,)?>)? { $($fields:tt)* }) => { - $crate::_try_pin_init!($(&$this in)? $t $(::<$($generics),* $(,)?>)? { + ::pin_init::try_pin_init!($(&$this in)? $t $(::<$($generics),* $(,)?>)? { $($fields)* }? $crate::error::Error) }; ($(&$this:ident in)? $t:ident $(::<$($generics:ty),* $(,)?>)? { $($fields:tt)* }? $err:ty) => { - $crate::_try_pin_init!($(&$this in)? $t $(::<$($generics),* $(,)?>)? { + ::pin_init::try_pin_init!($(&$this in)? $t $(::<$($generics),* $(,)?>)? { $($fields)* }? $err) }; diff --git a/rust/kernel/lib.rs b/rust/kernel/lib.rs index e3933f3dfc0b..c92497c7c655 100644 --- a/rust/kernel/lib.rs +++ b/rust/kernel/lib.rs @@ -50,11 +50,7 @@ pub mod faux; #[cfg(CONFIG_RUST_FW_LOADER_ABSTRACTIONS)] pub mod firmware; pub mod fs; -#[path = "../pin-init/src/lib.rs"] pub mod init; -// momentarily use the name `init_ext` and set the path manually -#[path = "init.rs"] -pub mod init_ext; pub mod io; pub mod ioctl; pub mod jump_label; @@ -116,11 +112,11 @@ pub trait InPlaceModule: Sync + Send { /// Creates an initialiser for the module. /// /// It is called when the module is loaded. - fn init(module: &'static ThisModule) -> impl init::PinInit<Self, error::Error>; + fn init(module: &'static ThisModule) -> impl pin_init::PinInit<Self, error::Error>; } impl<T: Module> InPlaceModule for T { - fn init(module: &'static ThisModule) -> impl init::PinInit<Self, error::Error> { + fn init(module: &'static ThisModule) -> impl pin_init::PinInit<Self, error::Error> { let initer = move |slot: *mut Self| { let m = <Self as Module>::init(module)?; @@ -130,7 +126,7 @@ impl<T: Module> InPlaceModule for T { }; // SAFETY: On success, `initer` always fully initialises an instance of `Self`. - unsafe { init::pin_init_from_closure(initer) } + unsafe { pin_init::pin_init_from_closure(initer) } } } diff --git a/rust/kernel/list.rs b/rust/kernel/list.rs index c0ed227b8a4f..a335c3b1ff5e 100644 --- a/rust/kernel/list.rs +++ b/rust/kernel/list.rs @@ -4,12 +4,12 @@ //! A linked list implementation. -use crate::init::PinInit; use crate::sync::ArcBorrow; use crate::types::Opaque; use core::iter::{DoubleEndedIterator, FusedIterator}; use core::marker::PhantomData; use core::ptr; +use pin_init::PinInit; mod impl_list_item_mod; pub use self::impl_list_item_mod::{ diff --git a/rust/kernel/prelude.rs b/rust/kernel/prelude.rs index ab7c07788a28..baa774a351ce 100644 --- a/rust/kernel/prelude.rs +++ b/rust/kernel/prelude.rs @@ -17,7 +17,9 @@ pub use core::pin::Pin; pub use crate::alloc::{flags::*, Box, KBox, KVBox, KVVec, KVec, VBox, VVec, Vec}; #[doc(no_inline)] -pub use macros::{export, module, pin_data, pinned_drop, vtable, Zeroable}; +pub use macros::{export, module, vtable}; + +pub use pin_init::{init, pin_data, pin_init, pinned_drop, InPlaceWrite, Init, PinInit, Zeroable}; pub use super::{build_assert, build_error}; @@ -28,7 +30,7 @@ pub use super::fmt; pub use super::{dev_alert, dev_crit, dev_dbg, dev_emerg, dev_err, dev_info, dev_notice, dev_warn}; pub use super::{pr_alert, pr_crit, pr_debug, pr_emerg, pr_err, pr_info, pr_notice, pr_warn}; -pub use super::{init, pin_init, try_init, try_pin_init}; +pub use super::{try_init, try_pin_init}; pub use super::static_assert; @@ -36,7 +38,6 @@ pub use super::error::{code::*, Error, Result}; pub use super::{str::CStr, ThisModule}; -pub use super::init::{InPlaceWrite, Init, PinInit}; -pub use super::init_ext::InPlaceInit; +pub use super::init::InPlaceInit; pub use super::current; diff --git a/rust/kernel/sync/arc.rs b/rust/kernel/sync/arc.rs index 31c26b692c6d..c64eac8b4235 100644 --- a/rust/kernel/sync/arc.rs +++ b/rust/kernel/sync/arc.rs @@ -19,8 +19,7 @@ use crate::{ alloc::{AllocError, Flags, KBox}, bindings, - init::{self, InPlaceWrite, Init, PinInit}, - init_ext::InPlaceInit, + init::InPlaceInit, try_init, types::{ForeignOwnable, Opaque}, }; @@ -33,7 +32,7 @@ use core::{ pin::Pin, ptr::NonNull, }; -use macros::pin_data; +use pin_init::{self, pin_data, InPlaceWrite, Init, PinInit}; mod std_vendor; @@ -738,7 +737,7 @@ impl<T> UniqueArc<T> { try_init!(ArcInner { // SAFETY: There are no safety requirements for this FFI call. refcount: Opaque::new(unsafe { bindings::REFCOUNT_INIT(1) }), - data <- init::uninit::<T, AllocError>(), + data <- pin_init::uninit::<T, AllocError>(), }? AllocError), flags, )?; diff --git a/rust/kernel/sync/condvar.rs b/rust/kernel/sync/condvar.rs index 5aa7fa7c7002..c2535db9e0f8 100644 --- a/rust/kernel/sync/condvar.rs +++ b/rust/kernel/sync/condvar.rs @@ -8,8 +8,6 @@ use super::{lock::Backend, lock::Guard, LockClassKey}; use crate::{ ffi::{c_int, c_long}, - init::PinInit, - pin_init, str::CStr, task::{MAX_SCHEDULE_TIMEOUT, TASK_INTERRUPTIBLE, TASK_NORMAL, TASK_UNINTERRUPTIBLE}, time::Jiffies, @@ -17,7 +15,7 @@ use crate::{ }; use core::marker::PhantomPinned; use core::ptr; -use macros::pin_data; +use pin_init::{pin_data, pin_init, PinInit}; /// Creates a [`CondVar`] initialiser with the given name and a newly-created lock class. #[macro_export] diff --git a/rust/kernel/sync/lock.rs b/rust/kernel/sync/lock.rs index eb80048e0110..7f611b59ac57 100644 --- a/rust/kernel/sync/lock.rs +++ b/rust/kernel/sync/lock.rs @@ -7,13 +7,11 @@ use super::LockClassKey; use crate::{ - init::PinInit, - pin_init, str::CStr, types::{NotThreadSafe, Opaque, ScopeGuard}, }; use core::{cell::UnsafeCell, marker::PhantomPinned}; -use macros::pin_data; +use pin_init::{pin_data, pin_init, PinInit}; pub mod mutex; pub mod spinlock; diff --git a/rust/kernel/sync/lock/mutex.rs b/rust/kernel/sync/lock/mutex.rs index 70cadbc2e8e2..581cee7ab842 100644 --- a/rust/kernel/sync/lock/mutex.rs +++ b/rust/kernel/sync/lock/mutex.rs @@ -26,7 +26,7 @@ pub use new_mutex; /// Since it may block, [`Mutex`] needs to be used with care in atomic contexts. /// /// Instances of [`Mutex`] need a lock class and to be pinned. The recommended way to create such -/// instances is with the [`pin_init`](crate::pin_init) and [`new_mutex`] macros. +/// instances is with the [`pin_init`](pin_init::pin_init) and [`new_mutex`] macros. /// /// # Examples /// diff --git a/rust/kernel/sync/lock/spinlock.rs b/rust/kernel/sync/lock/spinlock.rs index ab2f8d075311..d7be38ccbdc7 100644 --- a/rust/kernel/sync/lock/spinlock.rs +++ b/rust/kernel/sync/lock/spinlock.rs @@ -24,7 +24,7 @@ pub use new_spinlock; /// unlocked, at which point another CPU will be allowed to make progress. /// /// Instances of [`SpinLock`] need a lock class and to be pinned. The recommended way to create such -/// instances is with the [`pin_init`](crate::pin_init) and [`new_spinlock`] macros. +/// instances is with the [`pin_init`](pin_init::pin_init) and [`new_spinlock`] macros. /// /// # Examples /// diff --git a/rust/kernel/types.rs b/rust/kernel/types.rs index 7237b2224680..9d0471afc964 100644 --- a/rust/kernel/types.rs +++ b/rust/kernel/types.rs @@ -2,7 +2,6 @@ //! Kernel types. -use crate::init::{self, PinInit, Zeroable}; use core::{ cell::UnsafeCell, marker::{PhantomData, PhantomPinned}, @@ -10,6 +9,7 @@ use core::{ ops::{Deref, DerefMut}, ptr::NonNull, }; +use pin_init::{PinInit, Zeroable}; /// Used to transfer ownership to and from foreign (non-Rust) languages. /// @@ -336,7 +336,7 @@ impl<T> Opaque<T> { // - `ptr` is a valid pointer to uninitialized memory, // - `slot` is not accessed on error; the call is infallible, // - `slot` is pinned in memory. - let _ = unsafe { init::PinInit::<T>::__pinned_init(slot, ptr) }; + let _ = unsafe { PinInit::<T>::__pinned_init(slot, ptr) }; }) } @@ -352,7 +352,7 @@ impl<T> Opaque<T> { // SAFETY: We contain a `MaybeUninit`, so it is OK for the `init_func` to not fully // initialize the `T`. unsafe { - init::pin_init_from_closure::<_, ::core::convert::Infallible>(move |slot| { + pin_init::pin_init_from_closure::<_, ::core::convert::Infallible>(move |slot| { init_func(Self::raw_get(slot)); Ok(()) }) @@ -372,7 +372,9 @@ impl<T> Opaque<T> { ) -> impl PinInit<Self, E> { // SAFETY: We contain a `MaybeUninit`, so it is OK for the `init_func` to not fully // initialize the `T`. - unsafe { init::pin_init_from_closure::<_, E>(move |slot| init_func(Self::raw_get(slot))) } + unsafe { + pin_init::pin_init_from_closure::<_, E>(move |slot| init_func(Self::raw_get(slot))) + } } /// Returns a raw pointer to the opaque data. diff --git a/rust/macros/helpers.rs b/rust/macros/helpers.rs index 141d8476c197..a3ee27e29a6f 100644 --- a/rust/macros/helpers.rs +++ b/rust/macros/helpers.rs @@ -86,5 +86,3 @@ pub(crate) fn function_name(input: TokenStream) -> Option<Ident> { } None } - -include!("../pin-init/internal/src/helpers.rs"); diff --git a/rust/macros/lib.rs b/rust/macros/lib.rs index ba93dd686e38..f0f8c9232748 100644 --- a/rust/macros/lib.rs +++ b/rust/macros/lib.rs @@ -13,13 +13,7 @@ mod export; mod helpers; mod module; mod paste; -#[path = "../pin-init/internal/src/pin_data.rs"] -mod pin_data; -#[path = "../pin-init/internal/src/pinned_drop.rs"] -mod pinned_drop; mod vtable; -#[path = "../pin-init/internal/src/zeroable.rs"] -mod zeroable; use proc_macro::TokenStream; @@ -398,5 +392,3 @@ pub fn paste(input: TokenStream) -> TokenStream { paste::expand(&mut tokens); tokens.into_iter().collect() } - -include!("../pin-init/internal/src/lib.rs"); diff --git a/rust/macros/module.rs b/rust/macros/module.rs index 42ed16c48b37..46f20682a7a9 100644 --- a/rust/macros/module.rs +++ b/rust/macros/module.rs @@ -244,7 +244,7 @@ pub(crate) fn module(ts: TokenStream) -> TokenStream { mod __module_init {{ mod __module_init {{ use super::super::{type_}; - use kernel::init::PinInit; + use pin_init::PinInit; /// The \"Rust loadable module\" mark. // diff --git a/rust/macros/quote.rs b/rust/macros/quote.rs index 31b7ebe504f4..92cacc4067c9 100644 --- a/rust/macros/quote.rs +++ b/rust/macros/quote.rs @@ -2,6 +2,7 @@ use proc_macro::{TokenStream, TokenTree}; +#[allow(dead_code)] pub(crate) trait ToTokens { fn to_tokens(&self, tokens: &mut TokenStream); } diff --git a/rust/pin-init/internal/src/_lib.rs b/rust/pin-init/internal/src/_lib.rs deleted file mode 100644 index 0874cf04e4cb..000000000000 --- a/rust/pin-init/internal/src/_lib.rs +++ /dev/null @@ -1,3 +0,0 @@ -// SPDX-License-Identifier: Apache-2.0 OR MIT - -//! Will be removed in a future commit, only exists to prevent compilation errors. diff --git a/rust/pin-init/internal/src/helpers.rs b/rust/pin-init/internal/src/helpers.rs index 2f4fc75c014e..78521ba19d0b 100644 --- a/rust/pin-init/internal/src/helpers.rs +++ b/rust/pin-init/internal/src/helpers.rs @@ -1,5 +1,7 @@ // SPDX-License-Identifier: Apache-2.0 OR MIT +use proc_macro::{TokenStream, TokenTree}; + /// Parsed generics. /// /// See the field documentation for an explanation what each of the fields represents. diff --git a/rust/pin-init/internal/src/lib.rs b/rust/pin-init/internal/src/lib.rs index 3146da5cc47c..c201b8a53915 100644 --- a/rust/pin-init/internal/src/lib.rs +++ b/rust/pin-init/internal/src/lib.rs @@ -4,6 +4,22 @@ // and thus add a dependency on `include/config/RUSTC_VERSION_TEXT`, which is // touched by Kconfig when the version string from the compiler changes. +//! `pin-init` proc macros. + +#![cfg_attr(not(RUSTC_LINT_REASONS_IS_STABLE), feature(lint_reasons))] + +use proc_macro::TokenStream; + +#[cfg(kernel)] +#[path = "../../../macros/quote.rs"] +#[macro_use] +mod quote; + +mod helpers; +mod pin_data; +mod pinned_drop; +mod zeroable; + #[allow(missing_docs)] #[proc_macro_attribute] pub fn pin_data(inner: TokenStream, item: TokenStream) -> TokenStream { diff --git a/rust/pin-init/internal/src/pin_data.rs b/rust/pin-init/internal/src/pin_data.rs index 1d4a3547c684..9b974498f4a8 100644 --- a/rust/pin-init/internal/src/pin_data.rs +++ b/rust/pin-init/internal/src/pin_data.rs @@ -5,7 +5,7 @@ use proc_macro::{Group, Punct, Spacing, TokenStream, TokenTree}; pub(crate) fn pin_data(args: TokenStream, input: TokenStream) -> TokenStream { // This proc-macro only does some pre-parsing and then delegates the actual parsing to - // `kernel::__pin_data!`. + // `pin_init::__pin_data!`. let ( Generics { @@ -71,7 +71,7 @@ pub(crate) fn pin_data(args: TokenStream, input: TokenStream) -> TokenStream { .collect::<Vec<_>>(); // This should be the body of the struct `{...}`. let last = rest.pop(); - let mut quoted = quote!(::kernel::__pin_data! { + let mut quoted = quote!(::pin_init::__pin_data! { parse_input: @args(#args), @sig(#(#rest)*), diff --git a/rust/pin-init/internal/src/pinned_drop.rs b/rust/pin-init/internal/src/pinned_drop.rs index 88fb72b20660..386f52f73c06 100644 --- a/rust/pin-init/internal/src/pinned_drop.rs +++ b/rust/pin-init/internal/src/pinned_drop.rs @@ -35,11 +35,11 @@ pub(crate) fn pinned_drop(_args: TokenStream, input: TokenStream) -> TokenStream let idx = pinned_drop_idx .unwrap_or_else(|| panic!("Expected an `impl` block implementing `PinnedDrop`.")); // Fully qualify the `PinnedDrop`, as to avoid any tampering. - toks.splice(idx..idx, quote!(::kernel::init::)); + toks.splice(idx..idx, quote!(::pin_init::)); // Take the `{}` body and call the declarative macro. if let Some(TokenTree::Group(last)) = toks.pop() { let last = last.stream(); - quote!(::kernel::__pinned_drop! { + quote!(::pin_init::__pinned_drop! { @impl_sig(#(#toks)*), @impl_body(#last), }) diff --git a/rust/pin-init/internal/src/zeroable.rs b/rust/pin-init/internal/src/zeroable.rs index cfee2cec18d5..0cf6732f27dc 100644 --- a/rust/pin-init/internal/src/zeroable.rs +++ b/rust/pin-init/internal/src/zeroable.rs @@ -27,7 +27,7 @@ pub(crate) fn derive(input: TokenStream) -> TokenStream { // If we find a `,`, then we have finished a generic/constant/lifetime parameter. TokenTree::Punct(p) if nested == 0 && p.as_char() == ',' => { if in_generic && !inserted { - new_impl_generics.extend(quote! { : ::kernel::init::Zeroable }); + new_impl_generics.extend(quote! { : ::pin_init::Zeroable }); } in_generic = true; inserted = false; @@ -41,7 +41,7 @@ pub(crate) fn derive(input: TokenStream) -> TokenStream { TokenTree::Punct(p) if nested == 0 && p.as_char() == ':' => { new_impl_generics.push(tt); if in_generic { - new_impl_generics.extend(quote! { ::kernel::init::Zeroable + }); + new_impl_generics.extend(quote! { ::pin_init::Zeroable + }); inserted = true; } } @@ -59,10 +59,10 @@ pub(crate) fn derive(input: TokenStream) -> TokenStream { } assert_eq!(nested, 0); if in_generic && !inserted { - new_impl_generics.extend(quote! { : ::kernel::init::Zeroable }); + new_impl_generics.extend(quote! { : ::pin_init::Zeroable }); } quote! { - ::kernel::__derive_zeroable!( + ::pin_init::__derive_zeroable!( parse_input: @sig(#(#rest)*), @impl_generics(#(#new_impl_generics)*), diff --git a/rust/pin-init/src/_lib.rs b/rust/pin-init/src/_lib.rs deleted file mode 100644 index e0918fd8e9e7..000000000000 --- a/rust/pin-init/src/_lib.rs +++ /dev/null @@ -1,5 +0,0 @@ -// SPDX-License-Identifier: Apache-2.0 OR MIT - -//! Will be removed in a future commit, only exists to prevent compilation errors. - -#![no_std] diff --git a/rust/pin-init/src/lib.rs b/rust/pin-init/src/lib.rs index 5f1afd3abb56..41bfb35c7a2c 100644 --- a/rust/pin-init/src/lib.rs +++ b/rust/pin-init/src/lib.rs @@ -209,9 +209,21 @@ //! [`impl PinInit<Foo>`]: PinInit //! [`impl PinInit<T, E>`]: PinInit //! [`impl Init<T, E>`]: Init -//! [`pin_data`]: ::macros::pin_data +//! [`pin_data`]: crate::pin_data //! [`pin_init!`]: crate::pin_init! +#![cfg_attr(not(RUSTC_LINT_REASONS_IS_STABLE), feature(lint_reasons))] +#![cfg_attr( + all( + any(feature = "alloc", feature = "std"), + not(RUSTC_NEW_UNINIT_IS_STABLE) + ), + feature(new_uninit) +)] +#![forbid(missing_docs, unsafe_op_in_unsafe_fn)] +#![cfg_attr(not(feature = "std"), no_std)] +#![cfg_attr(feature = "alloc", feature(allocator_api))] + use core::{ cell::UnsafeCell, convert::Infallible, @@ -288,7 +300,7 @@ pub mod macros; /// ``` /// /// [`pin_init!`]: crate::pin_init -pub use ::macros::pin_data; +pub use ::pin_init_internal::pin_data; /// Used to implement `PinnedDrop` safely. /// @@ -322,7 +334,7 @@ pub use ::macros::pin_data; /// } /// } /// ``` -pub use ::macros::pinned_drop; +pub use ::pin_init_internal::pinned_drop; /// Derives the [`Zeroable`] trait for the given struct. /// @@ -340,7 +352,7 @@ pub use ::macros::pinned_drop; /// len: usize, /// } /// ``` -pub use ::macros::Zeroable; +pub use ::pin_init_internal::Zeroable; /// Initialize and pin a type directly on the stack. /// @@ -385,8 +397,8 @@ pub use ::macros::Zeroable; macro_rules! stack_pin_init { (let $var:ident $(: $t:ty)? = $val:expr) => { let val = $val; - let mut $var = ::core::pin::pin!($crate::init::__internal::StackInit$(::<$t>)?::uninit()); - let mut $var = match $crate::init::__internal::StackInit::init($var, val) { + let mut $var = ::core::pin::pin!($crate::__internal::StackInit$(::<$t>)?::uninit()); + let mut $var = match $crate::__internal::StackInit::init($var, val) { Ok(res) => res, Err(x) => { let x: ::core::convert::Infallible = x; @@ -463,13 +475,13 @@ macro_rules! stack_pin_init { macro_rules! stack_try_pin_init { (let $var:ident $(: $t:ty)? = $val:expr) => { let val = $val; - let mut $var = ::core::pin::pin!($crate::init::__internal::StackInit$(::<$t>)?::uninit()); - let mut $var = $crate::init::__internal::StackInit::init($var, val); + let mut $var = ::core::pin::pin!($crate::__internal::StackInit$(::<$t>)?::uninit()); + let mut $var = $crate::__internal::StackInit::init($var, val); }; (let $var:ident $(: $t:ty)? =? $val:expr) => { let val = $val; - let mut $var = ::core::pin::pin!($crate::init::__internal::StackInit$(::<$t>)?::uninit()); - let mut $var = $crate::init::__internal::StackInit::init($var, val)?; + let mut $var = ::core::pin::pin!($crate::__internal::StackInit$(::<$t>)?::uninit()); + let mut $var = $crate::__internal::StackInit::init($var, val)?; }; } @@ -670,7 +682,7 @@ macro_rules! pin_init { ($(&$this:ident in)? $t:ident $(::<$($generics:ty),* $(,)?>)? { $($fields:tt)* }) => { - $crate::_try_pin_init!($(&$this in)? $t $(::<$($generics),*>)? { + $crate::try_pin_init!($(&$this in)? $t $(::<$($generics),*>)? { $($fields)* }? ::core::convert::Infallible) }; @@ -716,7 +728,7 @@ macro_rules! pin_init { // For a detailed example of how this macro works, see the module documentation of the hidden // module `__internal` inside of `init/__internal.rs`. #[macro_export] -macro_rules! _try_pin_init { +macro_rules! try_pin_init { ($(&$this:ident in)? $t:ident $(::<$($generics:ty),* $(,)?>)? { $($fields:tt)* }? $err:ty) => { @@ -755,7 +767,7 @@ macro_rules! init { ($(&$this:ident in)? $t:ident $(::<$($generics:ty),* $(,)?>)? { $($fields:tt)* }) => { - $crate::_try_init!($(&$this in)? $t $(::<$($generics),*>)? { + $crate::try_init!($(&$this in)? $t $(::<$($generics),*>)? { $($fields)* }? ::core::convert::Infallible) } @@ -798,7 +810,7 @@ macro_rules! init { // For a detailed example of how this macro works, see the module documentation of the hidden // module `__internal` inside of `init/__internal.rs`. #[macro_export] -macro_rules! _try_init { +macro_rules! try_init { ($(&$this:ident in)? $t:ident $(::<$($generics:ty),* $(,)?>)? { $($fields:tt)* }? $err:ty) => { @@ -868,8 +880,8 @@ macro_rules! assert_pinned { ($ty:ty, $field:ident, $field_ty:ty, inline) => { let _ = move |ptr: *mut $field_ty| { // SAFETY: This code is unreachable. - let data = unsafe { <$ty as $crate::init::__internal::HasPinData>::__pin_data() }; - let init = $crate::init::__internal::AlwaysFail::<$field_ty>::new(); + let data = unsafe { <$ty as $crate::__internal::HasPinData>::__pin_data() }; + let init = $crate::__internal::AlwaysFail::<$field_ty>::new(); // SAFETY: This code is unreachable. unsafe { data.$field(ptr, init) }.ok(); }; @@ -1262,7 +1274,7 @@ pub trait InPlaceWrite<T> { /// /// This trait must be implemented via the [`pinned_drop`] proc-macro attribute on the impl. /// -/// [`pinned_drop`]: crate::macros::pinned_drop +/// [`pinned_drop`]: crate::pinned_drop pub unsafe trait PinnedDrop: __internal::HasPinData { /// Executes the pinned destructor of this type. /// diff --git a/rust/pin-init/src/macros.rs b/rust/pin-init/src/macros.rs index c45ad6af5ca0..d41c4f198c42 100644 --- a/rust/pin-init/src/macros.rs +++ b/rust/pin-init/src/macros.rs @@ -19,7 +19,7 @@ //! We will look at the following example: //! //! ```rust,ignore -//! # use kernel::init::*; +//! # use pin_init::*; //! # use core::pin::Pin; //! #[pin_data] //! #[repr(C)] @@ -75,7 +75,7 @@ //! Here is the definition of `Bar` from our example: //! //! ```rust,ignore -//! # use kernel::init::*; +//! # use pin_init::*; //! #[pin_data] //! #[repr(C)] //! struct Bar<T> { @@ -121,22 +121,22 @@ //! self, //! slot: *mut T, //! // Since `t` is `#[pin]`, this is `PinInit`. -//! init: impl ::kernel::init::PinInit<T, E>, +//! init: impl ::pin_init::PinInit<T, E>, //! ) -> ::core::result::Result<(), E> { -//! unsafe { ::kernel::init::PinInit::__pinned_init(init, slot) } +//! unsafe { ::pin_init::PinInit::__pinned_init(init, slot) } //! } //! pub unsafe fn x<E>( //! self, //! slot: *mut usize, //! // Since `x` is not `#[pin]`, this is `Init`. -//! init: impl ::kernel::init::Init<usize, E>, +//! init: impl ::pin_init::Init<usize, E>, //! ) -> ::core::result::Result<(), E> { -//! unsafe { ::kernel::init::Init::__init(init, slot) } +//! unsafe { ::pin_init::Init::__init(init, slot) } //! } //! } //! // Implement the internal `HasPinData` trait that associates `Bar` with the pin-data struct //! // that we constructed above. -//! unsafe impl<T> ::kernel::init::__internal::HasPinData for Bar<T> { +//! unsafe impl<T> ::pin_init::__internal::HasPinData for Bar<T> { //! type PinData = __ThePinData<T>; //! unsafe fn __pin_data() -> Self::PinData { //! __ThePinData { @@ -147,7 +147,7 @@ //! // Implement the internal `PinData` trait that marks the pin-data struct as a pin-data //! // struct. This is important to ensure that no user can implement a rogue `__pin_data` //! // function without using `unsafe`. -//! unsafe impl<T> ::kernel::init::__internal::PinData for __ThePinData<T> { +//! unsafe impl<T> ::pin_init::__internal::PinData for __ThePinData<T> { //! type Datee = Bar<T>; //! } //! // Now we only want to implement `Unpin` for `Bar` when every structurally pinned field is @@ -191,7 +191,7 @@ //! #[expect(non_camel_case_types)] //! trait UselessPinnedDropImpl_you_need_to_specify_PinnedDrop {} //! impl< -//! T: ::kernel::init::PinnedDrop, +//! T: ::pin_init::PinnedDrop, //! > UselessPinnedDropImpl_you_need_to_specify_PinnedDrop for T {} //! impl<T> UselessPinnedDropImpl_you_need_to_specify_PinnedDrop for Bar<T> {} //! }; @@ -227,11 +227,11 @@ //! // - we `use` the `HasPinData` trait in the block, it is only available in that //! // scope. //! let data = unsafe { -//! use ::kernel::init::__internal::HasPinData; +//! use ::pin_init::__internal::HasPinData; //! Self::__pin_data() //! }; //! // Ensure that `data` really is of type `PinData` and help with type inference: -//! let init = ::kernel::init::__internal::PinData::make_closure::< +//! let init = ::pin_init::__internal::PinData::make_closure::< //! _, //! __InitOk, //! ::core::convert::Infallible, @@ -262,7 +262,7 @@ //! } //! // We again create a `DropGuard`. //! let __x_guard = unsafe { -//! ::kernel::init::__internal::DropGuard::new(::core::addr_of_mut!((*slot).x)) +//! ::pin_init::__internal::DropGuard::new(::core::addr_of_mut!((*slot).x)) //! }; //! // Since initialization has successfully completed, we can now forget //! // the guards. This is not `mem::forget`, since we only have @@ -303,7 +303,7 @@ //! }; //! // Construct the initializer. //! let init = unsafe { -//! ::kernel::init::pin_init_from_closure::< +//! ::pin_init::pin_init_from_closure::< //! _, //! ::core::convert::Infallible, //! >(init) @@ -350,19 +350,19 @@ //! unsafe fn b<E>( //! self, //! slot: *mut Bar<u32>, -//! init: impl ::kernel::init::PinInit<Bar<u32>, E>, +//! init: impl ::pin_init::PinInit<Bar<u32>, E>, //! ) -> ::core::result::Result<(), E> { -//! unsafe { ::kernel::init::PinInit::__pinned_init(init, slot) } +//! unsafe { ::pin_init::PinInit::__pinned_init(init, slot) } //! } //! unsafe fn a<E>( //! self, //! slot: *mut usize, -//! init: impl ::kernel::init::Init<usize, E>, +//! init: impl ::pin_init::Init<usize, E>, //! ) -> ::core::result::Result<(), E> { -//! unsafe { ::kernel::init::Init::__init(init, slot) } +//! unsafe { ::pin_init::Init::__init(init, slot) } //! } //! } -//! unsafe impl ::kernel::init::__internal::HasPinData for Foo { +//! unsafe impl ::pin_init::__internal::HasPinData for Foo { //! type PinData = __ThePinData; //! unsafe fn __pin_data() -> Self::PinData { //! __ThePinData { @@ -370,7 +370,7 @@ //! } //! } //! } -//! unsafe impl ::kernel::init::__internal::PinData for __ThePinData { +//! unsafe impl ::pin_init::__internal::PinData for __ThePinData { //! type Datee = Foo; //! } //! #[allow(dead_code)] @@ -394,8 +394,8 @@ //! let pinned = unsafe { ::core::pin::Pin::new_unchecked(self) }; //! // Create the unsafe token that proves that we are inside of a destructor, this //! // type is only allowed to be created in a destructor. -//! let token = unsafe { ::kernel::init::__internal::OnlyCallFromDrop::new() }; -//! ::kernel::init::PinnedDrop::drop(pinned, token); +//! let token = unsafe { ::pin_init::__internal::OnlyCallFromDrop::new() }; +//! ::pin_init::PinnedDrop::drop(pinned, token); //! } //! } //! }; @@ -421,8 +421,8 @@ //! //! ```rust,ignore //! // `unsafe`, full path and the token parameter are added, everything else stays the same. -//! unsafe impl ::kernel::init::PinnedDrop for Foo { -//! fn drop(self: Pin<&mut Self>, _: ::kernel::init::__internal::OnlyCallFromDrop) { +//! unsafe impl ::pin_init::PinnedDrop for Foo { +//! fn drop(self: Pin<&mut Self>, _: ::pin_init::__internal::OnlyCallFromDrop) { //! pr_info!("{self:p} is getting dropped."); //! } //! } @@ -448,10 +448,10 @@ //! let initializer = { //! struct __InitOk; //! let data = unsafe { -//! use ::kernel::init::__internal::HasPinData; +//! use ::pin_init::__internal::HasPinData; //! Foo::__pin_data() //! }; -//! let init = ::kernel::init::__internal::PinData::make_closure::< +//! let init = ::pin_init::__internal::PinData::make_closure::< //! _, //! __InitOk, //! ::core::convert::Infallible, @@ -462,12 +462,12 @@ //! unsafe { ::core::ptr::write(::core::addr_of_mut!((*slot).a), a) }; //! } //! let __a_guard = unsafe { -//! ::kernel::init::__internal::DropGuard::new(::core::addr_of_mut!((*slot).a)) +//! ::pin_init::__internal::DropGuard::new(::core::addr_of_mut!((*slot).a)) //! }; //! let init = Bar::new(36); //! unsafe { data.b(::core::addr_of_mut!((*slot).b), b)? }; //! let __b_guard = unsafe { -//! ::kernel::init::__internal::DropGuard::new(::core::addr_of_mut!((*slot).b)) +//! ::pin_init::__internal::DropGuard::new(::core::addr_of_mut!((*slot).b)) //! }; //! ::core::mem::forget(__b_guard); //! ::core::mem::forget(__a_guard); @@ -492,13 +492,16 @@ //! init(slot).map(|__InitOk| ()) //! }; //! let init = unsafe { -//! ::kernel::init::pin_init_from_closure::<_, ::core::convert::Infallible>(init) +//! ::pin_init::pin_init_from_closure::<_, ::core::convert::Infallible>(init) //! }; //! init //! }; //! ``` +#[cfg(kernel)] pub use ::macros::paste; +#[cfg(not(kernel))] +pub use ::paste::paste; /// Creates a `unsafe impl<...> PinnedDrop for $type` block. /// @@ -519,7 +522,7 @@ macro_rules! __pinned_drop { unsafe $($impl_sig)* { // Inherit all attributes and the type/ident tokens for the signature. $(#[$($attr)*])* - fn drop($($sig)*, _: $crate::init::__internal::OnlyCallFromDrop) { + fn drop($($sig)*, _: $crate::__internal::OnlyCallFromDrop) { $($inner)* } } @@ -865,7 +868,7 @@ macro_rules! __pin_data { // SAFETY: We have added the correct projection functions above to `__ThePinData` and // we also use the least restrictive generics possible. unsafe impl<$($impl_generics)*> - $crate::init::__internal::HasPinData for $name<$($ty_generics)*> + $crate::__internal::HasPinData for $name<$($ty_generics)*> where $($whr)* { type PinData = __ThePinData<$($ty_generics)*>; @@ -877,7 +880,7 @@ macro_rules! __pin_data { // SAFETY: TODO. unsafe impl<$($impl_generics)*> - $crate::init::__internal::PinData for __ThePinData<$($ty_generics)*> + $crate::__internal::PinData for __ThePinData<$($ty_generics)*> where $($whr)* { type Datee = $name<$($ty_generics)*>; @@ -936,7 +939,7 @@ macro_rules! __pin_data { // `PinnedDrop` as the parameter to `#[pin_data]`. #[expect(non_camel_case_types)] trait UselessPinnedDropImpl_you_need_to_specify_PinnedDrop {} - impl<T: $crate::init::PinnedDrop> + impl<T: $crate::PinnedDrop> UselessPinnedDropImpl_you_need_to_specify_PinnedDrop for T {} impl<$($impl_generics)*> UselessPinnedDropImpl_you_need_to_specify_PinnedDrop for $name<$($ty_generics)*> @@ -959,8 +962,8 @@ macro_rules! __pin_data { let pinned = unsafe { ::core::pin::Pin::new_unchecked(self) }; // SAFETY: Since this is a drop function, we can create this token to call the // pinned destructor of this type. - let token = unsafe { $crate::init::__internal::OnlyCallFromDrop::new() }; - $crate::init::PinnedDrop::drop(pinned, token); + let token = unsafe { $crate::__internal::OnlyCallFromDrop::new() }; + $crate::PinnedDrop::drop(pinned, token); } } }; @@ -1000,10 +1003,10 @@ macro_rules! __pin_data { $pvis unsafe fn $p_field<E>( self, slot: *mut $p_type, - init: impl $crate::init::PinInit<$p_type, E>, + init: impl $crate::PinInit<$p_type, E>, ) -> ::core::result::Result<(), E> { // SAFETY: TODO. - unsafe { $crate::init::PinInit::__pinned_init(init, slot) } + unsafe { $crate::PinInit::__pinned_init(init, slot) } } )* $( @@ -1011,10 +1014,10 @@ macro_rules! __pin_data { $fvis unsafe fn $field<E>( self, slot: *mut $type, - init: impl $crate::init::Init<$type, E>, + init: impl $crate::Init<$type, E>, ) -> ::core::result::Result<(), E> { // SAFETY: TODO. - unsafe { $crate::init::Init::__init(init, slot) } + unsafe { $crate::Init::__init(init, slot) } } )* } @@ -1131,15 +1134,15 @@ macro_rules! __init_internal { // // SAFETY: TODO. let data = unsafe { - use $crate::init::__internal::$has_data; + use $crate::__internal::$has_data; // Here we abuse `paste!` to retokenize `$t`. Declarative macros have some internal // information that is associated to already parsed fragments, so a path fragment // cannot be used in this position. Doing the retokenization results in valid rust // code. - $crate::init::macros::paste!($t::$get_data()) + $crate::macros::paste!($t::$get_data()) }; // Ensure that `data` really is of type `$data` and help with type inference: - let init = $crate::init::__internal::$data::make_closure::<_, __InitOk, $err>( + let init = $crate::__internal::$data::make_closure::<_, __InitOk, $err>( data, move |slot| { { @@ -1149,7 +1152,7 @@ macro_rules! __init_internal { // error when fields are missing (since they will be zeroed). We also have to // check that the type actually implements `Zeroable`. $({ - fn assert_zeroable<T: $crate::init::Zeroable>(_: *mut T) {} + fn assert_zeroable<T: $crate::Zeroable>(_: *mut T) {} // Ensure that the struct is indeed `Zeroable`. assert_zeroable(slot); // SAFETY: The type implements `Zeroable` by the check above. @@ -1186,7 +1189,7 @@ macro_rules! __init_internal { init(slot).map(|__InitOk| ()) }; // SAFETY: TODO. - let init = unsafe { $crate::init::$construct_closure::<_, $err>(init) }; + let init = unsafe { $crate::$construct_closure::<_, $err>(init) }; init }}; (init_slot($($use_data:ident)?): @@ -1217,10 +1220,10 @@ macro_rules! __init_internal { // // We rely on macro hygiene to make it impossible for users to access this local variable. // We use `paste!` to create new hygiene for `$field`. - $crate::init::macros::paste! { + $crate::macros::paste! { // SAFETY: We forget the guard later when initialization has succeeded. let [< __ $field _guard >] = unsafe { - $crate::init::__internal::DropGuard::new(::core::ptr::addr_of_mut!((*$slot).$field)) + $crate::__internal::DropGuard::new(::core::ptr::addr_of_mut!((*$slot).$field)) }; $crate::__init_internal!(init_slot($use_data): @@ -1243,15 +1246,15 @@ macro_rules! __init_internal { // // SAFETY: `slot` is valid, because we are inside of an initializer closure, we // return when an error/panic occurs. - unsafe { $crate::init::Init::__init(init, ::core::ptr::addr_of_mut!((*$slot).$field))? }; + unsafe { $crate::Init::__init(init, ::core::ptr::addr_of_mut!((*$slot).$field))? }; // Create the drop guard: // // We rely on macro hygiene to make it impossible for users to access this local variable. // We use `paste!` to create new hygiene for `$field`. - $crate::init::macros::paste! { + $crate::macros::paste! { // SAFETY: We forget the guard later when initialization has succeeded. let [< __ $field _guard >] = unsafe { - $crate::init::__internal::DropGuard::new(::core::ptr::addr_of_mut!((*$slot).$field)) + $crate::__internal::DropGuard::new(::core::ptr::addr_of_mut!((*$slot).$field)) }; $crate::__init_internal!(init_slot(): @@ -1280,10 +1283,10 @@ macro_rules! __init_internal { // // We rely on macro hygiene to make it impossible for users to access this local variable. // We use `paste!` to create new hygiene for `$field`. - $crate::init::macros::paste! { + $crate::macros::paste! { // SAFETY: We forget the guard later when initialization has succeeded. let [< __ $field _guard >] = unsafe { - $crate::init::__internal::DropGuard::new(::core::ptr::addr_of_mut!((*$slot).$field)) + $crate::__internal::DropGuard::new(::core::ptr::addr_of_mut!((*$slot).$field)) }; $crate::__init_internal!(init_slot($($use_data)?): @@ -1317,7 +1320,7 @@ macro_rules! __init_internal { // information that is associated to already parsed fragments, so a path fragment // cannot be used in this position. Doing the retokenization results in valid rust // code. - $crate::init::macros::paste!( + $crate::macros::paste!( ::core::ptr::write($slot, $t { $($acc)* ..zeroed @@ -1341,7 +1344,7 @@ macro_rules! __init_internal { // information that is associated to already parsed fragments, so a path fragment // cannot be used in this position. Doing the retokenization results in valid rust // code. - $crate::init::macros::paste!( + $crate::macros::paste!( ::core::ptr::write($slot, $t { $($acc)* }); @@ -1396,12 +1399,12 @@ macro_rules! __derive_zeroable { ) => { // SAFETY: Every field type implements `Zeroable` and padding bytes may be zero. #[automatically_derived] - unsafe impl<$($impl_generics)*> $crate::init::Zeroable for $name<$($ty_generics)*> + unsafe impl<$($impl_generics)*> $crate::Zeroable for $name<$($ty_generics)*> where $($($whr)*)? {} const _: () = { - fn assert_zeroable<T: ?::core::marker::Sized + $crate::init::Zeroable>() {} + fn assert_zeroable<T: ?::core::marker::Sized + $crate::Zeroable>() {} fn ensure_zeroable<$($impl_generics)*>() where $($($whr)*)? { diff --git a/scripts/generate_rust_analyzer.py b/scripts/generate_rust_analyzer.py index a44a4475d11f..54228e87e577 100755 --- a/scripts/generate_rust_analyzer.py +++ b/scripts/generate_rust_analyzer.py @@ -95,7 +95,7 @@ def generate_crates(srctree, objtree, sysroot_src, external_src, cfgs): append_crate( "pin_init_internal", - srctree / "rust" / "pin-init" / "internal" / "src" / "_lib.rs", + srctree / "rust" / "pin-init" / "internal" / "src" / "lib.rs", [], cfg=["kernel"], is_proc_macro=True, @@ -103,7 +103,7 @@ def generate_crates(srctree, objtree, sysroot_src, external_src, cfgs): append_crate( "pin_init", - srctree / "rust" / "pin-init" / "src" / "_lib.rs", + srctree / "rust" / "pin-init" / "src" / "lib.rs", ["core", "pin_init_internal", "macros"], cfg=["kernel"], ) -- cgit v1.2.3 From 35dac71cff8f68f065a6719631db919d0640d5e5 Mon Sep 17 00:00:00 2001 From: "Guilherme G. Piccoli" <gpiccoli@igalia.com> Date: Mon, 20 Jan 2025 16:04:26 -0300 Subject: scripts: add script to extract built-in firmware blobs There is currently no tool to extract a firmware blob that is built-in on vmlinux to the best of my knowledge. So if we have a kernel image containing the blobs, and we want to rebuild the kernel with some debug patches for example (and given that the image also has IKCONFIG=y), we currently can't do that for the same versions for all the firmware blobs, _unless_ we have exact commits of linux-firmware for the specific versions for each firmware included. Through the options CONFIG_EXTRA_FIRMWARE{_DIR} one is able to build a kernel including firmware blobs in a built-in fashion. This is usually the case of built-in drivers that require some blobs in order to work properly, for example, like in non-initrd based systems. Add hereby a script to extract these blobs from a non-stripped vmlinux, similar to the idea of "extract-ikconfig". The firmware loader interface saves such built-in blobs as rodata entries, having a field for the FW name as "_fw_<module_name>_<firmware_name>_bin"; the tool extracts files named "<module_name>_<firmware_name>" for each rodata firmware entry detected. It makes use of awk, bash, dd and readelf, pretty standard tooling for Linux development. With this tool, we can blindly extract the FWs and easily re-add them in the new debug kernel build, allowing a more deterministic testing without the burden of "hunting down" the proper version of each firmware binary. Link: https://lkml.kernel.org/r/20250120190436.127578-1-gpiccoli@igalia.com Signed-off-by: Guilherme G. Piccoli <gpiccoli@igalia.com> Suggested-by: Thadeu Lima de Souza Cascardo <cascardo@igalia.com> Reviewed-by: Thadeu Lima de Souza Cascardo <cascardo@igalia.com> Cc: Danilo Krummrich <dakr@kernel.org> Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org> Cc: Luis Chamberalin <mcgrof@kernel.org> Cc: Masahiro Yamada <masahiroy@kernel.org> Cc: Nathan Chancellor <nathan@kernel.org> Cc: Nicolas Schier <nicolas@fjasle.eu> Cc: "Rafael J. Wysocki" <rafael@kernel.org> Cc: Russ Weight <russ.weight@linux.dev> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> --- scripts/extract-fwblobs | 30 ++++++++++++++++++++++++++++++ 1 file changed, 30 insertions(+) create mode 100755 scripts/extract-fwblobs (limited to 'scripts') diff --git a/scripts/extract-fwblobs b/scripts/extract-fwblobs new file mode 100755 index 000000000000..53729124e5a0 --- /dev/null +++ b/scripts/extract-fwblobs @@ -0,0 +1,30 @@ +#!/bin/bash +# SPDX-License-Identifier: GPL-2.0 +# +# ----------------------------------------------------------------------------- +# Extracts the vmlinux built-in firmware blobs - requires a non-stripped image +# ----------------------------------------------------------------------------- + +if [ -z "$1" ]; then + echo "Must provide a non-stripped vmlinux as argument" + exit 1 +fi + +read -r RD_ADDR_HEX RD_OFF_HEX <<< "$( readelf -SW "$1" |\ +grep -w rodata | awk '{print "0x"$5" 0x"$6}' )" + +FW_SYMS="$(readelf -sW "$1" |\ +awk -n '/fw_end/ { end=$2 ; print name " 0x" start " 0x" end; } { start=$2; name=$8; }')" + +while IFS= read -r entry; do + read -r FW_NAME FW_ADDR_ST_HEX FW_ADDR_END_HEX <<< "$entry" + + # Notice kernel prepends _fw_ and appends _bin to the FW name + # in rodata; hence we hereby filter that out. + FW_NAME=${FW_NAME:4:-4} + + FW_OFFSET="$(printf "%d" $((FW_ADDR_ST_HEX - RD_ADDR_HEX + RD_OFF_HEX)))" + FW_SIZE="$(printf "%d" $((FW_ADDR_END_HEX - FW_ADDR_ST_HEX)))" + + dd if="$1" of="./${FW_NAME}" bs="${FW_SIZE}" count=1 iflag=skip_bytes skip="${FW_OFFSET}" +done <<< "${FW_SYMS}" -- cgit v1.2.3 From 9ad18c855518161d0a80f6a7de3ed495f746cb5d Mon Sep 17 00:00:00 2001 From: Vlastimil Babka <vbabka@suse.cz> Date: Mon, 3 Feb 2025 12:13:16 +0100 Subject: get_maintainer: add --substatus for reporting subsystem status Patch series "get_maintainer: report subsystem status separately", v2. The subsystem status (S: field) can inform a patch submitter if the subsystem is well maintained or e.g. maintainers are missing. In get_maintainer, it is currently reported with --role(stats) by adjusting the maintainer role for any status different from Maintained. This has two downsides: - if a subsystem has only reviewers or mailing lists and no maintainers, the status is not reported. For example Orphan subsystems typically have no maintainers so there's nobody to report as orphan minder. - the Supported status means that someone is paid for maintaining, but it is reported as "supporter" for all the maintainers, which can be incorrect (only some of them may be paid). People (including myself) have been also confused about what "supporter" means. The second point has been brought up in 2022 and the discussion in the end resulted in adjusting documentation only [1]. I however agree with Ted's points that it's misleading to take the subsystem status and apply it to all maintainers [2]. The attempt to modify get_maintainer output was retracted after Joe objected that the status becomes not reported at all [3]. This series addresses that concern by reporting the status (unless it's the most common Maintained one) on separate lines that follow the reported emails, using a new --substatus parameter. Care is taken to reduce the noise to minimum by not reporting the most common Maintained status, by default require no opt-in that would need the users to discover the new parameter, and at the same time not to break existing git --cc-cmd usage. [1] https://lore.kernel.org/all/20221006162413.858527-1-bryan.odonoghue@linaro.org/ [2] https://lore.kernel.org/all/Yzen4X1Na0MKXHs9@mit.edu/ [3] https://lore.kernel.org/all/30776fe75061951777da8fa6618ae89bea7a8ce4.camel@perches.com/ This patch (of 2): The subsystem status is currently reported with --role(stats) by adjusting the maintainer role for any status different from Maintained. This has two downsides: - if a subsystem has only reviewers or mailing lists and no maintainers, the status is not reported (i.e. typically, Orphan subsystems have no maintainers) - the Supported status means that someone is paid for maintaining, but it is reported as "supporter" for all the maintainers, which can be incorrect. People have been also confused about what "supporter" means. This patch introduces a new --substatus option and functionality aimed to report the subsystem status separately, without adjusting the reported maintainer role. After the e-mails are output, the status of subsystems will follow, for example: ... linux-kernel@vger.kernel.org (open list:LIBRARY CODE) LIBRARY CODE status: Supported In order to allow replacing the role rewriting seamlessly, the new option works as follows: - it is automatically enabled when --email and --role are enabled (the defaults include --email and --rolestats which implies --role) - usages with --norolestats e.g. for git's --cc-cmd will thus need no adjustments - the most common Maintained status is not reported at all, to reduce unnecessary noise - THE REST catch-all section (contains lkml) status is not reported - the existing --subsystem and --status options are unaffected so their users will need no adjustments [vbabka@suse.cz: require that script output goes to a terminal] Link: https://lkml.kernel.org/r/66c2bf7a-9119-4850-b6b8-ac8f426966e1@suse.cz Link: https://lkml.kernel.org/r/20250203-b4-get_maintainer-v2-0-83ba008b491f@suse.cz Link: https://lkml.kernel.org/r/20250203-b4-get_maintainer-v2-1-83ba008b491f@suse.cz Fixes: c1565b6f7b53 ("get_maintainer: add --substatus for reporting subsystem status") Closes: https://lore.kernel.org/all/7aodxv46lj6rthjo4i5zhhx2lybrhb4uknpej2dyz3e7im5w3w@w23bz6fx3jnn/ Signed-off-by: Vlastimil Babka <vbabka@suse.cz> Acked-by: Lorenzo Stoakes <lorenzo.stoakes@oracle.com> Tested-by: Geert Uytterhoeven <geert+renesas@glider.be> Tested-by: Uwe Kleine-K=F6nig <u.kleine-koenig@baylibre.com> Cc: Bryan O'Donoghue <bryan.odonoghue@linaro.org> Cc: Joe Perches <joe@perches.com> Cc: Kees Cook <kees@kernel.org> Cc: Ted Ts'o <tytso@mit.edu> Cc: Thorsten Leemhuis <linux@leemhuis.info> Cc: Vlastimil Babka <vbabka@suse.cz> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> --- scripts/get_maintainer.pl | 28 +++++++++++++++++++++++++--- 1 file changed, 25 insertions(+), 3 deletions(-) (limited to 'scripts') diff --git a/scripts/get_maintainer.pl b/scripts/get_maintainer.pl index 5ac02e198737..3eb922b4d22c 100755 --- a/scripts/get_maintainer.pl +++ b/scripts/get_maintainer.pl @@ -50,6 +50,7 @@ my $output_multiline = 1; my $output_separator = ", "; my $output_roles = 0; my $output_rolestats = 1; +my $output_substatus = undef; my $output_section_maxlen = 50; my $scm = 0; my $tree = 1; @@ -269,6 +270,7 @@ if (!GetOptions( 'separator=s' => \$output_separator, 'subsystem!' => \$subsystem, 'status!' => \$status, + 'substatus!' => \$output_substatus, 'scm!' => \$scm, 'tree!' => \$tree, 'web!' => \$web, @@ -314,6 +316,10 @@ $output_multiline = 0 if ($output_separator ne ", "); $output_rolestats = 1 if ($interactive); $output_roles = 1 if ($output_rolestats); +if (!defined $output_substatus) { + $output_substatus = $email && $output_roles && -t STDOUT; +} + if ($sections || $letters ne "") { $sections = 1; $email = 0; @@ -637,6 +643,7 @@ my @web = (); my @bug = (); my @subsystem = (); my @status = (); +my @substatus = (); my %deduplicate_name_hash = (); my %deduplicate_address_hash = (); @@ -651,6 +658,11 @@ if ($scm) { output(@scm); } +if ($output_substatus) { + @substatus = uniq(@substatus); + output(@substatus); +} + if ($status) { @status = uniq(@status); output(@status); @@ -859,6 +871,7 @@ sub get_maintainers { @bug = (); @subsystem = (); @status = (); + @substatus = (); %deduplicate_name_hash = (); %deduplicate_address_hash = (); if ($email_git_all_signature_types) { @@ -1073,6 +1086,7 @@ MAINTAINER field selection options: --remove-duplicates => minimize duplicate email names/addresses --roles => show roles (status:subsystem, git-signer, list, etc...) --rolestats => show roles and statistics (commits/total_commits, %) + --substatus => show subsystem status if not Maintained (default: match --roles when output is tty)" --file-emails => add email addresses found in -f file (default: 0 (off)) --fixes => for patches, add signatures of commits with 'Fixes: <commit>' (default: 1 (on)) --scm => print SCM tree(s) if any @@ -1335,7 +1349,9 @@ sub add_categories { my $start = find_starting_index($index); my $end = find_ending_index($index); - push(@subsystem, $typevalue[$start]); + my $subsystem = $typevalue[$start]; + push(@subsystem, $subsystem); + my $status = "Unknown"; for ($i = $start + 1; $i < $end; $i++) { my $tv = $typevalue[$i]; @@ -1386,8 +1402,8 @@ sub add_categories { } } elsif ($ptype eq "R") { if ($email_reviewer) { - my $subsystem = get_subsystem_name($i); - push_email_addresses($pvalue, "reviewer:$subsystem" . $suffix); + my $subs = get_subsystem_name($i); + push_email_addresses($pvalue, "reviewer:$subs" . $suffix); } } elsif ($ptype eq "T") { push(@scm, $pvalue . $suffix); @@ -1397,9 +1413,14 @@ sub add_categories { push(@bug, $pvalue . $suffix); } elsif ($ptype eq "S") { push(@status, $pvalue . $suffix); + $status = $pvalue; } } } + + if ($subsystem ne "THE REST" and $status ne "Maintained") { + push(@substatus, $subsystem . " status: " . $status . $suffix) + } } sub email_inuse { @@ -1903,6 +1924,7 @@ EOT $done = 1; $output_rolestats = 0; $output_roles = 0; + $output_substatus = 0; last; } elsif ($nr =~ /^\d+$/ && $nr > 0 && $nr <= $count) { $selected{$nr - 1} = !$selected{$nr - 1}; -- cgit v1.2.3 From 6ba31721aeef048922672d49d48a7f3826005f7d Mon Sep 17 00:00:00 2001 From: Vlastimil Babka <vbabka@suse.cz> Date: Mon, 3 Feb 2025 12:13:17 +0100 Subject: get_maintainer: stop reporting subsystem status as maintainer role After introducing the --substatus option, we can stop adjusting the reported maintainer role by the subsystem's status. For compatibility with the --git-chief-penguins option, keep the "chief penguin" role. Link: https://lkml.kernel.org/r/20250203-b4-get_maintainer-v2-2-83ba008b491f@suse.cz Signed-off-by: Vlastimil Babka <vbabka@suse.cz> Reviewed-by: Lorenzo Stoakes <lorenzo.stoakes@oracle.com> Tested-by: Lorenzo Stoakes <lorenzo.stoakes@oracle.com> Cc: Bryan O'Donoghue <bryan.odonoghue@linaro.org> Cc: Joe Perches <joe@perches.com> Cc: Kees Cook <kees@kernel.org> Cc: Ted Ts'o <tytso@mit.edu> Cc: Thorsten Leemhuis <linux@leemhuis.info> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> --- scripts/get_maintainer.pl | 21 ++++++--------------- 1 file changed, 6 insertions(+), 15 deletions(-) (limited to 'scripts') diff --git a/scripts/get_maintainer.pl b/scripts/get_maintainer.pl index 3eb922b4d22c..4414194bedcf 100755 --- a/scripts/get_maintainer.pl +++ b/scripts/get_maintainer.pl @@ -1084,7 +1084,7 @@ MAINTAINER field selection options: --moderated => include moderated lists(s) if any (default: true) --s => include subscriber only list(s) if any (default: false) --remove-duplicates => minimize duplicate email names/addresses - --roles => show roles (status:subsystem, git-signer, list, etc...) + --roles => show roles (role:subsystem, git-signer, list, etc...) --rolestats => show roles and statistics (commits/total_commits, %) --substatus => show subsystem status if not Maintained (default: match --roles when output is tty)" --file-emails => add email addresses found in -f file (default: 0 (off)) @@ -1298,8 +1298,9 @@ sub get_maintainer_role { my $start = find_starting_index($index); my $end = find_ending_index($index); - my $role = "unknown"; + my $role = "maintainer"; my $subsystem = get_subsystem_name($index); + my $status = "unknown"; for ($i = $start + 1; $i < $end; $i++) { my $tv = $typevalue[$i]; @@ -1307,23 +1308,13 @@ sub get_maintainer_role { my $ptype = $1; my $pvalue = $2; if ($ptype eq "S") { - $role = $pvalue; + $status = $pvalue; } } } - $role = lc($role); - if ($role eq "supported") { - $role = "supporter"; - } elsif ($role eq "maintained") { - $role = "maintainer"; - } elsif ($role eq "odd fixes") { - $role = "odd fixer"; - } elsif ($role eq "orphan") { - $role = "orphan minder"; - } elsif ($role eq "obsolete") { - $role = "obsolete minder"; - } elsif ($role eq "buried alive in reporters") { + $status = lc($status); + if ($status eq "buried alive in reporters") { $role = "chief penguin"; } -- cgit v1.2.3 From 4022918876f9a28fe5379e2d7b7840e84f7b56ed Mon Sep 17 00:00:00 2001 From: Brendan Jackman <jackmanb@google.com> Date: Thu, 20 Feb 2025 12:23:40 +0000 Subject: scripts/gdb: add $lx_per_cpu_ptr() We currently have $lx_per_cpu() which works fine for stuff that kernel code would access via per_cpu(). But this doesn't work for stuff that kernel code accesses via per_cpu_ptr(): (gdb) p $lx_per_cpu(node_data[1].node_zones[2]->per_cpu_pageset) Cannot access memory at address 0xffff11105fbd6c28 This is because we take the address of the pointer and use that as the offset, instead of using the stored value. Add a GDB version that mirrors the kernel API, which uses the pointer value. To be consistent with per_cpu_ptr(), we need to return the pointer value instead of dereferencing it for the user. Therefore, move the existing dereference out of the per_cpu() Python helper and do that only in the $lx_per_cpu() implementation. Link: https://lkml.kernel.org/r/20250220-lx-per-cpu-ptr-v2-1-945dee8d8d38@google.com Signed-off-by: Brendan Jackman <jackmanb@google.com> Reviewed-by: Jan Kiszka <jan.kiszka@siemens.com> Cc: Florian Rommel <mail@florommel.de> Cc: Kieran Bingham <kbingham@kernel.org> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> --- scripts/gdb/linux/cpus.py | 22 ++++++++++++++++++++-- 1 file changed, 20 insertions(+), 2 deletions(-) (limited to 'scripts') diff --git a/scripts/gdb/linux/cpus.py b/scripts/gdb/linux/cpus.py index 13eb8b3901b8..1a50a4195def 100644 --- a/scripts/gdb/linux/cpus.py +++ b/scripts/gdb/linux/cpus.py @@ -46,7 +46,7 @@ def per_cpu(var_ptr, cpu): # !CONFIG_SMP case offset = 0 pointer = var_ptr.cast(utils.get_long_type()) + offset - return pointer.cast(var_ptr.type).dereference() + return pointer.cast(var_ptr.type) cpu_mask = {} @@ -149,11 +149,29 @@ Note that VAR has to be quoted as string.""" super(PerCpu, self).__init__("lx_per_cpu") def invoke(self, var, cpu=-1): - return per_cpu(var.address, cpu) + return per_cpu(var.address, cpu).dereference() PerCpu() + +class PerCpuPtr(gdb.Function): + """Return per-cpu pointer. + +$lx_per_cpu_ptr("VAR"[, CPU]): Return the per-cpu pointer called VAR for the +given CPU number. If CPU is omitted, the CPU of the current context is used. +Note that VAR has to be quoted as string.""" + + def __init__(self): + super(PerCpuPtr, self).__init__("lx_per_cpu_ptr") + + def invoke(self, var, cpu=-1): + return per_cpu(var, cpu) + + +PerCpuPtr() + + def get_current_task(cpu): task_ptr_type = task_type.get_type().pointer() -- cgit v1.2.3 From 15c200eaf569f804ba684cefbae437a6c731ba95 Mon Sep 17 00:00:00 2001 From: Easwar Hariharan <eahariha@linux.microsoft.com> Date: Tue, 25 Feb 2025 20:17:15 +0000 Subject: coccinelle: misc: secs_to_jiffies: Patch expressions too Patch series "Converge on using secs_to_jiffies() part two", v3. This is the second series that converts users of msecs_to_jiffies() that either use the multiply pattern of either of: - msecs_to_jiffies(N*1000) or - msecs_to_jiffies(N*MSEC_PER_SEC) where N is a constant or an expression, to avoid the multiplication. The conversion is made with Coccinelle with the secs_to_jiffies() script in scripts/coccinelle/misc. Attention is paid to what the best change can be rather than restricting to what the tool provides. This patch (of 16): Teach the script to suggest conversions for timeout patterns where the arguments to msecs_to_jiffies() are expressions as well. Link: https://lkml.kernel.org/r/20250225-converge-secs-to-jiffies-part-two-v3-0-a43967e36c88@linux.microsoft.com Link: https://lkml.kernel.org/r/20250225-converge-secs-to-jiffies-part-two-v3-1-a43967e36c88@linux.microsoft.com Signed-off-by: Easwar Hariharan <eahariha@linux.microsoft.com> Cc: Mark Brown <broonie@kernel.org> Cc: Carlos Maiolino <cem@kernel.org> Cc: Chris Mason <clm@fb.com> Cc: Christoph Hellwig <hch@lst.de> Cc: Damien Le Maol <dlemoal@kernel.org> Cc: "Darrick J. Wong" <djwong@kernel.org> Cc: David Sterba <dsterba@suse.com> Cc: Dick Kennedy <dick.kennedy@broadcom.com> Cc: Dongsheng Yang <dongsheng.yang@easystack.cn> Cc: Fabio Estevam <festevam@gmail.com> Cc: Frank Li <frank.li@nxp.com> Cc: Hans de Goede <hdegoede@redhat.com> Cc: Henrique de Moraes Holschuh <hmh@hmh.eng.br> Cc: Ilpo Jarvinen <ilpo.jarvinen@linux.intel.com> Cc: Ilya Dryomov <idryomov@gmail.com> Cc: James Bottomley <james.bottomley@HansenPartnership.com> Cc: James Smart <james.smart@broadcom.com> Cc: Jaroslav Kysela <perex@perex.cz> Cc: Jason Gunthorpe <jgg@ziepe.ca> Cc: Jens Axboe <axboe@kernel.dk> Cc: Josef Bacik <josef@toxicpanda.com> Cc: Julia Lawall <julia.lawall@inria.fr> Cc: Kalesh Anakkur Purayil <kalesh-anakkur.purayil@broadcom.com> Cc: Keith Busch <kbusch@kernel.org> Cc: Leon Romanovsky <leon@kernel.org> Cc: "Martin K. Petersen" <martin.petersen@oracle.com> Cc: Nicolas Palix <nicolas.palix@imag.fr> Cc: Niklas Cassel <cassel@kernel.org> Cc: Oded Gabbay <ogabbay@kernel.org> Cc: Takashi Iwai <tiwai@suse.de> Cc: Sagi Grimberg <sagi@grimberg.me> Cc: Sascha Hauer <s.hauer@pengutronix.de> Cc: Sebastian Reichel <sre@kernel.org> Cc: Selvin Thyparampil Xavier <selvin.xavier@broadcom.com> Cc: Shawn Guo <shawnguo@kernel.org> Cc: Shyam-sundar S-k <Shyam-sundar.S-k@amd.com> Cc: Takashi Iwai <tiwai@suse.com> Cc: Xiubo Li <xiubli@redhat.com> Cc: Carlos Maiolino <cmaiolino@redhat.com> Cc: Marc Kleine-Budde <mkl@pengutronix.de> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> --- scripts/coccinelle/misc/secs_to_jiffies.cocci | 10 ++++++++++ 1 file changed, 10 insertions(+) (limited to 'scripts') diff --git a/scripts/coccinelle/misc/secs_to_jiffies.cocci b/scripts/coccinelle/misc/secs_to_jiffies.cocci index 8bbb2884ea5d..416f348174ca 100644 --- a/scripts/coccinelle/misc/secs_to_jiffies.cocci +++ b/scripts/coccinelle/misc/secs_to_jiffies.cocci @@ -20,3 +20,13 @@ virtual patch - msecs_to_jiffies(C * MSEC_PER_SEC) + secs_to_jiffies(C) + +@depends on patch@ expression E; @@ + +- msecs_to_jiffies(E * 1000) ++ secs_to_jiffies(E) + +@depends on patch@ expression E; @@ + +- msecs_to_jiffies(E * MSEC_PER_SEC) ++ secs_to_jiffies(E) -- cgit v1.2.3 From 36799069b48198e5ce92d99310060c4aecb4b3e3 Mon Sep 17 00:00:00 2001 From: Josh Poimboeuf <jpoimboe@kernel.org> Date: Fri, 14 Mar 2025 12:29:11 -0700 Subject: objtool: Add CONFIG_OBJTOOL_WERROR Objtool warnings can be indicative of crashes, broken live patching, or even boot failures. Ignoring them is not recommended. Add CONFIG_OBJTOOL_WERROR to upgrade objtool warnings to errors by enabling the objtool --Werror option. Also set --backtrace to print the branches leading up to the warning, which can help considerably when debugging certain warnings. To avoid breaking bots too badly for now, make it the default for real world builds only (!COMPILE_TEST). Co-developed-by: Brendan Jackman <jackmanb@google.com> Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org> Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org> Link: https://lore.kernel.org/r/3e7c109313ff15da6c80788965cc7450115b0196.1741975349.git.jpoimboe@kernel.org --- lib/Kconfig.debug | 11 +++++++++++ scripts/Makefile.lib | 1 + 2 files changed, 12 insertions(+) (limited to 'scripts') diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug index 35796c290ca3..a9709a6db30f 100644 --- a/lib/Kconfig.debug +++ b/lib/Kconfig.debug @@ -545,6 +545,17 @@ config FRAME_POINTER config OBJTOOL bool +config OBJTOOL_WERROR + bool "Upgrade objtool warnings to errors" + depends on OBJTOOL && !COMPILE_TEST + help + Fail the build on objtool warnings. + + Objtool warnings can indicate kernel instability, including boot + failures. This option is highly recommended. + + If unsure, say Y. + config STACK_VALIDATION bool "Compile-time stack metadata validation" depends on HAVE_STACK_VALIDATION && UNWINDER_FRAME_POINTER diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib index cad20f0e66ee..99e281966ba3 100644 --- a/scripts/Makefile.lib +++ b/scripts/Makefile.lib @@ -277,6 +277,7 @@ objtool-args-$(CONFIG_HAVE_STATIC_CALL_INLINE) += --static-call objtool-args-$(CONFIG_HAVE_UACCESS_VALIDATION) += --uaccess objtool-args-$(CONFIG_GCOV_KERNEL) += --no-unreachable objtool-args-$(CONFIG_PREFIX_SYMBOLS) += --prefix=$(CONFIG_FUNCTION_PADDING_BYTES) +objtool-args-$(CONFIG_OBJTOOL_WERROR) += --Werror --backtrace objtool-args = $(objtool-args-y) \ $(if $(delay-objtool), --link) \ -- cgit v1.2.3 From e0349c46cb4fbbb507fa34476bd70f9c82bad359 Mon Sep 17 00:00:00 2001 From: Antonio Quartulli <antonio@mandelbit.com> Date: Fri, 21 Feb 2025 21:40:34 +0100 Subject: scripts/gdb/linux/symbols.py: address changes to module_sect_attrs MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit When loading symbols from kernel modules we used to iterate from 0 to module_sect_attrs::nsections, in order to retrieve their name and address. However module_sect_attrs::nsections has been removed from the struct by a previous commit. Re-arrange the iteration by accessing all items in module_sect_attrs::grp::bin_attrs[] until NULL is found (it's a NULL terminated array). At the same time the symbol address cannot be extracted from module_sect_attrs::attrs[]::address anymore because it has also been deleted. Fetch it from module_sect_attrs::grp::bin_attrs[]::private as described in 4b2c11e4aaf7. Link: https://lkml.kernel.org/r/20250221204034.4430-1-antonio@mandelbit.com Fixes: d8959b947a8d ("module: sysfs: Drop member 'module_sect_attrs::nsections'") Fixes: 4b2c11e4aaf7 ("module: sysfs: Drop member 'module_sect_attr::address'") Signed-off-by: Antonio Quartulli <antonio@mandelbit.com> Reviewed-by: Jan Kiszka <jan.kiszka@siemens.com> Cc: Thomas Weißschuh <linux@weissschuh.net> Cc: Kieran Bingham <kbingham@kernel.org> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> --- scripts/gdb/linux/symbols.py | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-) (limited to 'scripts') diff --git a/scripts/gdb/linux/symbols.py b/scripts/gdb/linux/symbols.py index f6c1b063775a..15d76f7d8ebc 100644 --- a/scripts/gdb/linux/symbols.py +++ b/scripts/gdb/linux/symbols.py @@ -15,6 +15,7 @@ import gdb import os import re +from itertools import count from linux import modules, utils, constants @@ -95,10 +96,14 @@ lx-symbols command.""" except gdb.error: return str(module_addr) - attrs = sect_attrs['attrs'] - section_name_to_address = { - attrs[n]['battr']['attr']['name'].string(): attrs[n]['address'] - for n in range(int(sect_attrs['nsections']))} + section_name_to_address = {} + for i in count(): + # this is a NULL terminated array + if sect_attrs['grp']['bin_attrs'][i] == 0x0: + break + + attr = sect_attrs['grp']['bin_attrs'][i].dereference() + section_name_to_address[attr['attr']['name'].string()] = attr['private'] textaddr = section_name_to_address.get(".text", module_addr) args = [] -- cgit v1.2.3 From 28939c3e9925735166e7b6e42f9e1dc828093510 Mon Sep 17 00:00:00 2001 From: Ilya Leoshkevich <iii@linux.ibm.com> Date: Mon, 3 Mar 2025 12:03:58 +0100 Subject: scripts/gdb/symbols: determine KASLR offset on s390 Use QEMU's qemu.PhyMemMode [1] functionality to read vmcore from the physical memory the same way the existing dump tooling does this. Gracefully handle non-QEMU targets, early boot, and memory corruptions; print a warning if such situation is detected. [1] https://qemu-project.gitlab.io/qemu/system/gdb.html#examining-physical-memory Link: https://lkml.kernel.org/r/20250303110437.79070-1-iii@linux.ibm.com Signed-off-by: Ilya Leoshkevich <iii@linux.ibm.com> Acked-by: Jan Kiszka <jan.kiszka@siemens.com> Cc: Alexander Gordeev <agordeev@linux.ibm.com> Cc: Andrew Donnellan <ajd@linux.ibm.com> Cc: Christian Borntraeger <borntraeger@linux.ibm.com> Cc: Heiko Carstens <hca@linux.ibm.com> Cc: Kieran Bingham <kbingham@kernel.org> Cc: Nina Schoetterl-Glausch <nsg@linux.ibm.com> Cc: Vasily Gorbik <gor@linux.ibm.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> --- scripts/gdb/linux/symbols.py | 31 ++++++++++++++++++++++++++++++- scripts/gdb/linux/utils.py | 35 +++++++++++++++++++++++++++++++++++ 2 files changed, 65 insertions(+), 1 deletion(-) (limited to 'scripts') diff --git a/scripts/gdb/linux/symbols.py b/scripts/gdb/linux/symbols.py index 15d76f7d8ebc..b255177301e9 100644 --- a/scripts/gdb/linux/symbols.py +++ b/scripts/gdb/linux/symbols.py @@ -14,6 +14,7 @@ import gdb import os import re +import struct from itertools import count from linux import modules, utils, constants @@ -54,6 +55,29 @@ if hasattr(gdb, 'Breakpoint'): return False +def get_vmcore_s390(): + with utils.qemu_phy_mem_mode(): + vmcore_info = 0x0e0c + paddr_vmcoreinfo_note = gdb.parse_and_eval("*(unsigned long long *)" + + hex(vmcore_info)) + inferior = gdb.selected_inferior() + elf_note = inferior.read_memory(paddr_vmcoreinfo_note, 12) + n_namesz, n_descsz, n_type = struct.unpack(">III", elf_note) + desc_paddr = paddr_vmcoreinfo_note + len(elf_note) + n_namesz + 1 + return gdb.parse_and_eval("(char *)" + hex(desc_paddr)).string() + + +def get_kerneloffset(): + if utils.is_target_arch('s390'): + try: + vmcore_str = get_vmcore_s390() + except gdb.error as e: + gdb.write("{}\n".format(e)) + return None + return utils.parse_vmcore(vmcore_str).kerneloffset + return None + + class LxSymbols(gdb.Command): """(Re-)load symbols of Linux kernel and currently loaded modules. @@ -160,7 +184,12 @@ lx-symbols command.""" obj.filename.endswith('vmlinux.debug')): orig_vmlinux = obj.filename gdb.execute("symbol-file", to_string=True) - gdb.execute("symbol-file {0}".format(orig_vmlinux)) + kerneloffset = get_kerneloffset() + if kerneloffset is None: + offset_arg = "" + else: + offset_arg = " -o " + hex(kerneloffset) + gdb.execute("symbol-file {0}{1}".format(orig_vmlinux, offset_arg)) self.loaded_modules = [] module_list = modules.module_list() diff --git a/scripts/gdb/linux/utils.py b/scripts/gdb/linux/utils.py index 245ab297ea84..03ebdccf5f69 100644 --- a/scripts/gdb/linux/utils.py +++ b/scripts/gdb/linux/utils.py @@ -11,6 +11,11 @@ # This work is licensed under the terms of the GNU GPL version 2. # +import contextlib +import dataclasses +import re +import typing + import gdb @@ -216,3 +221,33 @@ def gdb_eval_or_none(expresssion): return gdb.parse_and_eval(expresssion) except gdb.error: return None + + +@contextlib.contextmanager +def qemu_phy_mem_mode(): + connection = gdb.selected_inferior().connection + orig = connection.send_packet("qqemu.PhyMemMode") + if orig not in b"01": + raise gdb.error("Unexpected qemu.PhyMemMode") + orig = orig.decode() + if connection.send_packet("Qqemu.PhyMemMode:1") != b"OK": + raise gdb.error("Failed to set qemu.PhyMemMode") + try: + yield + finally: + if connection.send_packet("Qqemu.PhyMemMode:" + orig) != b"OK": + raise gdb.error("Failed to restore qemu.PhyMemMode") + + +@dataclasses.dataclass +class VmCore: + kerneloffset: typing.Optional[int] + + +def parse_vmcore(s): + match = re.search(r"KERNELOFFSET=([0-9a-f]+)", s) + if match is None: + kerneloffset = None + else: + kerneloffset = int(match.group(1), 16) + return VmCore(kerneloffset=kerneloffset) -- cgit v1.2.3 From bc2f19d65373bc166c18c486e2ec2611f5a12d8f Mon Sep 17 00:00:00 2001 From: Philipp Hahn <phahn-oss@avm.de> Date: Mon, 10 Mar 2025 12:22:27 +0100 Subject: checkpatch: describe --min-conf-desc-length Neither the warning nor the help message gives any hint on the unit for length: Could be meters, inches, bytes, characters or ... lines. Extend the output of `--help` to name the unit "lines" and the default: - --min-conf-desc-length=n set the min description length, if shorter, warn + --min-conf-desc-length=n set the minimum description length for config symbols + in lines, if shorter, warn (default 4) Include the minimum number of lines as other error messages already do: - WARNING: please write a help paragraph that fully describes the config symbol + WARNING: please write a help paragraph that fully describes the config symbol with at least 4 lines Link: https://lkml.kernel.org/r/c71c170c90eba26265951e248adfedd3245fe575.1741605695.git.p.hahn@avm.de Signed-off-by: Philipp Hahn <p.hahn@avm.de> Cc: Andy Whitcroft <apw@canonical.com> Cc: Joe Perches <joe@perches.com> Cc: Dwaipayan Ray <dwaipayanray1@gmail.com> Cc: Lukas Bulwahn <lukas.bulwahn@gmail.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> --- scripts/checkpatch.pl | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) (limited to 'scripts') diff --git a/scripts/checkpatch.pl b/scripts/checkpatch.pl index 7b28ad331742..784912f570e9 100755 --- a/scripts/checkpatch.pl +++ b/scripts/checkpatch.pl @@ -113,7 +113,8 @@ Options: --max-line-length=n set the maximum line length, (default $max_line_length) if exceeded, warn on patches requires --strict for use with --file - --min-conf-desc-length=n set the min description length, if shorter, warn + --min-conf-desc-length=n set the minimum description length for config symbols + in lines, if shorter, warn (default $min_conf_desc_length) --tab-size=n set the number of spaces for tab (default $tabsize) --root=PATH PATH to the kernel tree root --no-summary suppress the per-file summary @@ -3645,7 +3646,7 @@ sub process { $help_length < $min_conf_desc_length) { my $stat_real = get_stat_real($linenr, $ln - 1); WARN("CONFIG_DESCRIPTION", - "please write a help paragraph that fully describes the config symbol\n" . "$here\n$stat_real\n"); + "please write a help paragraph that fully describes the config symbol with at least $min_conf_desc_length lines\n" . "$here\n$stat_real\n"); } } -- cgit v1.2.3 From a926d15a799acc6935820340b5a1428754f8ab45 Mon Sep 17 00:00:00 2001 From: Steven Rostedt <rostedt@goodmis.org> Date: Fri, 7 Mar 2025 10:39:41 -0500 Subject: scripts/tracing: Remove scripts/tracing/draw_functrace.py The draw_functrace.py hasn't worked in years. There's better ways to accomplish the same thing (via libtracefs). Remove it. Link: https://lore.kernel.org/linux-trace-kernel/20250210-debuginfo-v1-1-368feb58292a@purestorage.com/ Cc: Masami Hiramatsu <mhiramat@kernel.org> Cc: Mark Rutland <mark.rutland@arm.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Cc: Uday Shankar <ushankar@purestorage.com> Cc: Masahiro Yamada <masahiroy@kernel.org> Link: https://lore.kernel.org/20250307103941.070654e7@gandalf.local.home Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org> --- scripts/tracing/draw_functrace.py | 129 -------------------------------------- 1 file changed, 129 deletions(-) delete mode 100755 scripts/tracing/draw_functrace.py (limited to 'scripts') diff --git a/scripts/tracing/draw_functrace.py b/scripts/tracing/draw_functrace.py deleted file mode 100755 index 42fa87300941..000000000000 --- a/scripts/tracing/draw_functrace.py +++ /dev/null @@ -1,129 +0,0 @@ -#!/usr/bin/env python -# SPDX-License-Identifier: GPL-2.0-only - -""" -Copyright 2008 (c) Frederic Weisbecker <fweisbec@gmail.com> - -This script parses a trace provided by the function tracer in -kernel/trace/trace_functions.c -The resulted trace is processed into a tree to produce a more human -view of the call stack by drawing textual but hierarchical tree of -calls. Only the functions's names and the call time are provided. - -Usage: - Be sure that you have CONFIG_FUNCTION_TRACER - # mount -t tracefs nodev /sys/kernel/tracing - # echo function > /sys/kernel/tracing/current_tracer - $ cat /sys/kernel/tracing/trace_pipe > ~/raw_trace_func - Wait some times but not too much, the script is a bit slow. - Break the pipe (Ctrl + Z) - $ scripts/tracing/draw_functrace.py < ~/raw_trace_func > draw_functrace - Then you have your drawn trace in draw_functrace -""" - - -import sys, re - -class CallTree: - """ This class provides a tree representation of the functions - call stack. If a function has no parent in the kernel (interrupt, - syscall, kernel thread...) then it is attached to a virtual parent - called ROOT. - """ - ROOT = None - - def __init__(self, func, time = None, parent = None): - self._func = func - self._time = time - if parent is None: - self._parent = CallTree.ROOT - else: - self._parent = parent - self._children = [] - - def calls(self, func, calltime): - """ If a function calls another one, call this method to insert it - into the tree at the appropriate place. - @return: A reference to the newly created child node. - """ - child = CallTree(func, calltime, self) - self._children.append(child) - return child - - def getParent(self, func): - """ Retrieve the last parent of the current node that - has the name given by func. If this function is not - on a parent, then create it as new child of root - @return: A reference to the parent. - """ - tree = self - while tree != CallTree.ROOT and tree._func != func: - tree = tree._parent - if tree == CallTree.ROOT: - child = CallTree.ROOT.calls(func, None) - return child - return tree - - def __repr__(self): - return self.__toString("", True) - - def __toString(self, branch, lastChild): - if self._time is not None: - s = "%s----%s (%s)\n" % (branch, self._func, self._time) - else: - s = "%s----%s\n" % (branch, self._func) - - i = 0 - if lastChild: - branch = branch[:-1] + " " - while i < len(self._children): - if i != len(self._children) - 1: - s += "%s" % self._children[i].__toString(branch +\ - " |", False) - else: - s += "%s" % self._children[i].__toString(branch +\ - " |", True) - i += 1 - return s - -class BrokenLineException(Exception): - """If the last line is not complete because of the pipe breakage, - we want to stop the processing and ignore this line. - """ - pass - -class CommentLineException(Exception): - """ If the line is a comment (as in the beginning of the trace file), - just ignore it. - """ - pass - - -def parseLine(line): - line = line.strip() - if line.startswith("#"): - raise CommentLineException - m = re.match("[^]]+?\\] +([a-z.]+) +([0-9.]+): (\\w+) <-(\\w+)", line) - if m is None: - raise BrokenLineException - return (m.group(2), m.group(3), m.group(4)) - - -def main(): - CallTree.ROOT = CallTree("Root (Nowhere)", None, None) - tree = CallTree.ROOT - - for line in sys.stdin: - try: - calltime, callee, caller = parseLine(line) - except BrokenLineException: - break - except CommentLineException: - continue - tree = tree.getParent(caller) - tree = tree.calls(callee, calltime) - - print(CallTree.ROOT) - -if __name__ == "__main__": - main() -- cgit v1.2.3 From 7e752910b8acd1527af34b51851998feb33cc028 Mon Sep 17 00:00:00 2001 From: Masahiro Yamada <masahiroy@kernel.org> Date: Wed, 12 Mar 2025 04:01:33 +0900 Subject: kbuild: deb-pkg: fix versioning for -rc releases The version number with -rc should be considered older than the final release. For example, 6.14-rc1 should be older than 6.14, but to handle this correctly (just like Debian kernel), "-rc" must be replace with "~rc". $ dpkg --compare-versions 6.14-rc1 lt 6.14 $ echo $? 1 $ dpkg --compare-versions 6.14~rc1 lt 6.14 $ echo $? 0 Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Reviewed-by: Nathan Chancellor <nathan@kernel.org> --- scripts/package/mkdebian | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) (limited to 'scripts') diff --git a/scripts/package/mkdebian b/scripts/package/mkdebian index 0178000197fe..07ca3528e8ea 100755 --- a/scripts/package/mkdebian +++ b/scripts/package/mkdebian @@ -161,7 +161,9 @@ version=$KERNELRELEASE if [ "${KDEB_PKGVERSION:+set}" ]; then packageversion=$KDEB_PKGVERSION else - packageversion=$(${srctree}/scripts/setlocalversion --no-local ${srctree})-$($srctree/scripts/build-version) + upstream_version=$("${srctree}/scripts/setlocalversion" --no-local "${srctree}" | sed 's/-\(rc[1-9]\)/~\1/') + debian_revision=$("${srctree}/scripts/build-version") + packageversion=${upstream_version}-${debian_revision} fi sourcename=${KDEB_SOURCENAME:-linux-upstream} -- cgit v1.2.3 From 1c3107ec73921088e55b7ff35861b9303962fd34 Mon Sep 17 00:00:00 2001 From: Masahiro Yamada <masahiroy@kernel.org> Date: Wed, 12 Mar 2025 04:02:24 +0900 Subject: kbuild: deb-pkg: remove "version" variable in mkdebian ${version} and ${KERNELRELEASE} are the same. Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Reviewed-by: Nathan Chancellor <nathan@kernel.org> --- scripts/package/mkdebian | 17 ++++++++--------- 1 file changed, 8 insertions(+), 9 deletions(-) (limited to 'scripts') diff --git a/scripts/package/mkdebian b/scripts/package/mkdebian index 07ca3528e8ea..744ddba01d93 100755 --- a/scripts/package/mkdebian +++ b/scripts/package/mkdebian @@ -157,7 +157,6 @@ while [ $# -gt 0 ]; do done # Some variables and settings used throughout the script -version=$KERNELRELEASE if [ "${KDEB_PKGVERSION:+set}" ]; then packageversion=$KDEB_PKGVERSION else @@ -216,11 +215,11 @@ Build-Depends-Arch: bc, bison, flex, python3:native, rsync Homepage: https://www.kernel.org/ -Package: $packagename-$version +Package: $packagename-${KERNELRELEASE} Architecture: $debarch -Description: Linux kernel, version $version +Description: Linux kernel, version ${KERNELRELEASE} This package contains the Linux kernel, modules and corresponding other - files, version: $version. + files, version: ${KERNELRELEASE}. EOF if [ "${SRCARCH}" != um ]; then @@ -239,11 +238,11 @@ EOF if is_enabled CONFIG_MODULES; then cat <<EOF >> debian/control -Package: linux-headers-$version +Package: linux-headers-${KERNELRELEASE} Architecture: $debarch Build-Profiles: <!pkg.${sourcename}.nokernelheaders> -Description: Linux kernel headers for $version on $debarch - This package provides kernel header files for $version on $debarch +Description: Linux kernel headers for ${KERNELRELEASE} on $debarch + This package provides kernel header files for ${KERNELRELEASE} on $debarch . This is useful for people who need to build external modules EOF @@ -253,11 +252,11 @@ fi if is_enabled CONFIG_DEBUG_INFO; then cat <<EOF >> debian/control -Package: linux-image-$version-dbg +Package: linux-image-${KERNELRELEASE}-dbg Section: debug Architecture: $debarch Build-Profiles: <!pkg.${sourcename}.nokerneldbg> -Description: Linux kernel debugging symbols for $version +Description: Linux kernel debugging symbols for ${KERNELRELEASE} This package will come in handy if you need to debug the kernel. It provides all the necessary debug symbols for the kernel and its modules. EOF -- cgit v1.2.3 From a7a05b1b2739b94870b499818986b82974839fe0 Mon Sep 17 00:00:00 2001 From: Masahiro Yamada <masahiroy@kernel.org> Date: Fri, 14 Mar 2025 18:53:35 +0900 Subject: kbuild: deb-pkg: add comment about future removal of KDEB_COMPRESS 'man dpkg-deb' describes as follows: DPKG_DEB_COMPRESSOR_TYPE Sets the compressor type to use (since dpkg 1.21.10). The -Z option overrides this value. When commit 1a7f0a34ea7d ("builddeb: allow selection of .deb compressor") was applied, dpkg-deb did not support this environment variable. Later, dpkg commit c10aeffc6d71 ("dpkg-deb: Add support for DPKG_DEB_COMPRESSOR_TYPE/LEVEL") introduced support for DPKG_DEB_COMPRESSOR_TYPE, which provides the same functionality as KDEB_COMPRESS. KDEB_COMPRESS is still useful for users of older dpkg versions, but I would like to remove this redundant functionality in the future. This commit adds comments to notify users of the planned removal and to encourage migration to DPKG_DEB_COMPRESSOR_TYPE where possible. Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> --- lib/Kconfig.debug | 6 +++--- scripts/package/debian/rules | 4 ++++ 2 files changed, 7 insertions(+), 3 deletions(-) (limited to 'scripts') diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug index 17ccd913975d..be9f5af4c05c 100644 --- a/lib/Kconfig.debug +++ b/lib/Kconfig.debug @@ -335,12 +335,12 @@ config DEBUG_INFO_COMPRESSED_ZLIB Compress the debug information using zlib. Requires GCC 5.0+ or Clang 5.0+, binutils 2.26+, and zlib. - Users of dpkg-deb via scripts/package/builddeb may find an increase in + Users of dpkg-deb via debian/rules may find an increase in size of their debug .deb packages with this config set, due to the debug info being compressed with zlib, then the object files being recompressed with a different compression scheme. But this is still - preferable to setting $KDEB_COMPRESS to "none" which would be even - larger. + preferable to setting KDEB_COMPRESS or DPKG_DEB_COMPRESSOR_TYPE to + "none" which would be even larger. config DEBUG_INFO_COMPRESSED_ZSTD bool "Compress debugging information with zstd" diff --git a/scripts/package/debian/rules b/scripts/package/debian/rules index ca07243bd5cd..33bfd00974b3 100755 --- a/scripts/package/debian/rules +++ b/scripts/package/debian/rules @@ -41,6 +41,10 @@ package = $($(@:binary-%=%-package)) # which package is being processed in the build log. DH_OPTIONS = -p$(package) +# Note: future removal of KDEB_COMPRESS +# dpkg-deb >= 1.21.10 supports the DPKG_DEB_COMPRESSOR_TYPE environment +# variable, which provides the same functionality as KDEB_COMPRESS. The +# KDEB_COMPRESS variable will be removed in the future. define binary $(Q)dh_testdir $(DH_OPTIONS) $(Q)dh_testroot $(DH_OPTIONS) -- cgit v1.2.3 From 6c6c1fc09de35f409f6971cb9e881103afe5dbe0 Mon Sep 17 00:00:00 2001 From: Jeff Johnson <jeff.johnson@oss.qualcomm.com> Date: Tue, 11 Mar 2025 12:49:02 -0700 Subject: modpost: require a MODULE_DESCRIPTION() Since commit 1fffe7a34c89 ("script: modpost: emit a warning when the description is missing"), a module without a MODULE_DESCRIPTION() has resulted in a warning with make W=1. Since that time, all known instances of this issue have been fixed. Therefore, now make it an error if a MODULE_DESCRIPTION() is not present. Signed-off-by: Jeff Johnson <jeff.johnson@oss.qualcomm.com> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> --- scripts/mod/modpost.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) (limited to 'scripts') diff --git a/scripts/mod/modpost.c b/scripts/mod/modpost.c index 7f4c72eca72c..92627e8d0e16 100644 --- a/scripts/mod/modpost.c +++ b/scripts/mod/modpost.c @@ -1602,8 +1602,8 @@ static void read_symbols(const char *modname) namespace); } - if (extra_warn && !get_modinfo(&info, "description")) - warn("missing MODULE_DESCRIPTION() in %s\n", modname); + if (!get_modinfo(&info, "description")) + error("missing MODULE_DESCRIPTION() in %s\n", modname); } for (sym = info.symtab_start; sym < info.symtab_stop; sym++) { -- cgit v1.2.3 From 62604063621fb075c7966286bdddcb057d883fa8 Mon Sep 17 00:00:00 2001 From: Alexandru Gagniuc <alexandru.gagniuc@hp.com> Date: Fri, 14 Mar 2025 13:10:53 +0000 Subject: kbuild: deb-pkg: don't set KBUILD_BUILD_VERSION unconditionally In ThinPro, we use the convention <upstream_ver>+hp<patchlevel> for the kernel package. This does not have a dash in the name or version. This is built by editing ".version" before a build, and setting EXTRAVERSION="+hp" and KDEB_PKGVERSION make variables: echo 68 > .version make -j<n> EXTRAVERSION="+hp" bindeb-pkg KDEB_PKGVERSION=6.12.2+hp69 .deb name: linux-image-6.12.2+hp_6.12.2+hp69_amd64.deb Since commit 7d4f07d5cb71 ("kbuild: deb-pkg: squash scripts/package/deb-build-option to debian/rules"), this no longer works. The deb build logic changed, even though, the commit message implies that the logic should be unmodified. Before, KBUILD_BUILD_VERSION was not set if the KDEB_PKGVERSION did not contain a dash. After the change KBUILD_BUILD_VERSION is always set to KDEB_PKGVERSION. Since this determines UTS_VERSION, the uname output to look off: (now) uname -a: version 6.12.2+hp ... #6.12.2+hp69 (expected) uname -a: version 6.12.2+hp ... #69 Update the debian/rules logic to restore the original behavior. Fixes: 7d4f07d5cb71 ("kbuild: deb-pkg: squash scripts/package/deb-build-option to debian/rules") Signed-off-by: Alexandru Gagniuc <alexandru.gagniuc@hp.com> Reviewed-by: Nicolas Schier <nicolas@fjasle.eu> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> --- scripts/package/debian/rules | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) (limited to 'scripts') diff --git a/scripts/package/debian/rules b/scripts/package/debian/rules index 33bfd00974b3..a417a7f8bbc1 100755 --- a/scripts/package/debian/rules +++ b/scripts/package/debian/rules @@ -21,9 +21,11 @@ ifeq ($(origin KBUILD_VERBOSE),undefined) endif endif -revision = $(lastword $(subst -, ,$(shell dpkg-parsechangelog -S Version))) +revision = $(shell dpkg-parsechangelog -S Version | sed -n 's/.*-//p') CROSS_COMPILE ?= $(filter-out $(DEB_BUILD_GNU_TYPE)-, $(DEB_HOST_GNU_TYPE)-) -make-opts = ARCH=$(ARCH) KERNELRELEASE=$(KERNELRELEASE) KBUILD_BUILD_VERSION=$(revision) $(addprefix CROSS_COMPILE=,$(CROSS_COMPILE)) +make-opts = ARCH=$(ARCH) KERNELRELEASE=$(KERNELRELEASE) \ + $(addprefix KBUILD_BUILD_VERSION=,$(revision)) \ + $(addprefix CROSS_COMPILE=,$(CROSS_COMPILE)) binary-targets := $(addprefix binary-, image image-dbg headers libc-dev) -- cgit v1.2.3 From 8bdd53e066012bed431667393676d1b5e8cce153 Mon Sep 17 00:00:00 2001 From: Masahiro Yamada <masahiroy@kernel.org> Date: Sun, 16 Mar 2025 00:15:20 +0900 Subject: kbuild: pacman-pkg: hardcode module installation path MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 'make pacman-pkg' for architectures with device tree support (i.e., arm, arm64, etc.) shows logs like follows: Installing dtbs... INSTALL /home/masahiro/linux/pacman/linux-upstream/pkg/linux-upstream/usr//lib/modules/6.14.0-rc6+/dtb/actions/s700-cubieboard7.dtb INSTALL /home/masahiro/linux/pacman/linux-upstream/pkg/linux-upstream/usr//lib/modules/6.14.0-rc6+/dtb/actions/s900-bubblegum-96.dtb INSTALL /home/masahiro/linux/pacman/linux-upstream/pkg/linux-upstream/usr//lib/modules/6.14.0-rc6+/dtb/airoha/en7581-evb.dtb ... The double slashes ('//') between 'usr' and 'lib' are somewhat ugly. Let's hardcode the module installation path because the package contents should remain unaffected even if ${MODLIB} is overridden. Please note that scripts/packages/{builddeb,kernel.spec} also hardcode the module installation path. With this change, the log will look better, as follows: Installing dtbs... INSTALL /home/masahiro/linux/pacman/linux-upstream/pkg/linux-upstream/usr/lib/modules/6.14.0-rc6+/dtb/actions/s700-cubieboard7.dtb INSTALL /home/masahiro/linux/pacman/linux-upstream/pkg/linux-upstream/usr/lib/modules/6.14.0-rc6+/dtb/actions/s900-bubblegum-96.dtb INSTALL /home/masahiro/linux/pacman/linux-upstream/pkg/linux-upstream/usr/lib/modules/6.14.0-rc6+/dtb/airoha/en7581-evb.dtb ... Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Acked-by: Thomas Weißschuh <linux@weissschuh.net> Reviewed-by: Nathan Chancellor <nathan@kernel.org> --- scripts/package/PKGBUILD | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) (limited to 'scripts') diff --git a/scripts/package/PKGBUILD b/scripts/package/PKGBUILD index 0cf3a55b05e1..452374d63c24 100644 --- a/scripts/package/PKGBUILD +++ b/scripts/package/PKGBUILD @@ -53,7 +53,7 @@ build() { _package() { pkgdesc="The ${pkgdesc} kernel and modules" - local modulesdir="${pkgdir}/usr/${MODLIB}" + local modulesdir="${pkgdir}/usr/lib/modules/${KERNELRELEASE}" _prologue @@ -81,7 +81,7 @@ _package() { _package-headers() { pkgdesc="Headers and scripts for building modules for the ${pkgdesc} kernel" - local builddir="${pkgdir}/usr/${MODLIB}/build" + local builddir="${pkgdir}/usr/lib/modules/${KERNELRELEASE}/build" _prologue @@ -114,7 +114,7 @@ _package-debug(){ pkgdesc="Non-stripped vmlinux file for the ${pkgdesc} kernel" local debugdir="${pkgdir}/usr/src/debug/${pkgbase}" - local builddir="${pkgdir}/usr/${MODLIB}/build" + local builddir="${pkgdir}/usr/lib/modules/${KERNELRELEASE}/build" _prologue -- cgit v1.2.3 From e1dfaa33fd2d8d7141f2602c4dfa5540403d3c14 Mon Sep 17 00:00:00 2001 From: Antonio Hickey <contact@antoniohickey.com> Date: Wed, 19 Mar 2025 22:07:20 -0400 Subject: rust: enable `raw_ref_op` feature Since Rust 1.82.0 the `raw_ref_op` feature is stable [1]. By enabling this feature we can use `&raw const place` and `&raw mut place` instead of using `addr_of!(place)` and `addr_of_mut!(place)` macros. Allowing us to reduce macro complexity, and improve consistency with existing reference syntax as `&raw const`, `&raw mut` are similar to `&`, `&mut` making it fit more naturally with other existing code. Suggested-by: Benno Lossin <benno.lossin@proton.me> Link: https://github.com/Rust-for-Linux/linux/issues/1148 Link: https://blog.rust-lang.org/2024/10/17/Rust-1.82.0.html#native-syntax-for-creating-a-raw-pointer [1] Signed-off-by: Antonio Hickey <contact@antoniohickey.com> Reviewed-by: Benno Lossin <benno.lossin@proton.me> Reviewed-by: Andreas Hindborg <a.hindborg@kernel.org> Reviewed-by: Tamir Duberstein <tamird@gmail.com> Link: https://lore.kernel.org/r/20250320020740.1631171-2-contact@antoniohickey.com [ Removed dashed line change as discussed. Added Link to the explanation of the feature in the Rust 1.82.0 release blog post. - Miguel ] Signed-off-by: Miguel Ojeda <ojeda@kernel.org> --- rust/kernel/lib.rs | 2 ++ scripts/Makefile.build | 2 +- 2 files changed, 3 insertions(+), 1 deletion(-) (limited to 'scripts') diff --git a/rust/kernel/lib.rs b/rust/kernel/lib.rs index 001374a96d66..ba0f3b0297b2 100644 --- a/rust/kernel/lib.rs +++ b/rust/kernel/lib.rs @@ -19,6 +19,8 @@ #![cfg_attr(not(CONFIG_RUSTC_HAS_COERCE_POINTEE), feature(unsize))] #![feature(inline_const)] #![feature(lint_reasons)] +// Stable in Rust 1.82 +#![feature(raw_ref_op)] // Stable in Rust 1.83 #![feature(const_maybe_uninit_as_mut_ptr)] #![feature(const_mut_refs)] diff --git a/scripts/Makefile.build b/scripts/Makefile.build index 08b6380933f5..56be83024851 100644 --- a/scripts/Makefile.build +++ b/scripts/Makefile.build @@ -226,7 +226,7 @@ $(obj)/%.lst: $(obj)/%.c FORCE # Compile Rust sources (.rs) # --------------------------------------------------------------------------- -rust_allowed_features := asm_const,asm_goto,arbitrary_self_types,lint_reasons +rust_allowed_features := asm_const,asm_goto,arbitrary_self_types,lint_reasons,raw_ref_op # `--out-dir` is required to avoid temporaries being created by `rustc` in the # current working directory, which may be not accessible in the out-of-tree -- cgit v1.2.3 From 4759670bc3e670069c055c2b33174813099fea4f Mon Sep 17 00:00:00 2001 From: Josh Poimboeuf <jpoimboe@kernel.org> Date: Mon, 24 Mar 2025 14:55:55 -0700 Subject: objtool: Fix CONFIG_OBJTOOL_WERROR for vmlinux.o With (!X86_KERNEL_IBT && !LTO_CLANG && NOINSTR_VALIDATION), objtool runs on both translation units and vmlinux.o. With CONFIG_OBJTOOL_WERROR, the TUs get --Werror but vmlinux.o doesn't. Fix that. Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org> Signed-off-by: Ingo Molnar <mingo@kernel.org> Cc: Linus Torvalds <torvalds@linux-foundation.org> Link: https://lore.kernel.org/r/4f71ab9b947ffc47b6a87dd3b9aff4bb32b36d0a.1742852846.git.jpoimboe@kernel.org --- scripts/Makefile.vmlinux_o | 14 +++++++++++--- 1 file changed, 11 insertions(+), 3 deletions(-) (limited to 'scripts') diff --git a/scripts/Makefile.vmlinux_o b/scripts/Makefile.vmlinux_o index 0b6e2ebf60dc..f476f5605029 100644 --- a/scripts/Makefile.vmlinux_o +++ b/scripts/Makefile.vmlinux_o @@ -30,12 +30,20 @@ endif # objtool for vmlinux.o # --------------------------------------------------------------------------- # -# For LTO and IBT, objtool doesn't run on individual translation units. -# Run everything on vmlinux instead. +# For delay-objtool (IBT or LTO), objtool doesn't run on individual translation +# units. Instead it runs on vmlinux.o. +# +# For !delay-objtool + CONFIG_NOINSTR_VALIDATION, it runs on both translation +# units and vmlinux.o, with the latter only used for noinstr/unret validation. objtool-enabled := $(or $(delay-objtool),$(CONFIG_NOINSTR_VALIDATION)) -vmlinux-objtool-args-$(delay-objtool) += $(objtool-args-y) +ifeq ($(delay-objtool),y) +vmlinux-objtool-args-y += $(objtool-args-y) +else +vmlinux-objtool-args-$(CONFIG_OBJTOOL_WERROR) += --Werror +endif + vmlinux-objtool-args-$(CONFIG_GCOV_KERNEL) += --no-unreachable vmlinux-objtool-args-$(CONFIG_NOINSTR_VALIDATION) += --noinstr \ $(if $(or $(CONFIG_MITIGATION_UNRET_ENTRY),$(CONFIG_MITIGATION_SRSO)), --unret) -- cgit v1.2.3 From d39f82a058c0269392a797cb04f2a6064e5dcab6 Mon Sep 17 00:00:00 2001 From: Josh Poimboeuf <jpoimboe@kernel.org> Date: Mon, 24 Mar 2025 14:56:00 -0700 Subject: objtool: Reduce CONFIG_OBJTOOL_WERROR verbosity Remove the following from CONFIG_OBJTOOL_WERROR: * backtrace * "upgraded warnings to errors" message * cmdline args This makes the default output less cluttered and makes it easier to spot the actual warnings. Note the above options are still are available with --verbose or OBJTOOL_VERBOSE=1. Also, do the cmdline arg printing on all warnings, regardless of werror. Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org> Signed-off-by: Ingo Molnar <mingo@kernel.org> Cc: Linus Torvalds <torvalds@linux-foundation.org> Link: https://lore.kernel.org/r/d61df69f64b396fa6b2a1335588aad7a34ea9e71.1742852846.git.jpoimboe@kernel.org --- scripts/Makefile.lib | 2 +- tools/objtool/builtin-check.c | 113 ++++++++++++++++---------------- tools/objtool/check.c | 23 ++++--- tools/objtool/include/objtool/builtin.h | 6 +- 4 files changed, 73 insertions(+), 71 deletions(-) (limited to 'scripts') diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib index 57620b439a1f..b93597420daf 100644 --- a/scripts/Makefile.lib +++ b/scripts/Makefile.lib @@ -277,7 +277,7 @@ objtool-args-$(CONFIG_HAVE_STATIC_CALL_INLINE) += --static-call objtool-args-$(CONFIG_HAVE_UACCESS_VALIDATION) += --uaccess objtool-args-$(CONFIG_GCOV_KERNEL) += --no-unreachable objtool-args-$(CONFIG_PREFIX_SYMBOLS) += --prefix=$(CONFIG_FUNCTION_PADDING_BYTES) -objtool-args-$(CONFIG_OBJTOOL_WERROR) += --Werror --backtrace +objtool-args-$(CONFIG_OBJTOOL_WERROR) += --Werror objtool-args = $(objtool-args-y) \ $(if $(delay-objtool), --link) \ diff --git a/tools/objtool/builtin-check.c b/tools/objtool/builtin-check.c index c973a751fb7d..2bdff910430e 100644 --- a/tools/objtool/builtin-check.c +++ b/tools/objtool/builtin-check.c @@ -15,8 +15,11 @@ #include <objtool/objtool.h> #include <objtool/warn.h> -const char *objname; +#define ORIG_SUFFIX ".orig" +int orig_argc; +static char **orig_argv; +const char *objname; struct opts opts; static const char * const check_usage[] = { @@ -224,39 +227,73 @@ static int copy_file(const char *src, const char *dst) return 0; } -static char **save_argv(int argc, const char **argv) +static void save_argv(int argc, const char **argv) { - char **orig_argv; - orig_argv = calloc(argc, sizeof(char *)); if (!orig_argv) { WARN_GLIBC("calloc"); - return NULL; + exit(1); } for (int i = 0; i < argc; i++) { orig_argv[i] = strdup(argv[i]); if (!orig_argv[i]) { WARN_GLIBC("strdup(%s)", orig_argv[i]); - return NULL; + exit(1); } }; - - return orig_argv; } -#define ORIG_SUFFIX ".orig" +void print_args(void) +{ + char *backup = NULL; + + if (opts.output || opts.dryrun) + goto print; + + /* + * Make a backup before kbuild deletes the file so the error + * can be recreated without recompiling or relinking. + */ + backup = malloc(strlen(objname) + strlen(ORIG_SUFFIX) + 1); + if (!backup) { + WARN_GLIBC("malloc"); + goto print; + } + + strcpy(backup, objname); + strcat(backup, ORIG_SUFFIX); + if (copy_file(objname, backup)) { + backup = NULL; + goto print; + } + +print: + /* + * Print the cmdline args to make it easier to recreate. If '--output' + * wasn't used, add it to the printed args with the backup as input. + */ + fprintf(stderr, "%s", orig_argv[0]); + + for (int i = 1; i < orig_argc; i++) { + char *arg = orig_argv[i]; + + if (backup && !strcmp(arg, objname)) + fprintf(stderr, " %s -o %s", backup, objname); + else + fprintf(stderr, " %s", arg); + } + + fprintf(stderr, "\n"); +} int objtool_run(int argc, const char **argv) { struct objtool_file *file; - char *backup = NULL; - char **orig_argv; int ret = 0; - orig_argv = save_argv(argc, argv); - if (!orig_argv) - return 1; + orig_argc = argc; + save_argv(argc, argv); cmd_parse_options(argc, argv, check_usage); @@ -279,59 +316,19 @@ int objtool_run(int argc, const char **argv) file = objtool_open_read(objname); if (!file) - goto err; + return 1; if (!opts.link && has_multiple_files(file->elf)) { WARN("Linked object requires --link"); - goto err; + return 1; } ret = check(file); if (ret) - goto err; + return ret; if (!opts.dryrun && file->elf->changed && elf_write(file->elf)) - goto err; - - return 0; - -err: - if (opts.dryrun) - goto err_msg; - - if (opts.output) { - unlink(opts.output); - goto err_msg; - } - - /* - * Make a backup before kbuild deletes the file so the error - * can be recreated without recompiling or relinking. - */ - backup = malloc(strlen(objname) + strlen(ORIG_SUFFIX) + 1); - if (!backup) { - WARN_GLIBC("malloc"); return 1; - } - - strcpy(backup, objname); - strcat(backup, ORIG_SUFFIX); - if (copy_file(objname, backup)) - return 1; - -err_msg: - fprintf(stderr, "%s", orig_argv[0]); - - for (int i = 1; i < argc; i++) { - char *arg = orig_argv[i]; - if (backup && !strcmp(arg, objname)) - fprintf(stderr, " %s -o %s", backup, objname); - else - fprintf(stderr, " %s", arg); - } - - fprintf(stderr, "\n"); - - return 1; + return 0; } diff --git a/tools/objtool/check.c b/tools/objtool/check.c index f4e7ee8e8fb5..ac21f2846ebc 100644 --- a/tools/objtool/check.c +++ b/tools/objtool/check.c @@ -4732,9 +4732,6 @@ int check(struct objtool_file *file) free_insns(file); - if (opts.verbose) - disas_warned_funcs(file); - if (opts.stats) { printf("nr_insns_visited: %ld\n", nr_insns_visited); printf("nr_cfi: %ld\n", nr_cfi); @@ -4743,19 +4740,25 @@ int check(struct objtool_file *file) } out: + if (!ret && !warnings) + return 0; + + if (opts.verbose) { + if (opts.werror && warnings) + WARN("%d warning(s) upgraded to errors", warnings); + print_args(); + disas_warned_funcs(file); + } + /* * CONFIG_OBJTOOL_WERROR upgrades all warnings (and errors) to actual * errors. * - * Note that even "fatal" type errors don't actually return an error - * without CONFIG_OBJTOOL_WERROR. That probably needs improved at some - * point. + * Note that even fatal errors don't yet actually return an error + * without CONFIG_OBJTOOL_WERROR. That will be fixed soon-ish. */ - if (opts.werror && (ret || warnings)) { - if (warnings) - WARN("%d warning(s) upgraded to errors", warnings); + if (opts.werror) return 1; - } return 0; } diff --git a/tools/objtool/include/objtool/builtin.h b/tools/objtool/include/objtool/builtin.h index 0fafd0f7a209..6b08666fa69d 100644 --- a/tools/objtool/include/objtool/builtin.h +++ b/tools/objtool/include/objtool/builtin.h @@ -43,8 +43,10 @@ struct opts { extern struct opts opts; -extern int cmd_parse_options(int argc, const char **argv, const char * const usage[]); +int cmd_parse_options(int argc, const char **argv, const char * const usage[]); -extern int objtool_run(int argc, const char **argv); +int objtool_run(int argc, const char **argv); + +void print_args(void); #endif /* _BUILTIN_H */ -- cgit v1.2.3 From 876a4bce3849b235d752d13ec3180e15a35e52de Mon Sep 17 00:00:00 2001 From: Josh Poimboeuf <jpoimboe@kernel.org> Date: Mon, 24 Mar 2025 14:56:02 -0700 Subject: objtool: Remove --no-unreachable for noinstr-only vmlinux.o runs For (!X86_KERNEL_IBT && !LTO_CLANG && NOINSTR_VALIDATION), objtool runs on both translation units and vmlinux.o. The vmlinux.o run only does noinstr/noret validation. In that case --no-unreachable has no effect. Remove it. Note that for ((X86_KERNEL_IBT || LTO_CLANG) && KCOV), --no-unreachable still gets set in objtool-args-y by scripts/Makefile.lib. Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org> Signed-off-by: Ingo Molnar <mingo@kernel.org> Cc: Linus Torvalds <torvalds@linux-foundation.org> Link: https://lore.kernel.org/r/05414246a0124db2f21b0d071b652aa9043d039d.1742852847.git.jpoimboe@kernel.org --- scripts/Makefile.vmlinux_o | 1 - 1 file changed, 1 deletion(-) (limited to 'scripts') diff --git a/scripts/Makefile.vmlinux_o b/scripts/Makefile.vmlinux_o index f476f5605029..938c7457717e 100644 --- a/scripts/Makefile.vmlinux_o +++ b/scripts/Makefile.vmlinux_o @@ -44,7 +44,6 @@ else vmlinux-objtool-args-$(CONFIG_OBJTOOL_WERROR) += --Werror endif -vmlinux-objtool-args-$(CONFIG_GCOV_KERNEL) += --no-unreachable vmlinux-objtool-args-$(CONFIG_NOINSTR_VALIDATION) += --noinstr \ $(if $(or $(CONFIG_MITIGATION_UNRET_ENTRY),$(CONFIG_MITIGATION_SRSO)), --unret) -- cgit v1.2.3 From ccb8ce526807fcbd4578d6619100d8ec48769ea8 Mon Sep 17 00:00:00 2001 From: Christian Schrrefl <chrisi.schrefl@gmail.com> Date: Fri, 31 Jan 2025 00:03:45 +0100 Subject: ARM: 9441/1: rust: Enable Rust support for ARMv7 This commit allows building ARMv7 kernels with Rust support. The rust core library expects some __eabi_... functions that are not implemented in the kernel. Those functions are some float operations and __aeabi_uldivmod. For now those are implemented with define_panicking_intrinsics!. This is based on the code by Sven Van Asbroeck from the original rust branch and inspired by the AArch version by Jamie Cunliffe. I have tested the rust samples and a custom simple MMIO module on hardware (De1SoC FPGA + Arm A9 CPU). Tested-by: Rudraksha Gupta <guptarud@gmail.com> Reviewed-by: Alice Ryhl <aliceryhl@google.com> Acked-by: Miguel Ojeda <ojeda@kernel.org> Tested-by: Miguel Ojeda <ojeda@kernel.org> Acked-by: Ard Biesheuvel <ardb@kernel.org> Signed-off-by: Christian Schrefl <chrisi.schrefl@gmail.com> Signed-off-by: Russell King (Oracle) <rmk+kernel@armlinux.org.uk> --- Documentation/rust/arch-support.rst | 1 + arch/arm/Kconfig | 1 + arch/arm/Makefile | 1 + rust/Makefile | 8 ++++++++ rust/compiler_builtins.rs | 24 ++++++++++++++++++++++++ scripts/generate_rust_target.rs | 4 +++- 6 files changed, 38 insertions(+), 1 deletion(-) (limited to 'scripts') diff --git a/Documentation/rust/arch-support.rst b/Documentation/rust/arch-support.rst index 54be7ddf3e57..6e6a515d0899 100644 --- a/Documentation/rust/arch-support.rst +++ b/Documentation/rust/arch-support.rst @@ -15,6 +15,7 @@ support corresponds to ``S`` values in the ``MAINTAINERS`` file. ============= ================ ============================================== Architecture Level of support Constraints ============= ================ ============================================== +``arm`` Maintained ARMv7 Little Endian only. ``arm64`` Maintained Little Endian only. ``loongarch`` Maintained \- ``riscv`` Maintained ``riscv64`` and LLVM/Clang only. diff --git a/arch/arm/Kconfig b/arch/arm/Kconfig index 835b5f100e92..202dbd17ad2f 100644 --- a/arch/arm/Kconfig +++ b/arch/arm/Kconfig @@ -133,6 +133,7 @@ config ARM select MMU_GATHER_RCU_TABLE_FREE if SMP && ARM_LPAE select HAVE_REGS_AND_STACK_ACCESS_API select HAVE_RSEQ + select HAVE_RUST if CPU_LITTLE_ENDIAN && CPU_32v7 select HAVE_STACKPROTECTOR select HAVE_SYSCALL_TRACEPOINTS select HAVE_UID16 diff --git a/arch/arm/Makefile b/arch/arm/Makefile index 00ca7886b18e..4808d3ed98e4 100644 --- a/arch/arm/Makefile +++ b/arch/arm/Makefile @@ -150,6 +150,7 @@ endif KBUILD_CPPFLAGS +=$(cpp-y) KBUILD_CFLAGS +=$(CFLAGS_ABI) $(CFLAGS_ISA) $(arch-y) $(tune-y) $(call cc-option,-mshort-load-bytes,$(call cc-option,-malignment-traps,)) -msoft-float -Uarm KBUILD_AFLAGS +=$(CFLAGS_ABI) $(AFLAGS_ISA) -Wa,$(arch-y) $(tune-y) -include asm/unified.h -msoft-float +KBUILD_RUSTFLAGS += --target=arm-unknown-linux-gnueabi CHECKFLAGS += -D__arm__ diff --git a/rust/Makefile b/rust/Makefile index 8fcfd60447bc..5b45f760f462 100644 --- a/rust/Makefile +++ b/rust/Makefile @@ -245,6 +245,7 @@ bindgen_skip_c_flags := -mno-fp-ret-in-387 -mpreferred-stack-boundary=% \ # Derived from `scripts/Makefile.clang`. BINDGEN_TARGET_x86 := x86_64-linux-gnu BINDGEN_TARGET_arm64 := aarch64-linux-gnu +BINDGEN_TARGET_arm := arm-linux-gnueabi BINDGEN_TARGET := $(BINDGEN_TARGET_$(SRCARCH)) # All warnings are inhibited since GCC builds are very experimental, @@ -397,6 +398,13 @@ redirect-intrinsics = \ __muloti4 __multi3 \ __udivmodti4 __udivti3 __umodti3 +ifdef CONFIG_ARM + # Add eabi initrinsics for ARM 32-bit + redirect-intrinsics += \ + __aeabi_fadd __aeabi_fmul __aeabi_fcmpeq __aeabi_fcmple __aeabi_fcmplt __aeabi_fcmpun \ + __aeabi_dadd __aeabi_dmul __aeabi_dcmple __aeabi_dcmplt __aeabi_dcmpun \ + __aeabi_uldivmod +endif ifneq ($(or $(CONFIG_ARM64),$(and $(CONFIG_RISCV),$(CONFIG_64BIT))),) # These intrinsics are defined for ARM64 and RISCV64 redirect-intrinsics += \ diff --git a/rust/compiler_builtins.rs b/rust/compiler_builtins.rs index f14b8d7caf89..dd16c1dc899c 100644 --- a/rust/compiler_builtins.rs +++ b/rust/compiler_builtins.rs @@ -73,5 +73,29 @@ define_panicking_intrinsics!("`u128` should not be used", { __umodti3, }); +#[cfg(target_arch = "arm")] +define_panicking_intrinsics!("`f32` should not be used", { + __aeabi_fadd, + __aeabi_fmul, + __aeabi_fcmpeq, + __aeabi_fcmple, + __aeabi_fcmplt, + __aeabi_fcmpun, +}); + +#[cfg(target_arch = "arm")] +define_panicking_intrinsics!("`f64` should not be used", { + __aeabi_dadd, + __aeabi_dmul, + __aeabi_dcmple, + __aeabi_dcmplt, + __aeabi_dcmpun, +}); + +#[cfg(target_arch = "arm")] +define_panicking_intrinsics!("`u64` division/modulo should not be used", { + __aeabi_uldivmod, +}); + // NOTE: if you are adding a new intrinsic here, you should also add it to // `redirect-intrinsics` in `rust/Makefile`. diff --git a/scripts/generate_rust_target.rs b/scripts/generate_rust_target.rs index 0d00ac3723b5..f8e7fb38bf16 100644 --- a/scripts/generate_rust_target.rs +++ b/scripts/generate_rust_target.rs @@ -172,7 +172,9 @@ fn main() { let mut ts = TargetSpec::new(); // `llvm-target`s are taken from `scripts/Makefile.clang`. - if cfg.has("ARM64") { + if cfg.has("ARM") { + panic!("arm uses the builtin rustc target"); + } else if cfg.has("ARM64") { panic!("arm64 uses the builtin rustc aarch64-unknown-none target"); } else if cfg.has("RISCV") { if cfg.has("64BIT") { -- cgit v1.2.3 From 314655d41e650b3d72c60aa80a449e0ab22e2ffd Mon Sep 17 00:00:00 2001 From: "J. Neuschäfer" <j.ne@posteo.net> Date: Sun, 9 Feb 2025 17:55:28 +0100 Subject: scripts/make_fit: Print DT name before libfdt errors MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit This makes it easier to pinpoint where the error happened. For example: FIT arch/powerpc/boot/image.fit Error processing arch/powerpc/boot/dts/microwatt.dtb: Traceback (most recent call last): File "/home/jn/dev/linux/linux-git/build-mpc83xx/../scripts/make_fit.py", line 335, in <module> sys.exit(run_make_fit()) ^^^^^^^^^^^^^^ File "/home/jn/dev/linux/linux-git/build-mpc83xx/../scripts/make_fit.py", line 309, in run_make_fit out_data, count, size = build_fit(args) ^^^^^^^^^^^^^^^ File "/home/jn/dev/linux/linux-git/build-mpc83xx/../scripts/make_fit.py", line 286, in build_fit raise e File "/home/jn/dev/linux/linux-git/build-mpc83xx/../scripts/make_fit.py", line 283, in build_fit (model, compat, files) = process_dtb(fname, args) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/jn/dev/linux/linux-git/build-mpc83xx/../scripts/make_fit.py", line 231, in process_dtb model = fdt.getprop(0, 'model').as_str() ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/libfdt.py", line 448, in getprop pdata = check_err_null(fdt_getprop(self._fdt, nodeoffset, prop_name), ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3/dist-packages/libfdt.py", line 153, in check_err_null raise FdtException(val) libfdt.FdtException: pylibfdt error -1: FDT_ERR_NOTFOUND Signed-off-by: J. Neuschäfer <j.ne@posteo.net> Link: https://lore.kernel.org/r/20250209-makefit-v1-1-bfe6151e8f0a@posteo.net Signed-off-by: Rob Herring (Arm) <robh@kernel.org> --- scripts/make_fit.py | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) (limited to 'scripts') diff --git a/scripts/make_fit.py b/scripts/make_fit.py index 4a1bb2f55861..1683e5ec6e67 100755 --- a/scripts/make_fit.py +++ b/scripts/make_fit.py @@ -279,7 +279,11 @@ def build_fit(args): if os.path.splitext(fname)[1] != '.dtb': continue - (model, compat, files) = process_dtb(fname, args) + try: + (model, compat, files) = process_dtb(fname, args) + except Exception as e: + sys.stderr.write(f"Error processing {fname}:\n") + raise e for fn in files: if fn not in fdts: -- cgit v1.2.3 From c1f4534b213d7be41b5d8b815a42d201a8f2978f Mon Sep 17 00:00:00 2001 From: Andrei Lalaev <andrei.lalaev@anton-paar.com> Date: Mon, 31 Mar 2025 06:17:52 +0000 Subject: scripts: generate_rust_analyzer: fix pin-init name in kernel deps Because of different crate names ("pin-init" and "pin_init") passed to "append_crate" and "append_crate_with_generated", the script fails with "KeyError: 'pin-init'". To overcome the issue, pass the same name to both functions. Signed-off-by: Andrei Lalaev <andrei.lalaev@anton-paar.com> Link: https://lore.kernel.org/r/AM9PR03MB7074692E5D24C288D2BBC801C8AD2@AM9PR03MB7074.eurprd03.prod.outlook.com Fixes: 4e82c87058f4 ("Merge tag 'rust-6.15' of git://git.kernel.org/pub/scm/linux/kernel/git/ojeda/linux") [ Made author match the Signed-off-by one. Added newline. - Miguel ] Signed-off-by: Miguel Ojeda <ojeda@kernel.org> --- scripts/generate_rust_analyzer.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'scripts') diff --git a/scripts/generate_rust_analyzer.py b/scripts/generate_rust_analyzer.py index b0d7dc1e9267..cd41bc906fbd 100755 --- a/scripts/generate_rust_analyzer.py +++ b/scripts/generate_rust_analyzer.py @@ -133,7 +133,7 @@ def generate_crates(srctree, objtree, sysroot_src, external_src, cfgs): append_crate_with_generated("bindings", ["core"]) append_crate_with_generated("uapi", ["core"]) - append_crate_with_generated("kernel", ["core", "macros", "build_error", "pin-init", "bindings", "uapi"]) + append_crate_with_generated("kernel", ["core", "macros", "build_error", "pin_init", "bindings", "uapi"]) def is_root_crate(build_file, target): try: -- cgit v1.2.3 From 55c78035a1a8dfb05f1472018ce2a651701adb7d Mon Sep 17 00:00:00 2001 From: Josh Poimboeuf <jpoimboe@kernel.org> Date: Mon, 31 Mar 2025 21:26:36 -0700 Subject: objtool: Silence more KCOV warnings, part 2 Similar to GCOV, KCOV can leave behind dead code and undefined behavior. Warnings related to those should be ignored. The previous commit: 6b023c784204 ("objtool: Silence more KCOV warnings") ... only did so for CONFIG_CGOV_KERNEL. Also do it for CONFIG_KCOV, but for real this time. Fixes the following warning: vmlinux.o: warning: objtool: synaptics_report_mt_data: unexpected end of section .text.synaptics_report_mt_data Fixes: 6b023c784204 ("objtool: Silence more KCOV warnings") Reported-by: kernel test robot <lkp@intel.com> Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org> Signed-off-by: Ingo Molnar <mingo@kernel.org> Cc: Linus Torvalds <torvalds@linux-foundation.org> Link: https://lore.kernel.org/r/a44ba16e194bcbc52c1cef3d3cd9051a62622723.1743481539.git.jpoimboe@kernel.org Closes: https://lore.kernel.org/oe-kbuild-all/202503282236.UhfRsF3B-lkp@intel.com/ --- scripts/Makefile.lib | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'scripts') diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib index b93597420daf..4d543054f723 100644 --- a/scripts/Makefile.lib +++ b/scripts/Makefile.lib @@ -275,7 +275,7 @@ objtool-args-$(CONFIG_MITIGATION_SLS) += --sls objtool-args-$(CONFIG_STACK_VALIDATION) += --stackval objtool-args-$(CONFIG_HAVE_STATIC_CALL_INLINE) += --static-call objtool-args-$(CONFIG_HAVE_UACCESS_VALIDATION) += --uaccess -objtool-args-$(CONFIG_GCOV_KERNEL) += --no-unreachable +objtool-args-$(or $(CONFIG_GCOV_KERNEL),$(CONFIG_KCOV)) += --no-unreachable objtool-args-$(CONFIG_PREFIX_SYMBOLS) += --prefix=$(CONFIG_FUNCTION_PADDING_BYTES) objtool-args-$(CONFIG_OBJTOOL_WERROR) += --Werror -- cgit v1.2.3 From 023f124a64174c47e18340ded7e2a39b96eb9523 Mon Sep 17 00:00:00 2001 From: Vasily Gorbik <gor@linux.ibm.com> Date: Wed, 2 Apr 2025 03:15:35 +0200 Subject: scripts/sorttable: Fix endianness handling in build-time mcount sort Kernel cross-compilation with BUILDTIME_MCOUNT_SORT produces zeroed mcount values if the build-host endianness does not match the ELF file endianness. The mcount values array is converted from ELF file endianness to build-host endianness during initialization in fill_relocs()/fill_addrs(). Avoid extra conversion of these values during weak-function zeroing; otherwise, they do not match nm-parsed addresses and all mcount values are zeroed out. Cc: Masami Hiramatsu <mhiramat@kernel.org> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Nathan Chancellor <nathan@kernel.org> Cc: Heiko Carstens <hca@linux.ibm.com> Cc: Alexander Gordeev <agordeev@linux.ibm.com> Link: https://lore.kernel.org/patch.git-dca31444b0f1.your-ad-here.call-01743554658-ext-8692@work.hours Fixes: ef378c3b8233 ("scripts/sorttable: Zero out weak functions in mcount_loc table") Reported-by: Ilya Leoshkevich <iii@linux.ibm.com> Reported-by: Ihor Solodrai <ihor.solodrai@linux.dev> Closes: https://lore.kernel.org/all/your-ad-here.call-01743522822-ext-4975@work.hours/ Signed-off-by: Vasily Gorbik <gor@linux.ibm.com> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org> --- scripts/sorttable.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'scripts') diff --git a/scripts/sorttable.c b/scripts/sorttable.c index 7b4b3714b1af..deed676bfe38 100644 --- a/scripts/sorttable.c +++ b/scripts/sorttable.c @@ -857,7 +857,7 @@ static void *sort_mcount_loc(void *arg) for (void *ptr = vals; ptr < vals + size; ptr += long_size) { uint64_t key; - key = long_size == 4 ? r((uint32_t *)ptr) : r8((uint64_t *)ptr); + key = long_size == 4 ? *(uint32_t *)ptr : *(uint64_t *)ptr; if (!find_func(key)) { if (long_size == 4) *(uint32_t *)ptr = 0; -- cgit v1.2.3 From a26fe287eed112b4e21e854f173c8918a6a8596d Mon Sep 17 00:00:00 2001 From: Daniel Gomez <da.gomez@samsung.com> Date: Fri, 28 Mar 2025 14:28:37 +0000 Subject: kconfig: merge_config: use an empty file as initfile The scripts/kconfig/merge_config.sh script requires an existing $INITFILE (or the $1 argument) as a base file for merging Kconfig fragments. However, an empty $INITFILE can serve as an initial starting point, later referenced by the KCONFIG_ALLCONFIG Makefile variable if -m is not used. This variable can point to any configuration file containing preset config symbols (the merged output) as stated in Documentation/kbuild/kconfig.rst. When -m is used $INITFILE will contain just the merge output requiring the user to run make (i.e. KCONFIG_ALLCONFIG=<$INITFILE> make <allnoconfig/alldefconfig> or make olddefconfig). Instead of failing when `$INITFILE` is missing, create an empty file and use it as the starting point for merges. Signed-off-by: Daniel Gomez <da.gomez@samsung.com> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> --- scripts/kconfig/merge_config.sh | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) (limited to 'scripts') diff --git a/scripts/kconfig/merge_config.sh b/scripts/kconfig/merge_config.sh index 0b7952471c18..79c09b378be8 100755 --- a/scripts/kconfig/merge_config.sh +++ b/scripts/kconfig/merge_config.sh @@ -112,8 +112,8 @@ INITFILE=$1 shift; if [ ! -r "$INITFILE" ]; then - echo "The base file '$INITFILE' does not exist. Exit." >&2 - exit 1 + echo "The base file '$INITFILE' does not exist. Creating one..." >&2 + touch "$INITFILE" fi MERGE_LIST=$* -- cgit v1.2.3 From a7c699d090a1f3795c3271c2b399230e182db06e Mon Sep 17 00:00:00 2001 From: Uday Shankar <ushankar@purestorage.com> Date: Mon, 31 Mar 2025 16:46:32 -0600 Subject: kbuild: rpm-pkg: build a debuginfo RPM The rpm-pkg make target currently suffers from a few issues related to debuginfo: 1. debuginfo for things built into the kernel (vmlinux) is not available in any RPM produced by make rpm-pkg. This makes using tools like systemtap against a make rpm-pkg kernel impossible. 2. debug source for the kernel is not available. This means that commands like 'disas /s' in gdb, which display source intermixed with assembly, can only print file names/line numbers which then must be painstakingly resolved to actual source in a separate editor. 3. debuginfo for modules is available, but it remains bundled with the .ko files that contain module code, in the main kernel RPM. This is a waste of space for users who do not need to debug the kernel (i.e. most users). Address all of these issues by additionally building a debuginfo RPM when the kernel configuration allows for it, in line with standard patterns followed by RPM distributors. With these changes: 1. systemtap now works (when these changes are backported to 6.11, since systemtap lags a bit behind in compatibility), as verified by the following simple test script: # stap -e 'probe kernel.function("do_sys_open").call { printf("%s\n", $$parms); }' dfd=0xffffffffffffff9c filename=0x7fe18800b160 flags=0x88800 mode=0x0 ... 2. disas /s works correctly in gdb, with source and disassembly interspersed: # gdb vmlinux --batch -ex 'disas /s blk_op_str' Dump of assembler code for function blk_op_str: block/blk-core.c: 125 { 0xffffffff814c8740 <+0>: endbr64 127 128 if (op < ARRAY_SIZE(blk_op_name) && blk_op_name[op]) 0xffffffff814c8744 <+4>: mov $0xffffffff824a7378,%rax 0xffffffff814c874b <+11>: cmp $0x23,%edi 0xffffffff814c874e <+14>: ja 0xffffffff814c8768 <blk_op_str+40> 0xffffffff814c8750 <+16>: mov %edi,%edi 126 const char *op_str = "UNKNOWN"; 0xffffffff814c8752 <+18>: mov $0xffffffff824a7378,%rdx 127 128 if (op < ARRAY_SIZE(blk_op_name) && blk_op_name[op]) 0xffffffff814c8759 <+25>: mov -0x7dfa0160(,%rdi,8),%rax 126 const char *op_str = "UNKNOWN"; 0xffffffff814c8761 <+33>: test %rax,%rax 0xffffffff814c8764 <+36>: cmove %rdx,%rax 129 op_str = blk_op_name[op]; 130 131 return op_str; 132 } 0xffffffff814c8768 <+40>: jmp 0xffffffff81d01360 <__x86_return_thunk> End of assembler dump. 3. The size of the main kernel package goes down substantially, especially if many modules are built (quite typical). Here is a comparison of installed size of the kernel package (configured with allmodconfig, dwarf4 debuginfo, and module compression turned off) before and after this patch: # rpm -qi kernel-6.13* | grep -E '^(Version|Size)' Version : 6.13.0postpatch+ Size : 1382874089 Version : 6.13.0prepatch+ Size : 17870795887 This is a ~92% size reduction. Note that a debuginfo package can only be produced if the following configs are set: - CONFIG_DEBUG_INFO=y - CONFIG_MODULE_COMPRESS=n - CONFIG_DEBUG_INFO_SPLIT=n The first of these is obvious - we can't produce debuginfo if the build does not generate it. The second two requirements can in principle be removed, but doing so is difficult with the current approach, which uses a generic rpmbuild script find-debuginfo.sh that processes all packaged executables. If we want to remove those requirements the best path forward is likely to add some debuginfo extraction/installation logic to the modules_install target (controllable by flags). That way, it's easier to operate on modules before they're compressed, and the logic can be reused by all packaging targets. Signed-off-by: Uday Shankar <ushankar@purestorage.com> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> --- scripts/package/kernel.spec | 46 +++++++++++++++++++++++++++++++++++++++++++-- scripts/package/mkspec | 10 ++++++++++ 2 files changed, 54 insertions(+), 2 deletions(-) (limited to 'scripts') diff --git a/scripts/package/kernel.spec b/scripts/package/kernel.spec index ac3e5ac01d8a..726f34e11960 100644 --- a/scripts/package/kernel.spec +++ b/scripts/package/kernel.spec @@ -2,8 +2,6 @@ %{!?_arch: %define _arch dummy} %{!?make: %define make make} %define makeflags %{?_smp_mflags} ARCH=%{ARCH} -%define __spec_install_post /usr/lib/rpm/brp-compress || : -%define debug_package %{nil} Name: kernel Summary: The Linux Kernel @@ -46,6 +44,36 @@ This package provides kernel headers and makefiles sufficient to build modules against the %{version} kernel package. %endif +%if %{with_debuginfo} +# list of debuginfo-related options taken from distribution kernel.spec +# files +%undefine _include_minidebuginfo +%undefine _find_debuginfo_dwz_opts +%undefine _unique_build_ids +%undefine _unique_debug_names +%undefine _unique_debug_srcs +%undefine _debugsource_packages +%undefine _debuginfo_subpackages +%global _find_debuginfo_opts -r +%global _missing_build_ids_terminate_build 1 +%global _no_recompute_build_ids 1 +%{debug_package} +%endif +# some (but not all) versions of rpmbuild emit %%debug_package with +# %%install. since we've already emitted it manually, that would cause +# a package redefinition error. ensure that doesn't happen +%define debug_package %{nil} + +# later, we make all modules executable so that find-debuginfo.sh strips +# them up. but they don't actually need to be executable, so remove the +# executable bit, taking care to do it _after_ find-debuginfo.sh has run +%define __spec_install_post \ + %{?__debug_package:%{__debug_install_post}} \ + %{__arch_install_post} \ + %{__os_install_post} \ + find %{buildroot}/lib/modules/%{KERNELRELEASE} -name "*.ko" -type f \\\ + | xargs --no-run-if-empty chmod u-x + %prep %setup -q -n linux cp %{SOURCE1} .config @@ -89,8 +117,22 @@ ln -fns /usr/src/kernels/%{KERNELRELEASE} %{buildroot}/lib/modules/%{KERNELRELEA echo "%exclude /lib/modules/%{KERNELRELEASE}/build" } > %{buildroot}/kernel.list +# make modules executable so that find-debuginfo.sh strips them. this +# will be undone later in %%__spec_install_post +find %{buildroot}/lib/modules/%{KERNELRELEASE} -name "*.ko" -type f \ + | xargs --no-run-if-empty chmod u+x + +%if %{with_debuginfo} +# copying vmlinux directly to the debug directory means it will not get +# stripped (but its source paths will still be collected + fixed up) +mkdir -p %{buildroot}/usr/lib/debug/lib/modules/%{KERNELRELEASE} +cp vmlinux %{buildroot}/usr/lib/debug/lib/modules/%{KERNELRELEASE} +%endif + %clean rm -rf %{buildroot} +rm -f debugfiles.list debuglinks.list debugsourcefiles.list debugsources.list \ + elfbins.list %post if [ -x /usr/bin/kernel-install ]; then diff --git a/scripts/package/mkspec b/scripts/package/mkspec index 4dc1466dfc81..c7375bfc25a9 100755 --- a/scripts/package/mkspec +++ b/scripts/package/mkspec @@ -23,6 +23,16 @@ else echo '%define with_devel 0' fi +# debuginfo package generation uses find-debuginfo.sh under the hood, +# which only works on uncompressed modules that contain debuginfo +if grep -q CONFIG_DEBUG_INFO=y include/config/auto.conf && + (! grep -q CONFIG_MODULE_COMPRESS=y include/config/auto.conf) && + (! grep -q CONFIG_DEBUG_INFO_SPLIT=y include/config/auto.conf); then +echo '%define with_debuginfo %{?_without_debuginfo: 0} %{?!_without_debuginfo: 1}' +else +echo '%define with_debuginfo 0' +fi + cat<<EOF %define ARCH ${ARCH} %define KERNELRELEASE ${KERNELRELEASE} -- cgit v1.2.3 From 6f110a5e4f9977c31ce76fefbfef6fd4eab6bfb7 Mon Sep 17 00:00:00 2001 From: Linus Torvalds <torvalds@linux-foundation.org> Date: Sun, 6 Apr 2025 10:00:04 -0700 Subject: Disable SLUB_TINY for build testing ... and don't error out so hard on missing module descriptions. Before commit 6c6c1fc09de3 ("modpost: require a MODULE_DESCRIPTION()") we used to warn about missing module descriptions, but only when building with extra warnigns (ie 'W=1'). After that commit the warning became an unconditional hard error. And it turns out not all modules have been converted despite the claims to the contrary. As reported by Damian Tometzki, the slub KUnit test didn't have a module description, and apparently nobody ever really noticed. The reason nobody noticed seems to be that the slub KUnit tests get disabled by SLUB_TINY, which also ends up disabling a lot of other code, both in tests and in slub itself. And so anybody doing full build tests didn't actually see this failre. So let's disable SLUB_TINY for build-only tests, since it clearly ends up limiting build coverage. Also turn the missing module descriptions error back into a warning, but let's keep it around for non-'W=1' builds. Reported-by: Damian Tometzki <damian@riscv-rocks.de> Link: https://lore.kernel.org/all/01070196099fd059-e8463438-7b1b-4ec8-816d-173874be9966-000000@eu-central-1.amazonses.com/ Cc: Masahiro Yamada <masahiroy@kernel.org> Cc: Jeff Johnson <jeff.johnson@oss.qualcomm.com> Fixes: 6c6c1fc09de3 ("modpost: require a MODULE_DESCRIPTION()") Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org> --- mm/Kconfig | 2 +- scripts/mod/modpost.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) (limited to 'scripts') diff --git a/mm/Kconfig b/mm/Kconfig index d3fb3762887b..e113f713b493 100644 --- a/mm/Kconfig +++ b/mm/Kconfig @@ -201,7 +201,7 @@ config KVFREE_RCU_BATCHED config SLUB_TINY bool "Configure for minimal memory footprint" - depends on EXPERT + depends on EXPERT && !COMPILE_TEST select SLAB_MERGE_DEFAULT help Configures the slab allocator in a way to achieve minimal memory diff --git a/scripts/mod/modpost.c b/scripts/mod/modpost.c index 92627e8d0e16..be89921d60b6 100644 --- a/scripts/mod/modpost.c +++ b/scripts/mod/modpost.c @@ -1603,7 +1603,7 @@ static void read_symbols(const char *modname) } if (!get_modinfo(&info, "description")) - error("missing MODULE_DESCRIPTION() in %s\n", modname); + warn("missing MODULE_DESCRIPTION() in %s\n", modname); } for (sym = info.symtab_start; sym < info.symtab_stop; sym++) { -- cgit v1.2.3