diff options
author | Andrew Geissler <geissonator@yahoo.com> | 2021-09-25 00:47:35 +0300 |
---|---|---|
committer | Andrew Geissler <geissonator@yahoo.com> | 2021-10-05 22:27:21 +0300 |
commit | 5199d831602da71945df7cef62eb3c01183cf20e (patch) | |
tree | 0e4c75d4ac0f346489cb92fa4ccccf0a0aa56fe1 /poky/meta/lib | |
parent | e9e982486160a1d724bf30f21167d72dfbcb84ce (diff) | |
download | openbmc-5199d831602da71945df7cef62eb3c01183cf20e.tar.xz |
subtree updates
meta-security: 1f18c623e9..de6712a806:
Armin Kuster (8):
cryfs: drop recipe
trousers: set precise BSD license
ibmtpm2tss: set precise BSD license
ibmswtpm2: set precise BSD license
opendnssec: set precise BSD license
checksec: set precise BSD license
isic: set precise BSD license
tpm-quote-tools: Update SRC_URI
Christer Fletcher (1):
dm-verity-img.bbclass: Expose --data-block-size for configuration
Kai Kang (1):
sssd: 2.5.1 -> 2.5.2
meta-raspberrypi: a6fa6b3aec..9eb4879cf4:
Andrew Penner (1):
rpi-cmdline: Support ethernet over USB
Khem Raj (2):
linux-raspberrypi: Update to 5.10.63
raspberrypi-firmware: Update to latest
meta-openembedded: e4a3c66505..cff8331f96:
Armin Kuster (21):
python3-cycler: set precise BSD license
python3-dill: set precise BSD license
python3-ipython-genutils: set precise BSD license
python3-traitlets: set precise BSD license
python3-parallax: set precise BSD license
python3-ipython:set precise BSD license
python3-mpmath: set precise BSD license
python3-sympy: set precise BSD license
python3-sqlparse: set precise BSD license
python3-webencodings: set precise BSD license
python3-pyperclip:set precise BSD license
python3-geojson: set precise BSD license
python3-aenum: set precise BSD license
python3-gnupg: set precise BSD license
python3-kiwisolver: set precise BSD license
python3-jdcal: set precise BSD license
python3-send2trash: set precise BSD license
python3-flask-bootstrap: Update LICENSES
autossh: set precise BSD licenses
jemalloc: set precise BSD license
gpsd-machine-conf: set precise BSD license
Bruce Ashfield (1):
vboxguestdrivers: fix build against 5.14+
Ed Tanous (1):
Boost-url Move to latest version
Khem Raj (57):
gdm: Add polkit to required distro features
python3-lxml: Inherit pkgconfig
python3-icu: Inherit pkgconfig
python3-h5py: Inherit pkgconfig
python3-pyparted: Inherit pkgconfig
python3-systemd: Inherit pkgconfig
rp-pppoe: Add configure cached variable via recipe
site: Remove local site files
postfix: Inherit pkgconfig
emacs: Inherit pkgconfig
libgnt: Inherit pkgconfig
libgnt: Inherit pkgconfig
portaudio-v19: Inherit pkgconfig
sshfs-fuse: Inherit pkgconfig
appstream-glib: Inherit pkgconfig
volume-key: Inherit pkgconfig
kronosnet: Inherit pkgconfig
rrdtool: Inherit pkgconfig
libbytesize: Inherit pkgconfig
dlt-daemon: Inherit pkgconfig
libmypaint: Inherit pkgconfig
libubox: Inherit pkgconfig
xfsprogs: Inherit pkgconfig
pavucontrol: Inherit pkgconfig
blueman: Inherit pkgconfig
mimic: Inherit pkgconfig
libchamplain: Inherit pkgconfig
gst-shark: Inherit pkgconfig
zchunk: Inherit pkgconfig
libvdpau: Inherit pkgconfig
tigervnc: Inherit pkgconfig
mpc: Inherit pkgconfig
avro-c: Inherit pkgconfig
udevil: Inherit pkgconfig
remmina: Inherit pkgconfig
transmission: Inherit pkgconfig
libuvc: Inherit pkgconfig
crda: Inherit pkgconfig
wxwidgets: Inherit pkgconfig
mdbus2: Inherit pkgconfig
firewalld: Inherit pkgconfig
renderdoc: Inherit pkgconfig
fetchmail: Inherit pkgconfig
ncmpc: Inherit pkgconfig
yad: Inherit pkgconfig
mscgen: Inherit pkgconfig
libldb: Inherit pkgconfig
pahole: Inherit missing pkgconfig
gerbera: Inherit pkgconfig
xfce4-datetime-setter: Inherit pkgconfig
libblockdev: Inherit pkgconfig
ntopng: Inherit pkgconfig
mosquitto: Inherit pkgconfig
samba: Inherit pkgconfig
fio: Upgrade to 3.28
rdma-core: Inherit pkgconfig
postfix: Add missing dependency on m4
Marek Vasut (1):
dstat: Add missing python-six runtime dependency
Matteo Croce (1):
pahole: call python via env in the shebang
Pascal Bach (1):
poco: update to 1.11.0
Peter Kjellerstedt (1):
libiio: Make libiio-python3 depend on python3-core
Pierre-Jean Texier (1):
cppzmq: upgrade 4.8.0 -> 4.8.1
Sakib Sajal (3):
bats: source files from correct directory
gd: upgrade 2.3.2 -> 2.3.3
lmdb: replace tag with commit id in SRCREV
Trevor Woerner (2):
vk-gl-cts: allow the user to specify the target
vk-gl-cts: fix soname linking
Yi Zhao (2):
samba: upgrade 4.14.5 -> 4.14.7
net-snmp: remove perllocal.pod when enable packageconfig[perl]
jan (1):
netdata: Fixed the recipe.
wangmy (3):
byacc: upgrade 20200910 -> 20210808
nghttp2: upgrade 1.44.0 -> 1.45.1
apache2: upgrade 2.4.48 -> 2.4.49
zangrc (5):
python3-beautifulsoup4: upgrade 4.9.3 -> 4.10.0
python3-bitarray: upgrade 2.3.3 -> 2.3.4
python3-decorator: upgrade 5.0.9 -> 5.1.0
python3-grpcio-tools: upgrade 1.39.0 -> 1.40.0
python3-grpcio: upgrade 1.39.0 -> 1.40.0
zhengruoqin (5):
python3-openpyxl: upgrade 3.0.7 -> 3.0.8
python3-pandas: upgrade 1.3.2 -> 1.3.3
python3-pulsectl: upgrade 21.5.18 -> 21.9.1
protobuf: upgrade 3.17.3 -> 3.18.0
span-lite: upgrade 0.10.0 -> 0.10.1
poky: 359e1cb62f..06dcace68b:
Alexander Kanavin (13):
lttng: update 2.12 -> 2.13.0
core-image-ptest-all: bump RAM requirement to 4G
bitbake: bitbake: drop old rules for python warnings
bitbake: bitbake: correct the collections vs collections.abc deprecation
bitbake: bitbake: fix regexp deprecation warnings
bitbake: bitbake: do not import imp in layerindexlib
bitbake: bitbake: adjust parser error check for python 3.10 compatibility
bitbake: bitbake: correct deprecation warning in process.py
bitbake: bitbake: enable python warnings at the first opportunity
meta: correct collections vs collections.abc deprecation
wic: keep rootfs_size as integer
cpan-base.bbclass: use raw string for regexp
testimage: symlink the task log and qemu console log to tmp/log/oeqa
Armin Kuster (2):
apr: Security fix for CVE-2021-35940
tar: ignore node-tar CVEs
Bruce Ashfield (11):
linux-yocto/5.13: update to v5.13.13
linux-yocto/5.13: update to v5.13.15
linux-yocto/5.10: update to v5.10.61
linux-yocto/5.10: update to v5.10.63
yocto-bsp/5.10: update to v5.10.63
yocto-bsp/5.13: update to v5.13.15
libc-headers: bump to v5.14
linux-yocto: introduce 5.14 reference kernel
systemtap: update to 4.5-latest
conf/machine: bump qemu preferred versions to 5.14
poky: set default kernel to 5.14
Changqing Li (1):
lttng-ust: fix do_compile error when PACKAGECONFIG examples is enabled
Chanho Park (1):
binutils: inherit pkgconfig to address libdebuginfod depdency
Claudius Heine (1):
rng-tools: add systemd-udev-settle wants to service
Daniel Ammann (1):
bitbake: fetch2/wget: Enable ftps
Daniel Wagenknecht (2):
mirrors.bbclass: provide additional rule for git repo fallbacks
mirrors.bbclass: remove redundant server-specific mirrors
Denys Dmytriyenko (1):
readline: correct pkg-config dependency for termcap
Hsia-Jun(Randy) Li (1):
cross-canadian: make android pass target sys check
Jon Mason (6):
Update mailing list address
README: update mailing list address
dev-manual: update mailing list address
core-image-sato: Fix runqemu error for qemuarmv5
machine/qemuarm*: use virtio graphics
testimage: remove aarch64 xorg exclusion
Joshua Watt (17):
Add SPDX licenses
classes/package: Add extended packaged data
classes/create-spdx: Add class
classes/create-spdx: Change creator
classes/create-spdx: Add SHA1 to index file
classes/create-spdx: Add index to DEPLOYDIR
classes/create-spdx: Add runtime dependency mapping
classes/create-spdx: Add NOASSERTION for unknown debug sources
classes/create-spdx: Fix another creator
classes/create-spdx: Fix up license reporting
classes/create-spdx: Speed up hash calculations
classes/create-spdx: Fix file:// in downloadLocation
classes/create-spdx: Add special exception for Public Domain license
classes/create-spdx: Collect all task dependencies
classes/create-spdx: Skip package processing for native recipes
classes/create-spdx: Comment out placeholder license warning
bitbake: cooker: Allow upstream for local hash equivalence server
Kai Kang (2):
perl: fix CVE-2021-36770
rust-common.bbclass: make sure ccache exist
Kevin Hao (1):
meta-yocto-bsp: Update the default kernel to v5.14
Khem Raj (3):
vim: Add packageconfig for sound notification support
site: Drop caching libIDL_cv_long_long_format
site: Drop ORBit2 relared cached variables
Konrad Weihmann (1):
expat: pull from github releases
Kristian Klausen (3):
systemd: Add homed PACKAGECONFIG
wic: Add extra-space argument
systemd: Add tpm2 PACKAGECONFIG
Mark Hatle (3):
reproducible_build: Remove BUILD_REPRODUCIBLE_BINARIES checking
externalsrc: Work with reproducible_build
tcf-agent: Move to the latest master version
Markus Volk (1):
util-linux: disable raw
Martin Jansa (3):
default-distrovars.inc: Set BBINCLUDELOGS to empty to disable printing failed task output multiple times
bitbake: bitbake.conf: fix vars_from_file() call
qemu-native: add direct dependency on ninja-native and meson-native
Michael Halstead (1):
releases: update to include 3.3.3
Michael Opdenacker (9):
dev-manual: explicit that devpyshell is a task
bitbake: bitbake-user-manual: replace "file name" by "filename"
manuals: replace Freenode by Libera Chat as IRC host
manuals: delete unmaintained history sections
ref-manual: document UPSTREAM_CHECK_COMMITS and UPSTREAM_VERSION_UNKNOWN
ref-manual: remove checkpkg task
ref-manual: improve "devtool check-upgrade-status" details
ref-manual: improve documentation for RECIPE_NO_UPDATE_REASON
ref-manual: update "devtool check-upgrade-status" output
Mingli Yu (6):
coreutils: add pkgconfig for selinux
findutils: add pkgconfig for selinux
tar: add pkgconfig for selinux
multilib.bbclass: add RDEPENDS related check back
insane.bbclass: add FILERDEPENDS related check back
python3: fix multilib qa issue
Peter Bergin (1):
systemd: add packageconfig for wheel-group
Peter Kjellerstedt (2):
common-licenses, licenses.conf: Remove duplicate licenses
create-spdx.bbclass: Search all license directories for licenses
Quentin Schulz (3):
bitbake: doc: bitbake-user-manual-execution: remove mention to long-gone BBHASHDEPS variable
conf/mips: mips16e: prepend override to MACHINEOVERRIDES
bitbake: doc: bitbake-user-manual-fetching: S should be set to WORKDIR/git for git fetcher
Randy MacLeod (1):
tcmode-default: add rust to the default toolchains
Ranjitsinh Rathod (1):
rpm: Handle proper return value to avoid major issues
Richard Purdie (67):
oeqa/runtime/parselogs: Make DVD ata error apply to all qemux86 machines
tcl: Exclude CVE-2021-35331 from checks
xdg-utils: Add fix for CVE-2020-27748
build-appliance-image: Update to master head revision
utils: Drop unused variable staging_install from oe_libinstall
utils: Drop obsolete oe_machinstall function
flex: Add CVE-2019-6293 to exclusions for checks
go: Exclude CVE-2021-29923 from report list
bitbake: runqueue: Avoid deadlock avoidance task graph corruption
bitbake: runqueue: Fix issues with multiconfig deferred task deadlock messages
oeqa/oescripts: Fix after tar recipe changes
pseudo: Update with fcntl and glibc 2.34 fixes
bitbake: persist_data: Drop deprecated/unused function
bitbake: parse_py: Drop deprecated function reference
bitbake: build: Match markup to real function name
bitbake: build: Handle SystemExit in python tasks correctly
bitbake: process: Don't include logs in error message if piping them
bitbake: build: Avoid duplicating logs in verbose mode
bitbake: data_smart: Make ExpansionErrors more readable
bitbake: build: Catch and error upon circular task references
bitbake: data_smart: Improve error display for handled exceptions
bitbake: fetch2: Add recursion guard
bitbake: cookerdata: Improve missing core layer error message
bitbake: cookerdata: Show error for no BBLAYERS in bblayers.conf
bitbake: runqueue: Clean up task stats handling
Revert "default-distrovars.inc: Set BBINCLUDELOGS to empty to disable printing failed task output multiple times"
bitbake.conf: Ensure XZ_THREADS doesn't change sstate checksums
sstate: Avoid problems with recipes using SRCPV when fetching sstate
local.conf.sample: Update sstate mirror entry with new hash equivalence setting
useradd: Ensure preinst data is expanded correctly in pkgdata
package: Fix pkgdata determinism issues
sstate: Ensure SDE is accounted for in package task timestamps
bash: Ensure deterministic build
sstatesig: Allow exclusion of the root directory for do_package
bitbake: bitbake-worker: Improve error handling
bitbake: runqueue/knotty: Improve UI handling of setscene task counting
bitbake: fetch2/git: Avoid races over mirror tarball creation
README: Update email address for Bruce
bitbake: cookerdata: Show a readable error for invalid multiconfig name
bitbake: fetch2/git: Use os.rename instead of mv
bitbake: tests/fetch2: Fix quoting warning
bitbake: data_smart: Don't add None to ExpansionError varlist
bitbake: fetch2/svn: Allow peg-revision functionality to be disabled
vim: Backport fix for CVE-2021-3770
libgcrypt: Upgrade 1.9.3 -> 1.9.4
sqlite3: Exclude CVE-2021-36690 from cve checks
recipes: Add missing pkgconfig inherit
lttng-tools: Add missing DEPENDS on bison-native
cross: Drop unused do_install
pybootchart: Avoid divide by zero
bitbake: tests/fetch2: Use our own git server for dtc test repo
scripts/oe-publish-sdk: Disable git gc to avoid build errors
image/qemu: Add explict depends for qemu-helper addto_recipe_sysroot task
siteinfo/autotools: Ensure task checksums reflect site files
package_ipk/deb/rpm: Drop recursive do_build task dependencies
reproducible_build/package_XXX: Ensure SDE task is in dependency chain
populate_sdk_base/images: Drop use of 'meta' class and hence do_build dependencies
buildtools-tarball/uninative-tarball/meta-ide-support: Drop useless meta class
meta: Drop useless class
staging: Mark deploy an sstate task
sstate: Ensure deploy tasks don't pull in toolchains
sstate: Avoid deploy_source_date_epoch sstate when unneeded
ssate: Cleanup directtasks handling
bitbake: build: Ensure python stdout/stderr is logged correctly
bitbake: build: Make exception printing clearer
bitbake: build: Fix log flushing race
oeqa/selftest: Add tests for bitbake shell/python task output
Robert P. J. Day (16):
dev-manual: pass False to d.getVar() for devpyshell example
ref-manual: add missing "${PN}-src" to default PACKAGES list
dev-manual: small number of minor aesthetic tweaks
dev-manual: various pedantic nitpickery
dev-manual: drop "three" since there are four requirements
ref-manual: update SYSROOT_DIRS_* variable entries
README: update manual list and names, online docs URL
image_types_wic.bbclass: alphabetize list of WICVARS
systemd: '${systemd_unitdir}/system' => '${systemd_system_unitdir}'
ref-manual: render options in monospace to show quotes properly
ref-manual: remove mention of obsolete devtool "--any-recipe" option
ref-manual: correct typo in "classes" section, "${BPN}/{PV}"
ref-manual: add potential of parallelism to defn of "Task"
ref-manual: couple minor tweaks to Chapter 1
dev-manual: emphasize that new layers live outside of poky
dev-manual: update output of "wic list images"
Robert Yang (1):
assimp: Remove it
Ross Burton (40):
lz4: remove redundant BSD license
python3-numpy: remove redundant BSD license
quota: remove BSD license
nfs-utils: set precise BSD license
dtc: set precise BSD license
acpica: set precise BSD license
libevent: set precise BSD license
openssh: remove redundant BSD license
python3-packaging: fix license statement
iputils: set precise BSD license
libx11-compose-data: set precise BSD license
webkitgtk: set precise BSD license
libwpe: set precise BSD license
wpebackend-fdo: set precise BSD license
common-licenses: add missing SPDX licences
dev-manual/common-tasks: sync libxpm fragment with the recipe
lsof: correct LICENSE
selftest/python-async-test: set precise BSD license
lsof: add upstream check
xinetd: correct LICENSE
oeqa/recipeutils: update for license change to python-async-test
libxfont: set precise BSD license
valgrind: set precise BSD license
shadow-sysroot: sync license with shadow
ovmf: set precise BSD license
ppp: set precise BSD license
ffmpeg: update LICENSE
hdparm: set correct license
recipetool/create_buildsys_python: treat BSD as BSD-3-Clause
oeqa/selftest/recipetool: update for license changes
create-spdx: transform license list into a dict for faster lookups
create-spdx: remove redundant test
create-spdx: embed unknown license texts
create-spdx: don't duplicate license texts in each package
create-spdx: handle CLOSED license
ffmpeg: fix LICENSE
avahi: remove obsolete intltool-native dependency
shared-mime-info: use a more concise description
libsoup-2.4: remove obsolete intltool dependency
oeqa/target/ssh: don't assume target_dumper is set
Sakib Sajal (1):
go: upgrade 1.16.5 -> 1.16.7
Saul Wold (2):
classes/create-spdx: extend DocumentRef to include name
create-spdx: remove trailing comma
Scott Weaver (3):
bitbake: bitbake: fetch2: fix premirror URI when downloadfilename defined
bitbake: bitbake: tests/fetch: add downloadfilename tests
bitbake: bitbake: tests/fetch: add and fix npm tests
Steve Sakoman (1):
connman: add CVE_PRODUCT
Tom Rini (1):
common-tasks: Add an example of using bbappends to add a file
Trevor Woerner (1):
hello-mod/hello.c: convert to module_init/module_exit
Valentin Danaila (1):
bitbake: fetch2/s3: allow to switch profile from environment variable
Vyacheslav Yurkov (1):
ref-manual: add overlayfs class
Signed-off-by: Andrew Geissler <geissonator@yahoo.com>
Change-Id: I194b13991cbaac7ae9e20cc2b552b508ab879905
Diffstat (limited to 'poky/meta/lib')
-rw-r--r-- | poky/meta/lib/oe/maketype.py | 7 | ||||
-rw-r--r-- | poky/meta/lib/oe/packagedata.py | 12 | ||||
-rw-r--r-- | poky/meta/lib/oe/rootfs.py | 2 | ||||
-rw-r--r-- | poky/meta/lib/oe/sbom.py | 74 | ||||
-rw-r--r-- | poky/meta/lib/oe/spdx.py | 271 | ||||
-rw-r--r-- | poky/meta/lib/oe/sstatesig.py | 5 | ||||
-rw-r--r-- | poky/meta/lib/oeqa/core/target/ssh.py | 4 | ||||
-rw-r--r-- | poky/meta/lib/oeqa/runtime/cases/parselogs.py | 2 | ||||
-rw-r--r-- | poky/meta/lib/oeqa/selftest/cases/bblogging.py | 104 | ||||
-rw-r--r-- | poky/meta/lib/oeqa/selftest/cases/devtool.py | 2 | ||||
-rw-r--r-- | poky/meta/lib/oeqa/selftest/cases/oescripts.py | 2 | ||||
-rw-r--r-- | poky/meta/lib/oeqa/selftest/cases/recipetool.py | 2 | ||||
-rw-r--r-- | poky/meta/lib/oeqa/selftest/cases/recipeutils.py | 2 | ||||
-rw-r--r-- | poky/meta/lib/oeqa/selftest/cases/wic.py | 11 |
14 files changed, 486 insertions, 14 deletions
diff --git a/poky/meta/lib/oe/maketype.py b/poky/meta/lib/oe/maketype.py index d929c8b3e5..d36082c535 100644 --- a/poky/meta/lib/oe/maketype.py +++ b/poky/meta/lib/oe/maketype.py @@ -10,12 +10,7 @@ the arguments of the type's factory for details. import inspect import oe.types as types -try: - # Python 3.7+ - from collections.abc import Callable -except ImportError: - # Python < 3.7 - from collections import Callable +from collections.abc import Callable available_types = {} diff --git a/poky/meta/lib/oe/packagedata.py b/poky/meta/lib/oe/packagedata.py index 0b17897e40..02c81e5a52 100644 --- a/poky/meta/lib/oe/packagedata.py +++ b/poky/meta/lib/oe/packagedata.py @@ -57,6 +57,18 @@ def read_subpkgdata_dict(pkg, d): ret[newvar] = subd[var] return ret +def read_subpkgdata_extended(pkg, d): + import json + import bb.compress.zstd + + fn = d.expand("${PKGDATA_DIR}/extended/%s.json.zstd" % pkg) + try: + num_threads = int(d.getVar("BB_NUMBER_THREADS")) + with bb.compress.zstd.open(fn, "rt", encoding="utf-8", num_threads=num_threads) as f: + return json.load(f) + except FileNotFoundError: + return None + def _pkgmap(d): """Return a dictionary mapping package to recipe name.""" diff --git a/poky/meta/lib/oe/rootfs.py b/poky/meta/lib/oe/rootfs.py index 4ea5adb9f4..b0dd625539 100644 --- a/poky/meta/lib/oe/rootfs.py +++ b/poky/meta/lib/oe/rootfs.py @@ -253,7 +253,7 @@ class Rootfs(object, metaclass=ABCMeta): # Remove the run-postinsts package if no delayed postinsts are found delayed_postinsts = self._get_delayed_postinsts() if delayed_postinsts is None: - if os.path.exists(self.d.expand("${IMAGE_ROOTFS}${sysconfdir}/init.d/run-postinsts")) or os.path.exists(self.d.expand("${IMAGE_ROOTFS}${systemd_unitdir}/system/run-postinsts.service")): + if os.path.exists(self.d.expand("${IMAGE_ROOTFS}${sysconfdir}/init.d/run-postinsts")) or os.path.exists(self.d.expand("${IMAGE_ROOTFS}${systemd_system_unitdir}/run-postinsts.service")): self.pm.remove(["run-postinsts"]) image_rorfs = bb.utils.contains("IMAGE_FEATURES", "read-only-rootfs", diff --git a/poky/meta/lib/oe/sbom.py b/poky/meta/lib/oe/sbom.py new file mode 100644 index 0000000000..848812c0b7 --- /dev/null +++ b/poky/meta/lib/oe/sbom.py @@ -0,0 +1,74 @@ +# +# SPDX-License-Identifier: GPL-2.0-only +# + +import collections + +DepRecipe = collections.namedtuple("DepRecipe", ("doc", "doc_sha1", "recipe")) +DepSource = collections.namedtuple("DepSource", ("doc", "doc_sha1", "recipe", "file")) + + +def get_recipe_spdxid(d): + return "SPDXRef-%s-%s" % ("Recipe", d.getVar("PN")) + + +def get_package_spdxid(pkg): + return "SPDXRef-Package-%s" % pkg + + +def get_source_file_spdxid(d, idx): + return "SPDXRef-SourceFile-%s-%d" % (d.getVar("PN"), idx) + + +def get_packaged_file_spdxid(pkg, idx): + return "SPDXRef-PackagedFile-%s-%d" % (pkg, idx) + + +def get_image_spdxid(img): + return "SPDXRef-Image-%s" % img + + +def write_doc(d, spdx_doc, subdir, spdx_deploy=None): + from pathlib import Path + + if spdx_deploy is None: + spdx_deploy = Path(d.getVar("SPDXDEPLOY")) + + dest = spdx_deploy / subdir / (spdx_doc.name + ".spdx.json") + dest.parent.mkdir(exist_ok=True, parents=True) + with dest.open("wb") as f: + doc_sha1 = spdx_doc.to_json(f, sort_keys=True) + + l = spdx_deploy / "by-namespace" / spdx_doc.documentNamespace.replace("/", "_") + l.parent.mkdir(exist_ok=True, parents=True) + l.symlink_to(os.path.relpath(dest, l.parent)) + + return doc_sha1 + + +def read_doc(fn): + import hashlib + import oe.spdx + import io + import contextlib + + @contextlib.contextmanager + def get_file(): + if isinstance(fn, io.IOBase): + yield fn + else: + with fn.open("rb") as f: + yield f + + with get_file() as f: + sha1 = hashlib.sha1() + while True: + chunk = f.read(4096) + if not chunk: + break + sha1.update(chunk) + + f.seek(0) + doc = oe.spdx.SPDXDocument.from_json(f) + + return (doc, sha1.hexdigest()) diff --git a/poky/meta/lib/oe/spdx.py b/poky/meta/lib/oe/spdx.py new file mode 100644 index 0000000000..9814fbfd66 --- /dev/null +++ b/poky/meta/lib/oe/spdx.py @@ -0,0 +1,271 @@ +# +# SPDX-License-Identifier: GPL-2.0-only +# + +import hashlib +import itertools +import json + +SPDX_VERSION = "2.2" + + +class _Property(object): + def __init__(self, *, default=None): + self.default = default + + def setdefault(self, dest, name): + if self.default is not None: + dest.setdefault(name, self.default) + + +class _String(_Property): + def __init__(self, **kwargs): + super().__init__(**kwargs) + + def set_property(self, attrs, name): + def get_helper(obj): + return obj._spdx[name] + + def set_helper(obj, value): + obj._spdx[name] = value + + def del_helper(obj): + del obj._spdx[name] + + attrs[name] = property(get_helper, set_helper, del_helper) + + def init(self, source): + return source + + +class _Object(_Property): + def __init__(self, cls, **kwargs): + super().__init__(**kwargs) + self.cls = cls + + def set_property(self, attrs, name): + def get_helper(obj): + if not name in obj._spdx: + obj._spdx[name] = self.cls() + return obj._spdx[name] + + def set_helper(obj, value): + obj._spdx[name] = value + + def del_helper(obj): + del obj._spdx[name] + + attrs[name] = property(get_helper, set_helper) + + def init(self, source): + return self.cls(**source) + + +class _ListProperty(_Property): + def __init__(self, prop, **kwargs): + super().__init__(**kwargs) + self.prop = prop + + def set_property(self, attrs, name): + def get_helper(obj): + if not name in obj._spdx: + obj._spdx[name] = [] + return obj._spdx[name] + + def del_helper(obj): + del obj._spdx[name] + + attrs[name] = property(get_helper, None, del_helper) + + def init(self, source): + return [self.prop.init(o) for o in source] + + +class _StringList(_ListProperty): + def __init__(self, **kwargs): + super().__init__(_String(), **kwargs) + + +class _ObjectList(_ListProperty): + def __init__(self, cls, **kwargs): + super().__init__(_Object(cls), **kwargs) + + +class MetaSPDXObject(type): + def __new__(mcls, name, bases, attrs): + attrs["_properties"] = {} + + for key in attrs.keys(): + if isinstance(attrs[key], _Property): + prop = attrs[key] + attrs["_properties"][key] = prop + prop.set_property(attrs, key) + + return super().__new__(mcls, name, bases, attrs) + + +class SPDXObject(metaclass=MetaSPDXObject): + def __init__(self, **d): + self._spdx = {} + + for name, prop in self._properties.items(): + prop.setdefault(self._spdx, name) + if name in d: + self._spdx[name] = prop.init(d[name]) + + def serializer(self): + return self._spdx + + def __setattr__(self, name, value): + if name in self._properties or name == "_spdx": + super().__setattr__(name, value) + return + raise KeyError("%r is not a valid SPDX property" % name) + + +class SPDXChecksum(SPDXObject): + algorithm = _String() + checksumValue = _String() + + +class SPDXRelationship(SPDXObject): + spdxElementId = _String() + relatedSpdxElement = _String() + relationshipType = _String() + comment = _String() + + +class SPDXExternalReference(SPDXObject): + referenceCategory = _String() + referenceType = _String() + referenceLocator = _String() + + +class SPDXPackageVerificationCode(SPDXObject): + packageVerificationCodeValue = _String() + packageVerificationCodeExcludedFiles = _StringList() + + +class SPDXPackage(SPDXObject): + name = _String() + SPDXID = _String() + versionInfo = _String() + downloadLocation = _String(default="NOASSERTION") + packageSupplier = _String(default="NOASSERTION") + homepage = _String() + licenseConcluded = _String(default="NOASSERTION") + licenseDeclared = _String(default="NOASSERTION") + summary = _String() + description = _String() + sourceInfo = _String() + copyrightText = _String(default="NOASSERTION") + licenseInfoFromFiles = _StringList(default=["NOASSERTION"]) + externalRefs = _ObjectList(SPDXExternalReference) + packageVerificationCode = _Object(SPDXPackageVerificationCode) + hasFiles = _StringList() + packageFileName = _String() + + +class SPDXFile(SPDXObject): + SPDXID = _String() + fileName = _String() + licenseConcluded = _String(default="NOASSERTION") + copyrightText = _String(default="NOASSERTION") + licenseInfoInFiles = _StringList(default=["NOASSERTION"]) + checksums = _ObjectList(SPDXChecksum) + fileTypes = _StringList() + + +class SPDXCreationInfo(SPDXObject): + created = _String() + licenseListVersion = _String() + comment = _String() + creators = _StringList() + + +class SPDXExternalDocumentRef(SPDXObject): + externalDocumentId = _String() + spdxDocument = _String() + checksum = _Object(SPDXChecksum) + + +class SPDXExtractedLicensingInfo(SPDXObject): + name = _String() + comment = _String() + licenseId = _String() + extractedText = _String() + + +class SPDXDocument(SPDXObject): + spdxVersion = _String(default="SPDX-" + SPDX_VERSION) + dataLicense = _String(default="CC0-1.0") + SPDXID = _String(default="SPDXRef-DOCUMENT") + name = _String() + documentNamespace = _String() + creationInfo = _Object(SPDXCreationInfo) + packages = _ObjectList(SPDXPackage) + files = _ObjectList(SPDXFile) + relationships = _ObjectList(SPDXRelationship) + externalDocumentRefs = _ObjectList(SPDXExternalDocumentRef) + hasExtractedLicensingInfos = _ObjectList(SPDXExtractedLicensingInfo) + + def __init__(self, **d): + super().__init__(**d) + + def to_json(self, f, *, sort_keys=False, indent=None, separators=None): + class Encoder(json.JSONEncoder): + def default(self, o): + if isinstance(o, SPDXObject): + return o.serializer() + + return super().default(o) + + sha1 = hashlib.sha1() + for chunk in Encoder( + sort_keys=sort_keys, + indent=indent, + separators=separators, + ).iterencode(self): + chunk = chunk.encode("utf-8") + f.write(chunk) + sha1.update(chunk) + + return sha1.hexdigest() + + @classmethod + def from_json(cls, f): + return cls(**json.load(f)) + + def add_relationship(self, _from, relationship, _to, *, comment=None): + if isinstance(_from, SPDXObject): + from_spdxid = _from.SPDXID + else: + from_spdxid = _from + + if isinstance(_to, SPDXObject): + to_spdxid = _to.SPDXID + else: + to_spdxid = _to + + r = SPDXRelationship( + spdxElementId=from_spdxid, + relatedSpdxElement=to_spdxid, + relationshipType=relationship, + ) + + if comment is not None: + r.comment = comment + + self.relationships.append(r) + + def find_by_spdxid(self, spdxid): + for o in itertools.chain(self.packages, self.files): + if o.SPDXID == spdxid: + return o + return None + + def find_external_document_ref(self, namespace): + for r in self.externalDocumentRefs: + if r.spdxDocument == namespace: + return r + return None diff --git a/poky/meta/lib/oe/sstatesig.py b/poky/meta/lib/oe/sstatesig.py index 78cdf878f1..dd6b9de7bb 100644 --- a/poky/meta/lib/oe/sstatesig.py +++ b/poky/meta/lib/oe/sstatesig.py @@ -491,8 +491,10 @@ def OEOuthashBasic(path, sigfile, task, d): if "package_write_" in task or task == "package_qa": include_owners = False include_timestamps = False + include_root = True if task == "package": include_timestamps = d.getVar('BUILD_REPRODUCIBLE_BINARIES') == '1' + include_root = False extra_content = d.getVar('HASHEQUIV_HASH_VERSION') try: @@ -603,7 +605,8 @@ def OEOuthashBasic(path, sigfile, task, d): update_hash("\n") # Process this directory and all its child files - process(root) + if include_root or root != ".": + process(root) for f in files: if f == 'fixmepath': continue diff --git a/poky/meta/lib/oeqa/core/target/ssh.py b/poky/meta/lib/oeqa/core/target/ssh.py index 923a223b25..f956a7744f 100644 --- a/poky/meta/lib/oeqa/core/target/ssh.py +++ b/poky/meta/lib/oeqa/core/target/ssh.py @@ -44,6 +44,7 @@ class OESSHTarget(OETarget): self.ssh = self.ssh + [ '-p', port ] self.scp = self.scp + [ '-P', port ] self._monitor_dumper = None + self.target_dumper = None def start(self, **kwargs): pass @@ -102,7 +103,8 @@ class OESSHTarget(OETarget): if self.monitor_dumper: self.monitor_dumper.dump_monitor() if status == 255: - self.target_dumper.dump_target() + if self.target_dumper: + self.target_dumper.dump_target() if self.monitor_dumper: self.monitor_dumper.dump_monitor() return (status, output) diff --git a/poky/meta/lib/oeqa/runtime/cases/parselogs.py b/poky/meta/lib/oeqa/runtime/cases/parselogs.py index 5db0216597..2b8893d842 100644 --- a/poky/meta/lib/oeqa/runtime/cases/parselogs.py +++ b/poky/meta/lib/oeqa/runtime/cases/parselogs.py @@ -91,6 +91,7 @@ qemux86_common = [ "glamor initialization failed", "blk_update_request: I/O error, dev fd0, sector 0 op 0x0:(READ)", "floppy: error", + 'failed to IDENTIFY (I/O error, err_mask=0x4)', ] + common_errors ignore_errors = { @@ -98,7 +99,6 @@ ignore_errors = { 'qemux86' : [ 'Failed to access perfctr msr (MSR', 'pci 0000:00:00.0: [Firmware Bug]: reg 0x..: invalid BAR (can\'t size)', - 'failed to IDENTIFY (I/O error, err_mask=0x4)', ] + qemux86_common, 'qemux86-64' : qemux86_common, 'qemumips' : [ diff --git a/poky/meta/lib/oeqa/selftest/cases/bblogging.py b/poky/meta/lib/oeqa/selftest/cases/bblogging.py new file mode 100644 index 0000000000..ea6c3c8c77 --- /dev/null +++ b/poky/meta/lib/oeqa/selftest/cases/bblogging.py @@ -0,0 +1,104 @@ +# +# SPDX-License-Identifier: MIT +# + + +from oeqa.selftest.case import OESelftestTestCase +from oeqa.utils.commands import bitbake + +class BitBakeLogging(OESelftestTestCase): + + def assertCount(self, item, entry, count): + self.assertEqual(item.count(entry), count, msg="Output:\n'''\n%s\n'''\ndoesn't contain %d copies of:\n'''\n%s\n'''\n" % (item, count, entry)) + + def test_shell_logging(self): + # no logs, no verbose + self.write_config('BBINCLUDELOGS = ""') + result = bitbake("logging-test -c shelltest -f", ignore_status = True) + self.assertIn("ERROR: Logfile of failure stored in:", result.output) + self.assertNotIn("This is shell stdout", result.output) + self.assertNotIn("This is shell stderr", result.output) + + # logs, no verbose + self.write_config('BBINCLUDELOGS = "yes"') + result = bitbake("logging-test -c shelltest -f", ignore_status = True) + self.assertIn("ERROR: Logfile of failure stored in:", result.output) + self.assertCount(result.output, "This is shell stdout", 1) + self.assertCount(result.output, "This is shell stderr", 1) + + # no logs, verbose + self.write_config('BBINCLUDELOGS = ""') + result = bitbake("logging-test -c shelltest -f -v", ignore_status = True) + self.assertIn("ERROR: Logfile of failure stored in:", result.output) + # two copies due to set +x + self.assertCount(result.output, "This is shell stdout", 2) + self.assertCount(result.output, "This is shell stderr", 2) + + # logs, verbose + self.write_config('BBINCLUDELOGS = "yes"') + result = bitbake("logging-test -c shelltest -f -v", ignore_status = True) + self.assertIn("ERROR: Logfile of failure stored in:", result.output) + # two copies due to set +x + self.assertCount(result.output, "This is shell stdout", 2) + self.assertCount(result.output, "This is shell stderr", 2) + + def test_python_exit_logging(self): + # no logs, no verbose + self.write_config('BBINCLUDELOGS = ""') + result = bitbake("logging-test -c pythontest_exit -f", ignore_status = True) + self.assertIn("ERROR: Logfile of failure stored in:", result.output) + self.assertNotIn("This is python stdout", result.output) + + # logs, no verbose + self.write_config('BBINCLUDELOGS = "yes"') + result = bitbake("logging-test -c pythontest_exit -f", ignore_status = True) + self.assertIn("ERROR: Logfile of failure stored in:", result.output) + # A sys.exit() should include the output + self.assertCount(result.output, "This is python stdout", 1) + + # no logs, verbose + self.write_config('BBINCLUDELOGS = ""') + result = bitbake("logging-test -c pythontest_exit -f -v", ignore_status = True) + self.assertIn("ERROR: Logfile of failure stored in:", result.output) + # python tasks don't log output with -v currently + #self.assertCount(result.output, "This is python stdout", 1) + + # logs, verbose + self.write_config('BBINCLUDELOGS = "yes"') + result = bitbake("logging-test -c pythontest_exit -f -v", ignore_status = True) + self.assertIn("ERROR: Logfile of failure stored in:", result.output) + # python tasks don't log output with -v currently + #self.assertCount(result.output, "This is python stdout", 1) + + def test_python_fatal_logging(self): + # no logs, no verbose + self.write_config('BBINCLUDELOGS = ""') + result = bitbake("logging-test -c pythontest_fatal -f", ignore_status = True) + self.assertIn("ERROR: Logfile of failure stored in:", result.output) + self.assertNotIn("This is python fatal test stdout", result.output) + self.assertCount(result.output, "This is a fatal error", 1) + + # logs, no verbose + self.write_config('BBINCLUDELOGS = "yes"') + result = bitbake("logging-test -c pythontest_fatal -f", ignore_status = True) + self.assertIn("ERROR: Logfile of failure stored in:", result.output) + # A bb.fatal() should not include the output + self.assertNotIn("This is python fatal test stdout", result.output) + self.assertCount(result.output, "This is a fatal error", 1) + + # no logs, verbose + self.write_config('BBINCLUDELOGS = ""') + result = bitbake("logging-test -c pythontest_fatal -f -v", ignore_status = True) + self.assertIn("ERROR: Logfile of failure stored in:", result.output) + # python tasks don't log output with -v currently + #self.assertCount(result.output, "This is python fatal test stdout", 1) + self.assertCount(result.output, "This is a fatal error", 1) + + # logs, verbose + self.write_config('BBINCLUDELOGS = "yes"') + result = bitbake("logging-test -c pythontest_fatal -f -v", ignore_status = True) + self.assertIn("ERROR: Logfile of failure stored in:", result.output) + # python tasks don't log output with -v currently + #self.assertCount(result.output, "This is python fatal test stdout", 1) + self.assertCount(result.output, "This is a fatal error", 1) + diff --git a/poky/meta/lib/oeqa/selftest/cases/devtool.py b/poky/meta/lib/oeqa/selftest/cases/devtool.py index 6d9cd46bf3..f495e84c79 100644 --- a/poky/meta/lib/oeqa/selftest/cases/devtool.py +++ b/poky/meta/lib/oeqa/selftest/cases/devtool.py @@ -649,7 +649,7 @@ class DevtoolModifyTests(DevtoolBase): self.track_for_cleanup(self.workspacedir) self.add_command_to_tearDown('bitbake-layers remove-layer */workspace') - testrecipes = 'perf kernel-devsrc package-index core-image-minimal meta-toolchain packagegroup-core-sdk meta-ide-support'.split() + testrecipes = 'perf kernel-devsrc package-index core-image-minimal meta-toolchain packagegroup-core-sdk'.split() # Find actual name of gcc-source since it now includes the version - crude, but good enough for this purpose result = runCmd('bitbake-layers show-recipes gcc-source*') for line in result.output.splitlines(): diff --git a/poky/meta/lib/oeqa/selftest/cases/oescripts.py b/poky/meta/lib/oeqa/selftest/cases/oescripts.py index 8a10ff357b..1decce39e9 100644 --- a/poky/meta/lib/oeqa/selftest/cases/oescripts.py +++ b/poky/meta/lib/oeqa/selftest/cases/oescripts.py @@ -150,7 +150,7 @@ class OEListPackageconfigTests(OEScriptTests): expected_endlines = [] expected_endlines.append("RECIPE NAME PACKAGECONFIG FLAGS") expected_endlines.append("pinentry gtk2 libcap ncurses qt secret") - expected_endlines.append("tar acl") + expected_endlines.append("tar acl selinux") self.check_endlines(results, expected_endlines) diff --git a/poky/meta/lib/oeqa/selftest/cases/recipetool.py b/poky/meta/lib/oeqa/selftest/cases/recipetool.py index 6f531dfa36..c2a53815d0 100644 --- a/poky/meta/lib/oeqa/selftest/cases/recipetool.py +++ b/poky/meta/lib/oeqa/selftest/cases/recipetool.py @@ -481,7 +481,7 @@ class RecipetoolTests(RecipetoolBase): result = runCmd('recipetool create -o %s %s' % (temprecipe, srcuri)) self.assertTrue(os.path.isfile(recipefile)) checkvars = {} - checkvars['LICENSE'] = set(['PSF', '&', 'BSD', 'GPL']) + checkvars['LICENSE'] = set(['PSF', '&', 'BSD-3-Clause', 'GPL']) checkvars['LIC_FILES_CHKSUM'] = 'file://COPYING.txt;md5=35a23d42b615470583563132872c97d6' checkvars['SRC_URI'] = 'https://files.pythonhosted.org/packages/84/f4/5771e41fdf52aabebbadecc9381d11dea0fa34e4759b4071244fa094804c/docutils-${PV}.tar.gz' checkvars['SRC_URI[md5sum]'] = 'c53768d63db3873b7d452833553469de' diff --git a/poky/meta/lib/oeqa/selftest/cases/recipeutils.py b/poky/meta/lib/oeqa/selftest/cases/recipeutils.py index 97edad88b5..f1dd63f65b 100644 --- a/poky/meta/lib/oeqa/selftest/cases/recipeutils.py +++ b/poky/meta/lib/oeqa/selftest/cases/recipeutils.py @@ -40,7 +40,7 @@ class RecipeUtilsTests(OESelftestTestCase): SUMMARY = "Python framework to process interdependent tasks in a pool of workers" HOMEPAGE = "http://github.com/gitpython-developers/async" SECTION = "devel/python" --LICENSE = "BSD" +-LICENSE = "BSD-3-Clause" +LICENSE = "something" LIC_FILES_CHKSUM = "file://PKG-INFO;beginline=8;endline=8;md5=88df8e78b9edfd744953862179f2d14e" diff --git a/poky/meta/lib/oeqa/selftest/cases/wic.py b/poky/meta/lib/oeqa/selftest/cases/wic.py index 3b4143414f..dc7b9e637e 100644 --- a/poky/meta/lib/oeqa/selftest/cases/wic.py +++ b/poky/meta/lib/oeqa/selftest/cases/wic.py @@ -744,6 +744,17 @@ part /etc --source rootfs --fstype=ext4 --change-directory=etc % (wks_file, self.resultdir), ignore_status=True).status) os.remove(wks_file) + def test_extra_space(self): + """Test --extra-space wks option.""" + extraspace = 1024**3 + runCmd("wic create wictestdisk " + "--image-name core-image-minimal " + "--extra-space %i -o %s" % (extraspace ,self.resultdir)) + wicout = glob(self.resultdir + "wictestdisk-*.direct") + self.assertEqual(1, len(wicout)) + size = os.path.getsize(wicout[0]) + self.assertTrue(size > extraspace) + class Wic2(WicTestCase): def test_bmap_short(self): |