| Age | Commit message (Collapse) | Author | Files | Lines |
|---|
|
Ignore mismatch errors for lcov 2.x in all locations that lcov
is run. When building more complex unit tests, these mismatch
errors are showing up in different phases of the code coverage
collection.
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
|
|
The environment variable `GTEST_CATCH_EXCEPTION` must be
set to `0` for so all exceptions are handled by the
address sanitizer and not GoogleTest. This allows stack
back trace and other details to be logged by the address
sanitizer so the source of the issue identified address
sanitizer can be determined.
The environment variable `ASAN_OPTIONS` must be set to
`detect_leaks=0` to disable memory leak detection. The
unit test frameworks may have memory leaks and some
firmware code under test use cases may perform a memory
allocation without a matching memory free as their
expected behavior.
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
|
|
Detect the version of lcov and only apply
version 2 workaround when needed.
Fixes 61c714285f8c
Signed-off-by: Oliver Steffen <osteffen@redhat.com>
|
|
lcov 2.0 and newer does additional error and consistency checking
compared to previous versions. This can lead to CI jobs failing due to
new/unexpected errors showing up.
See:
- https://edk2.groups.io/g/devel/message/116138
- https://github.com/linux-test-project/lcov/issues/209
- https://github.com/linux-test-project/lcov/issues/238
Signed-off-by: Oliver Steffen <osteffen@redhat.com>
|
|
Some unit tests would fail to execute or execute and not produce any output
logs. In these cases, the only output would be in the CI Log as
`UnitTest Execution Error`.
A UnitTest Execution Error should be considered the same as a unit tests
test failing.
Signed-off-by: Aaron Pop <aaronpop@microsoft.com>
|
|
Windows command prompt have 8191 characters limitation,
enhance it to make command too long can be resloved.
Provide an example, if have too many cov files, it causes to run single
command over the 8191 characters limitation.
> OpenCppCoverage
> --export_type binary:coverage.cov
> --working_dir={workspace}Build
> --input_coverage=AAA.cov
> ...
> --input_coverage=NNN.cov
The solution is passing many coverage files in single command line to
breaking it up into many command lines with one coverage file per
command line in order to prevent single line is over to 8191 characters.
- Command Line 1
> OpenCppCoverage
> --export_type binary:coverage.cov
> --working_dir={workspace}Build
> --input_coverage=AAA.cov
> --input_coverage=coverage.cov
...
- Command Line N
> OpenCppCoverage
> --export_type. binary:coverage.cov
> --working_dir={workspace}Build
> --input_coverage=NNN.cov
> --input_coverage=coverage.cov
Cc: Michael D Kinney <michael.d.kinney@intel.com>
Cc: Sean Brogan <sean.brogan@microsoft.com>
Cc: Michael Kubacki <mikuback@linux.microsoft.com>
Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Signed-off-by: Gua Guo <gua.guo@intel.com>
Reviewed-by: Michael Kubacki <mikuback@linux.microsoft.com>
Reviewed-by: Michael D Kinney <michael.d.kinney@intel.com>
|
|
If code coverage exist failure, CI/CD need to catch it
Cc: Michael D Kinney <michael.d.kinney@intel.com>
Cc: Sean Brogan <sean.brogan@microsoft.com>
Cc: Michael Kubacki <mikuback@linux.microsoft.com>
Signed-off-by: Gua Guo <gua.guo@intel.com>
Reviewed-by: Michael D Kinney <michael.d.kinney@intel.com>
Reviewed-by: Michael Kubacki <mikuback@linux.microsoft.com>
|
|
HostBasedUnitTestRunner.py is a build plugin responsible for locating
and executing host-based unit tests.
Recently, commit 6bb00aa introduced support for the plugin to
generate code coverage reports via lcov and OpenCppCoverage.
The plugin has discovered unit tests by searching for executables
with "Test" in the name for a while. However, the test coverage
change makes assumptions about test presence when crafting the
OpenCppCoverage command that ultimately fails with an ambiguous error
message if no host-based unit tests are discovered (see "ERROR").
```
SECTION - Run Host based Unit Tests
SUBSECTION - Testing for architecture: X64
ERROR - UnitTest Coverage: Failed to generate cobertura format xml in
single package.
PROGRESS - --->Test Success: Host Unit Test Compiler Plugin NOOPT
```
This change preempts that message with a check in the plugin to
determine if any host-based tests were discovered. If not, a message
is printed with more guidance about how the developer should proceed
to either (1) fix their tests so code coverage is generated as
expected or (2) prevent the error message.
New message:
```
SECTION - Run Host based Unit Tests
SUBSECTION - Testing for architecture: X64
WARNING - UnitTest Coverage:
No unit tests discovered. Test coverage will not be generated.
Prevent this message by:
1. Adding host-based unit tests to this package
2. Ensuring tests have the word "Test" in their name
3. Disabling HostUnitTestCompilerPlugin in the package CI YAML file
PROGRESS - --->Test Success: Host Unit Test Compiler Plugin NOOPT
```
Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Cc: Michael D Kinney <michael.d.kinney@intel.com>
Cc: Sean Brogan <sean.brogan@microsoft.com>
Signed-off-by: Michael Kubacki <michael.kubacki@microsoft.com>
Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>
Reviewed-by: Rebecca Cran <rebecca@bsdio.com>
|
|
For GCC, use lcov to generate Unit Test code coverage
report
For VS2019, use OpenCppCoverage to generate code
coverage report
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Bret Barkelew <Bret.Barkelew@microsoft.com>
Cc: Liming Gao <gaoliming@bysoft.com.cn>
Reviewed-by: Michael D Kinney <michael.d.kinney@intel.com>
Cc: Sean Brogan <sean.brogan@microsoft.com>
Reviewed-by: Michael Kubacki <mikuback@linux.microsoft.com>
Signed-off-by: Gua Guo <gua.guo@intel.com>
|
|
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=4134
Set environment variable GTEST_OUTPUT to specify the output
format of XML and the output file name. Both CMOCKA_XML_FILE
and GTEST_OUTPUT are set for each host based unit test to
support both cmocka unit tests and gtest unit tests.
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
Reviewed-by: Bob Feng <bob.c.feng@intel.com>
|
|
REF:https://bugzilla.tianocore.org/show_bug.cgi?id=2513
Update HostBasedUnitTestRunner plugin to support the Linux environment
and remove any Windows only logic.
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Bret Barkelew <Bret.Barkelew@microsoft.com>
Cc: Michael D Kinney <michael.d.kinney@intel.com>
Signed-off-by: Sean Brogan <sean.brogan@microsoft.com>
Signed-off-by: Michael Kubacki <michael.kubacki@microsoft.com>
Reviewed-by: Shenglei Zhang <shenglei.zhang@intel.com>
Reviewed-by: Liming Gao <liming.gao@intel.com>
|
|
https://bugzilla.tianocore.org/show_bug.cgi?id=2505
Add plugin to BaseTools to run host based unit tests.
Cc: Sean Brogan <sean.brogan@microsoft.com>
Cc: Bret Barkelew <Bret.Barkelew@microsoft.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
Reviewed-by: Bret Barkelew <Bret.Barkelew@microsoft.com>
Acked-by: Bob Feng <bob.c.feng@intel.com>
|
