Skip to content

Commit bd087fc

Browse files
fmeumfmeum
committed
[llvm-cov] Add support for baseline coverage
When no profile is provided via --instr-profile, the export/report/show commands now emit coverage data equivalent to that obtained from a profile with all zero counters ("baseline coverage"). This is useful for build systems (e.g. Bazel) that can track coverage information for each build target, even those that are never linked into tests and thus don't have runtime coverage data recorded. By merging in baseline coverage, lines in files that aren't linked into tests are correctly reported as uncovered.
1 parent ccc18ca commit bd087fc

File tree

6 files changed

+187
-80
lines changed

6 files changed

+187
-80
lines changed

‎llvm/docs/CommandGuide/llvm-cov.rst‎

Lines changed: 12 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -188,7 +188,7 @@ SHOW COMMAND
188188
SYNOPSIS
189189
^^^^^^^^
190190

191-
:program:`llvm-cov show` [*options*] -instr-profile *PROFILE* [*BIN*] [*-object BIN*]... [*-sources*] [*SOURCE*]...
191+
:program:`llvm-cov show` [*options*] [-instr-profile *PROFILE*] [*BIN*] [*-object BIN*]... [*-sources*] [*SOURCE*]...
192192

193193
DESCRIPTION
194194
^^^^^^^^^^^
@@ -200,6 +200,9 @@ filtered to only show the coverage for the files listed in *SOURCE*....
200200
*BIN* may be an executable, object file, dynamic library, or archive (thin or
201201
otherwise).
202202

203+
If *PROFILE* is not provided, the command displays the baseline coverage of the
204+
binaries with all zero execution counts.
205+
203206
To use :program:`llvm-cov show`, you need a program that is compiled with
204207
instrumentation to emit profile and coverage data. To build such a program with
205208
``clang`` use the ``-fprofile-instr-generate`` and ``-fcoverage-mapping``
@@ -390,7 +393,7 @@ REPORT COMMAND
390393
SYNOPSIS
391394
^^^^^^^^
392395

393-
:program:`llvm-cov report` [*options*] -instr-profile *PROFILE* [*BIN*] [*-object BIN*]... [*-sources*] [*SOURCE*]...
396+
:program:`llvm-cov report` [*options*] [-instr-profile *PROFILE*] [*BIN*] [*-object BIN*]... [*-sources*] [*SOURCE*]...
394397

395398
DESCRIPTION
396399
^^^^^^^^^^^
@@ -402,6 +405,9 @@ filtered to only show the coverage for the files listed in *SOURCE*....
402405
*BIN* may be an executable, object file, dynamic library, or archive (thin or
403406
otherwise).
404407

408+
If *PROFILE* is not provided, the command displays the baseline coverage of the
409+
binaries with all zero execution counts.
410+
405411
If no source files are provided, a summary line is printed for each file in the
406412
coverage data. If any files are provided, summaries can be shown for each
407413
function in the listed files if the ``-show-functions`` option is enabled.
@@ -480,7 +486,7 @@ EXPORT COMMAND
480486
SYNOPSIS
481487
^^^^^^^^
482488

483-
:program:`llvm-cov export` [*options*] -instr-profile *PROFILE* [*BIN*] [*-object BIN*]... [*-sources*] [*SOURCE*]...
489+
:program:`llvm-cov export` [*options*] [-instr-profile *PROFILE*] [*BIN*] [*-object BIN*]... [*-sources*] [*SOURCE*]...
484490

485491
DESCRIPTION
486492
^^^^^^^^^^^
@@ -489,6 +495,9 @@ The :program:`llvm-cov export` command exports coverage data of the binaries
489495
*BIN*... using the profile data *PROFILE* in either JSON or lcov trace file
490496
format.
491497

498+
If *PROFILE* is not provided, the command exports baseline coverage data
499+
with all zero execution counts.
500+
492501
When exporting JSON, the regions, functions, branches, expansions, and
493502
summaries of the coverage data will be exported. When exporting an lcov trace
494503
file, the line-based coverage, branch coverage, and summaries will be exported.

‎llvm/include/llvm/ProfileData/Coverage/CoverageMapping.h‎

Lines changed: 15 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -990,18 +990,23 @@ class CoverageMapping {
990990
// Load coverage records from readers.
991991
static Error loadFromReaders(
992992
ArrayRef<std::unique_ptr<CoverageMappingReader>> CoverageReaders,
993-
IndexedInstrProfReader &ProfileReader, CoverageMapping &Coverage);
993+
std::optional<std::reference_wrapper<IndexedInstrProfReader>>
994+
&ProfileReader,
995+
CoverageMapping &Coverage);
994996

995997
// Load coverage records from file.
996998
static Error
997999
loadFromFile(StringRef Filename, StringRef Arch, StringRef CompilationDir,
998-
IndexedInstrProfReader &ProfileReader, CoverageMapping &Coverage,
999-
bool &DataFound,
1000+
std::optional<std::reference_wrapper<IndexedInstrProfReader>>
1001+
&ProfileReader,
1002+
CoverageMapping &Coverage, bool &DataFound,
10001003
SmallVectorImpl<object::BuildID> *FoundBinaryIDs = nullptr);
10011004

10021005
/// Add a function record corresponding to \p Record.
1003-
Error loadFunctionRecord(const CoverageMappingRecord &Record,
1004-
IndexedInstrProfReader &ProfileReader);
1006+
Error loadFunctionRecord(
1007+
const CoverageMappingRecord &Record,
1008+
const std::optional<std::reference_wrapper<IndexedInstrProfReader>>
1009+
&ProfileReader);
10051010

10061011
/// Look up the indices for function records which are at least partially
10071012
/// defined in the specified file. This is guaranteed to return a superset of
@@ -1017,15 +1022,16 @@ class CoverageMapping {
10171022
/// Load the coverage mapping using the given readers.
10181023
static Expected<std::unique_ptr<CoverageMapping>>
10191024
load(ArrayRef<std::unique_ptr<CoverageMappingReader>> CoverageReaders,
1020-
IndexedInstrProfReader &ProfileReader);
1025+
std::optional<std::reference_wrapper<IndexedInstrProfReader>>
1026+
&ProfileReader);
10211027

10221028
/// Load the coverage mapping from the given object files and profile. If
10231029
/// \p Arches is non-empty, it must specify an architecture for each object.
10241030
/// Ignores non-instrumented object files unless all are not instrumented.
10251031
static Expected<std::unique_ptr<CoverageMapping>>
1026-
load(ArrayRef<StringRef> ObjectFilenames, StringRef ProfileFilename,
1027-
vfs::FileSystem &FS, ArrayRef<StringRef> Arches = {},
1028-
StringRef CompilationDir = "",
1032+
load(ArrayRef<StringRef> ObjectFilenames,
1033+
std::optional<StringRef> ProfileFilename, vfs::FileSystem &FS,
1034+
ArrayRef<StringRef> Arches = {}, StringRef CompilationDir = "",
10291035
const object::BuildIDFetcher *BIDFetcher = nullptr,
10301036
bool CheckBinaryIDs = false);
10311037

‎llvm/lib/ProfileData/Coverage/CoverageMapping.cpp‎

Lines changed: 77 additions & 47 deletions
Original file line numberDiff line numberDiff line change
@@ -823,7 +823,8 @@ class MCDCDecisionRecorder {
823823

824824
Error CoverageMapping::loadFunctionRecord(
825825
const CoverageMappingRecord &Record,
826-
IndexedInstrProfReader &ProfileReader) {
826+
const std::optional<std::reference_wrapper<IndexedInstrProfReader>>
827+
&ProfileReader) {
827828
StringRef OrigFuncName = Record.FunctionName;
828829
if (OrigFuncName.empty())
829830
return make_error<CoverageMapError>(coveragemap_error::malformed,
@@ -837,35 +838,44 @@ Error CoverageMapping::loadFunctionRecord(
837838
CounterMappingContext Ctx(Record.Expressions);
838839

839840
std::vector<uint64_t> Counts;
840-
if (Error E = ProfileReader.getFunctionCounts(Record.FunctionName,
841-
Record.FunctionHash, Counts)) {
842-
instrprof_error IPE = std::get<0>(InstrProfError::take(std::move(E)));
843-
if (IPE == instrprof_error::hash_mismatch) {
844-
FuncHashMismatches.emplace_back(std::string(Record.FunctionName),
845-
Record.FunctionHash);
846-
return Error::success();
841+
if (ProfileReader) {
842+
if (Error E = ProfileReader.value().get().getFunctionCounts(
843+
Record.FunctionName, Record.FunctionHash, Counts)) {
844+
instrprof_error IPE = std::get<0>(InstrProfError::take(std::move(E)));
845+
if (IPE == instrprof_error::hash_mismatch) {
846+
FuncHashMismatches.emplace_back(std::string(Record.FunctionName),
847+
Record.FunctionHash);
848+
return Error::success();
849+
}
850+
if (IPE != instrprof_error::unknown_function)
851+
return make_error<InstrProfError>(IPE);
852+
Counts.assign(getMaxCounterID(Ctx, Record) + 1, 0);
847853
}
848-
if (IPE != instrprof_error::unknown_function)
849-
return make_error<InstrProfError>(IPE);
854+
} else {
850855
Counts.assign(getMaxCounterID(Ctx, Record) + 1, 0);
851856
}
852857
Ctx.setCounts(Counts);
853858

854859
bool IsVersion11 =
855-
ProfileReader.getVersion() < IndexedInstrProf::ProfVersion::Version12;
860+
ProfileReader && ProfileReader.value().get().getVersion() <
861+
IndexedInstrProf::ProfVersion::Version12;
856862

857863
BitVector Bitmap;
858-
if (Error E = ProfileReader.getFunctionBitmap(Record.FunctionName,
859-
Record.FunctionHash, Bitmap)) {
860-
instrprof_error IPE = std::get<0>(InstrProfError::take(std::move(E)));
861-
if (IPE == instrprof_error::hash_mismatch) {
862-
FuncHashMismatches.emplace_back(std::string(Record.FunctionName),
863-
Record.FunctionHash);
864-
return Error::success();
864+
if (ProfileReader) {
865+
if (Error E = ProfileReader.value().get().getFunctionBitmap(
866+
Record.FunctionName, Record.FunctionHash, Bitmap)) {
867+
instrprof_error IPE = std::get<0>(InstrProfError::take(std::move(E)));
868+
if (IPE == instrprof_error::hash_mismatch) {
869+
FuncHashMismatches.emplace_back(std::string(Record.FunctionName),
870+
Record.FunctionHash);
871+
return Error::success();
872+
}
873+
if (IPE != instrprof_error::unknown_function)
874+
return make_error<InstrProfError>(IPE);
875+
Bitmap = BitVector(getMaxBitmapSize(Record, IsVersion11));
865876
}
866-
if (IPE != instrprof_error::unknown_function)
867-
return make_error<InstrProfError>(IPE);
868-
Bitmap = BitVector(getMaxBitmapSize(Record, IsVersion11));
877+
} else {
878+
Bitmap = BitVector(getMaxBitmapSize(Record, false));
869879
}
870880
Ctx.setBitmap(std::move(Bitmap));
871881

@@ -960,10 +970,17 @@ Error CoverageMapping::loadFunctionRecord(
960970
// of CoverageMappingReader instances.
961971
Error CoverageMapping::loadFromReaders(
962972
ArrayRef<std::unique_ptr<CoverageMappingReader>> CoverageReaders,
963-
IndexedInstrProfReader &ProfileReader, CoverageMapping &Coverage) {
973+
std::optional<std::reference_wrapper<IndexedInstrProfReader>>
974+
&ProfileReader,
975+
CoverageMapping &Coverage) {
964976
assert(!Coverage.SingleByteCoverage ||
965-
*Coverage.SingleByteCoverage == ProfileReader.hasSingleByteCoverage());
966-
Coverage.SingleByteCoverage = ProfileReader.hasSingleByteCoverage();
977+
!ProfileReader ||
978+
*Coverage.SingleByteCoverage ==
979+
ProfileReader.value().get().hasSingleByteCoverage());
980+
if (ProfileReader) {
981+
Coverage.SingleByteCoverage =
982+
ProfileReader.value().get().hasSingleByteCoverage();
983+
}
967984
for (const auto &CoverageReader : CoverageReaders) {
968985
for (auto RecordOrErr : *CoverageReader) {
969986
if (Error E = RecordOrErr.takeError())
@@ -978,7 +995,8 @@ Error CoverageMapping::loadFromReaders(
978995

979996
Expected<std::unique_ptr<CoverageMapping>> CoverageMapping::load(
980997
ArrayRef<std::unique_ptr<CoverageMappingReader>> CoverageReaders,
981-
IndexedInstrProfReader &ProfileReader) {
998+
std::optional<std::reference_wrapper<IndexedInstrProfReader>>
999+
&ProfileReader) {
9821000
auto Coverage = std::unique_ptr<CoverageMapping>(new CoverageMapping());
9831001
if (Error E = loadFromReaders(CoverageReaders, ProfileReader, *Coverage))
9841002
return std::move(E);
@@ -987,18 +1005,19 @@ Expected<std::unique_ptr<CoverageMapping>> CoverageMapping::load(
9871005

9881006
// If E is a no_data_found error, returns success. Otherwise returns E.
9891007
static Error handleMaybeNoDataFoundError(Error E) {
990-
return handleErrors(
991-
std::move(E), [](const CoverageMapError &CME) {
992-
if (CME.get() == coveragemap_error::no_data_found)
993-
return static_cast<Error>(Error::success());
994-
return make_error<CoverageMapError>(CME.get(), CME.getMessage());
995-
});
1008+
return handleErrors(std::move(E), [](const CoverageMapError &CME) {
1009+
if (CME.get() == coveragemap_error::no_data_found)
1010+
return static_cast<Error>(Error::success());
1011+
return make_error<CoverageMapError>(CME.get(), CME.getMessage());
1012+
});
9961013
}
9971014

9981015
Error CoverageMapping::loadFromFile(
9991016
StringRef Filename, StringRef Arch, StringRef CompilationDir,
1000-
IndexedInstrProfReader &ProfileReader, CoverageMapping &Coverage,
1001-
bool &DataFound, SmallVectorImpl<object::BuildID> *FoundBinaryIDs) {
1017+
std::optional<std::reference_wrapper<IndexedInstrProfReader>>
1018+
&ProfileReader,
1019+
CoverageMapping &Coverage, bool &DataFound,
1020+
SmallVectorImpl<object::BuildID> *FoundBinaryIDs) {
10021021
auto CovMappingBufOrErr = MemoryBuffer::getFileOrSTDIN(
10031022
Filename, /*IsText=*/false, /*RequiresNullTerminator=*/false);
10041023
if (std::error_code EC = CovMappingBufOrErr.getError())
@@ -1034,13 +1053,23 @@ Error CoverageMapping::loadFromFile(
10341053
}
10351054

10361055
Expected<std::unique_ptr<CoverageMapping>> CoverageMapping::load(
1037-
ArrayRef<StringRef> ObjectFilenames, StringRef ProfileFilename,
1038-
vfs::FileSystem &FS, ArrayRef<StringRef> Arches, StringRef CompilationDir,
1056+
ArrayRef<StringRef> ObjectFilenames,
1057+
std::optional<StringRef> ProfileFilename, vfs::FileSystem &FS,
1058+
ArrayRef<StringRef> Arches, StringRef CompilationDir,
10391059
const object::BuildIDFetcher *BIDFetcher, bool CheckBinaryIDs) {
1040-
auto ProfileReaderOrErr = IndexedInstrProfReader::create(ProfileFilename, FS);
1041-
if (Error E = ProfileReaderOrErr.takeError())
1042-
return createFileError(ProfileFilename, std::move(E));
1043-
auto ProfileReader = std::move(ProfileReaderOrErr.get());
1060+
std::unique_ptr<IndexedInstrProfReader> ProfileReader;
1061+
if (ProfileFilename) {
1062+
auto ProfileReaderOrErr =
1063+
IndexedInstrProfReader::create(ProfileFilename.value(), FS);
1064+
if (Error E = ProfileReaderOrErr.takeError())
1065+
return createFileError(ProfileFilename.value(), std::move(E));
1066+
ProfileReader = std::move(ProfileReaderOrErr.get());
1067+
}
1068+
auto ProfileReaderRef =
1069+
ProfileReader
1070+
? std::optional<std::reference_wrapper<IndexedInstrProfReader>>(
1071+
*ProfileReader)
1072+
: std::nullopt;
10441073
auto Coverage = std::unique_ptr<CoverageMapping>(new CoverageMapping());
10451074
bool DataFound = false;
10461075

@@ -1054,16 +1083,17 @@ Expected<std::unique_ptr<CoverageMapping>> CoverageMapping::load(
10541083

10551084
SmallVector<object::BuildID> FoundBinaryIDs;
10561085
for (const auto &File : llvm::enumerate(ObjectFilenames)) {
1057-
if (Error E =
1058-
loadFromFile(File.value(), GetArch(File.index()), CompilationDir,
1059-
*ProfileReader, *Coverage, DataFound, &FoundBinaryIDs))
1086+
if (Error E = loadFromFile(File.value(), GetArch(File.index()),
1087+
CompilationDir, ProfileReaderRef, *Coverage,
1088+
DataFound, &FoundBinaryIDs))
10601089
return std::move(E);
10611090
}
10621091

10631092
if (BIDFetcher) {
10641093
std::vector<object::BuildID> ProfileBinaryIDs;
1065-
if (Error E = ProfileReader->readBinaryIds(ProfileBinaryIDs))
1066-
return createFileError(ProfileFilename, std::move(E));
1094+
if (ProfileReader)
1095+
if (Error E = ProfileReader->readBinaryIds(ProfileBinaryIDs))
1096+
return createFileError(ProfileFilename.value(), std::move(E));
10671097

10681098
SmallVector<object::BuildIDRef> BinaryIDsToFetch;
10691099
if (!ProfileBinaryIDs.empty()) {
@@ -1083,12 +1113,12 @@ Expected<std::unique_ptr<CoverageMapping>> CoverageMapping::load(
10831113
if (PathOpt) {
10841114
std::string Path = std::move(*PathOpt);
10851115
StringRef Arch = Arches.size() == 1 ? Arches.front() : StringRef();
1086-
if (Error E = loadFromFile(Path, Arch, CompilationDir, *ProfileReader,
1087-
*Coverage, DataFound))
1116+
if (Error E = loadFromFile(Path, Arch, CompilationDir, ProfileReaderRef,
1117+
*Coverage, DataFound))
10881118
return std::move(E);
10891119
} else if (CheckBinaryIDs) {
10901120
return createFileError(
1091-
ProfileFilename,
1121+
ProfileFilename.value(),
10921122
createStringError(errc::no_such_file_or_directory,
10931123
"Missing binary ID: " +
10941124
llvm::toHex(BinaryID, /*LowerCase=*/true)));

‎llvm/test/tools/llvm-cov/showLineExecutionCounts-lcov-baseline.test‎

Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,37 @@
1+
// FULL: SF:{{.*}}showLineExecutionCounts.cpp
2+
// FULL: FN:6,main
3+
// FULL: FNDA:0,main
4+
// FULL: FNF:1
5+
// FULL: FNH:0
6+
int main() { // FULL: DA:[[@LINE]],0
7+
int x = 0; // FULL: DA:[[@LINE]],0
8+
// FULL: DA:[[@LINE]],0
9+
if (x) { // FULL: DA:[[@LINE]],0
10+
x = 0; // FULL: DA:[[@LINE]],0
11+
} else { // FULL: DA:[[@LINE]],0
12+
x = 1; // FULL: DA:[[@LINE]],0
13+
} // FULL: DA:[[@LINE]],0
14+
// FULL: DA:[[@LINE]],0
15+
for (int i = 0; i < 100; ++i) { // FULL: DA:[[@LINE]],0
16+
x = 1; // FULL: DA:[[@LINE]],0
17+
} // FULL: DA:[[@LINE]],0
18+
// FULL: DA:[[@LINE]],0
19+
x = x < 10 ? x + 1 : x - 1; // FULL: DA:[[@LINE]],0
20+
x = x > 10 ? // FULL: DA:[[@LINE]],0
21+
x - 1: // FULL: DA:[[@LINE]],0
22+
x + 1; // FULL: DA:[[@LINE]],0
23+
// FULL: DA:[[@LINE]],0
24+
return 0; // FULL: DA:[[@LINE]],0
25+
} // FULL: DA:[[@LINE]],0
26+
// FULL: LF:20
27+
// FULL: LH:0
28+
// FULL: end_of_record
29+
// RUN: llvm-cov export -format=lcov %S/Inputs/lineExecutionCounts.covmapping %s | FileCheck -check-prefixes=FULL %s
30+
31+
// RUN: llvm-cov export -format=lcov -summary-only %S/Inputs/lineExecutionCounts.covmapping %s | FileCheck -check-prefixes=SUMMARYONLY %s
32+
// SUMMARYONLY: SF:{{.*}}showLineExecutionCounts.cpp
33+
// SUMMARYONLY: FNF:1
34+
// SUMMARYONLY: FNH:0
35+
// SUMMARYONLY: LF:20
36+
// SUMMARYONLY: LH:0
37+
// SUMMARYONLY: end_of_record

0 commit comments

Comments
 (0)