Skip to content

Sort input data #414

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Jun 29, 2025
Merged

Sort input data #414

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
172b618
Sort input files during data capture and/or aggregation - so certain …
henry2cox Jun 12, 2025
ecb02ec
Portability: tests should run successfully (if incompletely) with ol…
henry2cox Jun 12, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
42 changes: 29 additions & 13 deletions bin/genhtml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6817,7 +6817,7 @@ our $hierarchical = 0; # if true: show directory hierarchy
# default: legacy two-level report
our $flat = 0; # if true: single table of all files in top level

our $sort = 1; # If set, provide directory listings with sorted entries
our $sort_tables = 1; # If set, provide directory listings with sorted entries
our $no_sort; # Disable sort
our $frames; # If set, use frames for source code view
our $keep_descriptions; # If set, do not remove unused test case descriptions
Expand Down Expand Up @@ -6899,7 +6899,7 @@ my %genhtml_rc_opts = (
"genhtml_mcdc_field_width" => \$mcdc_field_width,
"genhtml_owner_field_width" => \$owner_field_width,
"genhtml_age_field_width" => \$age_field_width,
"genhtml_sort" => \$sort,
"genhtml_sort" => \$sort_tables,
"genhtml_charset" => \$charset,
"genhtml_desc_html" => \$rc_desc_html,
'merge_function_aliases' => \$merge_function_aliases,
Expand Down Expand Up @@ -6975,7 +6975,7 @@ my %genhtml_options = ("output-directory|o=s" => \$output_directory,
"html-gzip" => \$html_gzip,
"hierarchical" => \$hierarchical,
"flat" => \$flat,
"sort" => \$sort,
"sort-tables" => \$sort_tables,
"no-sort" => \$no_sort,
"precision=i" => \$lcovutil::default_precision,
"missed" => \$opt_missed,
Expand All @@ -6985,6 +6985,15 @@ my %genhtml_options = ("output-directory|o=s" => \$output_directory,
"merge-aliases" => \$merge_function_aliases,
"suppress-aliases" => \$suppress_function_aliases,
'validate' => \$validateHTML,);

# remove ambiguous entry from common table -
# (genhtml has '--sort-inputs' and '--sort-tables')
# handle the (no obsolete) '--sort' option
Getopt::Long::Configure("pass_through", "no_auto_abbrev");
my $obsoleteSort;
Getopt::Long::GetOptions('sort' => \$obsoleteSort);
Getopt::Long::Configure('default');

# Parse command line options
if (
!lcovutil::parseOptions(\%genhtml_rc_opts, \%genhtml_options,
Expand All @@ -6994,6 +7003,13 @@ if (
exit(1);
}

if (defined($obsoleteSort)) {
$sort_tables = $obsoleteSort;
lcovutil::ignorable_warning($lcovutil::ERROR_DEPRECATED,
"option '--sort' is deprecated and will be removed in a future release; please use '--sort-tables' instead."
);
}

$merge_function_aliases = 1
if ($suppress_function_aliases ||
defined($lcovutil::cov_filter[$lcovutil::FILTER_FUNCTION_ALIAS]));
Expand Down Expand Up @@ -7037,7 +7053,7 @@ if (defined($lcovutil::stop_on_error) &&
}

# Merge sort options
$sort = 0
$sort_tables = 0
if ($no_sort);
die(
"unsupported use of mutually exclusive '--flat' and '--hierachical' options")
Expand Down Expand Up @@ -7244,7 +7260,7 @@ if ($no_prefix && @dir_prefix) {
@fileview_sortlist = ($SORT_FILE);
@funcview_sortlist = ($SORT_FILE);

if ($sort) {
if ($sort_tables) {
push(@fileview_sortlist, $SORT_LINE);
push(@fileview_sortlist, $SORT_FUNC) if ($lcovutil::func_coverage);
push(@fileview_sortlist, $SORT_BRANCH) if ($lcovutil::br_coverage);
Expand Down Expand Up @@ -8659,7 +8675,7 @@ sub write_png_files()
0x45, 0x4e, 0x44, 0xae, 0x42, 0x60, 0x82
];

if ($sort) {
if ($sort_tables) {
$data{"updown.png"} =
$dark_mode ?
[0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a, 0x00, 0x00,
Expand Down Expand Up @@ -11960,7 +11976,7 @@ sub write_header(*$$$$$$$)
%$differentialFunctionMap) {
$view .= "<span style=\"font-size: 80%;\">";
if ($type == $HDR_SOURCE) {
my $suffix = $sort ? '-c' : '';
my $suffix = $sort_tables ? '-c' : '';
$view .=
" (source / <a href=\"$base_name.func$suffix.$html_ext\" title=\"Click to go to function table\">functions</a>)";
} elsif ($type == $HDR_FUNC) {
Expand Down Expand Up @@ -12652,7 +12668,7 @@ sub write_file_table(*$$$$$$)
}
my $file_col_title = ($fileview || $main::flat) ? 'File' : 'Directory';
# don't insert the 'sort' controls if there is just a single source file
my $use_sort_button = $sort && 1 < scalar($dirSummary->sources());
my $use_sort_button = $sort_tables && 1 < scalar($dirSummary->sources());
my $file_code =
get_file_code($hide, $file_col_title,
$use_sort_button && $sort_type != $SORT_FILE,
Expand Down Expand Up @@ -13372,23 +13388,23 @@ sub funcview_get_label($$$$)
my $link;

if (!defined($col)) {
if ($sort && $sort_type != $SORT_FILE) {
if ($sort_tables && $sort_type != $SORT_FILE) {
$link = "$name.func.$html_ext";
}
return "Function Name" . get_sort_code($link, "function name", $base);
} elsif ($col eq 'hit') {
if ($sort && $sort_type != $SORT_LINE) {
if ($sort_tables && $sort_type != $SORT_LINE) {
$link = "$name.func-c.$html_ext";
}
return "Hit count" . get_sort_code($link, "function hit count", $base);
} elsif ($col eq 'missed_line') {
if ($sort && $sort_type != $SORT_MISSING_LINE) {
if ($sort_tables && $sort_type != $SORT_MISSING_LINE) {
$link = "$name.func-l.$html_ext";
}
return "Lines" .
get_sort_code($link, "unexercised lines in function", $base);
} elsif ($col eq 'missed_mcdc') {
if ($sort && $sort_type != $SORT_MISSING_MCDC) {
if ($sort_tables && $sort_type != $SORT_MISSING_MCDC) {
$link = "$name.func-m.$html_ext";
}
return "MC/DC"
Expand All @@ -13397,7 +13413,7 @@ sub funcview_get_label($$$$)
$base);
} else {
die("unexpected sort $col") unless ($col eq 'missed_branch');
if ($sort && $sort_type != $SORT_MISSING_BRANCH) {
if ($sort_tables && $sort_type != $SORT_MISSING_BRANCH) {
$link = "$name.func-b.$html_ext";
}
return "Branches" .
Expand Down
15 changes: 13 additions & 2 deletions bin/geninfo
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1288,7 +1288,18 @@ sub gen_info(@)
$builder->find_files(2, @_);
}
my $filelist = $builder->worklist();
my $total = scalar(@$filelist);
my @sorted_filelist;
if ($lcovutil::sort_inputs) {
@sorted_filelist = sort({
my $na = $a->[0] .
(defined($a->[1]) ? $a->[1] : $a->[2]);
my $nb = $b->[0] .
(defined($b->[1]) ? $b->[1] : $b->[2]);
$na cmp $nb
} @$filelist);
$filelist = \@sorted_filelist;
}
my $total = scalar(@$filelist);
if (1 < $lcovutil::maxParallelism) {
my $floor =
$lcovutil::maxParallelism ?
Expand Down Expand Up @@ -1326,7 +1337,7 @@ sub gen_info(@)
FILE: foreach my $j (@$filelist) {
my ($dir, $gcda, $gcno) = @$j;
foreach my $f ($gcda, $gcno) {
next unless defined($f); # might not be a GCDA file
next unless defined($f); # might not be a GCDA file
my $filename = $dir . $lcovutil::dirseparator . $f;
if (grep({ $filename =~ $_ } @main::large_files)) {
lcovutil::info(1, "large file: $filename\n");
Expand Down
5 changes: 5 additions & 0 deletions lcovrc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -490,6 +490,11 @@ lcov_json_module = auto
# directives, etc
#case_insensitive = 0

# sort file names before capture and/or aggregation - to reduce certain
# types of processing order dependencies which can affect coverage results.
# see the lcovrc man page for more details.
#sort_input = 1

# override line default line exclusion regexp
#lcov_excl_line = LCOV_EXCL_LINE

Expand Down
18 changes: 14 additions & 4 deletions lib/lcovutil.pm
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -117,6 +117,7 @@ our $lcov_url = "https://github.com//linux-test-project/lcov";
our @temp_dirs;
our $tmp_dir = '/tmp'; # where to put temporary/intermediate files
our $preserve_intermediates; # this is useful only for debugging
our $sort_inputs; # sort input file lists - to reduce unpredictability
our $devnull = File::Spec->devnull(); # portable way to do it
our $dirseparator = ($^O =~ /Win/) ? '\\' : '/';
our $interp = ($^O =~ /Win/) ? $^X : undef;
Expand Down Expand Up @@ -1126,7 +1127,8 @@ my %rc_common = (
"demangle_cpp" => \@lcovutil::cpp_demangle,
'excessive_count_threshold' => \$excessive_count_threshold,

,
'sort_input' => \$lcovutil::sort_inputs,

"fail_under_lines" => \$fail_under_lines,
"fail_under_branches" => \$fail_under_branches,
'lcov_filter_parallel' => \$lcovutil::lcov_filter_parallel,
Expand Down Expand Up @@ -1214,7 +1216,8 @@ our %argCommon = ("tempdir=s" => \$tempdirname,
"parallel|j:i" => \$lcovutil::maxParallelism,
"memory=i" => \$lcovutil::maxMemory,
"forget-test-names" => \$TraceFile::ignore_testcase_name,
"preserve" => \$lcovutil::preserve_intermediates,);
"preserve" => \$lcovutil::preserve_intermediates,
'sort-input' => \$lcovutil::sort_inputs,);

sub warnDeprecated
{
Expand Down Expand Up @@ -9248,6 +9251,13 @@ sub merge
$lcovutil::maxParallelism = $num;
}
}
# use a particular file sort order - to somewhat minimize order effects
my $filelist = \@_;
my @sorted_filelist;
if ($lcovutil::sort_inputs) {
@sorted_filelist = sort({ $a cmp $b } @_);
$filelist = \@sorted_filelist;
}

if (1 != $lcovutil::maxParallelism &&
(exists($ENV{LCOV_FORCE_PARALLEL}) ||
Expand Down Expand Up @@ -9277,7 +9287,7 @@ sub merge
$lcovutil::maxParallelism) :
1;
my $idx = 0;
foreach my $tracefile (@_) {
foreach my $tracefile (@$filelist) {
my $seg = $idx / $testsPerSegment;
$seg -= 1 if $seg == $lcovutil::maxParallelism;
push(@segments, [])
Expand Down Expand Up @@ -9524,7 +9534,7 @@ sub merge
} while (@segments);
} else {
# sequential
@effective = _process_segment($total_trace, $readSourceFile, \@_);
@effective = _process_segment($total_trace, $readSourceFile, $filelist);
}
if (defined($lcovutil::tempdirname) &&
!$lcovutil::preserve_intermediates) {
Expand Down
31 changes: 27 additions & 4 deletions man/genhtml.1
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,7 @@ genhtml \- Generate HTML view from LCOV coverage data files
.IR extension ]
.br
.RB [ \-\-html\-gzip ]
.RB [ \-\-sort ]
.RB [ \-\-sort\-tables ]
.RB [ \-\-no\-sort ]
.br
.RB [ \-\-function\-coverage ]
Expand Down Expand Up @@ -213,6 +213,8 @@ genhtml \- Generate HTML view from LCOV coverage data files
.br
.RB [ \-\-save ]
.br
.RB [ \-\-sort\-input ]
.br
.RB [ \-\-serialize
.IR serialize_output_file ]
.br
Expand Down Expand Up @@ -2671,14 +2673,22 @@ option
.IR genhtml_html_gzip .

.RE
.B \-\-sort
.B \-\-sort-tables
.br
.B \-\-no\-sort
.RS
Specify whether to include sorted views of file and directory overviews.

This option replaces the deprecated
.I "\-\-sort"
option.
.I "\-\-sort"
is still supported and is treated as an alias of
.I "\-\-sort\-tables"
but will be removed in a subsequent LCOV release.

Use
.B \-\-sort
.B \-\-sort-tables
to include sorted views or
.B \-\-no\-sort
to not include them.
Expand Down Expand Up @@ -3356,7 +3366,7 @@ Show counts of missed lines, functions, branches, and MC/DC expressions.
Use this option to change overview pages to show the count of lines, functions,
branches, or MC/DC expressions that were not hit. These counts are represented by negative numbers.

When specified together with \-\-sort, file and directory views will be sorted
When specified together with \-\-sort\-tables, file and directory views will be sorted
by missed counts.

This option can also be configured permanently using the configuration file
Expand Down Expand Up @@ -3397,6 +3407,19 @@ Keeping copies of the input data files may help to debug any issues or to regene

.RE

.B \-\-sort\-input
.br
.RS
Specify whether to sort file names before capture and/or aggregation.
Sorting reduces certain types of processing order-dependent output differences.
See the
.BI sort_input
section in
man
.B lcovrc(5).

.RE

.BI "\-\-serialize " file_name
.RS
Save coverage database to
Expand Down
17 changes: 17 additions & 0 deletions man/geninfo.1
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -79,6 +79,8 @@ geninfo \- Generate tracefiles from GCOV coverage data files
.RB [ \-\-external ]
.RB [ \-\-no\-external ]
.br
.RB [ \-\-sort\-input ]
.br
.RB [ \-\-config\-file
.IR config\-file ]
.RB [ \-\-no\-markers ]
Expand Down Expand Up @@ -677,6 +679,21 @@ config file option. See man
.B lcovrc(5) for more information.
.RE

.RE

.B \-\-sort\-input
.br
.RS
Specify whether to sort file names before capture and/or aggregation.
Sorting reduces certain types of processing order-dependent output differences.
See the
.BI sort_input
section in
man
.B lcovrc(5).

.RE

.B \-\-gcov\-tool
.I tool
.br
Expand Down
17 changes: 17 additions & 0 deletions man/lcov.1
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,8 @@ format is described in man
.RB [ \-\-no\-recursion ]
.RB [ \-f | \-\-follow ]
.br
.RB [ \-\-sort\-input ]
.br
.RB [ \-\-compat\-libtool ]
.RB [ \-\-no\-compat\-libtool ]
.br
Expand Down Expand Up @@ -159,6 +161,8 @@ Aggregate multiple coverage tracefiles into one:
.RB [ \-\-checksum ]
.RB [ \-\-no\-checksum ]
.br
.RB [ \-\-sort\-input ]
.br
.RE

Depending on your use model, it may not be necessary to create aggregate coverage data files.
Expand Down Expand Up @@ -1213,6 +1217,19 @@ want to perform the .info file creation on the build machine. See
\-\-to\-package for more information.
.RE

.B \-\-sort\-input
.br
.RS
Specify whether to sort file names before capture and/or aggregation.
Sorting reduces certain types of processing order-dependent output differences.
See the
.BI sort_input
section in
man
.B lcovrc(5).

.RE

.B \-\-gcov\-tool
.I tool
.br
Expand Down
Loading