Skip to content

xfstests: get XFSTESTS_BLACKLIST from file #25023

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
lansuse wants to merge 1 commit into
base: master
Choose a base branch
from
Open

xfstests: get XFSTESTS_BLACKLIST from file #25023

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 23 additions & 0 deletions lib/filesystem_utils.pm
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,8 @@ use utils;
use testapi;

use Mojo::JSON 'decode_json';
use Mojo::UserAgent;
use YAML::PP;

our @EXPORT = qw(
str_to_mb
Expand All @@ -35,6 +37,7 @@ our @EXPORT = qw(
validate_lsblk
get_partition_table_via_blkid
is_lsblk_able_to_display_mountpoints
fetch_xfstests_blacklist_from_url
generate_xfstests_list);

=head2 str_to_mb
Expand Down Expand Up @@ -513,6 +516,26 @@ sub validate_lsblk {
return $errors;
}

=head2 fetch_xfstests_blacklist_from_url

Fetch the xfstests blacklist YAML file from a specified URL.

Returns hash mapping of the YAML.

=cut

sub fetch_xfstests_blacklist_from_url {
my $url = shift;
my $res = Mojo::UserAgent->new->get($url)->result;
unless ($res->is_success) {
record_info("$url not available.", $res->message, result => 'softfail');
return;
}
my $body = $res->body;
my $yp = YAML::PP->new;
return $yp->load_string($body);
}

=head2

Generate xfstests subtests list
Expand Down
10 changes: 8 additions & 2 deletions tests/xfstests/run.pm
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ use utils;
use Utils::Backends 'is_pvm';
use serial_terminal 'select_serial_terminal';
use power_action_utils qw(power_action prepare_system_shutdown);
use filesystem_utils qw(format_partition generate_xfstests_list);
use filesystem_utils qw(format_partition fetch_xfstests_blacklist_from_url generate_xfstests_list);
use lockapi;
use mmapi;
use version_utils 'is_public_cloud';
Expand Down Expand Up @@ -118,11 +118,17 @@ sub run {
}

# Generate xfstests blacklist
my %black_list = (generate_xfstests_list($BLACKLIST), exclude_grouplist($TEST_RANGES, $GROUPLIST, $FSTYPE));
my $whitelist;
if (my $issues = get_var('XFSTESTS_KNOWN_ISSUES')) {
$whitelist = LTP::WhiteList->new($issues);
}
my ($yaml, $blist);
if ($BLACKLIST =~ m{^https?://}i) {
$yaml = fetch_xfstests_blacklist_from_url($BLACKLIST);
$blist = join(',', map { $_->{items} } @{$yaml->{$TEST_SUITE}{xfstests_blacklist}}) if grep { defined $_->{items} } @{$yaml->{$TEST_SUITE}{xfstests_blacklist}};
$BLACKLIST = $blist;
}
my %black_list = (generate_xfstests_list($BLACKLIST), exclude_grouplist($TEST_RANGES, $GROUPLIST, $FSTYPE));

my $subtest_num = scalar @tests;
foreach my $index (0 .. $#tests) {
Expand Down
2 changes: 1 addition & 1 deletion variables.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -436,7 +436,7 @@ Variable | Type | Default value | Details
XFSTESTS | string | --- | Determines the action to be taken. If set to "installation", it runs xfstests installation and then shuts down. If set to <FS_TYPE> (e.g., "btrfs," "xfs," or "ext4"), the test will be executed on the specified file system.
XFSTESTS_RANGES | string | | sub-tests ranges. This setting is mandatory. Support using "-" to define a range, or use "," to list separate subtests(e.g. xfs/001-999,generic/001). But the final test range will also count subtests defined in XFSTESTS_GROUPLIST, a skill to set subtests only by XFSTESTS_GROUPLIST is to set a minimal XFSTESTS_RANGES with XFSTESTS_GROUPLIST
NO_SHUFFLE | boolean | 0 | the default sequence to run all subtests is a random sequence, it's designed to reduce the influence between each subtest. Set NO_SHUFFLE=1 to run in order
XFSTESTS_BLACKLIST | string | | set the sub-tests will not run. Mostly use in the feature not supported, and exclude some critical issues to make whole tests stable. The final skip test list will also count those defined in XFSTESTS_GROUPLIST. It's also support "-" and "," to set skip range
XFSTESTS_BLACKLIST | string | | set the sub-tests will not run. Mostly use in the feature not supported, and exclude some critical issues to make whole tests stable. The final skip test list will also count those defined in XFSTESTS_GROUPLIST. It's also support "-" and "," to set skip range. Or set the blacklist file url if starts with http:// or https://. For instance: XFSTESTS_BLACKLIST=http://example.com/blacklist.yaml
XFSTESTS_GROUPLIST | string | | it's an efficient way to set XFSTESTS_RANGES. Most likely use in test whole range in a single test, such as test special mount option. The range is supported in xfstests upstream, to know the whole range of group names could take a look at xfstests upstream README file. This parameter in openqa supports not only "include" tests, but also "exclude" tests. To add a "!" before a group name to exclude all subtests in that group. Here is an example: e.g. XFSTESTS_GROUPLIST=quick,!fuzz,!fuzzers,!realtime (Add all subtests in quick group, and exclude all dangerous subtests in fuzz, fuzzers, realtime groups)
XFSTESTS_KNOWN_ISSUES | string | | Used to specify a url for a json file with well known xfstests issues. If an error occur which is listed, then the result is overwritten with softfailure.

Expand Down
Loading