Skip to content

kola.json is not used on external tests #4392

@sulfin

Description

@sulfin

Bug Report

Environment

What operating system is being used to run coreos-assembler?
Fedora 43

What operating system is being assembled?
fedora coreos

Is coreos-assembler running in Podman or Docker?
podman

If Podman, is coreos-assembler running privileged or unprivileged?
rootless with --privileged

Expected Behavior

When defining external tests for kola, the file kola.json should be used as default metadata for all tests in the test folder.

Actual Behavior

Only in binary metadatas (json and yaml) are used and kola.json is parsed, but never used

Reproduction Steps

  1. Create an external test. Put metadata in the kola.json file
  2. Run kola list -E /path/to/project
  3. On the ext.* tests, the metadata are not used

This procedure is for quick check, running or listing the tests is the same. Using label is a quick way to check.

Other Information

After reading the code I found the issue. in mantle/kola/harness.go, the function registerExternalTest will check if the binary have metadata in the binary using the metadataFromTestBinary function. If the result is nil, then kola.json is used, if not, the in binary metadata are used.

The commit 119bc7f that introduced the exclusive metadata, modified the function metadataFromTestBinary by initializing the meta variable with this expression :

meta := &externalTestMeta{Exclusive: true}

Causing the function to never return a nil value, and thus kola.json will never be used.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions