Skip to content

CICD is not failing when listener is failing #31

@Tattoo

Description

@Tattoo

The CICD currently relies on Robot Framework exit code to know if the test execution failed or not. However, in cases where the Oxygen listener fails, but the test execution otherwise passes, RF will return 0 and the CICD passes.

For example:

==============================================================================
Atest                                                                 | PASS |
10 critical tests, 10 passed, 0 failed
10 tests total, 10 passed, 0 failed
==============================================================================
[ ERROR ] Calling method 'output_file' of listener 'oxygen.listener' failed: MismatchArgumentException: parse_results expects 3 arguments but got 1
Output:  /[...]/output.xml
Log:     /[...]/log.html
Report:  /[...]/report.html

[python38][robotframework==3.2] Exited with 0

Since Oxygen is, in part, a listener, we need CICD to detect that listener has failed during the test execution

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions