Skip to content

[CodeceptJS] Allure marks all test failures as "Broken" #1404

@hxllv

Description

@hxllv

Describe the bug
Allure will mark legit failures (that should be Failed) as Broken. It happens with all types of failures e.g. missing element for click(), timeout for waitForVisible() etc.

To Reproduce
Steps to reproduce the behavior:

Codecept config:

const { setHeadlessWhen, setCommonPlugins } = require('@codeceptjs/configure')
// turn on headless mode when running with HEADLESS=true environment variable
// export HEADLESS=true && npx codeceptjs run
setHeadlessWhen(process.env.HEADLESS)

// enable all common plugins https://github.com/codeceptjs/configure#setcommonplugins
setCommonPlugins()

/** @type {CodeceptJS.MainConfig} */
exports.config = {
	tests: './*_test.js',
	output: './output',
	helpers: {
		Playwright: {
			browser: 'chromium',
			url: 'http://google.com',
			show: true,
			restart: true
		},
		FileSystem: {}
	},
	plugins: {
		screenshotOnFail: {
			enabled: true
		},
		allure: {
			enabled: true,
			require: 'allure-codeceptjs'
		}
	},
	name: 'ccjs'
}

Scenario:

Feature('test')

Scenario('test something', async ({ I }) => {
	I.amOnPage('https://codecept.io/quickstart/')

	I.fillField('#algolia-search-input', 'codecept')
	I.pressKey('Enter')
	I.see('CodeceptJS')
	I.seeCheckboxIsChecked('.nocheckboxishere')
})

Afterwards run allure serve and the issue can be observed

Expected behavior
Expected behavior would be to mark these scenarios as Failed

Screenshots

Image

Desktop:

  • OS: example on macOS 26.2, happens also on GitHub actions

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions