Advisory UI tests by vobratil · Pull Request #64 · trustification/trustify-tests

vobratil · 2025-07-07T15:54:03Z

No description provided.

…nd some CSS.

Copilot

Pull Request Overview

This PR adds comprehensive UI testing capabilities for advisory exploration functionality, including search, download, and display features for security advisories.

Adds step definitions for advisory-specific UI tests including search, download, and verification of UI elements
Creates new helper methods in DetailsPage for button and panel visibility verification
Introduces BDD-style feature tests with multiple scenarios covering advisory search, display, and interaction

Reviewed Changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

File	Description
tests/ui/steps/details-page.ts	Adds new step definitions for button/panel verification and file download testing
tests/ui/helpers/DetailsPage.ts	Extends DetailsPage class with button clicking and element visibility verification methods
tests/ui/features/@advisory-explorer/advisory-explorer.step.ts	Creates advisory-specific step definitions for search and table verification
tests/ui/features/@advisory-explorer/advisory-explorer.feature	Defines BDD feature scenarios for advisory exploration functionality

Copilot · 2025-08-11T16:03:28Z

tests/ui/steps/details-page.ts

+  "File with the name {string} is downloaded",
+  async ({ page }, expectedFilename) => {
+    const downloadPromise = page.waitForEvent("download");


The download promise is created before the download action is triggered. This should be set up before the user action that triggers the download, but here it appears to be waiting for a download that may have already been initiated by a previous step.

Suggested change

"File with the name {string} is downloaded",

async ({ page }, expectedFilename) => {

const downloadPromise = page.waitForEvent("download");

"User downloads the file {string} by clicking the {string} button",

async ({ page }, expectedFilename, buttonName) => {

const downloadPromise = page.waitForEvent("download");

const detailsPage = new DetailsPage(page);

await detailsPage.clickOnPageButton(buttonName);

Copilot · 2025-08-11T16:03:28Z

tests/ui/helpers/DetailsPage.ts

+      this.page
+        .locator(".pf-v6-c-card__title-text")
+        .filter({ hasText: panel })
+        .first()


Using a specific CSS class selector .pf-v6-c-card__title-text makes the test brittle to UI framework changes. Consider using a more semantic selector like getByRole('heading') or adding a test-specific data attribute.

Suggested change

this.page

.locator(".pf-v6-c-card__title-text")

.filter({ hasText: panel })

.first()

this.page.getByRole("heading", { name: panel })

matejnesuta · 2025-08-11T16:33:07Z

So, I took a look and there is still a feature file, which seems to be an older version of the feature file, which is newly added in this PR.

We can keep the old file if it has additional tests, which are not implemented in this PR, but I think duplicate scenarios should be cleared from the older file to prevent confusion.

matejnesuta · 2025-08-11T17:11:58Z

I ran the tests against an environment, which has about 500 000 advisories in it and this was the result:

3) [firefox] › tests/ui/features/@advisory-explorer/advisory-explorer.feature.spec.js:61:9 › Advisory Explorer › Display the Vulnerabilities tab › Example #1 › When User visits Advisory details Page of "CVE-2023-3223"

Error: locator.click: Error: strict mode violation: getByRole('link', { name: 'CVE-2023-3223' }) resolved to 10 elements:
    1) <a class="" href="/advisories/urn:uuid:194fb255-a572-47fb-912d-bd9b3f039953">CVE-2023-32233</a> aka getByRole('link', { name: 'CVE-2023-32233' })
    2) <a class="" href="/advisories/urn:uuid:34f16f7c-93cd-4587-8458-3859d0d005e7">CVE-2023-32235</a> aka getByRole('link', { name: 'CVE-2023-32235' })
    3) <a class="" href="/advisories/urn:uuid:d6ddcef5-e8b1-40da-846f-ac1fd8511e18">CVE-2023-32232</a> aka getByRole('link', { name: 'CVE-2023-32232' })
    4) <a class="" href="/advisories/urn:uuid:14666690-94ad-47d2-913f-d982476c1711">CVE-2023-32231</a> aka getByRole('link', { name: 'CVE-2023-32231' })
    5) <a class="" href="/advisories/urn:uuid:ffe4e183-19fa-4a07-933a-8d2144a8d50e">CVE-2023-32239</a> aka getByRole('link', { name: 'CVE-2023-32239' })
    6) <a class="" href="/advisories/urn:uuid:f8f4d6bd-c7b3-492a-99a9-52f31053a2d0">CVE-2023-32236</a> aka getByRole('link', { name: 'CVE-2023-32236' })
    7) <a class="" href="/advisories/urn:uuid:0e5e5422-7ff6-4fc1-92f6-8d6fad3a61ff">CVE-2023-32237</a> aka getByRole('link', { name: 'CVE-2023-32237' })
    8) <a class="" href="/advisories/urn:uuid:178c9622-9cd0-4edf-a579-034c5e6099c6">CVE-2023-32230</a> aka getByRole('link', { name: 'CVE-2023-32230' })
    9) <a class="" href="/advisories/urn:uuid:2ebfb568-6115-4ad8-8212-27f330dde7e4">CVE-2023-3223</a> aka getByRole('row', { name: 'CVE-2023-3223 Undertow: outofmemoryerror due to @multipartconfig handling cve' }).getByRole('link')
    10) <a class="" href="/advisories/urn:uuid:525eb3be-6e4d-47a2-89ba-f07255ffa35b">CVE-2023-3223</a> aka getByRole('row', { name: 'CVE-2023-3223 undertow: OutOfMemoryError due to @MultipartConfig handling csaf' }).getByRole('link')

Call log:
  - waiting for getByRole('link', { name: 'CVE-2023-3223' })


   at ../tests/ui/features/@advisory-explorer/advisory-explorer.step.ts:15

  13 |     const searchPage = new SearchPage(page, "Advisories");
  14 |     await searchPage.dedicatedSearch(advisoryID);
> 15 |     await page.getByRole("link", { name: advisoryID }).click();
     |                                                        ^
  16 |   }
  17 | );
  18 |
    at Object.<anonymous> (/home/nesuta/work/trustify-tests/tests/ui/features/@advisory-explorer/advisory-explorer.step.ts:15:56)
    at /home/nesuta/work/trustify-tests/.features-gen/tests/ui/features/@advisory-explorer/advisory-explorer.feature.spec.js:62:7`

I think a more strict selector is needed in this case.

matejnesuta · 2025-08-11T17:23:05Z

Also, when searching an advisory, another strict mode violation is present. This is most likely due to new version of TPA.
1) [firefox] › tests/ui/features/@advisory-explorer/advisory-explorer.feature.spec.js:12:9 › Advisory Explorer › Search for an advisory using the general search bar › Example #1 › Then The advisory "CVE-2024-26308" shows in the results

Error: expect.toBeVisible: Error: strict mode violation: getByRole('gridcell').filter({ hasText: 'CVE-2024-26308' }) resolved to 2 elements:
    1) <td tabindex="-1" data-label="ID" class="pf-v6-c-table__td pf-m-break-word pf-m-width-15">…</td> aka getByRole('gridcell', { name: 'CVE-2024-' })
    2) <td tabindex="-1" data-label="Labels" class="pf-v6-c-table__td pf-m-width-20">…</td> aka getByRole('gridcell', { name: 'Label group category' })

Call log:
  - expect.toBeVisible with timeout 5000ms
  - waiting for getByRole('gridcell').filter({ hasText: 'CVE-2024-26308' })


   at ../tests/ui/features/@advisory-explorer/advisory-explorer.step.ts:41

  39 |     await expect(
  40 |       page.getByRole("gridcell").filter({ hasText: advisoryID })
> 41 |     ).toBeVisible();
     |       ^
  42 |   }
  43 | );
  44 |
    at Object.<anonymous> (/home/nesuta/work/trustify-tests/tests/ui/features/@advisory-explorer/advisory-explorer.step.ts:41:7)
    at /home/nesuta/work/trustify-tests/.features-gen/tests/ui/features/@advisory-explorer/advisory-explorer.feature.spec.js:14:7

2) [firefox] › tests/ui/features/@advisory-explorer/advisory-explorer.feature.spec.js:21:9 › Advisory Explorer › Search for an advisory using the dedicated search bar › Example #1 › Then The advisory "CVE-2024-26308" shows in the results

Error: expect.toBeVisible: Error: strict mode violation: getByRole('gridcell').filter({ hasText: 'CVE-2024-26308' }) resolved to 2 elements:
    1) <td tabindex="-1" data-label="ID" class="pf-v6-c-table__td pf-m-break-word pf-m-width-15">…</td> aka getByRole('gridcell', { name: 'CVE-2024-' })
    2) <td tabindex="-1" data-label="Labels" class="pf-v6-c-table__td pf-m-width-20">…</td> aka getByRole('gridcell', { name: 'Label group category' })

Call log:
  - expect.toBeVisible with timeout 5000ms
  - waiting for getByRole('gridcell').filter({ hasText: 'CVE-2024-26308' })


   at ../tests/ui/features/@advisory-explorer/advisory-explorer.step.ts:41

  39 |     await expect(
  40 |       page.getByRole("gridcell").filter({ hasText: advisoryID })
> 41 |     ).toBeVisible();
     |       ^
  42 |   }
  43 | );
  44 |
    at Object.<anonymous> (/home/nesuta/work/trustify-tests/tests/ui/features/@advisory-explorer/advisory-explorer.step.ts:41:7)
    at /home/nesuta/work/trustify-tests/.features-gen/tests/ui/features/@advisory-explorer/advisory-explorer.feature.spec.js:23:7

`

matejnesuta

See my comments.

vobratil force-pushed the advisory-ui-tests branch from 5a684b9 to f3f84b9 Compare July 29, 2025 09:06

Added Advisory Explorer and Advisory Search tests.

7cf9fbb

vobratil force-pushed the advisory-ui-tests branch from f3f84b9 to 7cf9fbb Compare July 29, 2025 09:17

Resolved conflicts with main branch.

0129ee4

vobratil force-pushed the advisory-ui-tests branch 4 times, most recently from d5def38 to a510274 Compare July 29, 2025 12:31

Fixed some references to dedicatedSearch in Advisory Explorer steps a…

048b804

…nd some CSS.

vobratil force-pushed the advisory-ui-tests branch from a510274 to 048b804 Compare July 29, 2025 13:08

Changed some of the data used in Advisory Explorer tests.

34097a4

vobratil marked this pull request as ready for review July 30, 2025 11:22

vobratil requested review from matejnesuta, mrrajan and queria July 30, 2025 11:23

matejnesuta requested a review from Copilot August 11, 2025 16:02

Copilot AI reviewed Aug 11, 2025

View reviewed changes

matejnesuta suggested changes Aug 11, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Comments

Advisory UI tests#64

Advisory UI tests#64
vobratil wants to merge 4 commits intotrustification:mainfrom
vobratil:advisory-ui-tests

vobratil commented Jul 7, 2025

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Aug 11, 2025

Uh oh!

Copilot AI Aug 11, 2025

Uh oh!

matejnesuta commented Aug 11, 2025 •

edited

Loading

Uh oh!

matejnesuta commented Aug 11, 2025 •

edited

Loading

Uh oh!

matejnesuta commented Aug 11, 2025

Uh oh!

matejnesuta left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Comments

Conversation

vobratil commented Jul 7, 2025

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Copilot AI Aug 11, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Aug 11, 2025

Choose a reason for hiding this comment

Uh oh!

matejnesuta commented Aug 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

matejnesuta commented Aug 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

matejnesuta commented Aug 11, 2025

Uh oh!

matejnesuta left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

matejnesuta commented Aug 11, 2025 •

edited

Loading

matejnesuta commented Aug 11, 2025 •

edited

Loading