Skip to content

Conversation

@ShaileshParmar11
Copy link
Contributor

@ShaileshParmar11 ShaileshParmar11 commented Jan 27, 2026

This pull request introduces a new GitHub Actions workflow to automate the generation and updating of Playwright E2E documentation, and updates the relevant documentation to reflect this automation. The workflow ensures that E2E documentation in the main branch is always up to date by generating docs, detecting changes, and opening a pull request automatically if updates are needed.

Automation of E2E Documentation Updates:

  • Added a new workflow file .github/workflows/update-playwright-e2e-docs.yml that installs dependencies, generates E2E docs, detects changes, and automatically creates a pull request to update documentation on the main branch when needed.

Documentation Updates:

  • Updated the Playwright doc-generator README.md to remove outdated manual CI instructions and describe the new automated workflow, including its trigger, logic, and usage.

Summary by Gitar

  • Workflow refinements:
    • Branch validation moved to explicit step with clear error message
    • Node.js version from .nvmrc file for consistency with project configuration
    • Playwright browsers installed explicitly with npx playwright install --with-deps
    • Permissions scoped at job level for better security practice
  • Change detection:
    • Scoped to playwright/docs directory only for precise detection
  • PR automation:
    • Improved error handling distinguishing between existing PR and creation failure
    • Git operations limited to docs directory for cleaner commits

This will update automatically on new commits.


Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This pull request introduces automation for Playwright E2E documentation updates through a new GitHub Actions workflow. The workflow runs on manual trigger, generates documentation from test files, detects changes, and automatically creates pull requests when documentation needs updating.

Changes:

  • Added a new workflow_dispatch-triggered GitHub Actions workflow to automate E2E documentation generation and PR creation
  • Updated the documentation generator README to reflect the new automated workflow approach and removed outdated manual CI instructions

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 8 comments.

File Description
.github/workflows/update-playwright-e2e-docs.yml New GitHub Actions workflow that installs dependencies, generates E2E docs, detects changes, and creates/updates PRs automatically
openmetadata-ui/src/main/resources/ui/playwright/doc-generator/README.md Updated workflow integration section to document the new automated workflow and simplify manual generation instructions

@github-actions
Copy link
Contributor

Jest test Coverage

UI tests summary

Lines Statements Branches Functions
Coverage: 65%
65.8% (55631/84544) 44.83% (28690/63997) 47.68% (8739/18328)

Copy link
Collaborator

@akash-jain-10 akash-jain-10 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Move the global permissions block under the Github Actions Job space as a best practice of scoping permissions at only the required scope!

Ref - https://docs.github.com/en/actions/reference/workflows-and-actions/workflow-syntax#jobsjob_idpermissions

@gitar-bot
Copy link

gitar-bot bot commented Jan 27, 2026

🔍 CI failure analysis for ba81a7e: CI failure unrelated to PR changes (workflow + docs only). 12 E2E tests failed due to locator/visibility issues with 99.7% pass rate, indicating test flakiness.

Issue

The playwright-ci-postgresql (3, 6) job failed with 12 out of 585 E2E tests failing.

Root Cause

These failures are unrelated to the PR changes. This PR only modifies:

  • .github/workflows/update-playwright-e2e-docs.yml (workflow file for docs automation)
  • openmetadata-ui/src/main/resources/ui/playwright/doc-generator/README.md (documentation)

Neither file affects the runtime behavior of the OpenMetadata application or the Playwright tests themselves.

Details

Test Failure Breakdown (12 failures, 583 passes = 99.7% pass rate):

Affected Test Files:

  • ServiceCreationPermissions.spec.ts (1 failure) - Service ownership permissions
  • CSVImportWithQuotesAndCommas.spec.ts (2 failures) - CSV import/export with special characters
  • GlossaryPermissions.spec.ts (1 failure) - Team-based glossary permissions
  • ServiceEntityPermissions.spec.ts (1 failure) - SearchIndex service permissions
  • SettingsNavigationPage.spec.ts (1 failure) - Navigation drag and drop
  • TagsSuggestion.spec.ts (1 failure) - Reject all tag suggestions
  • CustomizeWidgets.spec.ts (1 failure) - Data Products widget removal
  • ExploreDiscovery.spec.ts (2 failures) - Deleted assets display and search
  • ServiceForm.spec.ts (1 failure) - SSL cert upload with long filename
  • Bots.spec.ts (1 failure) - Bots page functionality

Failure Type:
All failures are locator assertion errors:

  • expect(locator).toBeVisible() failed
  • expect(locator).toBeEnabled() failed
  • expect(locator).toBeAttached() failed

Test Scenarios:

  • Service creation and ownership permissions
  • CSV import/export operations with commas and quotes
  • Glossary team-based access control
  • Settings navigation UI interactions (drag/drop)
  • Tag suggestions workflow
  • Widget customization
  • Explore discovery for deleted assets
  • Service form SSL certificate handling
  • Bots page operations

Flakiness indicators:

  • Very high pass rate (99.7% = 583/585 tests passed)
  • All failures are UI element visibility/state assertions
  • No compilation or syntax errors
  • PR changes only affect workflow and documentation files
  • Test failures span multiple unrelated feature areas
Code Review ✅ Approved 1 resolved / 1 findings

Well-structured GitHub Actions workflow with proper branch validation, scoped permissions, and improved error handling for PR creation. No significant issues found.

✅ 1 resolved
Quality: PR creation error handling masks all failures

📄 .github/workflows/update-playwright-e2e-docs.yml:76-80
The error handling || echo "PR already exists or failed to create" swallows all errors indiscriminately, making it difficult to distinguish between expected scenarios (PR already exists) and actual failures (authentication issues, rate limiting, network errors, permission problems).

Impact: When the workflow fails to create a PR for an unexpected reason, operators will see a generic message and may not realize the workflow didn't accomplish its goal.

Suggested Fix: Use GitHub CLI's built-in handling to distinguish between existing PR (exit code 1 with specific error) and other failures:

- name: Create Pull Request
  if: steps.git-check.outputs.changed == 'true'
  env:
    GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
  run: |
    BRANCH_NAME="chore/update-playwright-docs"
    git config user.name "github-actions[bot]"
    git config user.email "github-actions[bot]@users.noreply.github.com"

    git checkout -b $BRANCH_NAME
    git add .
    git commit -m "chore: update Playwright E2E documentation"
    git push origin $BRANCH_NAME --force

    # Check if PR already exists before attempting to create
    if gh pr list --head $BRANCH_NAME --state open --json number | jq -e 'length > 0' > /dev/null; then
      echo "PR already exists for branch $BRANCH_NAME"
    else
      gh pr create \
        --title "chore: update Playwright E2E documentation" \
        --body "This is an automated PR to update the Playwright E2E documentation. Generated by the \`Update Playwright E2E Documentation\` workflow." \
        --base main \
        --head $BRANCH_NAME
    fi

This approach explicitly checks for existing PRs and allows real errors to fail the workflow.

Rules 🎸 2 actions taken

Gitar Rules

🎸 Flaky Test Retry: Flaky tests detected and retried successfully
🎸 Summary Enhancement: PR description enhanced with technical details

1 rule not applicable. Show all rules by commenting gitar display:verbose.

Tip

Comment Gitar fix CI or enable auto-apply: gitar auto-apply:on

Options

Auto-apply is off → Gitar will not commit updates to this branch.
Display: compact → Showing less information.

Comment with these commands to change:

Auto-apply Compact
gitar auto-apply:on         
gitar display:verbose         

Was this helpful? React with 👍 / 👎 | Gitar

@sonarqubecloud
Copy link

@ShaileshParmar11
Copy link
Contributor Author

playwright is not needed for this PR

@ShaileshParmar11 ShaileshParmar11 merged commit 0422260 into main Jan 27, 2026
38 of 40 checks passed
@ShaileshParmar11 ShaileshParmar11 deleted the add-docs-update-workflow branch January 27, 2026 13:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

safe to test Add this label to run secure Github workflows on PRs UI UI specific issues

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants