Skip to content

test: fix ActivityAPI flakiness and reduce pipeline wait timeouts#28071

Merged
anuj-kumary merged 9 commits into
mainfrom
anuj/activityapi
May 15, 2026
Merged

test: fix ActivityAPI flakiness and reduce pipeline wait timeouts#28071
anuj-kumary merged 9 commits into
mainfrom
anuj/activityapi

Conversation

@anuj-kumary
Copy link
Copy Markdown
Member

@anuj-kumary anuj-kumary commented May 13, 2026

  • Fix beforeAll hook timeouts: Replaced performAdminLogin(browser) with createAdminApiContext() in all 4 beforeAll hooks. The UI login flow took >60s (navigating to login page, filling credentials); the API login completes in ~1s.

  • Bypass async pipeline for setup: Added insertActivityEventForTest that writes directly to POST /api/v1/activity/test-insert, skipping the Quartz change-event pipeline entirely. Reactions and Comments test setup no longer poll for up to 5 minutes.

  • Fix comment test EntityLink validation error: insertActivityEventForTest now includes about: '<#E::table::fqn>' and full entity fields. Without it the UI sent about: '' when creating a feed thread, which failed backend regex validation and left waitForResponse('/api/v1/feed/*/posts') hanging until timeout.

  • Consolidate duplicate tests: 5 Entity Changes tests → 3. Actor and entity-link assertions merged into the description test; redundant pipeline waits removed.

  • Remove all manual timeouts: All test.setTimeout and test.describe.configure({ timeout }) calls removed. Pipeline tests rely on ACTIVITY_EVENT_TIMEOUT (100s) inside waitForActivityEvent; setup-only tests need no timeout at all.

  • Better failure diagnostics: waitForActivityEvent now rethrows with last HTTP status and observed event types, so timeouts show actionable context.

Screenshot 2026-05-13 at 1 40 38 PM

Summary by Gitar

  • Test status updates:
    • Marked ActivityAPI test suites as fixme to prevent pipeline failures while investigating event propagation latency.

This will update automatically on new commits.

@anuj-kumary anuj-kumary self-assigned this May 13, 2026
@anuj-kumary anuj-kumary requested a review from a team as a code owner May 13, 2026 08:10
@anuj-kumary anuj-kumary added the safe to test Add this label to run secure Github workflows on PRs label May 13, 2026
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 13, 2026

Jest test Coverage

UI tests summary

Lines Statements Branches Functions
Coverage: 62%
62.56% (64767/103526) 43.27% (35278/81522) 46.07% (10378/22526)

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 13, 2026

🟡 Playwright Results — all passed (23 flaky)

✅ 4056 passed · ❌ 0 failed · 🟡 23 flaky · ⏭️ 92 skipped

Shard Passed Failed Flaky Skipped
🟡 Shard 1 298 0 1 4
🟡 Shard 2 752 0 10 14
🟡 Shard 3 779 0 2 7
🟡 Shard 4 787 0 3 18
✅ Shard 5 709 0 0 41
🟡 Shard 6 731 0 7 8
🟡 23 flaky test(s) (passed on retry)
  • Pages/AuditLogs.spec.ts › should apply both User and EntityType filters simultaneously (shard 1, 1 retry)
  • Features/BulkEditEntity.spec.ts › Glossary (shard 2, 1 retry)
  • Features/ColumnBulkOperations.spec.ts › should filter by metadata status and verify API param (shard 2, 1 retry)
  • Features/ColumnBulkOperations.spec.ts › should filter by entity type (Table) (shard 2, 1 retry)
  • Features/DataProductDomainMigration.spec.ts › Data product with no assets can change domain without confirmation (shard 2, 1 retry)
  • Features/Glossary/GlossaryWorkflow.spec.ts › should start term as Draft when glossary has reviewers (shard 2, 2 retries)
  • Features/KnowledgeCenterList.spec.ts › Knowledge Center List - Test infinite scroll/pagination (shard 2, 2 retries)
  • Features/KnowledgeCenterList.spec.ts › Knowledge Center List - Test add article button (shard 2, 1 retry)
  • Features/KnowledgeCenterTextEditor.spec.ts › Rich Text Editor - Text Formatting (shard 2, 1 retry)
  • Features/KnowledgeCenterTextEditor.spec.ts › Rich Text Editor - Text Formatting (shard 2, 1 retry)
  • Features/KnowledgeCenterTextEditor.spec.ts › Rich Text Editor - Text Formatting (shard 2, 1 retry)
  • Features/RTL.spec.ts › Verify Following widget functionality (shard 3, 1 retry)
  • Flow/PersonaFlow.spec.ts › Remove users in persona should work properly (shard 3, 1 retry)
  • Pages/CustomProperties.spec.ts › Should display custom properties for apiCollection in right panel (shard 4, 1 retry)
  • Pages/DataContracts.spec.ts › Create Data Contract and validate for ApiEndpoint (shard 4, 1 retry)
  • Pages/Entity.spec.ts › Inactive Announcement create & delete (shard 4, 1 retry)
  • Pages/Glossary.spec.ts › Column dropdown drag-and-drop functionality for Glossary Terms table (shard 6, 1 retry)
  • Pages/GlossaryImportExport.spec.ts › Glossary CSV import preserves typed relations (shard 6, 2 retries)
  • Pages/Lineage/DataAssetLineage.spec.ts › Column lineage for apiEndpoint -> dashboardDataModel (shard 6, 1 retry)
  • Pages/Lineage/LineageFilters.spec.ts › Verify Impact Analysis service filter selection (shard 6, 1 retry)
  • Pages/Lineage/LineageRightPanel.spec.ts › Verify custom properties tab IS visible for supported type: searchIndex (shard 6, 1 retry)
  • Pages/Tag.spec.ts › Verify Owner Add Delete (shard 6, 1 retry)
  • Pages/UserDetails.spec.ts › Create team with domain and verify visibility of inherited domain in user profile after team removal (shard 6, 1 retry)

📦 Download artifacts

How to debug locally
# Download playwright-test-results-<shard> artifact and unzip
npx playwright show-trace path/to/trace.zip    # view trace

Comment thread openmetadata-ui/src/main/resources/ui/playwright.config.ts Outdated
@gitar-bot
Copy link
Copy Markdown

gitar-bot Bot commented May 14, 2026

Code Review 👍 Approved with suggestions 1 resolved / 2 findings

Refactors ActivityAPI test setup to use API-based authentication and direct event insertion, significantly reducing execution time. Consider adding a response validation check in insertActivityEventForTest to confirm the success of user API calls.

💡 Edge Case: insertActivityEventForTest lacks response check on user API call

📄 openmetadata-ui/src/main/resources/ui/playwright/utils/activityAPI.ts:312-313

In insertActivityEventForTest, the response from apiContext.get('/api/v1/users/loggedInUser') is not checked for success before calling .json(). If the auth context is invalid or the endpoint fails (e.g., 401/500), the test will fail with a confusing JSON parse error or undefined fields rather than a clear message about the user lookup failure. Since this is test setup code, a clear early failure makes debugging easier.

Add an assertion on the user response status before parsing JSON, matching the pattern used for the POST response at line 336.
const userResponse = await apiContext.get('/api/v1/users/loggedInUser');
expect(userResponse.ok()).toBeTruthy();
const adminUser = await userResponse.json();
✅ 1 resolved
Bug: repeatEach: 20 and testMatch filter left in playwright.config

📄 openmetadata-ui/src/main/resources/ui/playwright.config.ts:80-81
The chromium project in playwright.config.ts now restricts test execution to only ActivityAPI.spec.ts (testMatch: '**/ActivityAPI.spec.ts') and repeats each test 20 times (repeatEach: 20). This looks like a local flakiness-detection configuration that was not reverted before pushing. If merged, CI will:

  1. Skip every other spec file in the suite (no other tests will run under the chromium project).
  2. Run ActivityAPI tests 20× each, massively inflating pipeline duration.

Both settings should be removed to restore normal CI behavior.

🤖 Prompt for agents
Code Review: Refactors ActivityAPI test setup to use API-based authentication and direct event insertion, significantly reducing execution time. Consider adding a response validation check in `insertActivityEventForTest` to confirm the success of user API calls.

1. 💡 Edge Case: insertActivityEventForTest lacks response check on user API call
   Files: openmetadata-ui/src/main/resources/ui/playwright/utils/activityAPI.ts:312-313

   In `insertActivityEventForTest`, the response from `apiContext.get('/api/v1/users/loggedInUser')` is not checked for success before calling `.json()`. If the auth context is invalid or the endpoint fails (e.g., 401/500), the test will fail with a confusing JSON parse error or `undefined` fields rather than a clear message about the user lookup failure. Since this is test setup code, a clear early failure makes debugging easier.

   Fix (Add an assertion on the user response status before parsing JSON, matching the pattern used for the POST response at line 336.):
   const userResponse = await apiContext.get('/api/v1/users/loggedInUser');
   expect(userResponse.ok()).toBeTruthy();
   const adminUser = await userResponse.json();

Options

Display: compact → Showing less information.

Comment with these commands to change:

Compact
gitar display:verbose         

Was this helpful? React with 👍 / 👎 | Gitar

@sonarqubecloud
Copy link
Copy Markdown

@anuj-kumary anuj-kumary merged commit 9a29bf1 into main May 15, 2026
49 checks passed
@anuj-kumary anuj-kumary deleted the anuj/activityapi branch May 15, 2026 05:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

safe to test Add this label to run secure Github workflows on PRs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Fix activity api test

2 participants