Skip to content

Commit c9beb5c

Browse files
authored
Merge branch 'main' into MINOR-Reindex-fix-incident
2 parents 12866d3 + 1f2c4fc commit c9beb5c

File tree

881 files changed

+54790
-15061
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

881 files changed

+54790
-15061
lines changed
File renamed without changes.

.claude/skills/playwright/SKILL.md

Lines changed: 123 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,123 @@
1+
---
2+
name: playwright-test
3+
description: Generate robust, zero-flakiness Playwright E2E tests following OpenMetadata patterns. Creates comprehensive test files with proper waits, API validation, multi-role permissions, and complete entity lifecycle management.
4+
user-invocable: true
5+
---
6+
7+
# Playwright Test Generator - OpenMetadata
8+
9+
Generate production-ready, zero-flakiness Playwright tests following OpenMetadata conventions.
10+
11+
## Usage
12+
13+
```
14+
/playwright-test
15+
Feature: <feature name>
16+
Category: <Features|Pages|Flow|VersionPages>
17+
Entity: <Table|Dashboard|Pipeline|Topic|Database|User|Team|Glossary|Other>
18+
Domain: <Governance|Discovery|Platform|Observability|Integration>
19+
Scenarios:
20+
- <scenario 1 description>
21+
- <scenario 2 description>
22+
- <scenario 3 description>
23+
Roles: <admin|dataConsumer|dataSteward|owner> (optional, defaults to admin)
24+
```
25+
26+
## Quick Example
27+
28+
```
29+
/playwright-test
30+
Feature: Data Quality Rules
31+
Category: Features
32+
Entity: Table
33+
Domain: Observability
34+
Scenarios:
35+
- Admin can create and configure data quality rules
36+
- Data consumer can view test results but not edit rules
37+
- Test results are persisted after page reload
38+
Roles: admin, dataConsumer
39+
```
40+
41+
---
42+
43+
## Instructions
44+
45+
### Step 1: Read the Handbook FIRST
46+
47+
**CRITICAL**: Before generating any tests, read and apply ALL patterns from:
48+
49+
```
50+
openmetadata-ui/src/main/resources/ui/playwright/PLAYWRIGHT_DEVELOPER_HANDBOOK.md
51+
```
52+
53+
The handbook contains:
54+
- Testing philosophy (user-centric, behavior-focused)
55+
- Anti-flakiness patterns (the :visible selector chain pattern, etc.)
56+
- Test timeout strategies (test.slow() vs test.setTimeout())
57+
- Common test patterns (form submission, dropdowns, multi-role testing)
58+
- Locator priority guidelines
59+
- Support classes reference
60+
- Domain tags
61+
- Validation checklist
62+
63+
**Apply ALL handbook principles before proceeding.**
64+
65+
---
66+
67+
### Step 2: Generate Test Using Handbook Template
68+
69+
Use the **Test File Structure Template** from the handbook. It includes:
70+
- Proper imports (performAdminLogin, entity classes, utilities)
71+
- test.describe with domain tags
72+
- beforeAll/afterAll for entity lifecycle via API
73+
- test.slow() for timeout handling
74+
- test.step() for clear organization
75+
- API response validation pattern
76+
77+
---
78+
79+
### Step 3: Apply Common Test Patterns from Handbook
80+
81+
Reference the **Common Test Patterns** section for:
82+
- Form submission with API validation
83+
- Dropdown selection (with :visible chain pattern)
84+
- Multi-role permission testing
85+
- Data persistence verification
86+
- Navigation patterns
87+
- Search and filter patterns
88+
89+
---
90+
91+
### Step 4: Validate Against Handbook Checklist
92+
93+
Before returning the generated test, verify ALL items from the handbook's **Validation Checklist**:
94+
95+
- ✅ Structure & Organization (test.step, domain tags, imports, beforeAll/afterAll)
96+
- ✅ Anti-Flakiness (no waitForTimeout, no networkidle, no force: true, no positional selectors, no stored :visible locators)
97+
- ✅ API & Network (waitForResponse before actions, status code validation)
98+
- ✅ Waits & Assertions (waitForAllLoadersToDisappear, semantic locators, proper assertions)
99+
- ✅ Coverage & Roles (multi-role tests, data persistence, error handling)
100+
101+
---
102+
103+
## Key Reminders
104+
105+
**All patterns, rules, and best practices are in the handbook.**
106+
107+
Read and apply the handbook sections in order:
108+
1. **Anti-Flakiness Patterns** (CRITICAL - #1 cause of flaky tests)
109+
2. **Test File Structure Template** (for proper test structure)
110+
3. **Common Test Patterns** (for specific scenarios)
111+
4. **Validation Checklist** (before returning generated test)
112+
113+
---
114+
115+
## Final Notes
116+
117+
- Generate **production-ready** tests that pass 10/10 times
118+
- Follow ALL patterns from the handbook exactly
119+
- No comments for obvious code (e.g., `// Create entity` before `entity.create()`)
120+
- Test independence - each test runs in any order
121+
- Reference examples: `playwright/e2e/Pages/DataContractInheritance.spec.ts`, `playwright/e2e/Features/Table.spec.ts`
122+
123+
**Generate tests that are production-ready, maintainable, and zero-flakiness by following the handbook patterns exactly.**
Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
---
2+
name: writing-playwright-tests
3+
description: Use when writing new Playwright E2E tests or adding test cases. Provides testing philosophy, patterns, and best practices from the Playwright Developer Handbook.
4+
---
5+
6+
# Writing Playwright Tests Skill
7+
8+
This skill guides you through writing Playwright E2E tests following OpenMetadata standards.
9+
10+
**Reference**: @openmetadata-ui/src/main/resources/ui/playwright/PLAYWRIGHT_DEVELOPER_HANDBOOK.md

.github/workflows/integration-tests-mysql-elasticsearch.yml

Lines changed: 11 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -50,11 +50,11 @@ jobs:
5050
- name: Free Disk Space (Ubuntu)
5151
uses: jlumbroso/free-disk-space@main
5252
with:
53-
tool-cache: false
53+
tool-cache: true
5454
android: true
5555
dotnet: true
5656
haskell: true
57-
large-packages: false
57+
large-packages: true
5858
docker-images: false
5959
swap-storage: true
6060

@@ -110,10 +110,17 @@ jobs:
110110
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
111111
run: mvn -DskipTests clean install -pl :openmetadata-integration-tests -am
112112

113+
- name: Free build artifacts
114+
run: |
115+
rm -rf openmetadata-service/target/lib openmetadata-service/target/classes
116+
rm -rf openmetadata-spec/target openmetadata-sdk/target common/target
117+
rm -rf openmetadata-shaded-deps/*/target
118+
df -h /
119+
113120
- name: Run Integration Tests (MySQL + Elasticsearch)
114121
env:
115122
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
116-
run: mvn test -pl :openmetadata-integration-tests -Pmysql-elasticsearch
123+
run: mvn verify -pl :openmetadata-integration-tests -Pmysql-elasticsearch
117124

118125
- name: Clean Up
119126
run: |
@@ -127,4 +134,4 @@ jobs:
127134
with:
128135
github_token: ${{ secrets.GITHUB_TOKEN }}
129136
fail_on_test_failures: true
130-
report_paths: 'openmetadata-integration-tests/target/surefire-reports/TEST-*.xml'
137+
report_paths: 'openmetadata-integration-tests/target/failsafe-reports/TEST-*.xml'

.github/workflows/integration-tests-postgres-opensearch.yml

Lines changed: 11 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -50,11 +50,11 @@ jobs:
5050
- name: Free Disk Space (Ubuntu)
5151
uses: jlumbroso/free-disk-space@main
5252
with:
53-
tool-cache: false
53+
tool-cache: true
5454
android: true
5555
dotnet: true
5656
haskell: true
57-
large-packages: false
57+
large-packages: true
5858
docker-images: false
5959
swap-storage: true
6060

@@ -110,10 +110,17 @@ jobs:
110110
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
111111
run: mvn -DskipTests clean install -pl :openmetadata-integration-tests -am
112112

113+
- name: Free build artifacts
114+
run: |
115+
rm -rf openmetadata-service/target/lib openmetadata-service/target/classes
116+
rm -rf openmetadata-spec/target openmetadata-sdk/target common/target
117+
rm -rf openmetadata-shaded-deps/*/target
118+
df -h /
119+
113120
- name: Run Integration Tests (PostgreSQL + OpenSearch)
114121
env:
115122
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
116-
run: mvn test -pl :openmetadata-integration-tests -Ppostgres-opensearch
123+
run: mvn verify -pl :openmetadata-integration-tests -Ppostgres-opensearch
117124

118125
- name: Clean Up
119126
run: |
@@ -127,4 +134,4 @@ jobs:
127134
with:
128135
github_token: ${{ secrets.GITHUB_TOKEN }}
129136
fail_on_test_failures: true
130-
report_paths: 'openmetadata-integration-tests/target/surefire-reports/TEST-*.xml'
137+
report_paths: 'openmetadata-integration-tests/target/failsafe-reports/TEST-*.xml'

.github/workflows/java-checkstyle.yml

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -20,11 +20,14 @@ on:
2020
- "0.[0-9]+.[0-9]+"
2121
pull_request_target:
2222
types: [opened, synchronize, reopened, labeled]
23+
paths-ignore:
24+
- "openmetadata-ui/src/main/resources/ui/playwright/doc-generator/**"
25+
- "openmetadata-ui/src/main/resources/ui/playwright/docs/**"
2326

2427
permissions:
2528
contents: read
2629

27-
concurrency:
30+
concurrency:
2831
group: java-checkstyle-${{ github.head_ref }}
2932
cancel-in-progress: true
3033
jobs:
@@ -45,7 +48,7 @@ jobs:
4548
swap-storage: true
4649
docker-images: false
4750
- name: Wait for the labeler
48-
uses: lewagon/wait-on-check-action@v1.3.4
51+
uses: lewagon/wait-on-check-action@v1.3.4
4952
if: ${{ github.event_name == 'pull_request_target' }}
5053
with:
5154
ref: ${{ github.event.pull_request.head.sha }}

.github/workflows/maven-build-collate.yml

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,8 @@ on:
1919
paths:
2020
- "openmetadata-service/**"
2121
- "openmetadata-ui/**"
22+
- "!openmetadata-ui/src/main/resources/ui/playwright/doc-generator/**"
23+
- "!openmetadata-ui/src/main/resources/ui/playwright/docs/**"
2224
- "openmetadata-spec/src/main/resources/json/schema/**"
2325
- "openmetadata-dist/**"
2426
- "openmetadata-clients/**"
@@ -38,7 +40,7 @@ permissions:
3840
contents: read
3941
checks: write
4042

41-
concurrency:
43+
concurrency:
4244
group: maven-build-collate-${{ github.head_ref || github.run_id }}
4345
cancel-in-progress: true
4446
jobs:

.github/workflows/maven-postgres-tests-build.yml

Lines changed: 5 additions & 30 deletions
Original file line numberDiff line numberDiff line change
@@ -10,42 +10,17 @@
1010
# limitations under the License.
1111

1212
name: Maven Postgres Tests CI
13+
# Disabled: Integration tests (openmetadata-integration-tests) now cover the same ground
14+
# with full Testcontainers-based PostgreSQL testing.
1315

1416
on:
15-
push:
16-
branches:
17-
- main
18-
paths:
19-
- "openmetadata-service/**"
20-
- "openmetadata-ui/**"
21-
- "openmetadata-spec/src/main/resources/json/schema/**"
22-
- "openmetadata-dist/**"
23-
- "openmetadata-clients/**"
24-
- "openmetadata-sdk/**"
25-
- "common/**"
26-
- "pom.xml"
27-
- "yarn.lock"
28-
- "Makefile"
29-
- "bootstrap/**"
30-
pull_request_target:
31-
types: [labeled, opened, synchronize, reopened, ready_for_review]
32-
paths:
33-
- "openmetadata-service/**"
34-
- "openmetadata-spec/src/main/resources/json/schema/**"
35-
- "openmetadata-dist/**"
36-
- "openmetadata-clients/**"
37-
- "openmetadata-sdk/**"
38-
- "common/**"
39-
- "pom.xml"
40-
- "yarn.lock"
41-
- "Makefile"
42-
- "bootstrap/**"
17+
workflow_dispatch:
4318

4419
permissions:
4520
contents: read
4621
checks: write
4722

48-
concurrency:
23+
concurrency:
4924
group: maven-build-postgres-${{ github.head_ref || github.run_id }}
5025
cancel-in-progress: true
5126
jobs:
@@ -94,7 +69,7 @@ jobs:
9469
key: ${{ runner.os }}-maven-${{ hashFiles('**/pom.xml') }}
9570
restore-keys: |
9671
${{ runner.os }}-maven-
97-
72+
9873
- name: Set up JDK 21
9974
if: steps.cache-output.outputs.exit-code == 0
10075
uses: actions/setup-java@v4

.github/workflows/monitor-slack-link.yml

Lines changed: 9 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -44,16 +44,20 @@ jobs:
4444
run: |
4545
python -m venv env
4646
source env/bin/activate
47-
pip install --upgrade pip requests types-requests slack_sdk==3.35.0
47+
pip install --upgrade pip playwright slack_sdk==3.35.0
4848
49-
# Step 4: Run the monitoring script
49+
# Step 4: Install Playwright browsers
50+
- name: Install Playwright Browsers
51+
run: |
52+
source env/bin/activate
53+
playwright install chromium --with-deps
54+
55+
# Step 5: Run the monitoring script
5056
- name: Monitor Slack Link
5157
id: monitor
52-
continue-on-error: true # Allow subsequent steps (notification) even if this fails
5358
env:
54-
PYTHONUNBUFFERED: "1" # Recommended for immediate log output
59+
PYTHONUNBUFFERED: "1"
5560
SLACK_WEBHOOK_URL: ${{ secrets.SLACK_MONITOR_SLACK_WEBHOOK }}
56-
SLACK_WEBHOOK_TYPE: INCOMING_WEBHOOK
5761
GITHUB_SERVER_URL: ${{ github.server_url }}
5862
GITHUB_REPOSITORY: ${{ github.repository }}
5963
GITHUB_RUN_ID: ${{ github.run_id }}

.github/workflows/mysql-nightly-e2e.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -108,7 +108,7 @@ jobs:
108108
PLAYWRIGHT_SNOWFLAKE_WAREHOUSE: ${{ secrets.TEST_SNOWFLAKE_WAREHOUSE }}
109109
PLAYWRIGHT_PROJECT_ID: ${{ steps.cypress-project-id.outputs.CYPRESS_PROJECT_ID }}
110110
PLAYWRIGHT_BQ_PRIVATE_KEY: ${{ secrets.TEST_BQ_PRIVATE_KEY }}
111-
PLAYWRIGHT_BQ_PROJECT_ID: ${{ secrets.TEST_BQ_PROJECT_ID }}
111+
PLAYWRIGHT_BQ_PROJECT_ID: ${{ secrets.PLAYWRIGHT_BQ_PROJECT_ID }}
112112
PLAYWRIGHT_BQ_PRIVATE_KEY_ID: ${{ secrets.TEST_BQ_PRIVATE_KEY_ID }}
113113
PLAYWRIGHT_BQ_PROJECT_ID_TAXONOMY: ${{ secrets.TEST_BQ_PROJECT_ID_TAXONOMY }}
114114
PLAYWRIGHT_BQ_CLIENT_EMAIL: ${{ secrets.TEST_BQ_CLIENT_EMAIL }}

0 commit comments

Comments
 (0)