Skip to content

chore: Experiment with flaky performance marks test #4228

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
pan-kot wants to merge 1 commit into
base: main
Choose a base branch
from
+174 −168
Draft

chore: Experiment with flaky performance marks test #4228

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
42 changes: 23 additions & 19 deletions src/button-dropdown/__integ__/performance-marks.test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,24 +25,28 @@
});
}

describe('ButtonDropdown', () => {
test(
'Emits a single mark',
setupTest('main-action', async ({ getMarks, getElementPerformanceMarkText }) => {
const marks = await getMarks();
for (let i = 0; i < 100; i++) {
describe('ButtonDropdown', () => {
test(
'Emits a single mark',
setupTest('main-action', async ({ getMarks, getElementPerformanceMarkText }) => {
const marks = await getMarks();

expect(marks).toHaveLength(1);
expect(marks[0].name).toBe('primaryButtonRendered');
expect(marks[0].detail).toMatchObject({
source: 'awsui',
instanceIdentifier: expect.any(String),
loading: false,
disabled: false,
inViewport: true,
text: 'Launch instance',
});
expect(marks).toHaveLength(1);

Check warning on line 35 in src/button-dropdown/__integ__/performance-marks.test.ts

View workflow job for this annotation

GitHub Actions / dry-run / Components integration tests shards (React 18, shard 3/4)

RETRY 2: ButtonDropdown › Emits a single mark 

expect(received).toHaveLength(expected)

Expected length: 1
Received length: 0
Received array:  []

    at src/button-dropdown/__integ__/performance-marks.test.ts:35:23
    at src/button-dropdown/__integ__/performance-marks.test.ts:24:5
    at Object.<anonymous> (node_modules/@cloudscape-design/browser-test-tools/use-browser.js:36:13)

Check warning on line 35 in src/button-dropdown/__integ__/performance-marks.test.ts

View workflow job for this annotation

GitHub Actions / dry-run / Components integration tests shards (React 18, shard 3/4)

RETRY 1: ButtonDropdown › Emits a single mark 

expect(received).toHaveLength(expected)

Expected length: 1
Received length: 0
Received array:  []

    at src/button-dropdown/__integ__/performance-marks.test.ts:35:23
    at src/button-dropdown/__integ__/performance-marks.test.ts:24:5
    at Object.<anonymous> (node_modules/@cloudscape-design/browser-test-tools/use-browser.js:36:13)

Check warning on line 35 in src/button-dropdown/__integ__/performance-marks.test.ts

View workflow job for this annotation

GitHub Actions / dry-run / Components integration tests shards (React 18, shard 3/4)

RETRY 1: ButtonDropdown › Emits a single mark 

expect(received).toHaveLength(expected)

Expected length: 1
Received length: 0
Received array:  []

    at src/button-dropdown/__integ__/performance-marks.test.ts:35:23
    at src/button-dropdown/__integ__/performance-marks.test.ts:24:5
    at Object.<anonymous> (node_modules/@cloudscape-design/browser-test-tools/use-browser.js:36:13)

Check failure on line 35 in src/button-dropdown/__integ__/performance-marks.test.ts

View workflow job for this annotation

GitHub Actions / dry-run / Components integration tests shards (React 18, shard 3/4)

ButtonDropdown › Emits a single mark 

expect(received).toHaveLength(expected)

Expected length: 1
Received length: 0
Received array:  []

    at src/button-dropdown/__integ__/performance-marks.test.ts:35:23
    at src/button-dropdown/__integ__/performance-marks.test.ts:24:5
    at Object.<anonymous> (node_modules/@cloudscape-design/browser-test-tools/use-browser.js:36:13)

Check warning on line 35 in src/button-dropdown/__integ__/performance-marks.test.ts

View workflow job for this annotation

GitHub Actions / dry-run / Components integration tests shards (React 18, shard 3/4)

RETRY 3: ButtonDropdown › Emits a single mark 

expect(received).toHaveLength(expected)

Expected length: 1
Received length: 0
Received array:  []

    at src/button-dropdown/__integ__/performance-marks.test.ts:35:23
    at src/button-dropdown/__integ__/performance-marks.test.ts:24:5
    at Object.<anonymous> (node_modules/@cloudscape-design/browser-test-tools/use-browser.js:36:13)

Check warning on line 35 in src/button-dropdown/__integ__/performance-marks.test.ts

View workflow job for this annotation

GitHub Actions / dry-run / Components integration tests shards (React 18, shard 3/4)

RETRY 2: ButtonDropdown › Emits a single mark 

expect(received).toHaveLength(expected)

Expected length: 1
Received length: 0
Received array:  []

    at src/button-dropdown/__integ__/performance-marks.test.ts:35:23
    at src/button-dropdown/__integ__/performance-marks.test.ts:24:5
    at Object.<anonymous> (node_modules/@cloudscape-design/browser-test-tools/use-browser.js:36:13)

Check warning on line 35 in src/button-dropdown/__integ__/performance-marks.test.ts

View workflow job for this annotation

GitHub Actions / dry-run / Components integration tests shards (React 18, shard 3/4)

RETRY 1: ButtonDropdown › Emits a single mark 

expect(received).toHaveLength(expected)

Expected length: 1
Received length: 0
Received array:  []

    at src/button-dropdown/__integ__/performance-marks.test.ts:35:23
    at src/button-dropdown/__integ__/performance-marks.test.ts:24:5
    at Object.<anonymous> (node_modules/@cloudscape-design/browser-test-tools/use-browser.js:36:13)
expect(marks[0].name).toBe('primaryButtonRendered');
expect(marks[0].detail).toMatchObject({
source: 'awsui',
instanceIdentifier: expect.any(String),
loading: false,
disabled: false,
inViewport: true,
text: 'Launch instance',
});

await expect(getElementPerformanceMarkText(marks[0].detail.instanceIdentifier)).resolves.toBe('Launch instance');
})
);
});
await expect(getElementPerformanceMarkText(marks[0].detail.instanceIdentifier)).resolves.toBe(
'Launch instance'
);
})
);
});
}
300 changes: 151 additions & 149 deletions src/flashbar/__integ__/collapsible.test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,176 +8,178 @@
import { setupTest } from './pages/interactive-page';
import { setupTest as setupStickyFlashbarTest } from './pages/sticky-page';

describe('Collapsible Flashbar', () => {
describe('Keyboard navigation', () => {
it(
'can expand and collapse stacked notifications with the keyboard',
setupTest(async page => {
await page.toggleStackingFeature();

// Navigate past all buttons to add and remove flashes
await page.click('#focus-before');
await page.keys('Tab');

await expect(page.countFlashes()).resolves.toBe(1);
await page.keys('Space');

await page.pause(FOCUS_DEBOUNCE_DELAY);
await expect(page.countFlashes()).resolves.toBe(5);
await expect(page.isFlashFocused(1)).resolves.toBe(true);

// Navigate past all flash buttons
await page.keys(new Array(11).fill('Tab'));

await page.keys('Space');
await expect(page.countFlashes()).resolves.toBe(1);
})
);
});

describe('Focus interactions', () => {
describe('in collapsed state', () => {
test(
'adding flash with ariaRole="status" does not move focus',
for (let i = 0; i < 50; i++) {
describe(`Collapsible Flashbar $i`, () => {
describe('Keyboard navigation', () => {
it(
'can expand and collapse stacked notifications with the keyboard',
setupTest(async page => {
await page.toggleStackingFeature();
await page.addInfoFlash();
return expect(page.isFlashFocused(1)).resolves.toBe(false);
})
);

test(
'adding flash with ariaRole="alert" moves focus to the new flash',
setupTest(async page => {
await page.toggleStackingFeature();
await page.addErrorFlash();
return expect(page.isFlashFocused(1)).resolves.toBe(true);
})
);
// Navigate past all buttons to add and remove flashes
await page.click('#focus-before');
await page.keys('Tab');

test(
'adding new non-alert flashes does not move focus to the new flash',
setupTest(async page => {
await page.toggleStackingFeature();
await page.addErrorFlash();
await expect(page.isFlashFocused(1)).resolves.toBe(true);
await page.addInfoFlash();
await expect(page.isFlashFocused(1)).resolves.toBe(false);
})
);
});
await expect(page.countFlashes()).resolves.toBe(1);
await page.keys('Space');

describe('in expanded state', () => {
test(
'adding flash with ariaRole="status" does not move focus',
setupTest(async page => {
await page.toggleStackingFeature();
await page.toggleCollapsedState();
await page.addInfoFlash();
return expect(page.isFlashFocused(1)).resolves.toBe(false);
})
);
await page.pause(FOCUS_DEBOUNCE_DELAY);
await expect(page.countFlashes()).resolves.toBe(5);
await expect(page.isFlashFocused(1)).resolves.toBe(true);

test(
'adding flash with ariaRole="alert" moves focus to the new flash',
setupTest(async page => {
await page.toggleStackingFeature();
await page.toggleCollapsedState();
await page.addErrorFlash();
return expect(page.isFlashFocused(1)).resolves.toBe(true);
})
);
// Navigate past all flash buttons
await page.keys(new Array(11).fill('Tab'));

test(
'adding new non-alert flashes does not move focus to the new flash',
setupTest(async page => {
await page.toggleStackingFeature();
await page.toggleCollapsedState();
await page.addErrorFlash();
await expect(page.isFlashFocused(1)).resolves.toBe(true);
await page.addInfoFlash();
await expect(page.isFlashFocused(1)).resolves.toBe(false);
await page.keys('Space');
await expect(page.countFlashes()).resolves.toBe(1);
})
);
});

test(
'adding new non-alert flashes does not move focus to previous alert flashes',
setupTest(async page => {
await page.toggleStackingFeature();
await page.toggleCollapsedState();
await page.addErrorFlash();
await expect(page.isFlashFocused(1)).resolves.toBe(true);
await page.addInfoFlash();
await expect(page.isFlashFocused(2)).resolves.toBe(false);
})
);
describe('Focus interactions', () => {
describe('in collapsed state', () => {
test(
'adding flash with ariaRole="status" does not move focus',
setupTest(async page => {
await page.toggleStackingFeature();
await page.addInfoFlash();
return expect(page.isFlashFocused(1)).resolves.toBe(false);
})
);

test(
'adding flash with ariaRole="alert" moves focus to the new flash',
setupTest(async page => {
await page.toggleStackingFeature();
await page.addErrorFlash();
return expect(page.isFlashFocused(1)).resolves.toBe(true);
})
);

test(
'adding new non-alert flashes does not move focus to the new flash',
setupTest(async page => {
await page.toggleStackingFeature();
await page.addErrorFlash();
await expect(page.isFlashFocused(1)).resolves.toBe(true);
await page.addInfoFlash();
await expect(page.isFlashFocused(1)).resolves.toBe(false);
})
);
});

describe('in expanded state', () => {
test(
'adding flash with ariaRole="status" does not move focus',
setupTest(async page => {
await page.toggleStackingFeature();
await page.toggleCollapsedState();
await page.addInfoFlash();
return expect(page.isFlashFocused(1)).resolves.toBe(false);
})
);

test(
'adding flash with ariaRole="alert" moves focus to the new flash',
setupTest(async page => {
await page.toggleStackingFeature();
await page.toggleCollapsedState();
await page.addErrorFlash();
return expect(page.isFlashFocused(1)).resolves.toBe(true);
})
);

test(
'adding new non-alert flashes does not move focus to the new flash',
setupTest(async page => {
await page.toggleStackingFeature();
await page.toggleCollapsedState();
await page.addErrorFlash();
await expect(page.isFlashFocused(1)).resolves.toBe(true);
await page.addInfoFlash();
await expect(page.isFlashFocused(1)).resolves.toBe(false);
})
);

test(
'adding new non-alert flashes does not move focus to previous alert flashes',
setupTest(async page => {
await page.toggleStackingFeature();
await page.toggleCollapsedState();
await page.addErrorFlash();
await expect(page.isFlashFocused(1)).resolves.toBe(true);
await page.addInfoFlash();
await expect(page.isFlashFocused(2)).resolves.toBe(false);
})
);
});

describe('on expand', () => {
test(
'focuses on the first flash',
setupTest(async page => {
await page.toggleStackingFeature();
await page.addErrorFlash();
await page.toggleCollapsedState();
await page.pause(FOCUS_DEBOUNCE_DELAY);
await expect(page.isFlashFocused(1)).resolves.toBe(true);
})
);
});
});

describe('on expand', () => {
describe('Sticky Flashbar', () => {
test(
'focuses on the first flash',
setupTest(async page => {
await page.toggleStackingFeature();
await page.addErrorFlash();
'keeps a space to the screen bottom to prevent the notification bar from getting cropped',
setupStickyFlashbarTest(async page => {
const windowDimensions = { width: 1000, height: 500 };
await page.setWindowSize(windowDimensions);
await page.toggleCollapsedState();
await page.pause(FOCUS_DEBOUNCE_DELAY);
await expect(page.isFlashFocused(1)).resolves.toBe(true);
expect(await page.getNotificationBarBottom()).toBeGreaterThan(windowDimensions.height);
await page.windowScrollTo({ top: 1200 });
await page.waitForAssertion(async () => {
await expect(page.getNotificationBarBottom()).resolves.toBeLessThan(windowDimensions.height);

Check warning on line 141 in src/flashbar/__integ__/collapsible.test.ts

View workflow job for this annotation

GitHub Actions / dry-run / Components integration tests shards (React 16, shard 4/4)

RETRY 1: Collapsible Flashbar $i › Sticky Flashbar › keeps a space to the screen bottom to prevent the notification bar from getting cropped 

expect(received).resolves.toBeLessThan(expected)

Expected: < 500
Received:   566

    at Object.toBeLessThan (node_modules/expect/build/index.js:174:22)
    at src/flashbar/__integ__/collapsible.test.ts:141:68
    at RetryOperation._fn (node_modules/p-retry/index.js:50:18)
    at Timeout._onTimeout (node_modules/retry/lib/retry_operation.js:85:10)

Check warning on line 141 in src/flashbar/__integ__/collapsible.test.ts

View workflow job for this annotation

GitHub Actions / dry-run / Components integration tests shards (React 16, shard 4/4)

RETRY 1: Collapsible Flashbar $i › Sticky Flashbar › keeps a space to the screen bottom to prevent the notification bar from getting cropped 

expect(received).resolves.toBeLessThan(expected)

Expected: < 500
Received:   566

    at Object.toBeLessThan (node_modules/expect/build/index.js:174:22)
    at src/flashbar/__integ__/collapsible.test.ts:141:68
    at RetryOperation._fn (node_modules/p-retry/index.js:50:18)
    at Timeout._onTimeout (node_modules/retry/lib/retry_operation.js:85:10)
});
})
);
});
});

describe('Sticky Flashbar', () => {
describe('Flashbar Style API', () => {
test(
'keeps a space to the screen bottom to prevent the notification bar from getting cropped',
setupStickyFlashbarTest(async page => {
const windowDimensions = { width: 1000, height: 500 };
await page.setWindowSize(windowDimensions);
await page.toggleCollapsedState();
expect(await page.getNotificationBarBottom()).toBeGreaterThan(windowDimensions.height);
await page.windowScrollTo({ top: 1200 });
await page.waitForAssertion(async () => {
await expect(page.getNotificationBarBottom()).resolves.toBeLessThan(windowDimensions.height);
});
})
);
});
});
'active, hover and focus states',
useBrowser(async browser => {
await browser.url('#/light/flashbar/style-custom');

describe('Flashbar Style API', () => {
test(
'active, hover and focus states',
useBrowser(async browser => {
await browser.url('#/light/flashbar/style-custom');
const page = new FlashbarBasePage(browser);
const dismissButton = page.getDismissButton();
const notificationBar = page.getNotificationBar();
const expandButton = page.getExpandButton();

const page = new FlashbarBasePage(browser);
const dismissButton = page.getDismissButton();
const notificationBar = page.getNotificationBar();
const expandButton = page.getExpandButton();
await page.hoverElement(dismissButton);
await expect((await browser.$(dismissButton).getCSSProperty('color')).value).toBe('rgba(63,125,74,1)');

await page.hoverElement(dismissButton);
await expect((await browser.$(dismissButton).getCSSProperty('color')).value).toBe('rgba(63,125,74,1)');
await page.buttonDownOnElement(dismissButton);
await expect((await browser.$(dismissButton).getCSSProperty('color')).value).toBe('rgba(46,72,50,1)');

await page.buttonDownOnElement(dismissButton);
await expect((await browser.$(dismissButton).getCSSProperty('color')).value).toBe('rgba(46,72,50,1)');

await page.click('[data-testid=collapsed]');
await page.keys('Tab');
await expect((await browser.$(dismissButton).getCSSProperty('box-shadow', '::before')).value).toBe(
'rgb(63,125,74)0px0px0px1px'
);

await page.click('[data-testid=collapsed]');
await page.keys(['Shift', 'Tab']);
await expect((await browser.$(expandButton).getCSSProperty('box-shadow', '::before')).value).toBe(
'rgb(239,240,240)0px0px0px1px'
);
await expect((await browser.$(expandButton).getCSSProperty('border-radius', '::before')).value).toBe('4px');

await page.buttonDownOnElement(notificationBar);
await expect((await browser.$(notificationBar).getCSSProperty('background-color')).value).toBe(
'rgba(92,102,112,1)'
);
})
);
});
await page.click('[data-testid=collapsed]');
await page.keys('Tab');
await expect((await browser.$(dismissButton).getCSSProperty('box-shadow', '::before')).value).toBe(
'rgb(63,125,74)0px0px0px1px'
);

await page.click('[data-testid=collapsed]');
await page.keys(['Shift', 'Tab']);
await expect((await browser.$(expandButton).getCSSProperty('box-shadow', '::before')).value).toBe(
'rgb(239,240,240)0px0px0px1px'
);
await expect((await browser.$(expandButton).getCSSProperty('border-radius', '::before')).value).toBe('4px');

await page.buttonDownOnElement(notificationBar);
await expect((await browser.$(notificationBar).getCSSProperty('background-color')).value).toBe(
'rgba(92,102,112,1)'
);
})
);
});
}
Loading