Skip to content

Commit a9cde9a

Browse files
andrii-ipre-commit-ci[bot]3coinsdlqqq
authored
Add UI tests (#387)
* add launcher UI test * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * stop waiting for launcher to be detached * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add back autoGoto: false * wait for launcher selector * use locator and waitFor * use jupyterlab/maintainer-tools binder-link packaged action * update integration test workflow * Update ui-tests/tests/jupyter_scheduler.spec.ts Co-authored-by: Piyush Jain <[email protected]> * Try adding a failing test * add exact option * use locator and expect element to be visible * remove failing test * remove trailing whitespaces * rename test * test visibility of create a job btn in notebook toolbar * Test for create job item on file browser notebook right click * refresh filebrowser * Add quotation mark to test description * add snapshot comparasion for launcher * add launcher snapshot * add snapshot comparasion for a notebook toolbar * add snapshot for notebook create job toolbar button * add snapshot check for notebook righclick menu in file browser * provide unique snapshotName * add flie browser righclick menu snapshot * wait until kernel is ready or idle * update notebook snapshot * create and use SchedulerHelper testing ckass * use schedulerHelper in launcher test * add and use notebookToolbarButton class properties * create and use file browser item helper methods * wait for hidden kernel status message * wait for debugger to be enabled * wait for debugger to be ready * await for execution indicator to become idle * fix DebuggerBugButton class name * move SchedulerHelper to a separate file * clarify snapshot var naming * add create view snapshot comparasion * remove empty test * wait until loading is done * add snapshot for empty create job page * add a test to check ability to create a job * Change describe string to more descriptive * save the notebook after creating * close filebrowser before every test * Click rename in the save dialog * have file browser closed on all snapshots * wait 2 seconds for page reload * update empty create view snapshot * update create view snapshot * specify pydantic version to be <2 (breaking changes) * close notebook tab before comparasion * close filebrowser before clicking reload * don't click reload * add in-progresds list view snapshot, clarify tab close selectors * use less strict text selectors * update snapshot, don't attempt to close a notebook * mask timestamp * Revert "specify pydantic version to be <2 (breaking changes)" This reverts commit 3581d5b. * use jobs panel not the whole page for screen comparasion * make job name selector more precise * Shorten function description * make timestamp selector more precise * make mask color white * update job list view snapshot * roll back clicking the job name * click jpb name link * update list view snapshot * use schedulerHelper to wait until loading is done * use whole shell for list view comparasion * make waitTextGone function async * await waitTextGone everywhere * remove waitTextGone * use jp-main-content-panel instead of lab shell * provide actual list view snapshot * test job deletion * wait until job is deleted before taking a screenshot * wait until Saving Completed is not visible * replace waitForFunction with waitForSelector * update MyTestJob selector * use waitFor to wait for hidden state of the deleted job * remove strict from MyTestJob locator * hit a reload button * remove delete job because of error * bump build.yml jupyterlab version to ~=3.6.5 * ignore jsonschema.RefResolver * update list-view-in-progress snapshot * bump base_setup python version to 3.11 * Update Integration tests documentation * maxDiffPixelRatio 0.01 to account for table shifting * add conda and pip reccomendation * add yarn.lock to fix playwright install * run prettier * remove binder badge wf as binder is not working * remove conda pip reccomendation * define harcoded snapshot names at top level in all caps * split in-notebook 'create a job' button test in 2 * close sidebar before taking a snapshot * Split filebrowser create a job item test in 2 --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Piyush Jain <[email protected]> Co-authored-by: David L. Qiu <[email protected]>
1 parent d637a7e commit a9cde9a

13 files changed

+3197
-34
lines changed

‎.github/workflows/binder-on-pr.yml

Lines changed: 0 additions & 30 deletions
This file was deleted.

‎.github/workflows/update-integration-tests.yml

Lines changed: 9 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,7 @@ jobs:
1717

1818
steps:
1919
- name: Checkout
20-
uses: actions/checkout@v2
20+
uses: actions/checkout@v3
2121
with:
2222
token: ${{ secrets.GITHUB_TOKEN }}
2323

@@ -29,7 +29,15 @@ jobs:
2929
env:
3030
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
3131

32+
- name: Base Setup
33+
uses: jupyterlab/maintainer-tools/.github/actions/base-setup@v1
34+
with:
35+
python_version: '3.11'
36+
3237
- name: Install dependencies
38+
run: python -m pip install -U jupyterlab~=3.1 jupyter-archive
39+
40+
- name: Install extension
3341
run: |
3442
set -eux
3543
jlpm

‎docs/contributors/index.md

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -111,6 +111,22 @@ More precisely, the JupyterLab helper
111111
[Galata](https://github.com/jupyterlab/jupyterlab/tree/master/galata) is used to
112112
test the extension in JupyterLab.
113113

114+
Install test dependencies (needed only once):
115+
116+
```sh
117+
cd ./ui-tests
118+
jlpm install
119+
jlpm playwright install
120+
cd ..
121+
```
122+
123+
To execute them, run:
124+
125+
```sh
126+
cd ./ui-tests
127+
jlpm playwright test
128+
```
129+
114130
You can find more information in the
115131
[ui-tests](https://github.com/jupyter-server/jupyter-scheduler/tree/main/ui-tests)
116132
README.

‎pyproject.toml

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -137,7 +137,8 @@ filterwarnings = [
137137
"ignore:Passing a schema to Validator.iter_errors:DeprecationWarning",
138138
"ignore:unclosed <socket.socket:ResourceWarning",
139139
"ignore:unclosed event loop:ResourceWarning",
140-
"ignore:run_pre_save_hook is deprecated:DeprecationWarning"
140+
"ignore:run_pre_save_hook is deprecated:DeprecationWarning",
141+
"ignore:jsonschema.RefResolver is deprecated as of v4.18.0, in favor of the:DeprecationWarning"
141142
]
142143

143144
[tool.jupyter_releaser]

‎ui-tests/helpers/SchedulerHelper.ts

Lines changed: 64 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,64 @@
1+
import { Page } from '@playwright/test';
2+
3+
/**
4+
* Helper class for Jupyter Scheduler testing in JupyterLab
5+
*/
6+
export class SchedulerHelper {
7+
constructor(readonly page: Page) {}
8+
9+
/**
10+
* JupyterLab launcher "Notebook Jobs" card selector
11+
*/
12+
get launcherCardSelector() {
13+
return 'div.jp-LauncherCard[title="Notebook Jobs"]';
14+
}
15+
16+
/**
17+
* JupyterLab launcher "Notebook Jobs" card locator
18+
*/
19+
get launcherCardLocator() {
20+
return this.page.locator(this.launcherCardSelector);
21+
}
22+
23+
/**
24+
* JupyterLab notebook toolbar "Create a notebook job" button selector
25+
*/
26+
get notebookToolbarButtonSelector() {
27+
return 'button.jp-ToolbarButtonComponent[data-command="scheduling:create-from-notebook"][title="Create a notebook job"]';
28+
}
29+
30+
/**
31+
* JupyterLab notebook toolbar "Create a notebook job" button locator
32+
*/
33+
get notebookToolbarButtonLocator() {
34+
return this.page.locator(this.notebookToolbarButtonSelector);
35+
}
36+
37+
/**
38+
* JupyterLab File Browser right-click menu "Create Notebook Job" item selector
39+
*/
40+
get filebrowserMenuItemSelector() {
41+
return 'li[data-type="command"][data-command="scheduling:create-from-filebrowser"] >> div:has-text("Create Notebook Job")';
42+
}
43+
44+
/**
45+
* JupyterLab File Browser right-click menu "Create Notebook Job" item locator
46+
*/
47+
get filebrowserMenuItemLocator() {
48+
return this.page.locator(this.filebrowserMenuItemSelector);
49+
}
50+
51+
/**
52+
* Notebook jobs panel selector
53+
*/
54+
get timestampSelector() {
55+
return 'td.MuiTableCell-body:has-text(" AM"), td.MuiTableCell-body:has-text(" PM")';
56+
}
57+
58+
/**
59+
* Notebook jobs panel locator
60+
*/
61+
get timestampLocator() {
62+
return this.page.locator(this.timestampSelector);
63+
}
64+
}

‎ui-tests/tests/jupyter_scheduler.spec.ts

Lines changed: 130 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,11 +1,139 @@
11
import { expect, test } from '@jupyterlab/galata';
2+
import { SchedulerHelper } from '../helpers/SchedulerHelper';
3+
4+
const CREATE_FROM_NOTEBOOK_SNAPSHOT_FILENAME = 'create-view-from-toolbar.png';
5+
const CREATE_VIEW_SNAPSHOT_FILENAME = 'create-view-empty.png';
6+
const LAUNCHER_SNAPSHOT_FILENAME = 'launcher-with-scheduler.png';
7+
const LIST_VIEW_IN_PROGRESS_SNAPSHOT_FILENAME = 'list-view-in-progress.png';
8+
const NOTEBOOK_SNAPSHOT_FILENAME= 'notebook-with-createjob-button.png';
9+
const RIGHTCLICK_MENU_SNAPSHOT_FILENAME = 'filebrowser-notebook-rightclick-menu.png';
210

311
/**
412
* Don't load JupyterLab webpage before running the tests.
513
* This is required to ensure we capture all log messages.
614
*/
715
test.use({ autoGoto: false });
816

9-
test('should emit an activation console message', async ({ page }) => {
10-
// TODO: Add real tests
17+
test.describe('Jupyter Scheduler integration tests for JupyterLab', () => {
18+
let schedulerHelper: SchedulerHelper;
19+
test.beforeEach(async ({ page }) => {
20+
schedulerHelper = new SchedulerHelper(page);
21+
await page.goto();
22+
await page.sidebar.close(
23+
(await page.sidebar.getTabPosition('filebrowser')) ?? undefined
24+
);
25+
});
26+
27+
test('"Notebook Jobs" card is visible in JupyterLab launcher', async ({
28+
page
29+
}) => {
30+
const launcher = page.locator('div[role="main"] >> text=Launcher');
31+
await launcher.waitFor();
32+
const launcherCard = schedulerHelper.launcherCardLocator;
33+
34+
await expect(launcherCard).toBeVisible();
35+
expect(await page.screenshot()).toMatchSnapshot(LAUNCHER_SNAPSHOT_FILENAME);
36+
});
37+
38+
test('"Create a notebook job" button in notebook toolbar is visible', async ({
39+
page
40+
}) => {
41+
await page.notebook.createNew();
42+
await page
43+
.locator('.jp-DebuggerBugButton[aria-disabled="false"]')
44+
.waitFor();
45+
await page
46+
.locator('.jp-Notebook-ExecutionIndicator[data-status="idle"]')
47+
.waitFor();
48+
const createJobButton = schedulerHelper.notebookToolbarButtonLocator;
49+
50+
await expect(createJobButton).toBeVisible();
51+
expect(await page.screenshot()).toMatchSnapshot(NOTEBOOK_SNAPSHOT_FILENAME);
52+
await page.menu.clickMenuItem('File>Save Notebook');
53+
await page.click('button:has-text("Rename")');
54+
});
55+
56+
test('"Create a notebook job" button in notebook toolbar leads to "Create a Job" page', async ({
57+
page
58+
}) => {
59+
await page.sidebar.openTab('filebrowser');
60+
expect(await page.sidebar.isTabOpen('filebrowser')).toBeTruthy();
61+
await page.filebrowser.refresh();
62+
await page.dblclick('.jp-DirListing-item[data-file-type="notebook"]');
63+
await page.sidebar.close(
64+
(await page.sidebar.getTabPosition('filebrowser')) ?? undefined
65+
);
66+
const createJobButton = schedulerHelper.notebookToolbarButtonLocator;
67+
await createJobButton.click();
68+
await page.waitForSelector('text=Loading …', { state: 'hidden' });
69+
70+
await page.waitForSelector('text=Saving Completed', { state: 'hidden' });
71+
expect(await page.screenshot()).toMatchSnapshot(CREATE_FROM_NOTEBOOK_SNAPSHOT_FILENAME);
72+
});
73+
74+
test('"Create Notebook Job" item is visible when right clicking a notebook in File Browser', async ({
75+
page
76+
}) => {
77+
await page.sidebar.openTab('filebrowser');
78+
expect(await page.sidebar.isTabOpen('filebrowser')).toBeTruthy();
79+
await page.filebrowser.refresh();
80+
await page.click('.jp-DirListing-item[data-file-type="notebook"]', {
81+
button: 'right'
82+
});
83+
84+
expect(await page.menu.isAnyOpen()).toBe(true);
85+
const righClickMenu = page.locator('ul.lm-Menu-content[role="menu"]');
86+
const createJobItem = schedulerHelper.filebrowserMenuItemLocator;
87+
await expect(createJobItem).toBeVisible();
88+
expect(await righClickMenu.screenshot()).toMatchSnapshot(
89+
RIGHTCLICK_MENU_SNAPSHOT_FILENAME
90+
);
91+
});
92+
93+
test('"Create Notebook Job" button from File Browser right-click menu leads to "Create a Job" page', async ({
94+
page
95+
}) => {
96+
await page.sidebar.openTab('filebrowser');
97+
expect(await page.sidebar.isTabOpen('filebrowser')).toBeTruthy();
98+
await page.filebrowser.refresh();
99+
await page.click('.jp-DirListing-item[data-file-type="notebook"]', {
100+
button: 'right'
101+
});
102+
expect(await page.menu.isAnyOpen()).toBe(true);
103+
const createJobItem = schedulerHelper.filebrowserMenuItemLocator;
104+
await createJobItem.click();
105+
await page.waitForSelector('text=Loading …', { state: 'hidden' });
106+
await page.sidebar.close(
107+
(await page.sidebar.getTabPosition('filebrowser')) ?? undefined
108+
);
109+
110+
expect(await page.screenshot()).toMatchSnapshot(CREATE_VIEW_SNAPSHOT_FILENAME);
111+
});
112+
113+
test('Create a job and see it in the list of jobs', async ({ page }) => {
114+
await page.sidebar.openTab('filebrowser');
115+
await page.filebrowser.refresh();
116+
await page.click('.jp-DirListing-item[data-file-type="notebook"]', {
117+
button: 'right'
118+
});
119+
const createJobItem = schedulerHelper.filebrowserMenuItemLocator;
120+
await createJobItem.click();
121+
await page.waitForSelector('text=Loading', { state: 'hidden' });
122+
123+
await page.fill('input[name=jobName]', 'MyTestJob');
124+
await page.click('button:has-text("Create")');
125+
const jobNameLink = page.getByText('MyTestJob', { exact: true });
126+
jobNameLink.waitFor();
127+
await page.sidebar.close(
128+
(await page.sidebar.getTabPosition('filebrowser')) ?? undefined
129+
);
130+
const timeStamp = schedulerHelper.timestampLocator;
131+
const contentPanel = page.locator('#jp-main-content-panel');
132+
133+
await expect(contentPanel).toHaveScreenshot(LIST_VIEW_IN_PROGRESS_SNAPSHOT_FILENAME, {
134+
mask: [timeStamp],
135+
maskColor: 'white',
136+
maxDiffPixelRatio: 0.01
137+
});
138+
});
11139
});

‎ui-tests/tests/jupyter_scheduler.spec.ts-snapshots/create-view-empty-linux.png

41.2 KB
Loading

‎ui-tests/tests/jupyter_scheduler.spec.ts-snapshots/create-view-from-toolbar-linux.png

43.1 KB
Loading

‎ui-tests/tests/jupyter_scheduler.spec.ts-snapshots/filebrowser-notebook-rightclick-menu-linux.png

27.7 KB
Loading

‎ui-tests/tests/jupyter_scheduler.spec.ts-snapshots/launcher-with-scheduler-linux.png

43.1 KB
Loading

0 commit comments

Comments
 (0)