Skip to content

Update UI tests for large notebooks #15

Update UI tests for large notebooks

Update UI tests for large notebooks #15

Triggered via pull request April 3, 2025 15:03
Status Failure
Total duration 4m 22s
Artifacts 1

build.yaml

on: pull_request
Fit to window
Zoom out
Zoom in

Annotations

10 errors and 1 notice
ui-tests/ipywidgets-test/ipywidgets.spec.ts:6:7 › ipywidgets Extension › should render widgets and interact: ui-tests/ipywidgets-test/ipywidgets.spec.ts#L35
1) ui-tests/ipywidgets-test/ipywidgets.spec.ts:6:7 › ipywidgets Extension › should render widgets and interact Error: expect(Buffer).toMatchSnapshot(expected) 22829 pixels (ratio 0.03 of all image pixels) are different. Expected: /home/runner/work/jupyterlab-meta/jupyterlab-meta/ui-tests/ipywidgets-test/ipywidgets.spec.ts-snapshots/ipywidgets-after-click-linux.png Received: /home/runner/work/jupyterlab-meta/jupyterlab-meta/test-results/ui-tests-ipywidgets-test-i-19fcd-render-widgets-and-interact/ipywidgets-after-click-actual.png Diff: /home/runner/work/jupyterlab-meta/jupyterlab-meta/test-results/ui-tests-ipywidgets-test-i-19fcd-render-widgets-and-interact/ipywidgets-after-click-diff.png 33 | await page.waitForTimeout(500); 34 | > 35 | expect(await page.screenshot()).toMatchSnapshot( | ^ 36 | "ipywidgets-after-click.png" 37 | ); 38 | }); at /home/runner/work/jupyterlab-meta/jupyterlab-meta/ui-tests/ipywidgets-test/ipywidgets.spec.ts:35:37
ui-tests/ipywidgets-test/ipywidgets.spec.ts:6:7 › ipywidgets Extension › should render widgets and interact: ui-tests/ipywidgets-test/ipywidgets.spec.ts#L35
1) ui-tests/ipywidgets-test/ipywidgets.spec.ts:6:7 › ipywidgets Extension › should render widgets and interact Retry #1 ─────────────────────────────────────────────────────────────────────────────────────── Error: expect(Buffer).toMatchSnapshot(expected) 22829 pixels (ratio 0.03 of all image pixels) are different. Expected: /home/runner/work/jupyterlab-meta/jupyterlab-meta/ui-tests/ipywidgets-test/ipywidgets.spec.ts-snapshots/ipywidgets-after-click-linux.png Received: /home/runner/work/jupyterlab-meta/jupyterlab-meta/test-results/ui-tests-ipywidgets-test-i-19fcd-render-widgets-and-interact-retry1/ipywidgets-after-click-actual.png Diff: /home/runner/work/jupyterlab-meta/jupyterlab-meta/test-results/ui-tests-ipywidgets-test-i-19fcd-render-widgets-and-interact-retry1/ipywidgets-after-click-diff.png 33 | await page.waitForTimeout(500); 34 | > 35 | expect(await page.screenshot()).toMatchSnapshot( | ^ 36 | "ipywidgets-after-click.png" 37 | ); 38 | }); at /home/runner/work/jupyterlab-meta/jupyterlab-meta/ui-tests/ipywidgets-test/ipywidgets.spec.ts:35:37
ui-tests/jupyterlab-code-formatter-test/jupyterlab_code_formatter.spec.ts:6:7 › JupyterLab Code Formatter › should format notebook: ui-tests/jupyterlab-code-formatter-test/jupyterlab_code_formatter.spec.ts#L38
2) ui-tests/jupyterlab-code-formatter-test/jupyterlab_code_formatter.spec.ts:6:7 › JupyterLab Code Formatter › should format notebook Error: expect(Buffer).toMatchSnapshot(expected) 24739 pixels (ratio 0.04 of all image pixels) are different. Expected: /home/runner/work/jupyterlab-meta/jupyterlab-meta/ui-tests/jupyterlab-code-formatter-test/jupyterlab_code_formatter.spec.ts-snapshots/after-format-notebook-linux.png Received: /home/runner/work/jupyterlab-meta/jupyterlab-meta/test-results/ui-tests-jupyterlab-code-f-0a1ad-tter-should-format-notebook/after-format-notebook-actual.png Diff: /home/runner/work/jupyterlab-meta/jupyterlab-meta/test-results/ui-tests-jupyterlab-code-f-0a1ad-tter-should-format-notebook/after-format-notebook-diff.png 36 | await page.waitForTimeout(2000); 37 | > 38 | expect(await page.screenshot()).toMatchSnapshot( | ^ 39 | "after_format_notebook.png" 40 | ); 41 | }); at /home/runner/work/jupyterlab-meta/jupyterlab-meta/ui-tests/jupyterlab-code-formatter-test/jupyterlab_code_formatter.spec.ts:38:37
ui-tests/jupyterlab-code-formatter-test/jupyterlab_code_formatter.spec.ts:6:7 › JupyterLab Code Formatter › should format notebook: ui-tests/jupyterlab-code-formatter-test/jupyterlab_code_formatter.spec.ts#L38
2) ui-tests/jupyterlab-code-formatter-test/jupyterlab_code_formatter.spec.ts:6:7 › JupyterLab Code Formatter › should format notebook Retry #1 ─────────────────────────────────────────────────────────────────────────────────────── Error: expect(Buffer).toMatchSnapshot(expected) 24739 pixels (ratio 0.04 of all image pixels) are different. Expected: /home/runner/work/jupyterlab-meta/jupyterlab-meta/ui-tests/jupyterlab-code-formatter-test/jupyterlab_code_formatter.spec.ts-snapshots/after-format-notebook-linux.png Received: /home/runner/work/jupyterlab-meta/jupyterlab-meta/test-results/ui-tests-jupyterlab-code-f-0a1ad-tter-should-format-notebook-retry1/after-format-notebook-actual.png Diff: /home/runner/work/jupyterlab-meta/jupyterlab-meta/test-results/ui-tests-jupyterlab-code-f-0a1ad-tter-should-format-notebook-retry1/after-format-notebook-diff.png 36 | await page.waitForTimeout(2000); 37 | > 38 | expect(await page.screenshot()).toMatchSnapshot( | ^ 39 | "after_format_notebook.png" 40 | ); 41 | }); at /home/runner/work/jupyterlab-meta/jupyterlab-meta/ui-tests/jupyterlab-code-formatter-test/jupyterlab_code_formatter.spec.ts:38:37
ui-tests/papermill-test/papermill.spec.ts:6:7 › Papermill Execution Test › should run notebook with papermill and verify execution: ui-tests/papermill-test/papermill.spec.ts#L28
3) ui-tests/papermill-test/papermill.spec.ts:6:7 › Papermill Execution Test › should run notebook with papermill and verify execution Error: expect(Buffer).toMatchSnapshot(expected) 42018 pixels (ratio 0.06 of all image pixels) are different. Expected: /home/runner/work/jupyterlab-meta/jupyterlab-meta/ui-tests/papermill-test/papermill.spec.ts-snapshots/papermill-output-linux.png Received: /home/runner/work/jupyterlab-meta/jupyterlab-meta/test-results/ui-tests-papermill-test-pa-66a6a-ermill-and-verify-execution/papermill-output-actual.png Diff: /home/runner/work/jupyterlab-meta/jupyterlab-meta/test-results/ui-tests-papermill-test-pa-66a6a-ermill-and-verify-execution/papermill-output-diff.png 26 | 27 | expect(outputPath).toBeTruthy(); > 28 | expect(await page.screenshot()).toMatchSnapshot("papermill-output.png", { | ^ 29 | maxDiffPixelRatio: 0.05, 30 | }); 31 | }); at /home/runner/work/jupyterlab-meta/jupyterlab-meta/ui-tests/papermill-test/papermill.spec.ts:28:37
ui-tests/papermill-test/papermill.spec.ts:6:7 › Papermill Execution Test › should run notebook with papermill and verify execution: ui-tests/papermill-test/papermill.spec.ts#L28
3) ui-tests/papermill-test/papermill.spec.ts:6:7 › Papermill Execution Test › should run notebook with papermill and verify execution Retry #1 ─────────────────────────────────────────────────────────────────────────────────────── Error: expect(Buffer).toMatchSnapshot(expected) 42011 pixels (ratio 0.06 of all image pixels) are different. Expected: /home/runner/work/jupyterlab-meta/jupyterlab-meta/ui-tests/papermill-test/papermill.spec.ts-snapshots/papermill-output-linux.png Received: /home/runner/work/jupyterlab-meta/jupyterlab-meta/test-results/ui-tests-papermill-test-pa-66a6a-ermill-and-verify-execution-retry1/papermill-output-actual.png Diff: /home/runner/work/jupyterlab-meta/jupyterlab-meta/test-results/ui-tests-papermill-test-pa-66a6a-ermill-and-verify-execution-retry1/papermill-output-diff.png 26 | 27 | expect(outputPath).toBeTruthy(); > 28 | expect(await page.screenshot()).toMatchSnapshot("papermill-output.png", { | ^ 29 | maxDiffPixelRatio: 0.05, 30 | }); 31 | }); at /home/runner/work/jupyterlab-meta/jupyterlab-meta/ui-tests/papermill-test/papermill.spec.ts:28:37
ui-tests/python-lsp-server-test/python_lsp_server.spec.ts:6:7 › Jupyterlab-lsp Extension Test › should open lsp popup: ui-tests/python-lsp-server-test/python_lsp_server.spec.ts#L31
4) ui-tests/python-lsp-server-test/python_lsp_server.spec.ts:6:7 › Jupyterlab-lsp Extension Test › should open lsp popup Error: expect(Buffer).toMatchSnapshot(expected) 27511 pixels (ratio 0.04 of all image pixels) are different. Expected: /home/runner/work/jupyterlab-meta/jupyterlab-meta/ui-tests/python-lsp-server-test/python_lsp_server.spec.ts-snapshots/lsp-popup-fully-initialized-linux.png Received: /home/runner/work/jupyterlab-meta/jupyterlab-meta/test-results/ui-tests-python-lsp-server-177d4--Test-should-open-lsp-popup/lsp-popup-fully-initialized-actual.png Diff: /home/runner/work/jupyterlab-meta/jupyterlab-meta/test-results/ui-tests-python-lsp-server-177d4--Test-should-open-lsp-popup/lsp-popup-fully-initialized-diff.png 29 | await page.waitForTimeout(500); 30 | > 31 | expect(await page.screenshot()).toMatchSnapshot( | ^ 32 | "lsp-popup-fully-initialized.png" 33 | ); 34 | }); at /home/runner/work/jupyterlab-meta/jupyterlab-meta/ui-tests/python-lsp-server-test/python_lsp_server.spec.ts:31:37
ui-tests/python-lsp-server-test/python_lsp_server.spec.ts:6:7 › Jupyterlab-lsp Extension Test › should open lsp popup: ui-tests/python-lsp-server-test/python_lsp_server.spec.ts#L31
4) ui-tests/python-lsp-server-test/python_lsp_server.spec.ts:6:7 › Jupyterlab-lsp Extension Test › should open lsp popup Retry #1 ─────────────────────────────────────────────────────────────────────────────────────── Error: expect(Buffer).toMatchSnapshot(expected) 27511 pixels (ratio 0.04 of all image pixels) are different. Expected: /home/runner/work/jupyterlab-meta/jupyterlab-meta/ui-tests/python-lsp-server-test/python_lsp_server.spec.ts-snapshots/lsp-popup-fully-initialized-linux.png Received: /home/runner/work/jupyterlab-meta/jupyterlab-meta/test-results/ui-tests-python-lsp-server-177d4--Test-should-open-lsp-popup-retry1/lsp-popup-fully-initialized-actual.png Diff: /home/runner/work/jupyterlab-meta/jupyterlab-meta/test-results/ui-tests-python-lsp-server-177d4--Test-should-open-lsp-popup-retry1/lsp-popup-fully-initialized-diff.png 29 | await page.waitForTimeout(500); 30 | > 31 | expect(await page.screenshot()).toMatchSnapshot( | ^ 32 | "lsp-popup-fully-initialized.png" 33 | ); 34 | }); at /home/runner/work/jupyterlab-meta/jupyterlab-meta/ui-tests/python-lsp-server-test/python_lsp_server.spec.ts:31:37
ui-tests/ui.spec.ts:8:7 › Large Notebook Performance Benchmark › should perform benchmark operations: ui-tests/ui.spec.ts#L32
5) ui-tests/ui.spec.ts:8:7 › Large Notebook Performance Benchmark › should perform benchmark operations Error: expect(Buffer).toMatchSnapshot(expected) 21815 pixels (ratio 0.03 of all image pixels) are different. Expected: /home/runner/work/jupyterlab-meta/jupyterlab-meta/ui-tests/ui.spec.ts-snapshots/toc-linux.png Received: /home/runner/work/jupyterlab-meta/jupyterlab-meta/test-results/ui-tests-ui-Large-Notebook-8efd2-erform-benchmark-operations/toc-actual.png Diff: /home/runner/work/jupyterlab-meta/jupyterlab-meta/test-results/ui-tests-ui-Large-Notebook-8efd2-erform-benchmark-operations/toc-diff.png 30 | await page.locator('li[data-id="table-of-contents"]').click(); 31 | await page.waitForTimeout(500); > 32 | expect(await page.screenshot()).toMatchSnapshot("toc.png"); | ^ 33 | await page.locator('li[data-id="table-of-contents"]').click(); 34 | }, 35 | testInfo, at /home/runner/work/jupyterlab-meta/jupyterlab-meta/ui-tests/ui.spec.ts:32:41 at timeit (/home/runner/work/jupyterlab-meta/jupyterlab-meta/ui-tests/utils.ts:7:3) at addBenchmarkToTest (/home/runner/work/jupyterlab-meta/jupyterlab-meta/ui-tests/utils.ts:33:22) at /home/runner/work/jupyterlab-meta/jupyterlab-meta/ui-tests/ui.spec.ts:25:5
ui-tests/ui.spec.ts:8:7 › Large Notebook Performance Benchmark › should perform benchmark operations: ui-tests/ui.spec.ts#L32
5) ui-tests/ui.spec.ts:8:7 › Large Notebook Performance Benchmark › should perform benchmark operations Retry #1 ─────────────────────────────────────────────────────────────────────────────────────── Error: expect(Buffer).toMatchSnapshot(expected) 21808 pixels (ratio 0.03 of all image pixels) are different. Expected: /home/runner/work/jupyterlab-meta/jupyterlab-meta/ui-tests/ui.spec.ts-snapshots/toc-linux.png Received: /home/runner/work/jupyterlab-meta/jupyterlab-meta/test-results/ui-tests-ui-Large-Notebook-8efd2-erform-benchmark-operations-retry1/toc-actual.png Diff: /home/runner/work/jupyterlab-meta/jupyterlab-meta/test-results/ui-tests-ui-Large-Notebook-8efd2-erform-benchmark-operations-retry1/toc-diff.png 30 | await page.locator('li[data-id="table-of-contents"]').click(); 31 | await page.waitForTimeout(500); > 32 | expect(await page.screenshot()).toMatchSnapshot("toc.png"); | ^ 33 | await page.locator('li[data-id="table-of-contents"]').click(); 34 | }, 35 | testInfo, at /home/runner/work/jupyterlab-meta/jupyterlab-meta/ui-tests/ui.spec.ts:32:41 at timeit (/home/runner/work/jupyterlab-meta/jupyterlab-meta/ui-tests/utils.ts:7:3) at addBenchmarkToTest (/home/runner/work/jupyterlab-meta/jupyterlab-meta/ui-tests/utils.ts:33:22) at /home/runner/work/jupyterlab-meta/jupyterlab-meta/ui-tests/ui.spec.ts:25:5
🎭 Playwright Run Summary
5 failed ui-tests/ipywidgets-test/ipywidgets.spec.ts:6:7 › ipywidgets Extension › should render widgets and interact ui-tests/jupyterlab-code-formatter-test/jupyterlab_code_formatter.spec.ts:6:7 › JupyterLab Code Formatter › should format notebook ui-tests/papermill-test/papermill.spec.ts:6:7 › Papermill Execution Test › should run notebook with papermill and verify execution ui-tests/python-lsp-server-test/python_lsp_server.spec.ts:6:7 › Jupyterlab-lsp Extension Test › should open lsp popup ui-tests/ui.spec.ts:8:7 › Large Notebook Performance Benchmark › should perform benchmark operations 11 passed (2.7m)

Artifacts

Produced during runtime
Name Size Digest
playwright-tests Expired
16.5 MB
sha256:3c3a5791c6351b2d93627fc3edeca89adf366717111b9086da00cfd4aff1f9b1