Skip to content

Conversation

@FilipPyrek
Copy link
Member

@FilipPyrek FilipPyrek commented Oct 21, 2025

Fixes GRN-4766

Summary

Adds integration tests for .deepnote file handling in the VS Code extension. This PR introduces a test suite that verifies loading, kernel startup, and code execution for .deepnote files.

Changes

Test Files

  • src/test/datascience/notebook/deepnote.vscode.test.ts: New integration test suite with 5 tests:

    1. Load .deepnote file and verify notebook type
    2. Verify kernel starts for .deepnote files
    3. Execute code block with output validation
    4. Execute multiple code blocks
    5. Verify cell execution order
  • src/test/datascience/notebook/test.deepnote: Test fixture with minimal .deepnote structure including:

    • Init notebook with initNotebookId reference
    • Main notebook with code and markdown blocks

CI Configuration

  • .github/workflows/ci.yml: New integration-tests job that:
    • Runs on ubicloud-standard-2
    • Uses Python 3.12 and xvfb for headless testing
    • Filters tests with VSC_JUPYTER_CI_TEST_GREP: 'Deepnote Integration Tests'
    • Uses pinned commit SHAs for all GitHub actions

Review Focus

⚠️ Critical items to review:

  1. Init notebook auto-execution: The requirements mentioned testing "Making sure the init notebook runs automatically" but there's no dedicated test for this feature. The fixture includes an initNotebookId, but we should verify if a test is needed to validate automatic execution.

  2. Test fixture validity: The .deepnote file structure was created from type definitions without real-world examples. Verify the YAML structure correctly matches what the deserializer expects.

  3. Test execution pattern: Tests use TestNotebookDocument and manually copy cells from the deserialized notebook. Confirm this accurately represents real .deepnote file usage in VS Code.

  4. Tests not run locally: These integration tests were compiled but not executed. First CI run will be the initial validation.

Testing

  • ✅ TypeScript compilation passes
  • ✅ ESLint passes
  • ✅ Prettier formatting passes
  • ✅ Type checking passes
  • ⏳ Integration tests will run in CI

Link to Devin run: https://app.devin.ai/sessions/e14ceb76944741e79823eb71a2088bd8

Requested by: Filip Pyrek ([email protected]) (@FilipPyrek)

Summary by CodeRabbit

  • Tests

    • Added comprehensive integration tests for Deepnote notebooks, validating kernel initialization, code cell execution, output correctness, and proper execution sequencing across multiple cells with robust error handling and logging.
  • Chores

    • Enhanced CI/CD pipeline with dedicated integration testing infrastructure for Deepnote notebooks, automating environment setup and test execution.

- Create test fixture test.deepnote with init and main notebooks
- Add deepnote.vscode.test.ts with tests for:
  - Loading/opening .deepnote files
  - Kernel startup for .deepnote files
  - Code block execution with output validation
  - Multiple code block execution
  - Cell execution order verification
- Add integration-tests CI job for running Deepnote tests on ubicloud
- Configure CI to use Python 3.12 and xvfb for headless testing

Co-Authored-By: Filip Pyrek <[email protected]>
@linear
Copy link

linear bot commented Oct 21, 2025

GRN-4766 Add basic integration tests for a .deepnote file

Just run a block after kernel is instantiated.

We need to be able to run some integrations tests with the .deepnote file in the deepnote/vscode-deepnote repo.

Your task is

  • research how integration testing is implemented in the repo and take inspiration how the tests look like so that you know how to write them
  • try to write integration tests which will test
    • Loading of the .deepnote file
    • Trying to run the .deepnote file
    • Making sure the kernel starts
    • Making sure the init notebook runs
    • Running the blocks and checking the output

Make sure that the newly created integration tests are passing in GitHub actions. We can add more tests later.

Here is a sample .deepnote file

metadata:
  createdAt: '2025-07-21T14:50:41.160Z'
  modifiedAt: '2025-10-13T18:25:48.045Z'
project:
  id: 4686ec79-9341-4ac4-8aba-ec0ea497f818
  name: Filip Pyrek’s Untitled project (ra-158)
  initNotebookId: a5356b1e77b34793a815faa71e75aad5
  notebooks:
    - blocks:
        - blockGroup: b4d73fbb4ab44563a34f2d1251cfd5e2
          content: |-
            ## Initialization Notebook
            This is your initialization notebook.

            **What's this for?**

            You can put custom code you want us to run every time we setup your environment in here. 

            **Awesome! Anything I should not put in here?**

            Please don't install `jupyter` or `jedi` packages, they would break your Deepnote environment. Also, no need to put `!pip install`s here, we already save those automatically!

            **I want to learn more!**

            Great! Just [head over to our docs](https://deepnote.com/docs/project-initialization).
          id: b4d73fbb4ab44563a34f2d1251cfd5e2
          metadata: {}
          sortingKey: a0
          type: markdown
        - blockGroup: c8c0deee42454742961d5d623385af0c
          content: |-
            %%bash
            # If your project has a 'requirements.txt' file, we'll install it here.
            if test -f requirements.txt
              then
                pip install -r ./requirements.txt
              else echo "There's no requirements.txt, so nothing to install."
            fi

            %%bash
            date
          id: c8c0deee42454742961d5d623385af0c
          metadata:
            execution_start: 1759911417964
            execution_millis: 3008
            execution_context_id: 2cd84c6a-d7a9-46cf-a8a3-9bdee70d6c26
          sortingKey: m
          type: code
          executionCount: 4
          outputs:
            - name: stdout
              text: "Requirement already satisfied: cowsay in /root/venv/lib/python3.10/site-packages (from -r ./requirements.txt (line 1)) (6.1)\nRequirement already satisfied: matplotlib in /root/venv/lib/python3.10/site-packages (from -r ./requirements.txt (line 2)) (3.6.3)\nRequirement already satisfied: wordcloud in /root/venv/lib/python3.10/site-packages (from -r ./requirements.txt (line 3)) (1.9.4)\nRequirement already satisfied: python-dateutil>=2.7 in /root/venv/lib/python3.10/site-packages (from matplotlib->-r ./requirements.txt (line 2)) (2.9.0.post0)\nRequirement already satisfied: kiwisolver>=1.0.1 in /root/venv/lib/python3.10/site-packages (from matplotlib->-r ./requirements.txt (line 2)) (1.4.9)\nRequirement already satisfied: fonttools>=4.22.0 in /root/venv/lib/python3.10/site-packages (from matplotlib->-r ./requirements.txt (line 2)) (4.59.2)\nRequirement already satisfied: contourpy>=1.0.1 in /root/venv/lib/python3.10/site-packages (from matplotlib->-r ./requirements.txt (line 2)) (1.3.2)\nRequirement already satisfied: packaging>=20.0 in /root/venv/lib/python3.10/site-packages (from matplotlib->-r ./requirements.txt (line 2)) (25.0)\nRequirement already satisfied: pillow>=6.2.0 in /root/venv/lib/python3.10/site-packages (from matplotlib->-r ./requirements.txt (line 2)) (11.3.0)\nRequirement already satisfied: pyparsing>=2.2.1 in /root/venv/lib/python3.10/site-packages (from matplotlib->-r ./requirements.txt (line 2)) (3.2.3)\nRequirement already satisfied: cycler>=0.10 in /root/venv/lib/python3.10/site-packages (from matplotlib->-r ./requirements.txt (line 2)) (0.12.1)\nRequirement already satisfied: numpy>=1.19 in /root/venv/lib/python3.10/site-packages (from matplotlib->-r ./requirements.txt (line 2)) (1.25.2)\nRequirement already satisfied: six>=1.5 in /root/venv/lib/python3.10/site-packages (from python-dateutil>=2.7->matplotlib->-r ./requirements.txt (line 2)) (1.17.0)\n\n\e[1m[\e[0m\e[34;49mnotice\e[0m\e[1;39;49m]\e[0m\e[39;49m A new release of pip is available: \e[0m\e[31;49m23.0.1\e[0m\e[39;49m -> \e[0m\e[32;49m25.2\e[0m\n\e[1m[\e[0m\e[34;49mnotice\e[0m\e[1;39;49m]\e[0m\e[39;49m To update, run: \e[0m\e[32;49mpip install --upgrade pip\e[0m\n"
              output_type: stream
      executionMode: block
      id: a5356b1e77b34793a815faa71e75aad5
      isModule: false
      name: Init
    - blocks:
        - blockGroup: e2e82bbb444341abaaf2d91523c597fe
          content: generate nice sample app using python libs
          id: 7b98bc6613e74dfb83e54f35c0de0893
          metadata:
            color: purple
          sortingKey: m
          type: text-cell-callout
        - blockGroup: 47a182784451441ea7f7fd07d1f797b1
          content: I'll create a simple but nice sample application using Python libraries\\\\\\\\\\\\\\\. I'll create a streamlit app that demonstrates some data visualization and interactivity\\\\\\\\\\\\\\\.
          id: 89202edc4e8449788f4915050afc451b
          metadata: {}
          sortingKey: s
          type: text-cell-p
        - blockGroup: de1c79adbc9c48e39aef3b0bd77c22dc
          content: |-
            import cowsay

            cowsay.cow('Hello World')
          id: a146301be41e41608c54d7100c74274c
          metadata:
            execution_start: 1759908563054
            execution_millis: 0
            execution_context_id: 89673497-22e7-4b79-b414-de052702cd5f
          sortingKey: v
          type: code
          executionCount: 1
          outputs:
            - name: stdout
              text: |2
                  ___________
                | Hello World |
                  ===========
                           \
                            \
                              ^__^
                              (oo)\_______
                              (__)\       )\/\
                                  ||----w |
                                  ||     ||
              output_type: stream
        - blockGroup: default-group
          content: |-
            import numpy as np


            # Create a 2-D array, set every second element in
            # some rows and find max per row:

            x = np.arange(15, dtype=np.int64).reshape(3, 5)
            x[1:, ::2] = -99
            x
            # array([[  0,   1,   2,   3,   4],
            #        [-99,   6, -99,   8, -99],
            #        [-99,  11, -99,  13, -99]])

            x.max(axis=1)
            # array([ 4,  8, 13])

            # Generate normally distributed random numbers:
            rng = np.random.default_rng()
            samples = rng.normal(size=2500)
            samples
          id: 15465f69465690fc73f358ec72862e51
          metadata: {}
          sortingKey: a3
          type: code
          outputs:
            - data:
                text/plain: |-
                  array([ 0.05253606, -0.3891007 ,  0.16905018, ..., -0.62507232,
                          0.89273893, -0.96078015])
              execution_count: 3
              output_type: execute_result
              metadata:
                outputType: execute_result
                metadata: {}
        - blockGroup: default-group
          content: |-
            import pandas as pd

            df = pd.DataFrame({"num_legs": [2, 4], "num_wings": [2, 0]}, index=["falcon", "dog"])

            df
          id: 6d7ab6e47660832391b810ffa918d82f
          metadata: {}
          sortingKey: a4
          type: code
          outputs:
            - data:
                application/vnd.deepnote.dataframe.v3+json:
                  column_count: 2
                  columns:
                    - name: num_legs
                      dtype: int64
                      stats:
                        unique_count: 2
                        nan_count: 0
                        min: '2'
                        max: '4'
                        histogram:
                          - bin_start: 2
                            bin_end: 2.2
                            count: 1
                          - bin_start: 2.2
                            bin_end: 2.4
                            count: 0
                          - bin_start: 2.4
                            bin_end: 2.6
                            count: 0
                          - bin_start: 2.6
                            bin_end: 2.8
                            count: 0
                          - bin_start: 2.8
                            bin_end: 3
                            count: 0
                          - bin_start: 3
                            bin_end: 3.2
                            count: 0
                          - bin_start: 3.2
                            bin_end: 3.4000000000000004
                            count: 0
                          - bin_start: 3.4000000000000004
                            bin_end: 3.6
                            count: 0
                          - bin_start: 3.6
                            bin_end: 3.8
                            count: 0
                          - bin_start: 3.8
                            bin_end: 4
                            count: 1
                        categories: null
                    - name: num_wings
                      dtype: int64
                      stats:
                        unique_count: 2
                        nan_count: 0
                        min: '0'
                        max: '2'
                        histogram:
                          - bin_start: 0
                            bin_end: 0.2
                            count: 1
                          - bin_start: 0.2
                            bin_end: 0.4
                            count: 0
                          - bin_start: 0.4
                            bin_end: 0.6000000000000001
                            count: 0
                          - bin_start: 0.6000000000000001
                            bin_end: 0.8
                            count: 0
                          - bin_start: 0.8
                            bin_end: 1
                            count: 0
                          - bin_start: 1
                            bin_end: 1.2000000000000002
                            count: 0
                          - bin_start: 1.2000000000000002
                            bin_end: 1.4000000000000001
                            count: 0
                          - bin_start: 1.4000000000000001
                            bin_end: 1.6
                            count: 0
                          - bin_start: 1.6
                            bin_end: 1.8
                            count: 0
                          - bin_start: 1.8
                            bin_end: 2
                            count: 1
                        categories: null
                    - name: _deepnote_index_column
                      dtype: object
                  row_count: 2
                  preview_row_count: 2
                  rows:
                    - num_legs: 2
                      num_wings: 2
                      _deepnote_index_column: falcon
                    - num_legs: 4
                      num_wings: 0
                      _deepnote_index_column: dog
                  type: dataframe
                text/html: |-
                  <div>
                  <style scoped>
                      .dataframe tbody tr th:only-of-type {
                          vertical-align: middle;
                      }

                      .dataframe tbody tr th {
                          vertical-align: top;
                      }

                      .dataframe thead th {
                          text-align: right;
                      }
                  </style>
                  <table border="1" class="dataframe">
                    <thead>
                      <tr style="text-align: right;">
                        <th></th>
                        <th>num_legs</th>
                        <th>num_wings</th>
                      </tr>
                    </thead>
                    <tbody>
                      <tr>
                        <th>falcon</th>
                        <td>2</td>
                        <td>2</td>
                      </tr>
                      <tr>
                        <th>dog</th>
                        <td>4</td>
                        <td>0</td>
                      </tr>
                    </tbody>
                  </table>
                  </div>
                text/plain: |2-
                          num_legs  num_wings
                  falcon         2          2
                  dog            4          0
              execution_count: 4
              output_type: execute_result
              metadata:
                outputType: execute_result
                metadata: {}
      executionMode: block
      id: d8403aaa3cd9462a8051a75b8c1eec42
      isModule: false
      name: Notebook 1
  settings:
    requirements:
      - cowsay
      - matplotlib
      - wordcloud
version: 1.0.0

@devin-ai-integration
Copy link

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR that start with 'DevinAI' or '@devin'.
  • Look at CI failures and help fix them

⚙️ Control Options:

  • Disable automatic comment and CI monitoring

@coderabbitai
Copy link
Contributor

coderabbitai bot commented Oct 21, 2025

📝 Walkthrough

Walkthrough

Adds integration tests for Deepnote notebook execution. A new CI job runs the test suite on a dedicated runner with 30-minute timeout. The test file loads a .deepnote fixture, initializes a Jupyter kernel, executes notebook cells, and validates outputs and execution order. Includes setup/teardown with screenshot capture on failures.

Sequence Diagram

sequenceDiagram
    participant Test
    participant TestEnv as Test Environment
    participant Kernel
    participant Notebook

    Test->>TestEnv: Initialize test environment
    TestEnv->>TestEnv: Start Jupyter server
    TestEnv->>Notebook: Load test.deepnote fixture
    
    Test->>Kernel: Create kernel instance
    Kernel->>Kernel: Execute init notebook
    Kernel-->>Test: Kernel ready
    
    Test->>Notebook: Execute code cell 1
    Notebook->>Kernel: Run code
    Kernel-->>Notebook: Output result
    Notebook-->>Test: Verify output
    
    Test->>Notebook: Execute code cell 2
    Notebook->>Kernel: Run code
    Kernel-->>Notebook: Output result
    Notebook-->>Test: Verify output & execution order
    
    Test->>TestEnv: Cleanup & capture logs
    TestEnv-->>Test: Test complete
Loading

Possibly related PRs

Suggested reviewers

  • Artmann
  • saltenasl

Pre-merge checks

❌ Failed checks (1 inconclusive)
Check name Status Explanation Resolution
Linked Issues Check ❓ Inconclusive The pull request addresses most objectives from GRN-4766: it adds a test suite exercising .deepnote file loading [deepnote.vscode.test.ts], includes a test fixture [test.deepnote], instantiates and starts kernels, executes code blocks with output assertions, integrates with CI via a new GitHub Actions job, and follows existing test conventions (setup/teardown, error handling). However, a specific concern noted in the PR description raises uncertainty: the requirement to "verify the project's init notebook runs on kernel startup" is listed as a review focus item, with the fixture containing initNotebookId but no explicit dedicated test mentioned in the summary. The author flags this as an open question for reviewers, creating ambiguity about whether this core requirement is satisfied. Review the test code to confirm whether the init notebook execution requirement is explicitly tested or implicitly validated during kernel startup. If no test covers automatic init notebook execution, request the author add a dedicated test case or clarify how the existing tests satisfy this objective.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title Check ✅ Passed The title "Add integration tests for .deepnote file handling (GRN-4766)" clearly and concisely summarizes the main change in the pull request. It is specific enough that someone reviewing the repository history would understand the primary focus (integration tests for .deepnote file handling), avoids vague terms, and directly corresponds to the three files modified: the CI workflow job, the test suite, and the test fixture.
Out of Scope Changes Check ✅ Passed All modified files are directly related to the integration test objective. Changes include: (1) the CI workflow job addition to execute the new tests in GitHub Actions; (2) the main test suite file implementing the integration tests; (3) the test fixture providing the .deepnote structure. No unrelated or tangential changes were introduced outside the stated scope of adding .deepnote integration tests.
Docstring Coverage ✅ Passed No functions found in the changes. Docstring coverage check skipped.

Comment @coderabbitai help to get the list of available commands and usage tips.

@codecov
Copy link

codecov bot commented Oct 21, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 71%. Comparing base (55d8454) to head (c6e16ec).
⚠️ Report is 28 commits behind head on main.
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@          Coverage Diff          @@
##            main    #103   +/-   ##
=====================================
  Coverage     71%     71%           
=====================================
  Files        527     527           
  Lines      39474   39474           
  Branches    4933    4933           
=====================================
  Hits       28277   28277           
  Misses      9563    9563           
  Partials    1634    1634           
🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 10

📜 Review details

Configuration used: CodeRabbit UI

Review profile: ASSERTIVE

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 55d8454 and c6e16ec.

📒 Files selected for processing (3)
  • .github/workflows/ci.yml (1 hunks)
  • src/test/datascience/notebook/deepnote.vscode.test.ts (1 hunks)
  • src/test/datascience/notebook/test.deepnote (1 hunks)
🧰 Additional context used
📓 Path-based instructions (4)
**/!(*.node|*.web).ts

📄 CodeRabbit inference engine (.github/copilot-instructions.md)

Place shared cross-platform logic in common .ts files (not suffixed with .node or .web)

Files:

  • src/test/datascience/notebook/deepnote.vscode.test.ts
**/*.{ts,tsx}

📄 CodeRabbit inference engine (.github/copilot-instructions.md)

**/*.{ts,tsx}: Inject interfaces, not concrete classes
Avoid circular dependencies
Use l10n.t() for user-facing strings
Use typed error classes from src/platform/errors/ when throwing or handling errors
Use the ILogger service instead of console.log
Preserve error details while scrubbing PII in messages and telemetry
Include the Microsoft copyright header in source files
Prefer async/await and handle cancellation with CancellationToken

**/*.{ts,tsx}: Order class members (methods, fields, properties) first by accessibility (public/protected/private) and then alphabetically
Do not add the Microsoft copyright header to new files
Use Uri.joinPath() to construct file paths instead of manual string concatenation
Add a blank line after groups of const declarations and before return statements for readability
Separate third-party imports from local file imports

Files:

  • src/test/datascience/notebook/deepnote.vscode.test.ts
**/!(*.unit).test.ts

📄 CodeRabbit inference engine (.github/copilot-instructions.md)

Place integration tests in *.test.ts files that are not *.unit.test.ts

Files:

  • src/test/datascience/notebook/deepnote.vscode.test.ts
**/*.{test,spec}.ts

📄 CodeRabbit inference engine (.github/instructions/typescript.instructions.md)

In unit tests, when a mock is returned from a promise, ensure the mocked instance has an undefined then property to avoid hanging tests

Files:

  • src/test/datascience/notebook/deepnote.vscode.test.ts
🧠 Learnings (1)
📓 Common learnings
Learnt from: CR
PR: deepnote/vscode-extension#0
File: .github/instructions/kernel-jupyter.instructions.md:0-0
Timestamp: 2025-09-03T12:56:39.535Z
Learning: Use real Jupyter servers for integration tests and mock jupyterlab/services for unit tests; cover both success and failure scenarios and verify cleanup
Learnt from: m1so
PR: deepnote/deepnote-toolkit#262
File: tests/unit/test_jinjasql.py:78-87
Timestamp: 2025-10-21T10:22:53.711Z
Learning: In the deepnote-toolkit repository, unit tests with 50,000 parameters in test_large_inclause (tests/unit/test_jinjasql.py) are acceptable; the full unit test suite runs in ~25 seconds.
🧬 Code graph analysis (1)
src/test/datascience/notebook/deepnote.vscode.test.ts (6)
src/platform/common/types.ts (1)
  • IDisposable (211-213)
src/test/constants.node.ts (1)
  • EXTENSION_ROOT_DIR_FOR_TESTS (11-11)
src/test/datascience/notebook/executionHelper.ts (2)
  • TestNotebookDocument (99-187)
  • createKernelController (25-31)
src/kernels/types.ts (2)
  • IKernel (439-446)
  • INotebookKernelExecution (459-495)
src/platform/logging/index.ts (1)
  • logger (35-48)
src/test/datascience/notebook/helper.ts (1)
  • getDefaultKernelConnection (612-616)
🪛 actionlint (1.7.8)
.github/workflows/ci.yml

148-148: label "ubicloud-standard-2" is unknown. available labels are "windows-latest", "windows-latest-8-cores", "windows-2025", "windows-2022", "windows-11-arm", "ubuntu-latest", "ubuntu-latest-4-cores", "ubuntu-latest-8-cores", "ubuntu-latest-16-cores", "ubuntu-24.04", "ubuntu-24.04-arm", "ubuntu-22.04", "ubuntu-22.04-arm", "macos-latest", "macos-latest-xl", "macos-latest-xlarge", "macos-latest-large", "macos-26-xlarge", "macos-26", "macos-15-intel", "macos-15-xlarge", "macos-15-large", "macos-15", "macos-14-xl", "macos-14-xlarge", "macos-14-large", "macos-14", "macos-13-xl", "macos-13-xlarge", "macos-13-large", "macos-13", "self-hosted", "x64", "arm", "arm64", "linux", "macos", "windows". if it is a custom label for self-hosted runner, set list of labels in actionlint.yaml config file

(runner-label)

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (4)
  • GitHub Check: Build & Package Extension
  • GitHub Check: Integration Tests - Deepnote
  • GitHub Check: Build & Test
  • GitHub Check: Lint & Format
🔇 Additional comments (2)
.github/workflows/ci.yml (1)

148-148: Ignore this review comment; ubicloud-standard-2 is a valid runner label in this codebase.

Per prior verification in this repository, ubicloud-* labels are recognized and accepted. The label is already in use without issues. No actionlint configuration is needed.

Likely an incorrect or invalid review comment.

src/test/datascience/notebook/deepnote.vscode.test.ts (1)

102-106: Remove the suggested test—it's incompatible with the test infrastructure.

The suggested diff calls notebook.insertCodeCell(), but notebook is a TestNotebookDocument mock that provides appendCodeCell(), not insertCodeCell(). The helper function insertCodeCell() requires real VS Code editor context (window.activeNotebookEditor), which this test doesn't have. Additionally, init_variable is undefined in the codebase—the init notebook mechanism doesn't expose such a variable.

If testing init notebook auto-execution is needed, use notebook.appendCodeCell() (line 53 in this file shows the pattern) and verify expected state after kernel startup instead.

Likely an incorrect or invalid review comment.

Comment on lines +166 to +170
- name: Setup Python
uses: actions/setup-python@0b93645e9fea7318ecaed2b359559ac225c90a2b # v5
with:
python-version: '3.12'

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🔴 Critical

🧩 Analysis chain

Ensure Jupyter is available on the runner.

Kernel startup will fail if Jupyter/ipykernel aren’t present. Install minimal deps before tests.

       - name: Setup Python
         uses: actions/setup-python@0b93645e9fea7318ecaed2b359559ac225c90a2b # v5
         with:
           python-version: '3.12'

+      - name: Install Python Jupyter deps
+        run: |
+          python -m pip install --upgrade pip
+          python -m pip install jupyter ipykernel
+
       - name: Install dependencies
         run: npm ci --prefer-offline --no-audit
         env:
           NODE_AUTH_TOKEN: ${{ secrets.GITHUB_TOKEN }}

Also applies to: 171-175



---

🏁 Script executed:

```shell
#!/bin/bash
# Check the full CI workflow to see what happens after Python setup
cat .github/workflows/ci.yml | grep -A 50 "Setup Python"

Length of output: 1472


🏁 Script executed:

#!/bin/bash
# Search for jupyter/ipykernel references in codebase
rg -i "jupyter|ipykernel" --type-list | head -20

Length of output: 389


🏁 Script executed:

#!/bin/bash
# Check package files for Jupyter dependencies
find . -name "package.json" -o -name "pyproject.toml" -o -name "requirements*.txt" -o -name "setup.py" | head -20

Length of output: 457


🏁 Script executed:

#!/bin/bash
# Search for kernel or Jupyter in test files
rg -i "kernel" -C3 --type ts --type js --type py | head -30

Length of output: 2003


🏁 Script executed:

#!/bin/bash
# Check pythonFiles/pyproject.toml to see what Python deps are specified
cat ./pythonFiles/pyproject.toml

Length of output: 79


🏁 Script executed:

#!/bin/bash
# Check package.json test script
rg '"test"' ./package.json -A 2 -B 2

Length of output: 388


🏁 Script executed:

#!/bin/bash
# Search for Jupyter/ipykernel without type restriction
rg "jupyter|ipykernel" -i

Length of output: 262776


🏁 Script executed:

#!/bin/bash
# Look for Python kernel startup in test code
rg "startKernel|kernelspec|launch.*kernel" -i -C 3

Length of output: 280309


Install Jupyter + ipykernel on the runner before running tests.

Many tests start a Jupyter server or assert ipykernel is installed (startJupyterServer, multiple datascience tests). The workflow sets up Python but does not install Jupyter/ipykernel — kernel startup will fail during xvfb-run npm test. Add a step right after "Setup Python" to install minimal Python Jupyter deps (add jupyterlab if you need jupyter lab):

  • File: .github/workflows/ci.yml (after Setup Python step)
  • Suggested step:
    python -m pip install --upgrade pip
    python -m pip install jupyter ipykernel
🤖 Prompt for AI Agents
.github/workflows/ci.yml around lines 166-170: the workflow sets up Python but
does not install Jupyter/ipykernel, causing tests that start a Jupyter server or
require ipykernel to fail; add a new step immediately after the "Setup Python"
step that upgrades pip and installs the minimal Jupyter dependencies (at least
jupyter and ipykernel, optionally jupyterlab if needed) using python -m pip
install --upgrade pip and python -m pip install jupyter ipykernel so kernel
startup succeeds during tests.

Comment on lines +179 to +183
- name: Run Deepnote integration tests
run: xvfb-run -a npm test
env:
VSC_JUPYTER_CI_TEST_GREP: 'Deepnote Integration Tests'

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick | 🔵 Trivial

Capture diagnostics on failures (screenshots/logs).

Tests take screenshots on failure; upload them as artifacts to debug CI.

       - name: Run Deepnote integration tests
         run: xvfb-run -a npm test
         env:
           VSC_JUPYTER_CI_TEST_GREP: 'Deepnote Integration Tests'
+
+      - name: Upload screenshots (on failure)
+        if: failure()
+        uses: actions/upload-artifact@694cdabd8bdb0f10b2cea11669e1bf5453eed0a6 # v4
+        with:
+          name: deepnote-integration-screenshots
+          path: |
+            **/*screenshot*.png
+            **/screenshots/**
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
- name: Run Deepnote integration tests
run: xvfb-run -a npm test
env:
VSC_JUPYTER_CI_TEST_GREP: 'Deepnote Integration Tests'
- name: Run Deepnote integration tests
run: xvfb-run -a npm test
env:
VSC_JUPYTER_CI_TEST_GREP: 'Deepnote Integration Tests'
- name: Upload screenshots (on failure)
if: failure()
uses: actions/upload-artifact@694cdabd8bdb0f10b2cea11669e1bf5453eed0a6 # v4
with:
name: deepnote-integration-screenshots
path: |
**/*screenshot*.png
**/screenshots/**
🤖 Prompt for AI Agents
.github/workflows/ci.yml around lines 179 to 183: the Deepnote integration test
step runs xvfb-run npm test but does not upload failure artifacts
(screenshots/logs); update the workflow to add a subsequent step that uses
actions/upload-artifact to upload the test failure artifacts (e.g., the
screenshots directory and any log files) with a clear artifact name, run it
conditioned to always() or at least on failure() so artifacts are preserved when
tests fail, and ensure paths match where tests write screenshots and logs and
that the step runs in the same job/workspace so the files are available.

Comment on lines +1 to +2
// Copyright (c) Microsoft Corporation.
// Licensed under the MIT License.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion | 🟠 Major

Remove header from new test file (guidelines).

New files shouldn’t include the Microsoft copyright header.

As per coding guidelines.

-// Copyright (c) Microsoft Corporation.
-// Licensed under the MIT License.
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
// Copyright (c) Microsoft Corporation.
// Licensed under the MIT License.
🤖 Prompt for AI Agents
In src/test/datascience/notebook/deepnote.vscode.test.ts around lines 1 to 2,
remove the Microsoft copyright header present at the top of this new test file;
simply delete those header lines so the file follows project guidelines that new
files must not include the Microsoft copyright header.

/* eslint-disable @typescript-eslint/no-require-imports, @typescript-eslint/no-var-requires */
import { assert } from 'chai';
import * as path from '../../../platform/vscode-path/path';
import { Uri, workspace } from 'vscode';
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion | 🟠 Major

Import NotebookCellKind and use constants, not magic numbers.

Avoid brittle numeric enums in tests.

-import { Uri, workspace } from 'vscode';
+import { NotebookCellKind, Uri, workspace } from 'vscode';

Committable suggestion skipped: line range outside the PR's diff.

🤖 Prompt for AI Agents
In src/test/datascience/notebook/deepnote.vscode.test.ts around line 7, the test
currently imports Uri and workspace and uses magic numeric values for notebook
cell kinds; update the import to also include NotebookCellKind from 'vscode' and
replace any numeric literals (e.g. 0/1 or other numbers used to denote cell
kinds) with the appropriate NotebookCellKind constants (NotebookCellKind.Markup,
NotebookCellKind.Code) so the test uses named enum values instead of brittle
magic numbers.

Comment on lines +27 to +29
const deepnoteFileUri = Uri.file(
path.join(EXTENSION_ROOT_DIR_FOR_TESTS, 'src', 'test', 'datascience', 'notebook', 'test.deepnote')
);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion | 🟠 Major

Use Uri.joinPath per guidelines.

Prefer Uri.joinPath over path.join + Uri.file.

As per coding guidelines.

-    const deepnoteFileUri = Uri.file(
-        path.join(EXTENSION_ROOT_DIR_FOR_TESTS, 'src', 'test', 'datascience', 'notebook', 'test.deepnote')
-    );
+    const deepnoteFileUri = Uri.joinPath(
+        Uri.file(EXTENSION_ROOT_DIR_FOR_TESTS),
+        'src',
+        'test',
+        'datascience',
+        'notebook',
+        'test.deepnote'
+    );
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
const deepnoteFileUri = Uri.file(
path.join(EXTENSION_ROOT_DIR_FOR_TESTS, 'src', 'test', 'datascience', 'notebook', 'test.deepnote')
);
const deepnoteFileUri = Uri.joinPath(
Uri.file(EXTENSION_ROOT_DIR_FOR_TESTS),
'src',
'test',
'datascience',
'notebook',
'test.deepnote'
);
🤖 Prompt for AI Agents
In src/test/datascience/notebook/deepnote.vscode.test.ts around lines 27 to 29,
replace the Uri.file(path.join(...)) usage with Uri.joinPath to construct the
file URI; specifically, call Uri.joinPath starting from
Uri.file(EXTENSION_ROOT_DIR_FOR_TESTS) and pass the path segments
('src','test','datascience','notebook','test.deepnote') as separate arguments so
the URI is built using VS Code's recommended API.

Comment on lines +52 to +60
if (cell.kind === 1) {
return notebook.appendCodeCell(
cell.document.getText(),
cell.document.languageId,
cell.metadata
);
} else {
return notebook.appendMarkdownCell(cell.document.getText(), cell.metadata);
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Replace magic numbers with NotebookCellKind.

1 = Code, 2 = Markup today, but using constants is safer and clearer.

-                    if (cell.kind === 1) {
+                    if (cell.kind === NotebookCellKind.Code) {
                         return notebook.appendCodeCell(
                             cell.document.getText(),
                             cell.document.languageId,
                             cell.metadata
                         );
                     } else {
                         return notebook.appendMarkdownCell(cell.document.getText(), cell.metadata);
                     }
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
if (cell.kind === 1) {
return notebook.appendCodeCell(
cell.document.getText(),
cell.document.languageId,
cell.metadata
);
} else {
return notebook.appendMarkdownCell(cell.document.getText(), cell.metadata);
}
if (cell.kind === NotebookCellKind.Code) {
return notebook.appendCodeCell(
cell.document.getText(),
cell.document.languageId,
cell.metadata
);
} else {
return notebook.appendMarkdownCell(cell.document.getText(), cell.metadata);
}
🤖 Prompt for AI Agents
In src/test/datascience/notebook/deepnote.vscode.test.ts around lines 52 to 60,
replace the magic numeric cell.kind checks with the NotebookCellKind enum to
make intent explicit and robust: use NotebookCellKind.Code instead of 1 and
NotebookCellKind.Markup (or NotebookCellKind.Markdown if your VSCode version
uses that name) instead of 2; ensure NotebookCellKind is imported from 'vscode'
at the top of the file if not already, and update the conditional branches to
compare against those enum members.

await waitForExecutionCompletedSuccessfully(cell!);

assert.isTrue(cell!.executionSummary?.success, 'Cell execution should succeed');
assert.isAtLeast(cell!.outputs.length, 0, 'Cell should have outputs');
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Vacuous assertion; require at least one output.

Length >= 0 is always true.

-        assert.isAtLeast(cell!.outputs.length, 0, 'Cell should have outputs');
+        assert.isAtLeast(cell!.outputs.length, 1, 'Cell should have outputs');
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
assert.isAtLeast(cell!.outputs.length, 0, 'Cell should have outputs');
assert.isAtLeast(cell!.outputs.length, 1, 'Cell should have outputs');
🤖 Prompt for AI Agents
In src/test/datascience/notebook/deepnote.vscode.test.ts around line 118, the
assertion assert.isAtLeast(cell!.outputs.length, 0, 'Cell should have outputs')
is vacuous (always true); change it to require at least one output by asserting
length >= 1 (e.g., use assert.isAtLeast(cell!.outputs.length, 1, 'Cell should
have at least one output') or assert.isAbove(cell!.outputs.length, 0, 'Cell
should have at least one output')).

Comment on lines +127 to +129
const codeCells = notebook.cells.filter((c) => c.kind === 2); // Code cells
assert.isAtLeast(codeCells.length, 1, 'Should have at least one code cell');

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🔴 Critical

Wrong kind constant; this filters markup cells, not code.

Use NotebookCellKind.Code.

-        const codeCells = notebook.cells.filter((c) => c.kind === 2); // Code cells
+        const codeCells = notebook.cells.filter((c) => c.kind === NotebookCellKind.Code); // Code cells
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
const codeCells = notebook.cells.filter((c) => c.kind === 2); // Code cells
assert.isAtLeast(codeCells.length, 1, 'Should have at least one code cell');
const codeCells = notebook.cells.filter((c) => c.kind === NotebookCellKind.Code); // Code cells
assert.isAtLeast(codeCells.length, 1, 'Should have at least one code cell');
🤖 Prompt for AI Agents
In src/test/datascience/notebook/deepnote.vscode.test.ts around lines 127 to
129, the test filters cells using the numeric constant 2 which corresponds to
markup cells; change the filter to use NotebookCellKind.Code instead (import
NotebookCellKind from 'vscode' if not already imported) so the test selects
actual code cells, and keep the assertion as-is.

Comment on lines +141 to +145
const codeCells = notebook.cells.filter((c) => c.kind === 2 && c.document.getText().trim().length > 0);
if (codeCells.length < 2) {
this.skip();
}

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🔴 Critical

Same issue here; correct the code-cell filter.

Use NotebookCellKind.Code and keep non-empty check.

-        const codeCells = notebook.cells.filter((c) => c.kind === 2 && c.document.getText().trim().length > 0);
+        const codeCells = notebook.cells.filter(
+            (c) => c.kind === NotebookCellKind.Code && c.document.getText().trim().length > 0
+        );
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
const codeCells = notebook.cells.filter((c) => c.kind === 2 && c.document.getText().trim().length > 0);
if (codeCells.length < 2) {
this.skip();
}
const codeCells = notebook.cells.filter(
(c) => c.kind === NotebookCellKind.Code && c.document.getText().trim().length > 0
);
🤖 Prompt for AI Agents
In src/test/datascience/notebook/deepnote.vscode.test.ts around lines 141 to
145, the notebook cell filter uses a magic number (2) for the cell kind; replace
that with NotebookCellKind.Code and keep the non-empty document text check. If
NotebookCellKind is not already imported from 'vscode', add the import. Ensure
the filter reads something like: notebook.cells.filter(c => c.kind ===
NotebookCellKind.Code && c.document.getText().trim().length > 0) and run tests.

Comment on lines +11 to +19
blocks:
- id: init-block-1
type: code
sortingKey: '0001'
content: |
# This is the init notebook
init_variable = "initialized"
print("Init notebook executed")
- id: main-notebook-id
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick | 🔵 Trivial

Validate fixture schema against the deserializer; consider adding language for code blocks.

If the deserializer keys differ (e.g., expects language), parsing may degrade. Add language to be explicit.

       blocks:
         - id: init-block-1
           type: code
+          language: python
           sortingKey: '0001'
           content: |
             # This is the init notebook
             init_variable = "initialized"
             print("Init notebook executed")
@@
         - id: block-1
           type: code
+          language: python
           sortingKey: '0001'
           content: |
             print("Hello World")
         - id: block-2
           type: code
+          language: python
           sortingKey: '0002'
           content: |
             x = 42
             print(f"The answer is {x}")

Also applies to: 22-33

🤖 Prompt for AI Agents
In src/test/datascience/notebook/test.deepnote around lines 11 to 19 (and also
apply the same change to 22-33), the fixture's code block objects lack an
explicit language key which can break the deserializer that expects a language
field; update each code block object to include a language property (e.g.,
language: "python") and ensure the fixture schema matches the deserializer's
expected keys, then run the deserializer unit tests to verify parsing succeeds.

@saltenasl
Copy link
Member

@FilipPyrek rabbit has some valid feedback please address that first before opening your PRs

@FilipPyrek
Copy link
Member Author

@FilipPyrek rabbit has some valid feedback please address that first before opening your PRs

@saltenasl probably some missclick, I'm not aware of marking this PR as ready for review

@FilipPyrek FilipPyrek marked this pull request as draft October 22, 2025 15:19
@FilipPyrek FilipPyrek closed this Oct 30, 2025
@FilipPyrek FilipPyrek deleted the devin/GRN-4766-1761059010 branch October 30, 2025 16:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants