High Memory Usage on Running Local Notebook

[Alan-Penkar]

Description

I'm using Windsurf and MacOS Tahoe. When running a notebook using the local python environment (.venv folder in working dir), running with the Datalayer runner is taking substantially more memory than with the standard Jupyter Notebook plugin.

Reproduce

I'm using pandas heavy code that I cannot share here, however running a notebook will cause a crash with high memory utilization on the system via Datalayer, but running with Jupyter notebook plugin gets nowhere close.

Expected behavior

Similar memory usage

Context

• Datalayer version: 0.0.17
• Operating System and version: MacOS Tahoe
• Browser and version: N/A

[echarles]

with the Datalayer runner

Datalayer launches with pyodide by default - are you using that (see top right of the notebook)? in which case pyodide will crash fast indeed

... or are you already launching a local kernel via the menu? (that menu supports local jupyter kernel or server, prolly not from a .venv listed in your current folder or remote runtimes)

You can launch that menu clicking on Pyodide

[Alan-Penkar]

This is how I'm selecting the venv -- it's under Python Environments.

[goanpeca]

Hi @Alan-Penkar thanks for the report. Would it be possible for you to share some fake example that mirrors the data you are dealing with so I can check locally what might be causing issues?

Thanks!

[Alan-Penkar]

Sorry for the delay - you caught me as I was going out of the country for a couple week. Let me look back into this and get back to you. I will try and get a minimal example set up that I can share code for.