How to benchmark lumino branches/forks?

[krassowski]

Problem

Running benchmarks against custom branches of JupyterLab is a pleasant experience, but benchmarking against custom lumino forks seems tricky. Can we benchmark JupyterLab against a lumino fork/branch?

Proposed Solution

We could add another parameter to the GitHub Action, which would result in using lumino from given fork/branch.

I initially thought about a local NPM registry overwriting the published lumino versions, but probably we should just make use of: https://jupyterlab.readthedocs.io/en/stable/developer/contributing.html#linking-unlinking-packages-to-jupyterlab

Additional Context

@fcollonval I wonder what was your workflow for jupyterlab/lumino#231 was your benchmarking automated or fully manual?

[welcome]

Thank you for opening your first issue in this project! Engagement like this is essential for open source projects! 🤗

If you haven't done so already, check out Jupyter's Code of Conduct. Also, please try to follow the issue template as it helps other other community members to contribute more effectively.

You can meet the other Jovyans by joining our Discourse forum. There is also an intro thread there where you can stop by and say Hi! 👋

Welcome to the Jupyter community! 🎉

[fcollonval]

@fcollonval I wonder what was your workflow for jupyterlab/lumino#231 was your benchmarking automated or fully manual?

At the time I ran tests locally. I did the benchmark CI work afterwards. But indeed, we could duplicate the existing workflow to address this case. It will also require defining test cases as the current ones are probably not highly impacted by lumino.