Skip to content

Conversation

@quffaro
Copy link
Collaborator

@quffaro quffaro commented Dec 10, 2024

This PR allows users to run a script for building a sysimage and installing it as a kernel. Currently this script builds a sysimage with all AlgebraicJuliaService dependencies, which can be time-consuming on slower machines. (a Thinkpad P15 with an i9 CPU can build this in ~2 minutes).

I need to make sure the Jupyter server chooses the installed kernel.

@quffaro quffaro requested a review from epatters December 10, 2024 17:50
@github-actions
Copy link

github-actions bot commented Dec 10, 2024

@epatters epatters added enhancement New feature or request external Work on interfacing with other tools labels Dec 10, 2024
@epatters
Copy link
Member

epatters commented Dec 11, 2024

Thanks for getting this started! Are you planning to add the test suites for CombinatorialSpaces and Decapodes to the precompile script, as explained here? https://julialang.github.io/PackageCompiler.jl/dev/sysimages.html#tracing

@epatters epatters changed the title Automate Building Sysimages Script to build sys image for AlgebraicJulia service Dec 11, 2024
@quffaro
Copy link
Collaborator Author

quffaro commented Dec 13, 2024

I've added a precompile.jl file loading the test statements, and verified on the frontend that this runs, with the AlgebraicJuliaService loading in ~9 seconds. Should the run_jupyter_service, which is currently hardcoding julia-1.11 kernel, be dynamic?

@quffaro quffaro marked this pull request as ready for review December 13, 2024 21:36
@quffaro quffaro self-assigned this Dec 13, 2024
Also, frontend now uses the default kernel set in the Jupyter server.
return createKernel(serverOptions, {
// XXX: Do I have to specify the Julia version?
name: "julia-1.11",
//name: "julia-1.11",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A comment on my own code: clearly this isn't ideal, but since in practice the only way to launch a Jupyter server that won't reject the frontend on cross-origin grounds is to run our script, this shouldn't be any less reliable than before, and it has the advantage of not baking the Julia version into the bundled JS.

Copy link
Member

@epatters epatters left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks Matt! I've confirmed that I can build the sys image and use it in the frontend.

At least on my machine, the performance benefits seem marginal, unfortunately. Loading the AlgJulia service is noticeably faster, but I can't tell a difference when running the simulation, which is far more time consuming. I haven't done a proper benchmark, though. Would be curious to see what difference others can tell.

Note that, unless you edit the config variables, the jupyter_server.sh uses the standard Julia kernel.

@epatters epatters merged commit 7c7f82c into main Dec 17, 2024
10 checks passed
@epatters epatters deleted the cm/sysimage branch December 17, 2024 06:31
@quffaro
Copy link
Collaborator Author

quffaro commented Dec 17, 2024

I think proper benchmarks are in order (I've created this issue to track it), but the sysimage won't improve simulation speed, just precompilation time. The simulation is generated code, so performance improvements must come from Decapodes itself.

@epatters
Copy link
Member

epatters commented Dec 17, 2024

Right, at most it would improve simulation speed on the first simulation, which I didn't observe.

quffaro added a commit that referenced this pull request Dec 20, 2024
* wip need to make sure jupyter server starts with sysimage

* 9s startup versus 21s startup. adding helper script

* added test precompilation statements

* removed kernel checking conditional

* Delete packages/sysimages/Manifest.toml

* CLEANUP: Move sys image scripts to main Julia dir.

Also, frontend now uses the default kernel set in the Jupyter server.

---------

Co-authored-by: Evan Patterson <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request external Work on interfacing with other tools

Projects

Development

Successfully merging this pull request may close these issues.

3 participants