Better workflow for using Pixi with Julia projects?

[matthewfeickert]

As Julia packages aren't for the most part distributed on conda-forge, what's the best course of action for trying to work on projects that might use Julia, but you'd still like to have controlled with Pixi? Install juliaup with Pixi and then just let Julia control all of its own packages through Project.toml (which then has a corresponding Manifest.toml lock file)? so this would result in keeping the pixi.toml, pixi.lock, Project.toml, and Manifest.toml under version control.

Example:

$ pixi init /tmp/julia-example && cd /tmp/julia-example
$ pixi add juliaup pyjuliacall
✔ Added juliaup >=1.17.21,<2
✔ Added pyjuliacall >=0.9.26,<0.10
$ pixi shell
(julia-example) $ command -v julia
/tmp/julia-example/.pixi/envs/default/bin/julia
(julia-example) $ julia --version
julia version 1.11.6
(julia-example) $ juliaup update
Checking for new Julia versions
(julia-example) $ julia --project=.
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.11.6 (2025-07-09)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |

(julia-example) pkg> add CUDA KernelAbstractions
...
(julia-example) pkg> ^C

julia> 

(julia-example) $ tree -L 1 .
.
├── Manifest.toml
├── pixi.lock
├── pixi.toml
└── Project.toml

1 directory, 4 files
(julia-example) $ cat Project.toml 
[deps]
CUDA = "052768ef-5323-5732-b1bb-66c8b64840ba"
KernelAbstractions = "63c18a36-062a-441e-b654-da1e3ab1ce7c"
$ head Manifest.toml 
# This file is machine-generated - editing it directly is not advised

julia_version = "1.11.6"
manifest_format = "2.0"
project_hash = "90022bbdfb63c9858762922e45e909e272436256"

[[deps.AbstractFFTs]]
deps = ["LinearAlgebra"]
git-tree-sha1 = "d92ad398961a3ed262d8bf04a1a2b8340f915fef"
uuid = "621f4979-c628-5d54-868e-fcf4e3e8185c"

Is there a better workflow than this in practice? Or is the workflow to just not use Pixi here given that the benefit is limited?

[lucascolley]

I think this is pretty much the workflow, yes. See for example how the Pixi repo is set up, we similarly install rust with Pixi and then have a separate Cargo.lock and Cargo.toml.

[wolfv]

I have started a little experiment a while ago (long while) to simply re-package Julia into Conda packages: https://github.com/wolfv/julia-forge

This package is available on https://prefix.dev/julia-forge

There was also some discussion here: https://discourse.julialang.org/t/managing-julia-versions-using-pixi/116165/15

I would love to have a better story for Julia, ideally on conda-forge. Packaging Julia from source is (afaik) quite difficult and also relies on patched LLVM versions (so we cannot simply rely on the LLVM which is already packaged on conda-forge).

[giordano]

For what is worth, most LLVM patches used by julia are backports, see for example the set of patches of llvm 19: llvm/llvm-project@release/19.x...JuliaLang:llvm-project:julia-release/19.x. Not having an army of engineers to constantly integrate LLVM main, upgrade to a newer version of LLVM usually takes a few weeks if not months, which means bugs (and their necessary fixes) are discovered much later than a major release.

[mkitti]

I'm one of the maintainers of the julia package on conda-forge. I have also assisted @MilesCranmer with Python-Julia hybrid packages on conda-forge.

Here is the correct julia-forge link:
https://prefix.dev/channels/julia-forge

If I understand, this is basically just a repackaging of the official package from julialang.org.

I think a relatively stable solution for Julia installation would be integration with the Rust based tool juliaup:
https://github.com/JuliaLang/juliaup

For Julia package management, some integration with Julia's Pkg.jl package manager via jlpkg would be great:
https://github.com/fredrikekre/jlpkg

On the Julia side, it would be nice if there were a nice way to install a package from a tarball. In lieu of this, installation from a shallow git copy of a Julia package repository works. One of the missing pieces is the calculation of a tree hash from a tarball, but I think we have advanced on that since my last examination of the issue.

[mkitti]

Could we just repackage the patched LLVM from these recipes:
https://github.com/JuliaPackaging/Yggdrasil/blob/master/L%2FLLVM%2FlibLLVM%4020%2Fbuild_tarballs.jl

The binaries aee posted here:
https://github.com/JuliaBinaryWrappers/LLVM_jll.jl

The main purpose this would be to allow conda packages to depend on the Julia fork of LLVM .

[matthewfeickert]

I have also assisted @MilesCranmer with Python-Julia hybrid packages on conda-forge.

Nice and thank you! 😎 We physicists have a lot of love for PySR. :)

Here is the correct julia-forge link: https://prefix.dev/channels/julia-forge

If I understand, this is basically just a repackaging of the official package from julialang.org.

This is pretty cool, @wolfv. :)

$ docker run --rm -ti ghcr.io/prefix-dev/pixi:latest 
root@13317b98dff9:/# pixi init example && cd example
Created /example/pixi.toml
root@13317b98dff9:/example# pixi workspace channel add --prepend https://prefix.dev/julia-forge
Added https://prefix.dev/julia-forge
root@13317b98dff9:/example# pixi add julia
Added julia >=1.10.4,<1.11
root@13317b98dff9:/example# pixi list
Package  Version  Build       Size       Kind   Source
julia    1.10.4   hb0f4dca_0  129.3 MiB  conda  https://prefix.dev/julia-forge/
root@13317b98dff9:/example# pixi run julia
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.10.4 (2024-06-04)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |

julia> 

It would be interesting to see how the packaging of Julia projects looks like, but I guess that as Julia needs (?) to do the precompilation locally that a "Julia package conda packag"e is actually just the source distribution (?) — kinda weird, but I guess there's no alternative.

Could we just repackage the patched LLVM from these recipes:
https://github.com/JuliaPackaging/Yggdrasil/blob/master/L%2FLLVM%2FlibLLVM%4020%2Fbuild_tarballs.jl

I think you would need a conda-forge/core person (e.g. @wolfv) to weigh in here, but as long as things are well defined and clearly distinct then probably?

[mkitti]

I think you would need a conda-forge/core person (e.g. @wolfv) to weigh in here, but as long as things are well defined and clearly distinct then probably?

The entirely point of julia-forge is that we cannot really satisfy conda-forge rules which requires from source build receipes. Essentially, conda-forge would need to duplicate what we're already doing in https://github.com/JuliaPackaging/Yggdrasil which already requires a substantial effort. Particularly, conda-forge would need to also produce packages for Julia components with Julia-specific patches.

Instead of duplicating this effort, we could just make julia-forge a conda repackaging of Yggdrasil / BinaryBuilder packages, particularly where Julia specific patches are needed since we are already building from source in Yggdrasil.

I'm not particularly sure if this is straight forward. Julia JLLs typically resolve dependencies by building up LD_LIBRARY_PATH whereas the conda approach is to use a placeholder path and replace that placeholder path in the binary upon installation into an environment.