Publishing pyfixest on conda-forge (roadmap and comments)

[timmens]

We publish on PyPI but not conda-forge, which is a gap for the econometrics ecosystem (see requests in previous issues like #549). Since #549 lets-plot has been made an optional dependency and the maketables package has been published on conda-forge (PR conda-forge/staged-recipes/32451), which is why we can tackle this issue now.

Publishing on conda-forge

What remains is to create a conda-forge feedstock via a submission to conda-forge/staged-recipes for pyfixest that builds the rust core using the rust compiler.

Roadmap and comments

In the current situation the pyfixest packaging requires building not only the python code but also compiling the rust kernels. An implication of this is that the rust compiler must be installed in the development environment, and working on the pure python parts of pyfixest always requires compilation of the rust kernels. It also means that even if the rust kernels stabilize, this more complex build process will remain.

One solution to this would be to split the package into a core (pyfixest-core) and a pure-python part (pyfixest). The rust kernels would live inside pyfixest-core which could also follow its own versioning cadence. pyfixest would become a pure-python package (noarch) simply depending on pyfixest-core.

I would only recommend such a split once the rust core is stable. A trigger could be once all numba kernels are ported to rust for instance.

For example, this is done by pydantic with pydantic and pydantic-core.

Open questions

• Should we aim for such a split in the future?
• If we aim for a split, should be publish to conda-forge before the split, or wait?

[s3alfisc]

Thanks @timmens!

I would be in favor of splitting the package into a Rust-core and python logic around it. Essentially, we only have to port the numba code in the randomization inference module (four small functions here) and the crv quantile regression (@abelaba already started this afaik).

As this could be done relatively quickly, I guess I'd say we should wait before we publish on coda-forge?