Release on R-universe

[eitsupi]

How about releasing on R-universe?
It should be possible to build on R-universe with a few modifications to the Makefile.
Like r-universe-org/help#219 (comment)

[sorhawell]

That would be very awesome. I read the discussion. I don't mind if we run msvc or gnu on windows if it works.

[eitsupi]

If polars build requires nightly toolchains, we may need to do the following on the Makefile anyway.

rustup:
	rustup toolchain install nightly || true

[eitsupi]

I looked into it and it seems that the dependency arrow2 cannot be used without installing the nightly tool chain, but the Linux builder in R-universe does not have rustup installed, so the nightly tool chain cannot be used......
This means that binaries cannot be distributed on R-universe, since mac and Windows jobs will not run without a successful build in the Linux builder.

Perhaps it is necessary to use the pre-build binaries and install method as string2path has done?
https://github.com/yutannihilation/string2path/blob/v0.1.0/tools/configure.R

[sorhawell]

Nice thanks for noticing.

Does R universe reject any build that fails R CMD check or can that be disabled?
I guess there could be a long term warning for macOS as described in #35. The package works just fine still.

[sorhawell]

oh btw I swapped to Swatinem/rust-cache@v2 in check-workflow, which main polars is also using. It brings down compile times a bit more and does not need much customizing. I flushes the cache once daily it seems, which is nice too.

[eitsupi]

Does R universe reject any build that fails R CMD check or can that be disabled?

If the first time Linux build failed, I believe it was not registered in R-universe.

If the Linux build is successful, the source will be released.
After that, if the mac build and Windows build are successful, a mac binary release and a Windows binary release will be made, respectively.
If the Linux build fails, all subsequent processes are skipped and nothing is released.

[sorhawell]

• I watched this tutorial video
• Signed in on r-universe with rpolars organization account (just activating all repos, even though they are old)
• Made a universe repo with the following file

The rpolars R-universe homepage

Some failed packages might appear within some hours

[sorhawell]

Now r-polars here has a branch "hello_r_universe" and in rpolars/universe packages.json points to that branch.

@eitsupi if it works R-universe should be building from that branch and you can freely push anything to that branch as you like

[sorhawell]

build status

[eitsupi]

Thank you very much. Just not sure how much I can do......

There are several ways to do this, I think.

1. Download binaries from elsewhere (same as old string2path).
2. Use old polars that can be built with old Rust.
3. Install the toolchain when the installation of rpolars.

1 maybe not worth doing since it is almost the same considering that it can currently be installed from GitHub Releases.
2 is clean but I think we need to use polars that are about 1 year old. (I'll look into it next time.)
3 is not good because it installs the rustup without the user's permission and is not allowed in CRAN, but may be feasible in R-universe...

@jeroen Sorry for tagging you, but do you have any advice? Thanks.

[jeroen]

There are two typos (trailing commas) in your packages.json and therefore it is not valid json and fails to parse, as you can also see from the ❌ in that repo.

[sorhawell]

like this? ... seems to work now thx

[jeroen]

Yes it now builds from that branch...
Build still fails with:

...
  cd ./rust/..
  cargo build --lib --release --manifest-path="./rust/Cargo.toml"
      Updating git repository `[https://github.com/extendr/extendr`](https://github.com/extendr/extendr%60)
      Updating git repository `[https://github.com/pola-rs/polars.git`](https://github.com/pola-rs/polars.git%60)
      Updating crates.io index
  error: no matching package named `polars-core` found
  location searched: https://github.com/pola-rs/polars.git?rev=6e64c8a626cd7a9e143e58939bf524153963237b#6e64c8a6
  required by package `rpolars v0.1.0 (/tmp/RtmpAAqQi1/R.INSTALL8163014e937/rpolars/src/rust)`

[eitsupi]

polars-core inherits versions, etc. from the workspace, and this feature is not available in Rust installed in the R-universe builder, so polars-core download from GitHub fails.

[sorhawell]

1 - As a prologue I try to resolve #4 in branch "long_arms64" based on @yutannihilation string2path recipe the following week. That is quite similar to option 1, and perhaps the makevars script can be adapted to do both. I don't mind having 1 as a backup option. It is not unlikely we have to support GPU's and/or quantum computers one day 😂
2 -
@ritchie46 suggested it would be possible to disable simd feature and compile on stable.

Build still fails with: error: no matching package named polars-core found

oh ok, r-polars and nodejs-polars subscribe to some non public features of the rust-polars crate and to bleeding edge. I guess it would be possible to make a PR to rust-polars exposing all non-public features, and only depend on the rust-polars crate. However that could slow down how fast r-polars can keep up with py-polars features.