Skip to content

RSN for conda package legacy labelling #570

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
bdice merged 14 commits into from
Apr 9, 2025
Merged

RSN for conda package legacy labelling #570

Changes from 5 commits
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
e20ea8c
Add notice for conda package `legacy` labelling
gforsyth Jan 27, 2025
e5bbcc6
Update _notices/rsn0042.md
gforsyth Jan 28, 2025
ba89cba
Use correct label command
gforsyth Jan 28, 2025
844022d
Update _notices/rsn0042.md
gforsyth Jan 28, 2025
2417de8
Remove blank line
bdice Jan 28, 2025
5259665
rename file
gforsyth Mar 4, 2025
2f49e64
Merge branch 'main' into channel_priority_notice
gforsyth Mar 4, 2025
15ebde2
address review comments
gforsyth Mar 4, 2025
acb3ac1
Merge branch 'channel_priority_notice' of https://github.com/gforsyth…
gforsyth Mar 4, 2025
2f961fc
Rename to RSN 44
bdice Apr 9, 2025
1b5756d
Merge branch 'main' into channel_priority_notice
bdice Apr 9, 2025
e8e6569
Update dates
bdice Apr 9, 2025
da941a9
Small updates to fix RSN 42 status and formatting.
bdice Apr 9, 2025
3fce008
Update notice status
bdice Apr 9, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
60 changes: 60 additions & 0 deletions _notices/rsn0042.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
---
layout: notice
parent: RAPIDS Support Notices
grand_parent: RAPIDS Notices
nav_exclude: true
notice_type: rsn
# Update meta-data for notice
notice_id: 42 # should match notice number
notice_pin: true # set to true to pin to notice page

title: "Moving older RAPIDS conda packages to a `legacy` label"
notice_author: RAPIDS Ops
notice_status: Proposal
notice_status_color: blue
# 'notice_status' and 'notice_status_color' combinations:
# "Proposal" - "blue"
# "Completed" - "green"
# "Review" - "purple"
# "In Progress" - "yellow"
# "Closed" - "red"
notice_topic: Platform Support Change
notice_rapids_version: "<=v23.02"
notice_created: 2025-01-27
# 'notice_updated' should match 'notice_created' until an update is made
notice_updated: 2025-01-27
---

## Overview

RAPIDS will move certain older conda packages in the `rapidsai` conda channel
under a new label: `legacy`. These packages will still be available by adding
`-c rapidsai/label/legacy` to installation and search commands.

By moving these packages out of the main portion of the channel, newer RAPIDS
conda environments will be solvable with `channel_priority: strict`, speeding up
environment solve times.

All RAPIDS conda install commands for RAPIDS versions `v23.04+` will continue to
work without any user intervention.

This change will occur concurrently with the release of RAPIDS `v25.04`.
Copy link
Contributor

@bdice bdice Jan 28, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just checking: We will be able to test strict channel priority in our CI before the 25.04 release, right? I want to be sure we minimize the risk of breaking recent releases (25.02 / 25.04) during this change.

Copy link
Contributor Author

@gforsyth gforsyth Jan 28, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We'll be able to test 25.02 once all of those packages are released using the same method I tested the previous releases with. Regarding 25.04, we can also release all of those packages, test, then move the selected older packages under the legacy label, or we can move them under the label, see if it works, and then undo the labeling if it breaks.

If 25.02 works, I'd feel confident moving the packages straight to the label for 25.04, and that would let us accurately advertise strict channel priority for the new install instructions.

Copy link
Contributor

@bdice bdice Jan 28, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's helpful. What about enabling strict priority in our CI for PR & nightly builds/tests, though? It sounds like you're discussing the user experience that we expect after the release is complete.

Copy link
Contributor Author

@gforsyth gforsyth Jan 28, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can't really enable them for CI until we move the older packages under legacy. The conda-metachannel package is quite brittle at this point and I don't think we want that spun up as part of our CI

Copy link
Contributor

@bdice bdice Jan 28, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does that mean we will make the legacy label along with the 25.04 release, and then enable strict priority in our CI during development for 25.06?

Copy link
Contributor Author

@gforsyth gforsyth Jan 28, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes

Copy link
Member

@jameslamb jameslamb Mar 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can't really enable them for CI until we move the older packages under legacy

Why is this? I'd think that in RAPIDS CI for 25.04, the pins like ==25.4.* we have for all RAPIDS packages would be sufficient to allow this.

Or said another way... if we're saying this -c rapidsai/label/legacy is only necessary for releases before RAPIDS v23.04, then I'd expect to be able to use strict channel priority with RAPIDS 25.04 today.

(sorry if I'm re-litigating things that have already been discussed, feel free to just respond with links to things I should go read)

Copy link
Contributor

@bdice bdice Mar 4, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're asking a good question. I think the only confusing part here is timing.

We can't enable strict channel priority until we move "legacy" packages (needed for releases prior to 23.04) to the legacy label. There are packages like datashader that are implicated here.

We plan to move things to the legacy label when 25.04 is released. Then we can enable strict priority in 25.06.

However, there is a path where we enable strict priority gradually in each repo's CI, which I think we could start in 25.04. It just won't work for every repo until after the "legacy" move occurs with the 25.04 release.

Copy link
Contributor Author

@gforsyth gforsyth Mar 4, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this? I'd think that in RAPIDS CI for 25.04, the pins like ==25.4.* we have for all RAPIDS packages would be sufficient to allow this.

channel priority doesn't consider versions -- if there is ANY version of a package in a higher priority channel, that is the only channel that gets considered for that package. because we have (especially) a patched version of datashader (although there are other problem packages) in rapidsai, most of our solves won't work because it'll see datashader in rapidsai but the single version in rapidsai won't satisfy the current dependency requirements and the solve will fail.

Copy link
Member

@jameslamb jameslamb Mar 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ahhhhhhh ok, thank you both for explaining. Ok then this makes sense to me as-is.

Agree with @bdice that it'd be really helpful to try to enable this repo-by-repo within the 25.04 cycle, to hopefully prevent accidental introduction of even more things that complicate the move to strict channel priority. And because it's a solution to the "did not install the package built in CI" problem: rapidsai/build-planning#14 (comment)



## Impact

RAPIDS environments installed using `conda` or `mamba`, for environments older
than RAPIDS `v23.04` will fail to install unless `-c rapidsai/label/legacy` is
added to the installation command.

e.g. if you are creating an environment like:

```python
mamba create -n rapids-23.02 rapids=23.02 python=3.10 'cuda-version>=11.4,<=11.8' -c rapidsai -c conda-forge -c nvidia
```

you will need to instead run

```python
mamba create -n rapids-23.02 rapids=23.02 python=3.10 'cuda-version>=11.4,<=11.8' -c rapidsai -c rapidsai/label/legacy -c conda-forge -c nvidia
```