Mark rc builds of arcticdb post 5.1.2 as broken

[G-D-Petrov]

Guidelines for marking packages as broken:

• We prefer to patch the repo data (see here)
  instead of marking packages as broken. This alternative workflow makes environments more reproducible.
• Packages with requirements/metadata that are too strict but otherwise work are
  not technically broken and should not be marked as such.
• Packages with missing metadata can be marked as broken on a temporary basis
  but should be patched in the repo data and be marked unbroken later.
• In some cases where the number of users of a package is small or it is used by
  the maintainers only, we can allow packages to be marked broken more liberally.
• We (conda-forge/core) try to make a decision on these requests within 24 hours.

What will happen when a package is marked broken?

• Our bots will add the broken label to the package. The main label will remain on the package and this is normal.
• Our bots will rebuild our repodata patches to remove this package from the repodata.
• In a few hours after the anaconda.org CDN picks up the new patches, you will no longer be able to install the package from the main channel.

Checklist:

• I want to mark a package as broken (or not broken):

  • Added a description of the problem with the package in the PR description.
  • Pinged the team for the package for their input.

• I want to archive a feedstock:

  • Pinged the team for that feedstock for their input.
  • Make sure you have opened an issue on the feedstock explaining why it was archived.
  • Linked that issue in this PR description.
  • Added links to any other relevant issues/PRs in the PR description.

• I want to request (or revoke) access to an opt-in CI resource:

  • Pinged the relevant feedstock team(s)
  • Added a small description explaining why access is needed

• I want to copy an artifact following CFEP-3:

  • Pinged the relevant feedstock team(s)
  • Added a reference to the original PR
  • Posted a link to the conda artifacts
  • Posted a link to the build logs

• I want to add a package output to a feedstock:

  • Pinged the relevant feedstock team(s)
  • Added a small description of why the output is being added.

Context
See conda-forge/arcticdb-feedstock#426.

Details for rc

These are just release candidate versions of the affected versions, so it is safe to remove them.

[xhochy]

Sames as in #1467

Also: You should not be uploading RC builds to the main label.

[jjerphan]

RC builds are published on conda-forge/label/arcticdb_rc, do we need to specify this channel explicitly?

[xhochy]

No, for that the mark broken automation doesn't work AFAIK.

[h-vetinari]

No, for that the mark broken automation doesn't work AFAIK.

I can mark them broken manually if you want (that's the only option for non-main channels)

[jjerphan]

It would be helpful. 🙏

All the rc builds in >=5.2.1rc0<5.3 might be affected.

[h-vetinari]

Oh wow, I did not see the number of builds involved 🤯

This is a non-batch process (needs to be done manually in UI), so I'm not going to click through 200+ packages. If someone has (or creates) a python script using anaconda-client that works (i.e. adds a broken label to a package in an existing channel), I'm happy to run that.

For now my suggestion would be to just rebuild all maintenance branches, and rely on the solver picking up the newest builds.

I don't understand though why you need to draw such a wide circle (rc-channel aside) - we tightened the run-exports of all packages on main to <11.1a0 (conda-forge/conda-forge-repodata-patches-feedstock#984; at least on windows, we can extend this to other platforms if necessary), so none of the old arcticdb builds should pull in fmt 11.1 anymore.

[jjerphan]

For now my suggestion would be to just rebuild all maintenance branches, and rely on the solver picking up the newest builds.

Yes, this is what we have been and are doing; FYI, see conda-forge/arcticdb-feedstock#426.

I don't understand though why you need to draw such a wide circle (rc-channel aside) - we tightened the run-exports of all packages on main to <11.1a0 (conda-forge/conda-forge-repodata-patches-feedstock#984; at least on windows, we can extend this to other platforms if necessary), so none of the old arcticdb builds should pull in fmt 11.1 anymore.

OK, I thought this was done for all platforms but it only has been the case for Windows, but users reported problems on Linux as well.

Could we also apply this repodata patch to linux-* and osx-*?

[h-vetinari]

Could we also apply this repodata patch to linux-* and osx-*?

Please open two separate PRs that include osx in the subdir filter for the yaml in conda-forge/conda-forge-repodata-patches-feedstock#984, resp. remove the filter entirely for the second PR. We need to do this in stages because we cannot patch ~10k packages at once.

[jjerphan]

I am not sure I understood.

Do you mean to open (up to 10) PRs for elements of:

{linux-64,osx-arm64,osx-64,linux-aarch64,linux-ppc64le} × {fmt >=11.0.1,<11.1a0,fmt >=11.0.2,<11.1a0}

?

[h-vetinari]

Two PRs. One that extends the subdir condition for the yaml in conda-forge/conda-forge-repodata-patches-feedstock#984 from windows to windows+osx, and then a second one that enables the patch on linux too by deleting the subdir condition. You can also switch the order of linux and osx if you want.

Repodata patches are against the current state of the repodata. Windows is already patched (hence why we keep patching it), now we need to add osx and linux in two waves, because there's a practical limit of ~5000 packages being patched in one go (second round a few hours later)

[jjerphan]

OK, this is much clearer, thank you.

Edit: done in conda-forge/conda-forge-repodata-patches-feedstock#988.