Add build constraints

[notatallshaw]

Fixes #13300

This adds a new --build-constraint flag to complement the existing --constraint flag, at a high level:

1. Build constraints are used as constraints for build requirements in isolated build environments
2. Regular constraints are no longer used as constraints in any isolated build environments (normally done via PIP_CONSTRAINT now requires PIP_BUILD_CONSTRAINT or simply --build-constraint)
3. Because this changes existing behavior the new behavior is only enabled when build constraints are passed or the --use-feature flag build-constraint is enabled, as not to break user workflows
4. There is a deprecation warning when using PIP_CONSTRAINT is present and in effect without using the new build constraints

This change in behavior was desired because:

1. It is a simpler UX than the existing behavior of specifying both regular and build time constraints (CLI flags for run time and ENV vars for isolated build vs. --constraint and --build-constraint)
2. This behavior will be easier to match with in-process installation (Install build dependencies in-process #13450) where as the existing behavior is trickier
3. It is the same behavior uv pip has for --constraint and --build-constraint so is well verified in the real world

[potiuk]

Very, very much needed. Thanks @notatallshaw -> today setuptools-scm 9.1.0 (2 hours ago) again broke source installation of xmlsec: pypa/setuptools-scm#1194

[potiuk]

Very, very much needed. Thanks @notatallshaw -> today setuptools-scm 9.1.0 (2 hours ago) again broke source installation of xmlsec: pypa/setuptools-scm#1194

Yanked now luckily.

Looking forward to this one - I really like how this one will help us in the future to make both - our PRs and installation mechanisms we tell our users, to make them resilient to similar accidents broken packages released on PyPI. I really look forward to this one to make it part of our regular process. The existing workarounds are complicated and non-end-user friendly at all and would complicate our CI process quite a lot as we woudl have to combine regular and build constraints dynamically on our CI.

[notatallshaw]

Thanks @potiuk, a few things worth nothing though:

This only allows users to protect themselves from breakages, not libraries. I feel like there is an open question about whether libraries (or popular applications) should provide upperbounds on their build dependencies. In fact it was recently discussed in pypa/packaging.python.org#1880 (comment)

This doesn't technically introduce any new functionality if you have full control of pip you can reproduce all of this:

• Install only constraints: --constraint
• Install and build time constraints single file: PIP_CONSTRAINT
• Install and build time constraints separate files: --constraint and PIP_CONSTRAINT
• Build time only constraints: --constraint pointed to an empty file and PIP_CONSTRAINT

But this is poor UX, it means you don't have the usual CLI and ENV flexibility, and the fact this works the way it does depends on multiple internal details of how pip is implemented. In fact what finally inspired me to write this PR is that one of those internal details (installing build dependencies via calling pip in a subprocess) is likely to change soon.

[potiuk]

Yep. I agree 100% with everythi g you wrote. I already started lookng at implementing a co bo of co ateaints solution in our CI this morning - bit I was so relieved I did not have to (yet) as today's aetuptools-scm 9.1.0 was yanked - precisely because of the poor UX and the undocumented, internal behaviour and potential conflicts between build and regular constraints which has kinda unspecified behaviour.

And - as for Airflow - we simply pin our build dependencies - because in our case reproducibility trumps potential security issues with build dependencies - and also it makes it easier for us because we are pure Python only so very few upstream users would actually use .sdist to build airflow - and if they do, they usually do it in controlled environments.