require --no-build-isolation in torchao builds

[vkuzo]

Summary:

In recent versions of setuptools, the behavior of using PEP 517 (enforce build isolation, https://peps.python.org/pep-0517/) flipped from False to True. This broke torchao builds as our setup.py file depends on not having build isolation, as it imports build extension class definitions from torch (

ao/setup.py

Line 106 in 94dee9c

BuildExtension,

).

A long term fix would be modernize the torchao build system to honor PEP 517. However, to unbreak CI and buy us some time, in the short term we can just require build isolation for building torchao. This PR implements the short term workaround.

An AI-generated writeup I used to debug this, with more details: https://gist.github.com/vkuzo/5e6f282fda816c2cd4bce8c7fc1f30e2

Note that I had to add a pyproject.toml file to make pip install . run without warnings.

Test Plan:

// both of the commands below work
with-proxy USE_CPP=0 pip install -e . --no-build-isolation
with-proxy USE_CPP=1 pip install -e . --no-build-isolation --verbose

// will check CI

Reviewers:

Subscribers:

Tasks:

Tags:

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/ao/3209

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 New Failure, 1 Cancelled Job, 1 Unrelated Failure

As of commit f58e799 with merge base 94dee9c ():

NEW FAILURE - The following job has failed:

• Run Regression Tests (aarch64) / test-cpu-ops (macos-14) (gh)
  /Users/runner/miniconda3/envs/venv/lib/python3.10/site-packages/executorch/share/cmake/../../include/executorch/runtime/core/memory_allocator.h:143:21: error: no matching function for call to 'mul_overflows'

CANCELLED JOB - The following job was cancelled. Please retry:

• Run Regression Tests (aarch64) / test-cpu-ops (linux.arm64.2xlarge) (gh)

BROKEN TRUNK - The following job failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• Run Regression Tests / test-nightly (CUDA Nightly, linux.g5.12xlarge.nvidia.gpu, --pre torch --index-url https://downloa... / linux-job (gh) (trunk failure)
  test/dtypes/test_nf4.py::TestComm::test_comm

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[vkuzo]

note: test-cpu-ops andtest-nightly are failing in main branch for other reasons, so looks like CI is as green as it can be

[vkuzo]

@atalman , I'm guessing if we go forward with this we'll also need to modify https://github.com/pytorch/test-infra/blob/a287d59209eb9221fa2428c3930efdc8643ad801/.github/workflows/build_wheels_linux.yml#L245 to use --no-build-isolation, gated to torchao only?