Do not install petsc4py using pip

[connorjward]

This PR is a first stab at an idea I've had to avoid what I think may be an upcoming versioning issue. Things are very rough but I think there's enough there to see what I mean.

The problem

PETSc and petsc4py must be installed using the exact same version but they are installed using different code paths. One from source and one from PyPI.

This is fine for most users currently because we fix the specific PETSc that they use, but I think this means we might hit issues with system installs of PETSc, both for HPC installs and a possible future apt package. I can easily imagine someone trying to install the latest Firedrake using a slightly out-of-date PETSc and thus having a mismatch between PETSc and petsc4py versions.

The solution

I think the way to fix this is to use a petsc4py installed alongside PETSc (--with-petsc4py=1) and we can tweak sys.path to find it (using PETSC_DIR and PETSC_ARCH). Then we have single source of truth for the version.

Feedback welcome.

---

As another benefit I think we are realising that petsc4py install isn't very robust as it's the cause of most of our current install problems.

TODOs

• Add numpy hack to the docs
• Fixup Dockerfile

[connorjward]

@dham made the excellent point that doing things this way means that the petsc4py wheel is tied to the system Python, which is easily not the same as the one used to install Firedrake.

This work came out of a desire to ensure compatibility between PETSc and petsc4py. I think it is sufficient to warn the user if this is the case and hence permit petsc4py inside the pyproject.toml.

For installs using main I think we will have to use --no-build-isolation, which I think is fine.

[francesco-ballarin]

For installs using main I think we will have to use --no-build-isolation, which I think is fine

Yes, you'll definitely need to disable build isolation, because petsc4py from pypi doesn't track main.

[connorjward]

Closing as the spirit of the PR is changing substantially so this will just get confusing. I'll open another.