Refactor CI job HF setup

[GregoryComer]

Refactor the HF setup to avoid duplicating the code everywhere. Keep requirements in one place. See #15403. Ideally, we want one definition of the relevant pins. We can likely use requirements files instead of explicitly calling pip install in a bunch of places.