Git dependencies

[rodneykinney]

https://github.com/allenai/astabench-issues/issues/283

PyPI does not allow git dependencies. Publish our own versions of inspect_evals and knowledge-storm so we can declare them as dependencies. Make a workflow file for each, which contains the specific git sha we will publish from. Need to manually override the name/version, which is done by making a static copy of the pyproject.toml/setup.py. Add docs for this procedure.

Not pretty! The only alternative I can think of is to fork the repos, modify the pyproject.toml in the fork, and add the workflow files there. I prefer this approach (slightly) because it keeps all of the activity in this repo. However, if we ever have another project that depends on those same sha-specific dependencies, then publishing from the fork seems like the only option.

[mdarcy220]

Trying to think if we have better options here...

For inspect_evals, the only thing we need from their huge repo is the core-bench task; I wanted it as a dep since that seemed simpler than pasting all their code in our repo, but if the alternative would be hosting their code on pypi anyway then I think we could just make it a submodule instead.

For STORM, since that's solver-specific and doesn't need to be installed by default for the evals, we can probably drop that dep for astabench and maybe kick the issue to agent-baselines? I'm inclined to say that users of agent-baselines could be instructed to do the pip .. git+.. install if they want to use the storm solver (or have it as part of the setup.sh).

Publishing these to pypi is maybe not the end of the world but doesn't feel like an intended workflow and I worry that there might be unforeseen headaches as a result. Lmk if those suggested alternatives seem harder though, in which case maybe we should just go forward with this.

[rodneykinney]

👍 to kicking the STORM can over to agent-baselines. That repo doesn't get published to pypi, so installing git+ is fine.

Using a submodule is a decent alternative. I'll try that instead.