Introduce build workflow with pixi.

[hameerabbasi]

No description provided.

[amjames]

@hameerabbasi I am holding this for now pending a discussion about the future of our tooling in here.

My thoughts:

• I don't really want to collect a bunch of stuff here to support the ins and outs of how everyone prefers to work. This should be a simple one stop shop to get anyone set up for development quickly, and it should support the variations on builds/environment that we need to cover the development we do.
• I would support going all in on Pixi, and making it the main driver for this system, refactor/remove some of our scripts and move forward with that.

[hameerabbasi]

This ideally requires pytorch/benchmark#2622 to be merged first.

One caveat here is that torchbenchmark mutates the environment; its dependencies aren't consistent; and so the environment may end up becoming inconsistent with the lock file.

One solution is to have a separate benchmark environment that can be mutated without affecting the others.

[hameerabbasi]

This should be a simple one stop shop to get anyone set up for development quickly, and it should support the variations on builds/environment that we need to cover the development we do.

Done! It now supports all the YAML files in the original repo.

I would support going all in on Pixi, and making it the main driver for this system, refactor/remove some of our scripts and move forward with that.

I would too, the only caveat being in #34 (comment).

[isuruf]

This is needed to make sure the environment is set up properly

[hameerabbasi]

With conda, yes. But if you look at the following lines; pixi will take care of that for us. I'm calling the script inside the activation.scripts section of pixi.toml.

torch-build/pixi.toml

Lines 18 to 19 in 9634e20

	[activation]
	scripts = ["./torch-common.sh"]

[isuruf]

What about deactivation?

[hameerabbasi]

Pixi tasks are generally run under a new shell process with pixi shell or pixi run instead of in the same process. Thus the deactivation cleanup isn't needed. One CAN use the same shell with eval "$(pixi shell-hook)", but that's a fringe use-case.