Package manager

[vwxyzjn]

Currently, the packages under this repository need to be installed in a python (virtual) environment one by one via pip install -e ..

To improve usability and reproducibility, we might want to use a package manager. I recommend using poetry, which also supports multiple sub-packages like in this repo (see here). In CleanRL and Gym-MicroRTS, poetry has greatly reduced the installation-related issues.

That said, some other promising package managers include:

• https://github.com/pypa/pipenv
• https://github.com/ofek/hatch
• https://github.com/spack/spack
• https://github.com/pdm-project/pdm

[vwxyzjn]

One significant benefit of packing managers is reproducible installation. The package manager resolves and locks the dependencies for packages (including the optional ones). For example, we might have dependencies like the following.

[tool.poetry.dependencies]
python = ">=3.7.1,<3.10"
torch = "^1.7.1"
tensorboard = "^2.5.0"
wandb = "^0.12.1"
pyglet = "^1.5.19"
opencv-python = "^4.5.3"
gym = "^0.21.0"

# Optinal dependencies
stable-baselines3 = {version = "^1.1.0", optional = true}
ale-py = {version = "^0.7", optional = true}
AutoROM = {version = "^0.4.2", optional = true, extras = ["accept-rom-license"]}
pybullet = {version = "^3.1.8", optional = true}
procgen = {version = "^0.10.4", optional = true}
pettingzoo = {version = "^1.11.2", optional = true}
pygame = {version = "^2.0.1", optional = true}
pymunk = {version = "^6.2.0", optional = true}
pandas = {version = "^1.3.3", optional = true}
seaborn = {version = "^0.11.2", optional = true}
boto3 = {version = "^1.18.57", optional = true}
awscli = {version = "^1.20.57", optional = true}
spyder = {version = "^5.1.5", optional = true}
pytest = {version = "^6.2.5", optional = true}
free-mujoco-py = {version = "^2.1.6", optional = true}
mkdocs-material = {version = "^7.3.4", optional = true}

[tool.poetry.extras]
atari = ["ale-py", "AutoROM", "stable-baselines3"]
pybullet = ["pybullet"]
procgen = ["procgen", "stable-baselines3"]
pettingzoo = ["pettingzoo", "stable-baselines3", "pygame", "pymunk"]
plot = ["pandas", "seaborn"]
cloud = ["boto3", "awscli"]
spyder = ["spyder"]
pytest = ["pytest"]
mujoco = ["free-mujoco-py"]
docs = ["mkdocs-material"]

Then we can install the optional dependencies like this:

poetry install -E atari
poetry install -E pybullet
poetry install -E procgen
poetry install -E pettingzoo
poetry install -E plot
poetry install -E cloud
poetry install -E docs
poetry install -E spyder

This installation is straightforward and highly reproducible. In comparison, when doing pip install -e . or pip install -e .[atari], we might override some dependencies on which the previous package depends, potentially resulting in installation issues.

[cswinter]

I'm not an expert on Python packagers so I'm happy to defer to your expertise here. I've heard good things about poetry, would love to get a chance to try it out!

There are two related issues I've not been able to figure out so far, maybe poetry could help us with those as well:

• On my local checkout, mypy is often really slow, even using dmypy, and I often get spurious errors in dependencies that don't show up on CI.
• CI is already quite slow, majority of the time is spent in pip installs. This is despite already caching dependencies.