First build the base docker image (includes conda/mamba and various other common packages)
docker build -t rfcl_base base
To setup a docker image for ManiSkill2:
docker build -t rfcl_ms2 ms2
To setup a docker image for Adroit:
docker build -t rfcl_adroit adroit
To setup a docker image for Metaworld
docker build -t rfcl_metaworld meta-world