Version 0.2.0 (#63)

* Update twine check

* Fix return type

* Annotate inits

* Annotate args

* carl/context mypy fixed

* Fix mypy

* Handle return types of gym.Env in reset

* Fix mypy for carl_env.py

* Correct reset function

* Fix mypy

* Fix mypy brax

* Fix mypy brax

* Moved demo to examples

* Update reset signature

* Remove rendering of score label

In the current gym version, there is no score label.

* Fix mypy box2d

* Move CARLVehicleRacing example

* Update carracing example

* Fix mypy box2d

* Fix mypy

* Fix mypy

* Update reset function

* mypy dmc fish

* mypy dmc quadruped

* mypy dmc context utils

* wrapper fix

* Fix mypy mario

* mypy dmc walker

* mypy dmc loader

* Fix mypy mario

* mypy dmc

* Add changelog.md

* Add S.D. to authors

* Update version

* mario everything except from env done

* mario_env done

* Update setup.py

Remove setup.cfg and fill init with more info.

* Update gym version

* Remove RNA

* mypy corrected for everything except carl_rna

* gitignore

* Fix mypy

* Change type of contexts to Contexts

* Change type of default_context to Context

* black, isort and flake8

* Update try_dm_control.py

* Update try_dm_control.py

* Fix pre-commit

* Update changelog.md

* Update changelog.md

* Update changelog.md

* Fix tests

* Fix dist check

* Fix mypy

* Format isort

* Remove setup packages for experiments

No experiments in this branch

* Remove requirements for rna

* Update requirements

* Remove RNA

* Fix tests.yaml

* Remove operating system warning

* Add test for dmc loader

* Move to examples

* Add requirement

* Remove requirement

* Update tests.yaml

* Fix whitespace in tests.yaml

Co-authored-by: Carolin Benjamins <benjamins@tnt.uni-hannover.de>
Co-authored-by: amsks <adityak735@gmail.com>
Co-authored-by: Sebastian Döhler <sebastiandoehler@hotmail.de>
Co-authored-by: sebidoe <45859263+sebidoe@users.noreply.github.com>

Author: 4 people

.github/workflows/dist.yaml

@@ -38,13 +38,18 @@ jobs:
     - name: Twine check
       run: |
         pip install twine
-        last_dist=$(ls -t dist/CARL-*.tar.gz | head -n 1)
+        last_dist=$(ls -t dist/carl-*.tar.gz | head -n 1)
         twine_output=`twine check "$last_dist"`
-        if [[ "$twine_output" != "Checking $last_dist: PASSED" ]]; then echo $twine_output && exit 1;fi
+        if [[ "$twine_output" != "Checking $last_dist: PASSED" ]]
+        then
+          echo $twine_output
+        else
+          pip install $last_dist
+        fi
 
     - name: Install dist
       run: |
-        last_dist=$(ls -t dist/CARL-*.tar.gz | head -n 1)
+        last_dist=$(ls -t dist/carl-*.tar.gz | head -n 1)
         pip install $last_dist
 
     - name: PEP 561 Compliance

.github/workflows/tests.yaml

@@ -85,21 +85,21 @@ jobs:
         # Miniconda is available in $CONDA env var
         $CONDA/bin/conda create -n testenv --yes pip wheel gxx_linux-64 gcc_linux-64 python=${{ matrix.python-version }}
         $CONDA/envs/testenv/bin/python3 -m pip install --upgrade pip
-        $CONDA/envs/testenv/bin/pip3 install -e .[dev]
+        $CONDA/envs/testenv/bin/pip3 install -e .[dev,dm_control]
 
     - name: Source install
       if: matrix.kind == 'source'
       run: |
         python -m pip install --upgrade pip
-        pip install -e .[dev]
+        pip install -e .[dev,dm_control]
 
     - name: Dist install
       if: matrix.kind == 'dist'
       run: |
         python -m pip install --upgrade pip
         python setup.py sdist
-        last_dist=$(ls -t dist/CARL-*.tar.gz | head -n 1)
-        pip install $last_dist[dev]
+        last_dist=$(ls -t dist/carl-*.tar.gz | head -n 1)
+        pip install $last_dist[dev,dm_control]
 
     - name: Tests
       timeout-minutes: 60

.gitignore

@@ -13,6 +13,7 @@ carl/runscripts/generated
 docs/html
 docs/apidoc
 CARL.egg-info
+carl.egg-info
 .mypy_cache
 .pytest_cache
 .coverage

.pre-commit-config.yaml

@@ -17,7 +17,7 @@ repos:
         args: [--check]
 
   - repo: https://github.com/ambv/black
-    rev: 21.12b0
+    rev: 22.6.0
     hooks:
       - id: black
         name: black formatter carl

README.md

@@ -21,9 +21,6 @@ Benchmarks include:
 - [Super Mario (TOAD-GAN)](https://github.com/Mawiszus/TOAD-GAN), a procedurally generated jump'n'run game with control
   over level similarity
 
-- [RNADesign](https://github.com/automl/learna/), an environment for RNA design given structure
-  constraints with structures from different datasets to choose from
-
 - [dm_control](https://github.com/deepmind/dm_control), environments based on the MuJoCo physics engine. The environments are extended with different context features.
 
 ![Screenshot of each environment included in CARL.](./docs/source/figures/envs_overview.png)
@@ -44,7 +41,7 @@ pip install .
 
 This will only install the basic classic control environments, which should run on most operating systems. For the full set of environments, use the install options:
 ```bash
-pip install -e .[box2d, brax, rna, mario, dm_control]
+pip install -e .[box2d, brax, mario, dm_control]
 ```
 
 These may not be compatible with Windows systems. Box2D environment may need to be installed via conda on MacOS systems:
@@ -60,9 +57,6 @@ To install the additional requirements for ToadGAN:
 javac carl/envs/mario/Mario-AI-Framework/**/*.java
 ```
 
-If you want to use the RNA design environment, you also need to download the RNA data
-in addition to installing the dependencies. Please follow the instructions in carl/envs/rna.
-
 ## CARL's Contextual Extension
 CARL contextually extends the environment by making the context visible and configurable.
 During training we therefore can encounter different contexts and train for generalization.
@@ -95,8 +89,6 @@ Benchmarking Track)](https://arxiv.org/pdf/2106.13281.pdf)
 [TOAD-GAN: Coherent Style Level Generation from a Single Example,
 Awiszus et al., AIIDE 2020](https://arxiv.org/pdf/2008.01531.pdf)
 
-[Learning to Design RNA, Runge et al., ICRL 2019](https://arxiv.org/pdf/1812.11951.pdf)
-
 [dm_control: Software and Tasks for Continuous Control](https://arxiv.org/pdf/2006.12983.pdf)
 
 ## License

carl/__init__.py

@@ -1,4 +1,22 @@
-__copyright__ = "Copyright 2021, AutoML.org Freiburg-Hannover"
 __license__ = "Apache-2.0 License"
-__version__ = "0.1"
-__author__ = "Carolin Benjamins, Theresa Eimer, Frederik Schubert, André Biedenkapp, Aditya Mohan"
+__version__ = "0.2.0"
+__author__ = "Carolin Benjamins, Theresa Eimer, Frederik Schubert, André Biedenkapp, Aditya Mohan, Sebastian Döhler"
+
+
+import datetime
+
+name = "CARL"
+package_name = "carl"
+author = __author__
+
+author_email = "benjamins@tnt.uni-hannover.de"
+description = "CARL- Contextually Adaptive Reinforcement Learning"
+url = "https://www.automl.org/"
+project_urls = {
+    "Documentation": "https://carl.readthedocs.io/en/latest/",
+    "Source Code": "https://github.com/https://github.com/automl/CARL",
+}
+copyright = f"""
+    Copyright {datetime.date.today().strftime('%Y')}, AutoML.org Freiburg-Hannover
+"""
+version = __version__

carl/context/augmentation.py

@@ -1,13 +1,13 @@
-from typing import List, Union
+from typing import Any, List, Union
 
 import numpy as np
 
 
 def add_gaussian_noise(
     default_value: Union[float, List[float]],
-    percentage_std: float = 0.01,
+    percentage_std: Union[float, Any] = 0.01,
     random_generator: np.random.Generator = None,
-) -> Union[float, List[float]]:
+) -> Union[float, Any]:
     """
     Add gaussian noise to default value.
 
@@ -45,7 +45,6 @@ def add_gaussian_noise(
 
     seed = 123456
     rng = np.random.default_rng(seed=seed)
-    default_value = 10
     default_value = list(np.arange(0, 4))
     percentage_std = 0.01
     n_samples = 1000

carl/context/sampling.py

@@ -5,6 +5,7 @@
 from scipy.stats import norm
 
 from carl import envs
+from carl.utils.types import Context, Contexts
 
 
 def get_default_context_and_bounds(
@@ -145,9 +146,9 @@ def sample_contexts(
             sample_dists[context_feature_name] = (random_variable, context_feature_type)
 
     # Sample contexts
-    contexts = {}
+    contexts: Contexts = {}
     for i in range(0, num_contexts):
-        c = {}
+        c: Context = {}
         # k = name of context feature
         for k in env_defaults.keys():
             if k in sample_dists.keys():
@@ -157,7 +158,7 @@ def sample_contexts(
                 lower_bound, upper_bound = env_bounds[k][0], env_bounds[k][1]
                 if context_feature_type == list:
                     length = np.random.randint(
-                        5e5
+                        500000
                     )  # TODO should we allow lists to be this long? or should we parametrize this?
                     arg_class = sample_dists[k][1][1]
                     context_list = random_variable.rvs(size=length)

carl/context/selection.py

@@ -1,9 +1,9 @@
 from abc import abstractmethod
-from typing import Any, Callable, Dict, List, Optional, Tuple
+from typing import Any, Callable, List, Optional, Tuple
 
 import numpy as np
 
-from carl.utils.types import Context
+from carl.utils.types import Context, Contexts
 
 
 class AbstractSelector(object):
@@ -15,13 +15,13 @@ class AbstractSelector(object):
 
     Parameters
     ----------
-    contexts: Dict[Any, Context]
+    contexts: Contexts
         Context set. A `Context` is a Dict[str, Any].
 
 
     Attributes
     ----------
-    contexts : Dict[Any, Context]
+    contexts : Contexts
         Context set.
     context_ids : List[int]
         Integer index for contexts.
@@ -34,8 +34,8 @@ class AbstractSelector(object):
 
     """
 
-    def __init__(self, contexts: Dict[Any, Context]):
-        self.contexts: Dict[Any, Context] = contexts
+    def __init__(self, contexts: Contexts):
+        self.contexts: Contexts = contexts
         self.context_ids: List[int] = list(np.arange(len(contexts)))
         self.contexts_keys: List[Any] = list(contexts.keys())
         self.n_calls: int = 0
@@ -79,7 +79,7 @@ class RandomSelector(AbstractSelector):
     Random Context Selector.
     """
 
-    def _select(self):
+    def _select(self) -> Tuple[Context, int]:
         # TODO seed?
         context_id = np.random.choice(self.context_ids)
         context = self.contexts[self.contexts_keys[context_id]]
@@ -93,7 +93,7 @@ class RoundRobinSelector(AbstractSelector):
     Iterate through all contexts and then start at the first again.
     """
 
-    def _select(self):
+    def _select(self) -> Tuple[Context, int]:
         if self.context_id is None:
             self.context_id = -1
         self.context_id = (self.context_id + 1) % len(self.contexts)
@@ -110,7 +110,7 @@ class CustomSelector(AbstractSelector):
 
     Parameters
     ----------
-    contexts: Dict[Any, Context]
+    contexts: Contexts
         Set of contexts.
     selector_function: callable
         Function receiving a pointer to the selector implementing selection logic.
@@ -133,13 +133,13 @@ class CustomSelector(AbstractSelector):
 
     def __init__(
         self,
-        contexts: Dict[Any, Context],
+        contexts: Contexts,
         selector_function: Callable[[AbstractSelector], Tuple[Context, int]],
     ):
         super().__init__(contexts=contexts)
         self.selector_function = selector_function
 
-    def _select(self):
-        context, context_id = self.selector_function(inst=self)
+    def _select(self) -> Tuple[Context, int]:
+        context, context_id = self.selector_function(self)
         self.context_id = context_id
         return context, context_id

carl/context/utils.py

@@ -1,10 +1,10 @@
-from typing import Dict, List, Tuple
+from typing import Any, Dict, List, Tuple, Type
 
 import numpy as np
 
 
 def get_context_bounds(
-    context_keys: List[str], context_bounds: Dict[str, Tuple[float, float, float]]
+    context_keys: List[str], context_bounds: Dict[str, Tuple[float, float, Type[Any]]]
 ) -> Tuple[np.ndarray, np.ndarray]:
     """
     Get context bounds for specific features.
@@ -15,7 +15,7 @@ def get_context_bounds(
     ----------
     context_keys: List[str]
         Names of context features.
-    context_bounds: Dict[str, Tuple[float, float, float]]
+    context_bounds: Dict[str, Tuple[float, float, type]]
         Dictionary containing lower and upper bound as a tuple, e.g., "context_feature_name": (-np.inf, np.inf)).
 
     Returns
@@ -28,7 +28,7 @@ def get_context_bounds(
     upper_bounds = np.empty(shape=len(context_keys))
 
     for i, context_key in enumerate(context_keys):
-        l, u, _ = context_bounds[context_key]
+        l, u, dtype = context_bounds[context_key]
         lower_bounds[i] = l
         upper_bounds[i] = u
 
@@ -51,16 +51,16 @@ def get_context_bounds(
     }
 
     CONTEXT_BOUNDS = {
-        "min_position": (-np.inf, np.inf),
-        "max_position": (-np.inf, np.inf),
-        "max_speed": (0, np.inf),
-        "goal_position": (-np.inf, np.inf),
-        "goal_velocity": (-np.inf, np.inf),
-        "force": (-np.inf, np.inf),
-        "gravity": (0, np.inf),
-        "min_position_start": (-np.inf, np.inf),
-        "max_position_start": (-np.inf, np.inf),
-        "min_velocity_start": (-np.inf, np.inf),
-        "max_velocity_start": (-np.inf, np.inf),
+        "min_position": (-np.inf, np.inf, float),
+        "max_position": (-np.inf, np.inf, float),
+        "max_speed": (0, np.inf, float),
+        "goal_position": (-np.inf, np.inf, float),
+        "goal_velocity": (-np.inf, np.inf, float),
+        "force": (-np.inf, np.inf, float),
+        "gravity": (0, np.inf, float),
+        "min_position_start": (-np.inf, np.inf, float),
+        "max_position_start": (-np.inf, np.inf, float),
+        "min_velocity_start": (-np.inf, np.inf, float),
+        "max_velocity_start": (-np.inf, np.inf, float),
     }
     lower, upper = get_context_bounds(list(DEFAULT_CONTEXT.keys()), CONTEXT_BOUNDS)