diff --git a/com.unity.ml-agents/Documentation~/Readme.md b/Readme.md similarity index 61% rename from com.unity.ml-agents/Documentation~/Readme.md rename to Readme.md index ed94116be9..20ad7ebad1 100644 --- a/com.unity.ml-agents/Documentation~/Readme.md +++ b/Readme.md @@ -2,27 +2,27 @@ [![docs badge](https://img.shields.io/badge/docs-reference-blue.svg)](https://docs.unity3d.com/Packages/com.unity.ml-agents@latest) -[![license badge](https://img.shields.io/badge/license-Apache--2.0-green.svg)](https://github.com/Unity-Technologies/ml-agents/blob/release_22/LICENSE.md) +[![license badge](https://img.shields.io/badge/license-Apache--2.0-green.svg)](https://github.com/Unity-Technologies/ml-agents/blob/release/4.0.0/LICENSE.md) ([latest release](https://github.com/Unity-Technologies/ml-agents/releases/tag/latest_release)) ([all releases](https://github.com/Unity-Technologies/ml-agents/releases)) **The Unity Machine Learning Agents Toolkit** (ML-Agents) is an open-source project that enables games and simulations to serve as environments for training intelligent agents. We provide implementations (based on PyTorch) of state-of-the-art algorithms to enable game developers and hobbyists to easily train intelligent agents for 2D, 3D and VR/AR games. Researchers can also use the provided simple-to-use Python API to train Agents using reinforcement learning, imitation learning, neuroevolution, or any other methods. These trained agents can be used for multiple purposes, including controlling NPC behavior (in a variety of settings such as multi-agent and adversarial), automated testing of game builds and evaluating different game design decisions pre-release. The ML-Agents Toolkit is mutually beneficial for both game developers and AI researchers as it provides a central platform where advances in AI can be evaluated on Unity’s rich environments and then made accessible to the wider research and game developer communities. ## Features -- 17+ [example Unity environments](Learning-Environment-Examples.md) +- 17+ [example Unity environments](https://docs.unity3d.com/Packages/com.unity.ml-agents@latest/index.html?subfolder=/manual/Learning-Environment-Examples.html) - Support for multiple environment configurations and training scenarios - Flexible Unity SDK that can be integrated into your game or custom Unity scene - Support for training single-agent, multi-agent cooperative, and multi-agent competitive scenarios via several Deep Reinforcement Learning algorithms (PPO, SAC, MA-POCA, self-play). - Support for learning from demonstrations through two Imitation Learning algorithms (BC and GAIL). -- Quickly and easily add your own [custom training algorithm](Python-Custom-Trainer-Plugin.md) and/or components. +- Quickly and easily add your own [custom training algorithm](https://docs.unity3d.com/Packages/com.unity.ml-agents@latest/index.html?subfolder=/manual/Python-Custom-Trainer-Plugin.html) and/or components. - Easily definable Curriculum Learning scenarios for complex tasks - Train robust agents using environment randomization - Flexible agent control with On Demand Decision Making - Train using multiple concurrent Unity environment instances -- Utilizes the [Inference Engine](Inference-Engine.md) to provide native cross-platform support -- Unity environment [control from Python](Python-LLAPI.md) -- Wrap Unity learning environments as a [gym](Python-Gym-API.md) environment -- Wrap Unity learning environments as a [PettingZoo](Python-PettingZoo-API.md) environment +- Utilizes the [Inference Engine](https://docs.unity3d.com/Packages/com.unity.ml-agents@latest/index.html?subfolder=/manual/Inference-Engine.html) to provide native cross-platform support +- Unity environment [control from Python](https://docs.unity3d.com/Packages/com.unity.ml-agents@latest/index.html?subfolder=/manual/Python-LLAPI.html) +- Wrap Unity learning environments as a [gym](https://docs.unity3d.com/Packages/com.unity.ml-agents@latest/index.html?subfolder=/manual/Python-Gym-API.html) environment +- Wrap Unity learning environments as a [PettingZoo](https://docs.unity3d.com/Packages/com.unity.ml-agents@latest/index.html?subfolder=/manual/Python-PettingZoo-API.html) environment ## Releases & Documentation @@ -31,15 +31,15 @@ The table below shows our latest release, including our `develop` branch which is under active development and may be unstable. A few helpful guidelines: -- The [Versioning page](Versioning.md) overviews how we manage our GitHub releases and the versioning process for each of the ML-Agents components. +- The [Versioning page](https://docs.unity3d.com/Packages/com.unity.ml-agents@latest/index.html?subfolder=/manual/Versioning.html) overviews how we manage our GitHub releases and the versioning process for each of the ML-Agents components. - The [Releases page](https://github.com/Unity-Technologies/ml-agents/releases) contains details of the changes between releases. -- The [Migration page](Migrating.md) contains details on how to upgrade from earlier releases of the ML-Agents Toolkit. +- The [Migration page](https://docs.unity3d.com/Packages/com.unity.ml-agents@latest/index.html?subfolder=/manual/Migrating.html) contains details on how to upgrade from earlier releases of the ML-Agents Toolkit. - The `com.unity.ml-agents` package is [verified](https://docs.unity3d.com/2020.1/Documentation/Manual/pack-safe.html) for Unity 2020.1 and later. Verified packages releases are numbered 1.0.x. -| **Version** | **Release Date** | **Source** | **Documentation** | **Download** | **Python Package** | **Unity Package** | -|:-----------:|:---------------:|:----------:|:-----------------:|:------------:|:------------------:|:-----------------:| -| **Release 22** | **October 5, 2024** | **[source](https://github.com/Unity-Technologies/ml-agents/tree/release_22)** | **[docs](https://unity-technologies.github.io/ml-agents/)** | **[download](https://github.com/Unity-Technologies/ml-agents/archive/release_22.zip)** | **[1.1.0](https://pypi.org/project/mlagents/1.1.0/)** | **[3.0.0](https://docs.unity3d.com/Packages/com.unity.ml-agents@3.0/manual/index.html)** | -| **develop (unstable)** | -- | [source](https://github.com/Unity-Technologies/ml-agents/tree/develop) | [docs](https://github.com/Unity-Technologies/ml-agents/tree/develop/com.unity.ml-agents/Documentation~/index.md) | [download](https://github.com/Unity-Technologies/ml-agents/archive/develop.zip) | -- | -- | +| **Version** | **Release Date** | **Source** | **Documentation** | **Download** | **Python Package** | **Unity Package** | +|:----------------------:|:-------------------:|:-----------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------------:|:--------------------------------------------------------------------------------------:|:-----------------------------------------------------:|:-------------------------------------------------------------------------------------:| +| **Release 23** | **August 15, 2025** | **[source](https://github.com/Unity-Technologies/ml-agents/tree/release_23)** | **[docs](https://docs.unity3d.com/Packages/com.unity.ml-agents@4.0/manual/index.html)** | **[download](https://github.com/Unity-Technologies/ml-agents/archive/release_23.zip)** | **[1.1.0](https://pypi.org/project/mlagents/1.1.0/)** | **4.0.0** | +| **develop (unstable)** | -- | [source](https://github.com/Unity-Technologies/ml-agents/tree/develop) | [docs](https://github.com/Unity-Technologies/ml-agents/tree/develop/com.unity.ml-agents/Documentation~/index.md) | [download](https://github.com/Unity-Technologies/ml-agents/archive/develop.zip) | -- | -- | @@ -77,12 +77,12 @@ Additionally, if you use the MA-POCA trainer in your research, we ask that you c * [Introduction to ML-Agents by Huggingface](https://huggingface.co/learn/deep-rl-course/en/unit5/introduction) * [Community created ML-Agents projects](https://discussions.unity.com/t/post-your-ml-agents-project/816756) * [ML-Agents models on Huggingface](https://huggingface.co/models?library=ml-agents) -* [Blog posts](Blog-posts.md) +* [Blog posts](https://docs.unity3d.com/Packages/com.unity.ml-agents@latest/index.html?subfolder=/manual/Blog-posts.html) * [Discord](https://discord.com/channels/489222168727519232/1202574086115557446) ## Community and Feedback -The ML-Agents Toolkit is an open-source project and we encourage and welcome contributions. If you wish to contribute, be sure to review our [contribution guidelines](CONTRIBUTING.md) and [code of conduct](https://github.com/Unity-Technologies/ml-agents/blob/release_22/CODE_OF_CONDUCT.md). +The ML-Agents Toolkit is an open-source project and we encourage and welcome contributions. If you wish to contribute, be sure to review our [contribution guidelines](https://docs.unity3d.com/Packages/com.unity.ml-agents@latest/index.html?subfolder=/manual/CONTRIBUTING.html) and [code of conduct](https://github.com/Unity-Technologies/ml-agents/blob/release/4.0.0/CODE_OF_CONDUCT.md). For problems with the installation and setup of the ML-Agents Toolkit, or discussions about how to best setup or train your agents, please create a new thread on the [Unity ML-Agents discussion forum](https://discussions.unity.com/tag/ml-agents). Be sure to include as many details as possible to help others assist you effectively. If you run into any other problems using the ML-Agents Toolkit or have a specific feature request, please [submit a GitHub issue](https://github.com/Unity-Technologies/ml-agents/issues). diff --git a/colab/Colab_UnityEnvironment_1_Run.ipynb b/colab/Colab_UnityEnvironment_1_Run.ipynb index b221ac8421..e70dca7b8a 100644 --- a/colab/Colab_UnityEnvironment_1_Run.ipynb +++ b/colab/Colab_UnityEnvironment_1_Run.ipynb @@ -32,7 +32,7 @@ }, "source": [ "# ML-Agents Open a UnityEnvironment\n", - "" + "" ] }, { diff --git a/colab/Colab_UnityEnvironment_2_Train.ipynb b/colab/Colab_UnityEnvironment_2_Train.ipynb index ca55f8a58e..840d0fb3eb 100644 --- a/colab/Colab_UnityEnvironment_2_Train.ipynb +++ b/colab/Colab_UnityEnvironment_2_Train.ipynb @@ -22,7 +22,7 @@ }, "source": [ "# ML-Agents Q-Learning with GridWorld\n", - "" + "" ] }, { @@ -190,7 +190,7 @@ "id": "pZhVRfdoyPmv" }, "source": [ - "The [GridWorld](https://github.com/Unity-Technologies/ml-agents/blob/release_22_docs/docs/Learning-Environment-Examples.md#gridworld) Environment is a simple Unity visual environment. The Agent is a blue square in a 3x3 grid that is trying to reach a green __`+`__ while avoiding a red __`x`__.\n", + "The [GridWorld](https://docs.unity3d.com/Packages/com.unity.ml-agents@latest/index.html?subfolder=/manual/Learning-Environment-Examples.html#gridworld) Environment is a simple Unity visual environment. The Agent is a blue square in a 3x3 grid that is trying to reach a green __`+`__ while avoiding a red __`x`__.\n", "\n", "The observation is an image obtained by a camera on top of the grid.\n", "\n", diff --git a/colab/Colab_UnityEnvironment_3_SideChannel.ipynb b/colab/Colab_UnityEnvironment_3_SideChannel.ipynb index 646a01defe..f74291fb17 100644 --- a/colab/Colab_UnityEnvironment_3_SideChannel.ipynb +++ b/colab/Colab_UnityEnvironment_3_SideChannel.ipynb @@ -23,7 +23,7 @@ }, "source": [ "# ML-Agents Use SideChannels\n", - "" + "" ] }, { @@ -176,7 +176,7 @@ "## Side Channel\n", "\n", "SideChannels are objects that can be passed to the constructor of a UnityEnvironment or the `make()` method of a registry entry to send non Reinforcement Learning related data.\n", - "More information available [here](https://github.com/Unity-Technologies/ml-agents/blob/release_22_docs/docs/Python-API.md#communicating-additional-information-with-the-environment)\n", + "More information available [here](https://docs.unity3d.com/Packages/com.unity.ml-agents@latest/index.html?subfolder=/manual/Python-LLAPI.html#communicating-additional-information-with-the-environment)\n", "\n", "\n", "\n" @@ -189,7 +189,7 @@ }, "source": [ "### Engine Configuration SideChannel\n", - "The [Engine Configuration Side Channel](https://github.com/Unity-Technologies/ml-agents/blob/release_22_docs/docs/Python-API.md#engineconfigurationchannel) is used to configure how the Unity Engine should run.\n", + "The [Engine Configuration Side Channel](https://docs.unity3d.com/Packages/com.unity.ml-agents@latest/index.html?subfolder=/manual/Python-LLAPI.html#engineconfigurationchannel) is used to configure how the Unity Engine should run.\n", "We will use the GridWorld environment to demonstrate how to use the EngineConfigurationChannel." ] }, @@ -282,7 +282,7 @@ }, "source": [ "### Environment Parameters Channel\n", - "The [Environment Parameters Side Channel](https://github.com/Unity-Technologies/ml-agents/blob/release_22_docs/docs/Python-API.md#environmentparameters) is used to modify environment parameters during the simulation.\n", + "The [Environment Parameters Side Channel](https://docs.unity3d.com/Packages/com.unity.ml-agents@latest/index.html?subfolder=/manual/Python-LLAPI.html#environmentparameters) is used to modify environment parameters during the simulation.\n", "We will use the GridWorld environment to demonstrate how to use the EngineConfigurationChannel." ] }, @@ -419,7 +419,7 @@ }, "source": [ "### Creating your own Side Channels\n", - "You can send various kinds of data between a Unity Environment and Python but you will need to [create your own implementation of a Side Channel](https://github.com/Unity-Technologies/ml-agents/blob/release_22_docs/docs/Custom-SideChannels.md#custom-side-channels) for advanced use cases.\n" + "You can send various kinds of data between a Unity Environment and Python but you will need to [create your own implementation of a Side Channel](https://docs.unity3d.com/Packages/com.unity.ml-agents@latest/index.html?subfolder=/manual/Custom-SideChannels.html#custom-side-channels) for advanced use cases.\n" ] }, { diff --git a/colab/Colab_UnityEnvironment_4_SB3VectorEnv.ipynb b/colab/Colab_UnityEnvironment_4_SB3VectorEnv.ipynb index e5d3d45c8b..84bb94c52c 100644 --- a/colab/Colab_UnityEnvironment_4_SB3VectorEnv.ipynb +++ b/colab/Colab_UnityEnvironment_4_SB3VectorEnv.ipynb @@ -7,7 +7,7 @@ }, "source": [ "# ML-Agents run with Stable Baselines 3\n", - "" + "" ] }, { diff --git a/com.unity.ml-agents/CHANGELOG.md b/com.unity.ml-agents/CHANGELOG.md index 48c9cccd39..85214a4356 100755 --- a/com.unity.ml-agents/CHANGELOG.md +++ b/com.unity.ml-agents/CHANGELOG.md @@ -6,7 +6,7 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/) and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.html). -## [Unreleased] +## [4.0.0] - 2025-08-15 ### Major Changes #### com.unity.ml-agents (C#) - Upgraded to Inference Engine 2.2.1 (#6212) diff --git a/com.unity.ml-agents/Documentation~/Inference-Engine.md b/com.unity.ml-agents/Documentation~/Inference-Engine.md index aca1dde648..6a45bf86d8 100644 --- a/com.unity.ml-agents/Documentation~/Inference-Engine.md +++ b/com.unity.ml-agents/Documentation~/Inference-Engine.md @@ -6,7 +6,7 @@ The ML-Agents Toolkit allows you to use pre-trained neural network models inside Inference Engine supports [all Unity runtime platforms](https://docs.unity3d.com/Manual/PlatformSpecific.html). -Scripting Backends : Inference Engine is generally faster with **IL2CPP** than with **Mono** for Standalone builds. In the Editor, It is not possible to use Inference Engine with GPU device selected when Editor Graphics Emulation is set to **OpenGL(ES) 3.0 or 2.0 emulation**. Also there might be non-fatal build time errors when target platform includes Graphics API that does not support **Unity Compute Shaders**. +Scripting Backends : Inference Engine is generally faster with **IL2CPP** than with **Mono** for Standalone builds. In the Editor, It is not possible to use Inference Engine with GPU device selected when Editor Graphics Emulation is set to **OpenGL(ES) 3.0 or 2.0 emulation**. Also, there might be non-fatal build time errors when target platform includes Graphics API that does not support **Unity Compute Shaders**. In cases when it is not possible to use compute shaders on the target platform, inference can be performed using **CPU** or **GPUPixel** Inference Engine backends. @@ -18,7 +18,7 @@ When using a model, drag the model file into the **Model** field in the Inspecto # Unsupported use cases ## Externally trained models -The ML-Agents Toolkit only supports the models created with our trainers. Model loading expects certain conventions for constants and tensor names. While it is possible to construct a model that follows these conventions, we don't provide any additional help for this. More details can be found in [TensorNames.cs](https://github.com/Unity-Technologies/ml-agents/blob/release_22_docs/com.unity.ml-agents/Runtime/Inference/TensorNames.cs) and [SentisModelParamLoader.cs](https://github.com/Unity-Technologies/ml-agents/blob/release_22_docs/com.unity.ml-agents/Runtime/Inference/SentisModelParamLoader.cs). +The ML-Agents Toolkit only supports the models created with our trainers. Model loading expects certain conventions for constants and tensor names. While it is possible to construct a model that follows these conventions, we don't provide any additional help for this. More details can be found in [TensorNames.cs](https://github.com/Unity-Technologies/ml-agents/blob/release/4.0.0/com.unity.ml-agents/Runtime/Inference/TensorNames.cs) and [SentisModelParamLoader.cs](https://github.com/Unity-Technologies/ml-agents/blob/release/4.0.0/com.unity.ml-agents/Runtime/Inference/SentisModelParamLoader.cs). If you wish to run inference on an externally trained model, you should use Inference Engine directly, instead of trying to run it through ML-Agents. diff --git a/com.unity.ml-agents/Documentation~/Installation.md b/com.unity.ml-agents/Documentation~/Installation.md index 921e4fa8b1..c28600fa2b 100644 --- a/com.unity.ml-agents/Documentation~/Installation.md +++ b/com.unity.ml-agents/Documentation~/Installation.md @@ -36,10 +36,10 @@ You will need to clone the repository if you plan to modify or extend the ML-Age Use the command below to clone the repository ```sh -git clone --branch release_22 https://github.com/Unity-Technologies/ml-agents.git +git clone --branch release_23 https://github.com/Unity-Technologies/ml-agents.git ``` -The `--branch release_22` option will switch to the tag of the latest stable release. Omitting that will get the `develop` branch which is potentially unstable. However, if you find that a release branch does not work, the recommendation is to use the `develop` branch as it may have potential fixes for bugs and dependency issues. +The `--branch release_23` option will switch to the tag of the latest stable release. Omitting that will get the `develop` branch which is potentially unstable. However, if you find that a release branch does not work, the recommendation is to use the `develop` branch as it may have potential fixes for bugs and dependency issues. (Optional to get bleeding edge) @@ -47,7 +47,7 @@ The `--branch release_22` option will switch to the tag of the latest stable rel git clone https://github.com/Unity-Technologies/ml-agents.git ``` -If you plan to contribute those changes back, make sure to clone the `develop` branch (by omitting `--branch release_22` from the command above). See our [Contributions Guidelines](CONTRIBUTING.md) for more information on contributing to the ML-Agents Toolkit. +If you plan to contribute those changes back, make sure to clone the `develop` branch (by omitting `--branch release_23` from the command above). See our [Contributions Guidelines](CONTRIBUTING.md) for more information on contributing to the ML-Agents Toolkit. You can [add the local](https://docs.unity3d.com/Manual/upm-ui-local.html) `com.unity.ml-agents` package (from the repository that you just cloned) to your project by: @@ -84,7 +84,7 @@ which will install the latest version of ML-Agents Python packages and associate conda install "grpcio=1.48.2" -c conda-forge ``` -When you install the Python package, the dependencies listed in the [setup.py file](https://github.com/Unity-Technologies/ml-agents/blob/release_22/ml-agents/setup.py) are also installed. These include [PyTorch](Background-PyTorch.md). +When you install the Python package, the dependencies listed in the [setup.py file](https://github.com/Unity-Technologies/ml-agents/blob/release/4.0.0/ml-agents/setup.py) are also installed. These include [PyTorch](Background-PyTorch.md). #### Advanced: Local Installation for Development diff --git a/com.unity.ml-agents/Documentation~/Learning-Environment-Design.md b/com.unity.ml-agents/Documentation~/Learning-Environment-Design.md index 42bc6b9bb7..2f72546a37 100644 --- a/com.unity.ml-agents/Documentation~/Learning-Environment-Design.md +++ b/com.unity.ml-agents/Documentation~/Learning-Environment-Design.md @@ -68,7 +68,7 @@ When you create a training environment in Unity, you must set up the scene so th Curriculum learning and environment parameter randomization are two training methods that control specific parameters in your environment. As such, it is important to ensure that your environment parameters are updated at each step to the correct values. To enable this, we expose a `EnvironmentParameters` C# class that you can use to retrieve the values of the parameters defined in the training configurations for both of those features. Please see our [documentation](Training-ML-Agents.md#environment-parameters) for curriculum learning and environment parameter randomization for details. -We recommend modifying the environment from the Agent's `OnEpisodeBegin()` function by leveraging `Academy.Instance.EnvironmentParameters`. See the WallJump example environment for a sample usage (specifically, [WallJumpAgent.cs](https://github.com/Unity-Technologies/ml-agents/blob/release_22/Project/Assets/ML-Agents/Examples/WallJump/Scripts/WallJumpAgent.cs) ). +We recommend modifying the environment from the Agent's `OnEpisodeBegin()` function by leveraging `Academy.Instance.EnvironmentParameters`. See the WallJump example environment for a sample usage (specifically, [WallJumpAgent.cs](https://github.com/Unity-Technologies/ml-agents/blob/release/4.0.0/Project/Assets/ML-Agents/Examples/WallJump/Scripts/WallJumpAgent.cs) ). ## Agent @@ -89,4 +89,4 @@ See [Agents](Learning-Environment-Design-Agents.md) for detailed information abo We offer developers a mechanism to record statistics from within their Unity environments. These statistics are aggregated and generated during the training process. To record statistics, see the `StatsRecorder` C# class. -See the FoodCollector example environment for a sample usage (specifically, [FoodCollectorSettings.cs](https://github.com/Unity-Technologies/ml-agents/blob/release_22/Project/Assets/ML-Agents/Examples/FoodCollector/Scripts/FoodCollectorSettings.cs) ). +See the FoodCollector example environment for a sample usage (specifically, [FoodCollectorSettings.cs](https://github.com/Unity-Technologies/ml-agents/blob/release/4.0.0/Project/Assets/ML-Agents/Examples/FoodCollector/Scripts/FoodCollectorSettings.cs) ). diff --git a/com.unity.ml-agents/Documentation~/Limitations.md b/com.unity.ml-agents/Documentation~/Limitations.md index 95e08f99a0..4f436fd43b 100644 --- a/com.unity.ml-agents/Documentation~/Limitations.md +++ b/com.unity.ml-agents/Documentation~/Limitations.md @@ -3,5 +3,5 @@ See the package-specific Limitations pages: - [`com.unity.mlagents` Unity package](Package-Limitations.md) -- [`mlagents` Python package](https://github.com/Unity-Technologies/ml-agents/blob/release_22/ml-agents/README.md#limitations) -- [`mlagents_envs` Python package](https://github.com/Unity-Technologies/ml-agents/blob/release_22/ml-agents-envs/README.md#limitations) +- [`mlagents` Python package](https://github.com/Unity-Technologies/ml-agents/blob/release/4.0.0/ml-agents/README.md#limitations) +- [`mlagents_envs` Python package](https://github.com/Unity-Technologies/ml-agents/blob/release/4.0.0/ml-agents-envs/README.md#limitations) diff --git a/com.unity.ml-agents/Documentation~/Migrating.md b/com.unity.ml-agents/Documentation~/Migrating.md index 2ae1c6e4ec..f2f21df31b 100644 --- a/com.unity.ml-agents/Documentation~/Migrating.md +++ b/com.unity.ml-agents/Documentation~/Migrating.md @@ -4,6 +4,9 @@ +## Migrating the package to version 4.x +- Unity ML-Agents now supports Inference Engine. If you run into issues, please refer to the [Upgrade to Inference Engine](https://docs.unity3d.com/Packages/com.unity.ai.inference@latest/index.html?subfolder=/manual/upgrade-guide.html) documentation. + ## Migrating to the ml-agents-envs 0.30.0 package - Python 3.10.12 is now the minimum version of python supported due to [python3.6 EOL](https://endoflife.date/python). Please update your python installation to 3.10.12 or higher. - The `gym-unity` package has been refactored into the `ml-agents-envs` package. Please update your imports accordingly. diff --git a/com.unity.ml-agents/Documentation~/Python-Custom-Trainer-Plugin.md b/com.unity.ml-agents/Documentation~/Python-Custom-Trainer-Plugin.md index 7038c6f6d4..8518f44262 100644 --- a/com.unity.ml-agents/Documentation~/Python-Custom-Trainer-Plugin.md +++ b/com.unity.ml-agents/Documentation~/Python-Custom-Trainer-Plugin.md @@ -4,7 +4,7 @@ As an attempt to bring a wider variety of reinforcement learning algorithms to o ## Overview Model-free RL algorithms generally fall into two broad categories: on-policy and off-policy. On-policy algorithms perform updates based on data gathered from the current policy. Off-policy algorithms learn a Q function from a buffer of previous data, then use this Q function to make decisions. Off-policy algorithms have three key benefits in the context of ML-Agents: They tend to use fewer samples than on-policy as they can pull and re-use data from the buffer many times. They allow player demonstrations to be inserted in-line with RL data into the buffer, enabling new ways of doing imitation learning by streaming player data. -To add new custom trainers to ML-agents, you would need to create a new python package. To give you an idea of how to structure your package, we have created a [mlagents_trainer_plugin](https://github.com/Unity-Technologies/ml-agents/tree/release_22/ml-agents-trainer-plugin) package ourselves as an example, with implementation of `A2c` and `DQN` algorithms. You would need a `setup.py` file to list extra requirements and register the new RL algorithm in ml-agents ecosystem and be able to call `mlagents-learn` CLI with your customized configuration. +To add new custom trainers to ML-agents, you would need to create a new python package. To give you an idea of how to structure your package, we have created a [mlagents_trainer_plugin](https://github.com/Unity-Technologies/ml-agents/tree/release/4.0.0/ml-agents-trainer-plugin) package ourselves as an example, with implementation of `A2c` and `DQN` algorithms. You would need a `setup.py` file to list extra requirements and register the new RL algorithm in ml-agents ecosystem and be able to call `mlagents-learn` CLI with your customized configuration. ```shell @@ -36,4 +36,4 @@ mlagents-learn ml-agents-trainer-plugin/mlagents_trainer_plugin/a2c/a2c_3DBall.y ``` ## Tutorial -Here’s a step-by-step [tutorial](Tutorial-Custom-Trainer-Plugin.md) on how to write a setup file and extend ml-agents trainers, optimizers, and hyperparameter settings.To extend ML-agents classes see references on [trainers](Python-On-Off-Policy-Trainer-Documentation.md) and [Optimizer](Python-Optimizer-Documentation.md). \ No newline at end of file +Here’s a step-by-step [tutorial](Tutorial-Custom-Trainer-Plugin.md) on how to write a setup file and extend ml-agents trainers, optimizers, and hyperparameter settings. To extend ML-agents classes, see references on [trainers](Python-On-Off-Policy-Trainer-Documentation.md) and [Optimizer](Python-Optimizer-Documentation.md). diff --git a/com.unity.ml-agents/Documentation~/Python-Gym-API.md b/com.unity.ml-agents/Documentation~/Python-Gym-API.md index d08caae44a..551316cd63 100644 --- a/com.unity.ml-agents/Documentation~/Python-Gym-API.md +++ b/com.unity.ml-agents/Documentation~/Python-Gym-API.md @@ -6,7 +6,7 @@ We provide a gym wrapper and instructions for using it with existing machine lea ## Installation -The gym wrapper is part of the `mlagents_envs` package. Please refer to the [mlagents_envs installation instructions](https://github.com/Unity-Technologies/ml-agents/blob/release_22/ml-agents-envs/README.md). +The gym wrapper is part of the `mlagents_envs` package. Please refer to the [mlagents_envs installation instructions](https://github.com/Unity-Technologies/ml-agents/blob/release/4.0.0/ml-agents-envs/README.md). ## Using the Gym Wrapper diff --git a/com.unity.ml-agents/Documentation~/Python-LLAPI.md b/com.unity.ml-agents/Documentation~/Python-LLAPI.md index 45f560ccea..c8ee1e7fc3 100644 --- a/com.unity.ml-agents/Documentation~/Python-LLAPI.md +++ b/com.unity.ml-agents/Documentation~/Python-LLAPI.md @@ -17,7 +17,7 @@ The key objects in the Python API include: - **TerminalSteps** — contains the data from Agents belonging to the same "Behavior" in the simulation, such as observations and rewards. Only Agents whose episode ended since the last call to `env.step()` are in the TerminalSteps object. - **BehaviorSpec** — describes the shape of the observation data inside DecisionSteps and TerminalSteps as well as the expected action shapes. -These classes are all defined in the [base_env](https://github.com/Unity-Technologies/ml-agents/blob/release_22/ml-agents-envs/mlagents_envs/base_env.py) script. +These classes are all defined in the [base_env](https://github.com/Unity-Technologies/ml-agents/blob/release/4.0.0/ml-agents-envs/mlagents_envs/base_env.py) script. An Agent "Behavior" is a group of Agents identified by a `BehaviorName` that share the same observations and action types (described in their `BehaviorSpec`). You can think about Agent Behavior as a group of agents that will share the same policy. All Agents with the same behavior have the same goal and reward signals. @@ -27,7 +27,7 @@ _Notice: Currently communication between Unity and Python takes place over an op ## Loading a Unity Environment -Python-side communication happens through `UnityEnvironment` which is located in [`environment.py`](https://github.com/Unity-Technologies/ml-agents/blob/release_22/ml-agents-envs/mlagents_envs/environment.py). To load a Unity environment from a built binary file, put the file in the same directory as `envs`. For example, if the filename of your Unity environment is `3DBall`, in python, run: +Python-side communication happens through `UnityEnvironment` which is located in [`environment.py`](https://github.com/Unity-Technologies/ml-agents/blob/release/4.0.0/ml-agents-envs/mlagents_envs/environment.py). To load a Unity environment from a built binary file, put the file in the same directory as `envs`. For example, if the filename of your Unity environment is `3DBall`, in python, run: ```python from mlagents_envs.environment import UnityEnvironment diff --git a/com.unity.ml-agents/Documentation~/Python-PettingZoo-API.md b/com.unity.ml-agents/Documentation~/Python-PettingZoo-API.md index ec6c3da3ba..dd3f5d26d8 100644 --- a/com.unity.ml-agents/Documentation~/Python-PettingZoo-API.md +++ b/com.unity.ml-agents/Documentation~/Python-PettingZoo-API.md @@ -4,7 +4,7 @@ With the increasing interest in multi-agent training with a gym-like API, we pro ## Installation and Examples -The PettingZoo wrapper is part of the `mlagents_envs` package. Please refer to the [mlagents_envs installation instructions](https://github.com/Unity-Technologies/ml-agents/blob/release_22/ml-agents-envs/README.md). +The PettingZoo wrapper is part of the `mlagents_envs` package. Please refer to the [mlagents_envs installation instructions](https://github.com/Unity-Technologies/ml-agents/blob/release/4.0.0/ml-agents-envs/README.md). [[Colab] PettingZoo Wrapper Example](https://colab.research.google.com/github/Unity-Technologies/ml-agents/blob/develop-python-api-ga/ml-agents-envs/colabs/Colab_PettingZoo.ipynb) diff --git a/com.unity.ml-agents/Documentation~/TableOfContents.md b/com.unity.ml-agents/Documentation~/TableOfContents.md index 7bfeac7d97..5f3b2d271c 100644 --- a/com.unity.ml-agents/Documentation~/TableOfContents.md +++ b/com.unity.ml-agents/Documentation~/TableOfContents.md @@ -26,9 +26,9 @@ * [On/Off Policy Trainer Documentation](Python-On-Off-Policy-Trainer-Documentation.md) * [Python Optimizer Documentation](Python-Optimizer-Documentation.md) * [Python Tutorial with Google Colab](Tutorial-Colab.md) - * [Using a UnityEnvironment](https://colab.research.google.com/github/Unity-Technologies/ml-agents/blob/release_22_docs/colab/Colab_UnityEnvironment_1_Run.ipynb) - * [Q-Learning with a UnityEnvironment](https://colab.research.google.com/github/Unity-Technologies/ml-agents/blob/release_22_docs/colab/Colab_UnityEnvironment_2_Train.ipynb) - * [Using Side Channels on a UnityEnvironment](https://colab.research.google.com/github/Unity-Technologies/ml-agents/blob/release_22_docs/colab/Colab_UnityEnvironment_3_SideChannel.ipynb) + * [Using a UnityEnvironment](https://colab.research.google.com/github/Unity-Technologies/ml-agents/blob/release/4.0.0/colab/Colab_UnityEnvironment_1_Run.ipynb) + * [Q-Learning with a UnityEnvironment](https://colab.research.google.com/github/Unity-Technologies/ml-agents/blob/release/4.0.0/colab/Colab_UnityEnvironment_2_Train.ipynb) + * [Using Side Channels on a UnityEnvironment](https://colab.research.google.com/github/Unity-Technologies/ml-agents/blob/release/4.0.0/colab/Colab_UnityEnvironment_3_SideChannel.ipynb) * [Advanced Features](Advanced-Features.md) * [Custom Side Channels](Custom-SideChannels.md) * [Custom Grid Sensors](Custom-GridSensors.md) diff --git a/com.unity.ml-agents/Documentation~/Training-ML-Agents.md b/com.unity.ml-agents/Documentation~/Training-ML-Agents.md index 02170c3428..e45df5499e 100644 --- a/com.unity.ml-agents/Documentation~/Training-ML-Agents.md +++ b/com.unity.ml-agents/Documentation~/Training-ML-Agents.md @@ -1,6 +1,6 @@ # Training ML-Agents -Once your learning environment has been created and is ready for training, the next step is to initiate a training run. Training in the ML-Agents Toolkit is powered by a dedicated Python package, `mlagents`. This package exposes a command `mlagents-learn` that is the single entry point for all training workflows (e.g. reinforcement leaning, imitation learning, curriculum learning). Its implementation can be found [here](https://github.com/Unity-Technologies/ml-agents/blob/release_22/ml-agents/mlagents/trainers/learn.py). +Once your learning environment has been created and is ready for training, the next step is to initiate a training run. Training in the ML-Agents Toolkit is powered by a dedicated Python package, `mlagents`. This package exposes a command `mlagents-learn` that is the single entry point for all training workflows (e.g. reinforcement leaning, imitation learning, curriculum learning). Its implementation can be found [here](https://github.com/Unity-Technologies/ml-agents/blob/release/4.0.0/ml-agents/mlagents/trainers/learn.py). ## Training with mlagents-learn @@ -62,7 +62,7 @@ More specifically, this section offers a detailed guide on the command-line flag - ``: defines the training hyperparameters for each Behavior in the scene, and the set-ups for the environment parameters (Curriculum Learning and Environment Parameter Randomization) -It is important to highlight that successfully training a Behavior in the ML-Agents Toolkit involves tuning the training hyperparameters and configuration. This guide contains some best practices for tuning the training process when the default parameters don't seem to be giving the level of performance you would like. We provide sample configuration files for our example environments in the [config](https://github.com/Unity-Technologies/ml-agents/tree/release_22/config/) directory. The `config/ppo/3DBall.yaml` was used to train the 3D Balance Ball in the [Running an Example](Sample.md) guide. That configuration file uses the PPO trainer, but we also have configuration files for SAC and GAIL. +It is important to highlight that successfully training a Behavior in the ML-Agents Toolkit involves tuning the training hyperparameters and configuration. This guide contains some best practices for tuning the training process when the default parameters don't seem to be giving the level of performance you would like. We provide sample configuration files for our example environments in the [config](https://github.com/Unity-Technologies/ml-agents/tree/release/4.0.0/config/) directory. The `config/ppo/3DBall.yaml` was used to train the 3D Balance Ball in the [Running an Example](Sample.md) guide. That configuration file uses the PPO trainer, but we also have configuration files for SAC and GAIL. Additionally, the set of configurations you provide depend on the training functionalities you use (see [ML-Agents Theory](ML-Agents-Overview.md) for a description of all the training functionalities). Each functionality you add typically has its own training configurations. For instance: diff --git a/com.unity.ml-agents/Documentation~/Training-Plugins.md b/com.unity.ml-agents/Documentation~/Training-Plugins.md index 934c2c1819..f29739e7d0 100644 --- a/com.unity.ml-agents/Documentation~/Training-Plugins.md +++ b/com.unity.ml-agents/Documentation~/Training-Plugins.md @@ -10,7 +10,7 @@ ML-Agents provides support for running your own python implementations of specif The `ml-agents-plugin-examples` directory contains a reference implementation of each plugin interface, so it's a good starting point. ### setup.py -If you don't already have a `setup.py` file for your python code, you'll need to add one. `ml-agents-plugin-examples` has a [minimal example](https://github.com/Unity-Technologies/ml-agents/blob/release_22/ml-agents-plugin-examples/setup.py) of this. +If you don't already have a `setup.py` file for your python code, you'll need to add one. `ml-agents-plugin-examples` has a [minimal example](https://github.com/Unity-Technologies/ml-agents/blob/release/4.0.0/ml-agents-plugin-examples/setup.py) of this. In the call to `setup()`, you'll need to add to the `entry_points` dictionary for each plugin interface that you implement. The form of this is `{entry point name}={plugin module}:{plugin function}`. For example, in `ml-agents-plugin-examples`: @@ -42,4 +42,4 @@ The StatsWriter class receives various information from the training process, su The `StatsWriter.write_stats()` method must be implemented in any derived classes. It takes a "category" parameter, which typically is the behavior name of the Agents being trained, and a dictionary of `StatSummary` values with string keys. Additionally, `StatsWriter.on_add_stat()` may be extended to register a callback handler for each stat emission. #### Registration -The `StatsWriter` registration function takes a `RunOptions` argument and returns a list of `StatsWriter`s. An example implementation is provided in [`mlagents_plugin_examples`](https://github.com/Unity-Technologies/ml-agents/blob/release_22/ml-agents-plugin-examples/mlagents_plugin_examples/example_stats_writer.py) +The `StatsWriter` registration function takes a `RunOptions` argument and returns a list of `StatsWriter`s. An example implementation is provided in [`mlagents_plugin_examples`](https://github.com/Unity-Technologies/ml-agents/blob/release/4.0.0/ml-agents-plugin-examples/mlagents_plugin_examples/example_stats_writer.py) diff --git a/com.unity.ml-agents/Documentation~/Training-on-Amazon-Web-Service.md b/com.unity.ml-agents/Documentation~/Training-on-Amazon-Web-Service.md index fd57c59ade..f6686355fa 100644 --- a/com.unity.ml-agents/Documentation~/Training-on-Amazon-Web-Service.md +++ b/com.unity.ml-agents/Documentation~/Training-on-Amazon-Web-Service.md @@ -58,7 +58,7 @@ After launching your EC2 instance using the ami and ssh into it: 2. Clone the ML-Agents repo and install the required Python packages ```sh - git clone --branch release_22 https://github.com/Unity-Technologies/ml-agents.git + git clone --branch release_23 https://github.com/Unity-Technologies/ml-agents.git cd ml-agents/ml-agents/ pip3 install -e . ``` diff --git a/com.unity.ml-agents/Documentation~/Tutorial-Colab.md b/com.unity.ml-agents/Documentation~/Tutorial-Colab.md index f91c847757..81934dfdd8 100644 --- a/com.unity.ml-agents/Documentation~/Tutorial-Colab.md +++ b/com.unity.ml-agents/Documentation~/Tutorial-Colab.md @@ -2,8 +2,8 @@ Interactive tutorials for using ML-Agents with Google Colab environments. -| **Tutorial** | **Description** | -|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------| -| [Using a UnityEnvironment](https://colab.research.google.com/github/Unity-Technologies/ml-agents/blob/release_22_docs/colab/Colab_UnityEnvironment_1_Run.ipynb) | Learn how to set up and interact with Unity environments in Colab. | -| [Q-Learning with a UnityEnvironment](https://colab.research.google.com/github/Unity-Technologies/ml-agents/blob/release_22_docs/colab/Colab_UnityEnvironment_2_Train.ipynb) | Implement Q-Learning algorithms with Unity ML-Agents in Colab. | -| [Using Side Channels on a UnityEnvironment](https://colab.research.google.com/github/Unity-Technologies/ml-agents/blob/release_22_docs/colab/Colab_UnityEnvironment_3_SideChannel.ipynb) | Explore side channel communication between Unity and Python. | +| **Tutorial** | **Description** | +|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------| +| [Using a UnityEnvironment](https://colab.research.google.com/github/Unity-Technologies/ml-agents/blob/release/4.0.0/colab/Colab_UnityEnvironment_1_Run.ipynb) | Learn how to set up and interact with Unity environments in Colab. | +| [Q-Learning with a UnityEnvironment](https://colab.research.google.com/github/Unity-Technologies/ml-agents/blob/release/4.0.0/colab/Colab_UnityEnvironment_2_Train.ipynb) | Implement Q-Learning algorithms with Unity ML-Agents in Colab. | +| [Using Side Channels on a UnityEnvironment](https://colab.research.google.com/github/Unity-Technologies/ml-agents/blob/release/4.0.0/colab/Colab_UnityEnvironment_3_SideChannel.ipynb) | Explore side channel communication between Unity and Python. | diff --git a/com.unity.ml-agents/Documentation~/Tutorial-Custom-Trainer-Plugin.md b/com.unity.ml-agents/Documentation~/Tutorial-Custom-Trainer-Plugin.md index bed6d1ef79..4797ce6c53 100644 --- a/com.unity.ml-agents/Documentation~/Tutorial-Custom-Trainer-Plugin.md +++ b/com.unity.ml-agents/Documentation~/Tutorial-Custom-Trainer-Plugin.md @@ -11,7 +11,7 @@ conda activate trainer-env Users of the plug-in system are responsible for implementing the trainer class subject to the API standard. Let us follow an example by implementing a custom trainer named "YourCustomTrainer". You can either extend `OnPolicyTrainer` or `OffPolicyTrainer` classes depending on the training strategies you choose. -Please refer to the internal [PPO implementation](https://github.com/Unity-Technologies/ml-agents/blob/release_22/ml-agents/mlagents/trainers/ppo/trainer.py) for a complete code example. We will not provide a workable code in the document. The purpose of the tutorial is to introduce you to the core components and interfaces of our plugin framework. We use code snippets and patterns to demonstrate the control and data flow. +Please refer to the internal [PPO implementation](https://github.com/Unity-Technologies/ml-agents/blob/release/4.0.0/ml-agents/mlagents/trainers/ppo/trainer.py) for a complete code example. We will not provide a workable code in the document. The purpose of the tutorial is to introduce you to the core components and interfaces of our plugin framework. We use code snippets and patterns to demonstrate the control and data flow. Your custom trainers are responsible for collecting experiences and training the models. Your custom trainer class acts like a coordinator to the policy and optimizer. To start implementing methods in the class, create a policy class objects from method `create_policy`: diff --git a/com.unity.ml-agents/Documentation~/images/anaconda_default.PNG b/com.unity.ml-agents/Documentation~/images/anaconda_default.png similarity index 100% rename from com.unity.ml-agents/Documentation~/images/anaconda_default.PNG rename to com.unity.ml-agents/Documentation~/images/anaconda_default.png diff --git a/com.unity.ml-agents/Documentation~/images/anaconda_install.PNG b/com.unity.ml-agents/Documentation~/images/anaconda_install.png similarity index 100% rename from com.unity.ml-agents/Documentation~/images/anaconda_install.PNG rename to com.unity.ml-agents/Documentation~/images/anaconda_install.png diff --git a/com.unity.ml-agents/Documentation~/images/conda_new.PNG b/com.unity.ml-agents/Documentation~/images/conda_new.png similarity index 100% rename from com.unity.ml-agents/Documentation~/images/conda_new.PNG rename to com.unity.ml-agents/Documentation~/images/conda_new.png diff --git a/com.unity.ml-agents/Documentation~/images/cuda_toolkit_directory.PNG b/com.unity.ml-agents/Documentation~/images/cuda_toolkit_directory.png similarity index 100% rename from com.unity.ml-agents/Documentation~/images/cuda_toolkit_directory.PNG rename to com.unity.ml-agents/Documentation~/images/cuda_toolkit_directory.png diff --git a/com.unity.ml-agents/Documentation~/images/cudnn_zip_files.PNG b/com.unity.ml-agents/Documentation~/images/cudnn_zip_files.png similarity index 100% rename from com.unity.ml-agents/Documentation~/images/cudnn_zip_files.PNG rename to com.unity.ml-agents/Documentation~/images/cudnn_zip_files.png diff --git a/com.unity.ml-agents/Documentation~/images/new_system_variable.PNG b/com.unity.ml-agents/Documentation~/images/new_system_variable.png similarity index 100% rename from com.unity.ml-agents/Documentation~/images/new_system_variable.PNG rename to com.unity.ml-agents/Documentation~/images/new_system_variable.png diff --git a/com.unity.ml-agents/Documentation~/images/path_variables.PNG b/com.unity.ml-agents/Documentation~/images/path_variables.png similarity index 100% rename from com.unity.ml-agents/Documentation~/images/path_variables.PNG rename to com.unity.ml-agents/Documentation~/images/path_variables.png diff --git a/com.unity.ml-agents/Documentation~/images/system_variable_name_value.PNG b/com.unity.ml-agents/Documentation~/images/system_variable_name_value.png similarity index 100% rename from com.unity.ml-agents/Documentation~/images/system_variable_name_value.PNG rename to com.unity.ml-agents/Documentation~/images/system_variable_name_value.png diff --git a/com.unity.ml-agents/Documentation~/index.md b/com.unity.ml-agents/Documentation~/index.md index 6d6292375b..70748a0c88 100644 --- a/com.unity.ml-agents/Documentation~/index.md +++ b/com.unity.ml-agents/Documentation~/index.md @@ -31,7 +31,7 @@ The package allows you to convert any Unity scene into a learning environment an ## Community and Feedback -The ML-Agents Toolkit is an open-source project, and we encourage and welcome contributions. If you wish to contribute, be sure to review our [contribution guidelines](CONTRIBUTING.md) and [code of conduct](https://github.com/Unity-Technologies/ml-agents/blob/release_22/CODE_OF_CONDUCT.md). +The ML-Agents Toolkit is an open-source project, and we encourage and welcome contributions. If you wish to contribute, be sure to review our [contribution guidelines](CONTRIBUTING.md) and [code of conduct](https://github.com/Unity-Technologies/ml-agents/blob/release/4.0.0/CODE_OF_CONDUCT.md). For problems with the installation and setup of the ML-Agents Toolkit, or discussions about how to best setup or train your agents, please create a new thread on the [Unity ML-Agents discussion forum](https://discussions.unity.com/tag/ml-agents). Be sure to include as many details as possible to help others assist you effectively. If you run into any other problems using the ML-Agents Toolkit or have a specific feature request, please [submit a GitHub issue](https://github.com/Unity-Technologies/ml-agents/issues). diff --git a/com.unity.ml-agents/Runtime/Academy.cs b/com.unity.ml-agents/Runtime/Academy.cs index bebb7174e4..3cdc15f64c 100644 --- a/com.unity.ml-agents/Runtime/Academy.cs +++ b/com.unity.ml-agents/Runtime/Academy.cs @@ -19,8 +19,8 @@ * manages the communication between the learning environment and the Python * API. For more information on each of these entities, in addition to how to * set-up a learning environment and train the behavior of characters in a - * Unity scene, please browse our documentation pages on GitHub: - * https://github.com/Unity-Technologies/ml-agents/tree/release_22_docs/docs/ + * Unity scene, please browse our documentation pages: + * https://docs.unity3d.com/Packages/com.unity.ml-agents@latest */ namespace Unity.MLAgents @@ -61,8 +61,8 @@ void FixedUpdate() /// fall back to inference or heuristic decisions. (You can also set agents to always use /// inference or heuristics.) /// - [HelpURL("https://github.com/Unity-Technologies/ml-agents/tree/release_22_docs/" + - "docs/Learning-Environment-Design.md")] + [HelpURL("https://docs.unity3d.com/Packages/com.unity.ml-agents@latest/index.html?subfolder=/manual/" + + "Learning-Environment-Design.html")] public class Academy : IDisposable { /// @@ -107,7 +107,7 @@ public class Academy : IDisposable /// Unity package version of com.unity.ml-agents. /// This must match the version string in package.json and is checked in a unit test. /// - internal const string k_PackageVersion = "3.0.0"; + internal const string k_PackageVersion = "4.0.0"; const int k_EditorTrainingPort = 5004; diff --git a/com.unity.ml-agents/Runtime/Actuators/IActionReceiver.cs b/com.unity.ml-agents/Runtime/Actuators/IActionReceiver.cs index 3dbb89d816..e92da804c5 100644 --- a/com.unity.ml-agents/Runtime/Actuators/IActionReceiver.cs +++ b/com.unity.ml-agents/Runtime/Actuators/IActionReceiver.cs @@ -184,7 +184,7 @@ public interface IActionReceiver /// /// See [Agents - Actions] for more information on masking actions. /// - /// [Agents - Actions]: https://github.com/Unity-Technologies/ml-agents/blob/release_22_docs/docs/Learning-Environment-Design-Agents.md#actions + /// [Agents - Actions]: https://docs.unity3d.com/Packages/com.unity.ml-agents@latest/index.html?subfolder=/manual/Learning-Environment-Design.html /// /// void WriteDiscreteActionMask(IDiscreteActionMask actionMask); diff --git a/com.unity.ml-agents/Runtime/Actuators/IDiscreteActionMask.cs b/com.unity.ml-agents/Runtime/Actuators/IDiscreteActionMask.cs index dfb33901ff..995a8a574a 100644 --- a/com.unity.ml-agents/Runtime/Actuators/IDiscreteActionMask.cs +++ b/com.unity.ml-agents/Runtime/Actuators/IDiscreteActionMask.cs @@ -16,7 +16,7 @@ public interface IDiscreteActionMask /// /// See [Agents - Actions] for more information on masking actions. /// - /// [Agents - Actions]: https://github.com/Unity-Technologies/ml-agents/blob/release_22_docs/docs/Learning-Environment-Design-Agents.md#masking-discrete-actions + /// [Agents - Actions]: https://docs.unity3d.com/Packages/com.unity.ml-agents@latest/index.html?subfolder=/manual/Learning-Environment-Design-Agents.html#masking-discrete-actions /// /// The branch for which the actions will be masked. /// Index of the action. diff --git a/com.unity.ml-agents/Runtime/Agent.cs b/com.unity.ml-agents/Runtime/Agent.cs index ea13bcfa6a..fbec5722ec 100644 --- a/com.unity.ml-agents/Runtime/Agent.cs +++ b/com.unity.ml-agents/Runtime/Agent.cs @@ -192,14 +192,14 @@ public override BuiltInActuatorType GetBuiltInActuatorType() /// [OnDisable()]: https://docs.unity3d.com/ScriptReference/MonoBehaviour.OnDisable.html] /// [OnBeforeSerialize()]: https://docs.unity3d.com/ScriptReference/MonoBehaviour.OnBeforeSerialize.html /// [OnAfterSerialize()]: https://docs.unity3d.com/ScriptReference/MonoBehaviour.OnAfterSerialize.html - /// [Agents]: https://github.com/Unity-Technologies/ml-agents/blob/release_22_docs/docs/Learning-Environment-Design-Agents.md - /// [Reinforcement Learning in Unity]: https://github.com/Unity-Technologies/ml-agents/blob/release_22_docs/docs/Learning-Environment-Design.md + /// [Agents]: https://docs.unity3d.com/Packages/com.unity.ml-agents@latest/index.html?subfolder=/manual/Learning-Environment-Design-Agents.html + /// [Reinforcement Learning in Unity]: https://docs.unity3d.com/Packages/com.unity.ml-agents@latest/index.html?subfolder=/manual/Learning-Environment-Design.html /// [Unity ML-Agents Toolkit]: https://github.com/Unity-Technologies/ml-agents - /// [Unity ML-Agents Toolkit manual]: https://github.com/Unity-Technologies/ml-agents/blob/release_22_docs/docs/Readme.md + /// [Unity ML-Agents Toolkit manual]: https://docs.unity3d.com/Packages/com.unity.ml-agents@latest /// /// - [HelpURL("https://github.com/Unity-Technologies/ml-agents/blob/release_22_docs/" + - "docs/Learning-Environment-Design-Agents.md")] + [HelpURL("https://docs.unity3d.com/Packages/com.unity.ml-agents@latest/index.html?subfolder=/manual/" + + "Learning-Environment-Design-Agents.html")] [Serializable] [RequireComponent(typeof(BehaviorParameters))] [DefaultExecutionOrder(-50)] @@ -728,8 +728,8 @@ public int CompletedEpisodes /// for information about mixing reward signals from curiosity and Generative Adversarial /// Imitation Learning (GAIL) with rewards supplied through this method. /// - /// [Agents - Rewards]: https://github.com/Unity-Technologies/ml-agents/blob/release_22_docs/docs/Learning-Environment-Design-Agents.md#rewards - /// [Reward Signals]: https://github.com/Unity-Technologies/ml-agents/blob/release_22_docs/docs/ML-Agents-Overview.md#a-quick-note-on-reward-signals + /// [Agents - Rewards]: https://docs.unity3d.com/Packages/com.unity.ml-agents@latest/index.html?subfolder=/manual/Learning-Environment-Design-Agents.html#rewards + /// [Reward Signals]: https://docs.unity3d.com/Packages/com.unity.ml-agents@latest/index.html?subfolder=/manual/ML-Agents-Overview.html#a-quick-note-on-reward-signals /// /// The new value of the reward. public void SetReward(float reward) @@ -756,8 +756,8 @@ public void SetReward(float reward) /// for information about mixing reward signals from curiosity and Generative Adversarial /// Imitation Learning (GAIL) with rewards supplied through this method. /// - /// [Agents - Rewards]: https://github.com/Unity-Technologies/ml-agents/blob/release_22_docs/docs/Learning-Environment-Design-Agents.md#rewards - /// [Reward Signals]: https://github.com/Unity-Technologies/ml-agents/blob/release_22_docs/docs/ML-Agents-Overview.md#a-quick-note-on-reward-signals + /// [Agents - Rewards]: https://docs.unity3d.com/Packages/com.unity.ml-agents@latest/index.html?subfolder=/manual/Learning-Environment-Design-Agents.html#rewards + /// [Reward Signals]: https://docs.unity3d.com/Packages/com.unity.ml-agents@latest/index.html?subfolder=/manual/ML-Agents-Overview.html#a-quick-note-on-reward-signals /// /// Incremental reward value. public void AddReward(float increment) @@ -945,8 +945,8 @@ public virtual void Initialize() { } /// implementing a simple heuristic function can aid in debugging agent actions and interactions /// with its environment. /// - /// [Demonstration Recorder]: https://github.com/Unity-Technologies/ml-agents/blob/release_22_docs/docs/Learning-Environment-Design-Agents.md#recording-demonstrations - /// [Actions]: https://github.com/Unity-Technologies/ml-agents/blob/release_22_docs/docs/Learning-Environment-Design-Agents.md#actions + /// [Demonstration Recorder]: https://docs.unity3d.com/Packages/com.unity.ml-agents@latest/index.html?subfolder=/manual/Learning-Environment-Design-Agents.html#recording-demonstrations + /// [Actions]: https://docs.unity3d.com/Packages/com.unity.ml-agents@latest/index.html?subfolder=/manual/Learning-Environment-Design-Agents.html#actions /// [GameObject]: https://docs.unity3d.com/Manual/GameObjects.html /// /// @@ -1203,7 +1203,7 @@ void ResetSensors() /// For more information about observations, see [Observations and Sensors]. /// /// [GameObject]: https://docs.unity3d.com/Manual/GameObjects.html - /// [Observations and Sensors]: https://github.com/Unity-Technologies/ml-agents/blob/release_22_docs/docs/Learning-Environment-Design-Agents.md#observations-and-sensors + /// [Observations and Sensors]: https://docs.unity3d.com/Packages/com.unity.ml-agents@latest/index.html?subfolder=/manual/Learning-Environment-Design-Agents.html#observations-and-sensors /// public virtual void CollectObservations(VectorSensor sensor) { @@ -1245,7 +1245,7 @@ public ReadOnlyCollection GetStackedObservations() /// /// See [Agents - Actions] for more information on masking actions. /// - /// [Agents - Actions]: https://github.com/Unity-Technologies/ml-agents/blob/release_22_docs/docs/Learning-Environment-Design-Agents.md#actions + /// [Agents - Actions]: https://docs.unity3d.com/Packages/com.unity.ml-agents@latest/index.html?subfolder=/manual/Learning-Environment-Design-Agents.html /// /// public virtual void WriteDiscreteActionMask(IDiscreteActionMask actionMask) { } @@ -1312,7 +1312,7 @@ public virtual void WriteDiscreteActionMask(IDiscreteActionMask actionMask) { } /// /// For more information about implementing agent actions see [Agents - Actions]. /// - /// [Agents - Actions]: https://github.com/Unity-Technologies/ml-agents/blob/release_22_docs/docs/Learning-Environment-Design-Agents.md#actions + /// [Agents - Actions]: https://docs.unity3d.com/Packages/com.unity.ml-agents@latest/index.html?subfolder=/manual/Learning-Environment-Design-Agents.html#actions /// /// /// diff --git a/com.unity.ml-agents/Runtime/Demonstrations/DemonstrationRecorder.cs b/com.unity.ml-agents/Runtime/Demonstrations/DemonstrationRecorder.cs index a4101f8976..a7f7627898 100644 --- a/com.unity.ml-agents/Runtime/Demonstrations/DemonstrationRecorder.cs +++ b/com.unity.ml-agents/Runtime/Demonstrations/DemonstrationRecorder.cs @@ -19,7 +19,7 @@ namespace Unity.MLAgents.Demonstrations /// See [Imitation Learning - Recording Demonstrations] for more information. /// /// [GameObject]: https://docs.unity3d.com/Manual/GameObjects.html - /// [Imitation Learning - Recording Demonstrations]: https://github.com/Unity-Technologies/ml-agents/blob/release_22_docs/docs//Learning-Environment-Design-Agents.md#recording-demonstrations + /// [Imitation Learning - Recording Demonstrations]: https://docs.unity3d.com/Packages/com.unity.ml-agents@latest/index.html?subfolder=/manual/Learning-Environment-Design-Agents.html#recording-demonstrations /// [RequireComponent(typeof(Agent))] [AddComponentMenu("ML Agents/Demonstration Recorder", (int)MenuGroup.Default)] diff --git a/com.unity.ml-agents/Runtime/Sensors/IJointExtractor.cs b/com.unity.ml-agents/Runtime/Sensors/IJointExtractor.cs index ed3d78d5f1..c2c0c1fbb0 100644 --- a/com.unity.ml-agents/Runtime/Sensors/IJointExtractor.cs +++ b/com.unity.ml-agents/Runtime/Sensors/IJointExtractor.cs @@ -10,16 +10,16 @@ public interface IJointExtractor /// Determine the number of observations that would be generated for the particular joint /// using the provided PhysicsSensorSettings. /// - /// + /// The settings used to configure the physics sensor. /// Number of floats that will be written. int NumObservations(PhysicsSensorSettings settings); /// /// Write the observations to the ObservationWriter, starting at the specified offset. /// - /// - /// - /// + /// The settings used to configure the physics sensor. + /// The writer to which observations are written. + /// The starting index in the writer to begin writing observations. /// Number of floats that were written. int Write(PhysicsSensorSettings settings, ObservationWriter writer, int offset); } diff --git a/com.unity.ml-agents/Runtime/Sensors/PhysicsBodySensor.cs b/com.unity.ml-agents/Runtime/Sensors/PhysicsBodySensor.cs index b343945460..b859d078b2 100644 --- a/com.unity.ml-agents/Runtime/Sensors/PhysicsBodySensor.cs +++ b/com.unity.ml-agents/Runtime/Sensors/PhysicsBodySensor.cs @@ -21,9 +21,9 @@ public class PhysicsBodySensor : ISensor, IBuiltInSensor /// /// Construct a new PhysicsBodySensor /// - /// - /// - /// + /// The pose extractor used to obtain rigid body poses. + /// The settings used to configure the physics sensor. + /// The name assigned to the sensor. public PhysicsBodySensor( RigidBodyPoseExtractor poseExtractor, PhysicsSensorSettings settings, diff --git a/com.unity.ml-agents/Runtime/Sensors/PoseExtractor.cs b/com.unity.ml-agents/Runtime/Sensors/PoseExtractor.cs index 98989c6499..b20adeaa78 100644 --- a/com.unity.ml-agents/Runtime/Sensors/PoseExtractor.cs +++ b/com.unity.ml-agents/Runtime/Sensors/PoseExtractor.cs @@ -30,6 +30,7 @@ public abstract class PoseExtractor /// /// Read iterator for the enabled model space transforms. /// + /// An enumerable of enabled model space poses. public IEnumerable GetEnabledModelSpacePoses() { if (m_ModelSpacePoses == null) @@ -49,6 +50,7 @@ public IEnumerable GetEnabledModelSpacePoses() /// /// Read iterator for the enabled local space transforms. /// + /// An enumerable of enabled local space poses. public IEnumerable GetEnabledLocalSpacePoses() { if (m_LocalSpacePoses == null) @@ -68,6 +70,7 @@ public IEnumerable GetEnabledLocalSpacePoses() /// /// Read iterator for the enabled model space linear velocities. /// + /// An enumerable of enabled model space linear velocities. public IEnumerable GetEnabledModelSpaceVelocities() { if (m_ModelSpaceLinearVelocities == null) @@ -87,6 +90,7 @@ public IEnumerable GetEnabledModelSpaceVelocities() /// /// Read iterator for the enabled local space linear velocities. /// + /// An enumerable of enabled local space linear velocities. public IEnumerable GetEnabledLocalSpaceVelocities() { if (m_LocalSpaceLinearVelocities == null) @@ -136,7 +140,7 @@ public int NumPoses /// /// Get the parent index of the body at the specified index. /// - /// + /// The index of the body whose parent index is to be retrieved. /// The parent index of the body at the specified index. public int GetParentIndex(int index) { @@ -151,13 +155,18 @@ public int GetParentIndex(int index) /// /// Set whether the pose at the given index is enabled or disabled for observations. /// - /// - /// + /// The index of the pose to enable or disable. + /// Whether the pose is enabled (true) or disabled (false). public void SetPoseEnabled(int index, bool val) { m_PoseEnabled[index] = val; } + /// + /// Returns whether the pose at the given index is enabled for observations. + /// + /// The index of the pose to check. + /// True if the pose is enabled; otherwise, false. public bool IsPoseEnabled(int index) { return m_PoseEnabled[index]; @@ -167,7 +176,7 @@ public bool IsPoseEnabled(int index) /// Initialize with the mapping of parent indices. /// The 0th element is assumed to be -1, indicating that it's the root. /// - /// + /// An array mapping each pose to its parent index. The root should be -1. protected void Setup(int[] parentIndices) { #if DEBUG @@ -195,14 +204,14 @@ protected void Setup(int[] parentIndices) /// /// Return the world space Pose of the i'th object. /// - /// + /// The index of the pose to retrieve. /// The world space Pose at given index. protected internal abstract Pose GetPoseAt(int index); /// /// Return the world space linear velocity of the i'th object. /// - /// + /// The index of the pose for which to get the linear velocity. /// The world space linear velocity at given index. protected internal abstract Vector3 GetLinearVelocityAt(int index); @@ -210,7 +219,7 @@ protected void Setup(int[] parentIndices) /// Return the underlying object at the given index. This is only /// used for display in the inspector. /// - /// + /// The index of the object to retrieve. /// The `Object` at given index. protected internal virtual Object GetObjectAt(int index) { @@ -285,7 +294,7 @@ public void UpdateLocalSpacePoses() /// /// Compute the number of floats needed to represent the poses for the given PhysicsSensorSettings. /// - /// + /// The settings used to configure the physics sensor. /// The number of floats needed to represent the poses for the given `PhysicsSensorSettings`. public int GetNumPoseObservations(PhysicsSensorSettings settings) { @@ -434,7 +443,7 @@ public static class PoseExtensions /// P.Inverse() * P /// will equal the identity pose (within tolerance). /// - /// + /// The pose to operate on. /// Inverse `Pose`. public static Pose Inverse(this Pose pose) { @@ -446,8 +455,8 @@ public static Pose Inverse(this Pose pose) /// /// This is equivalent to Pose.GetTransformedBy(), but keeps the order more intuitive. /// - /// - /// + /// The pose to transform by. + /// The pose to be transformed. /// Multiplied `Pose`. public static Pose Multiply(this Pose pose, Pose rhs) { @@ -459,8 +468,8 @@ public static Pose Multiply(this Pose pose, Pose rhs) /// as a 4x4 matrix and multiplying the augmented vector. /// See https://en.wikipedia.org/wiki/Affine_transformation#Augmented_matrix for more details. /// - /// - /// + /// The pose to transform by. + /// The vector to be transformed. /// Multiplied `Pose`. public static Vector3 Multiply(this Pose pose, Vector3 rhs) { diff --git a/com.unity.ml-agents/Runtime/Sensors/RigidBodyJointExtractor.cs b/com.unity.ml-agents/Runtime/Sensors/RigidBodyJointExtractor.cs index 1b30b94e25..d984c1f14d 100644 --- a/com.unity.ml-agents/Runtime/Sensors/RigidBodyJointExtractor.cs +++ b/com.unity.ml-agents/Runtime/Sensors/RigidBodyJointExtractor.cs @@ -2,23 +2,42 @@ namespace Unity.MLAgents.Sensors { + /// + /// Extracts joint and rigidbody information for physics-based sensors. + /// [UnityEngine.Scripting.APIUpdating.MovedFrom("Unity.MLAgents.Extensions.Sensors")] public class RigidBodyJointExtractor : IJointExtractor { Rigidbody m_Body; Joint m_Joint; + /// + /// Initializes a new instance of the class. + /// + /// The Rigidbody to extract joint information from. public RigidBodyJointExtractor(Rigidbody body) { m_Body = body; m_Joint = m_Body?.GetComponent(); } + /// + /// Gets the number of observations for this joint extractor using the provided settings. + /// + /// The physics sensor settings. + /// The number of observations for this joint extractor. public int NumObservations(PhysicsSensorSettings settings) { return NumObservations(m_Body, m_Joint, settings); } + /// + /// Gets the number of observations for the specified rigidbody and joint using the provided settings. + /// + /// The Rigidbody to extract from. + /// The Joint to extract from. + /// The physics sensor settings. + /// The number of observations for the specified rigidbody and joint. public static int NumObservations(Rigidbody body, Joint joint, PhysicsSensorSettings settings) { if (body == null || joint == null) @@ -36,6 +55,13 @@ public static int NumObservations(Rigidbody body, Joint joint, PhysicsSensorSett return numObservations; } + /// + /// Writes the joint observations to the provided writer using the given settings. + /// + /// The physics sensor settings. + /// The observation writer. + /// The offset in the writer to start writing at. + /// The number of floats written to the writer. public int Write(PhysicsSensorSettings settings, ObservationWriter writer, int offset) { if (m_Body == null || m_Joint == null) diff --git a/com.unity.ml-agents/package.json b/com.unity.ml-agents/package.json index 012037d361..394fd72689 100755 --- a/com.unity.ml-agents/package.json +++ b/com.unity.ml-agents/package.json @@ -1,7 +1,7 @@ { "name": "com.unity.ml-agents", "displayName": "ML Agents", - "version": "3.0.0", + "version": "4.0.0", "unity": "6000.0", "description": "Use state-of-the-art machine learning to create intelligent character behaviors in any Unity environment (games, robotics, film, etc.).", "dependencies": { diff --git a/ml-agents-envs/README.md b/ml-agents-envs/README.md index 2a2e0232f5..6920929abe 100644 --- a/ml-agents-envs/README.md +++ b/ml-agents-envs/README.md @@ -31,7 +31,7 @@ for more information on how to use the API to interact with a Unity environment. For more information on the ML-Agents Toolkit and how to instrument a Unity scene with the ML-Agents SDK, check out the main -[ML-Agents Toolkit documentation](../com.unity.ml-agents/Documentation~/Readme.md). +[ML-Agents Toolkit documentation](https://docs.unity3d.com/Packages/com.unity.ml-agents@latest). ## Limitations diff --git a/ml-agents/README.md b/ml-agents/README.md index 16ac4dd486..57447481a3 100644 --- a/ml-agents/README.md +++ b/ml-agents/README.md @@ -23,7 +23,7 @@ python -m pip install mlagents==1.1.0 For more information on the ML-Agents Toolkit and how to instrument a Unity scene with the ML-Agents SDK, check out the main -[ML-Agents Toolkit documentation](../com.unity.ml-agents/Documentation~/Readme.md). +[ML-Agents Toolkit documentation](https://docs.unity3d.com/Packages/com.unity.ml-agents@latest). ## Limitations diff --git a/utils/make_readme_table.py b/utils/make_readme_table.py index aefe0614be..c819ccdc27 100644 --- a/utils/make_readme_table.py +++ b/utils/make_readme_table.py @@ -146,6 +146,7 @@ def pypi_link(self): ReleaseInfo("release_20", "2.3.0", "0.30.0", "November 21, 2022"), ReleaseInfo("release_21", "3.0.0-exp.1", "1.0.0", "October 9, 2023"), ReleaseInfo("release_22", "3.0.0", "1.1.0", "October 5, 2024"), + ReleaseInfo("release_23", "4.0.0", "1.1.0", "August 15, 2025"), # Verified releases # ReleaseInfo("", "1.0.8", "0.16.1", "May 26, 2021", is_verified=True), # ReleaseInfo("", "1.0.7", "0.16.1", "March 8, 2021", is_verified=True),