Merge pull request #4235 from Unity-Technologies/r4-master-resolve

Release 4 -> master
5 年前 · 751c007d
--- a/.yamato/com.unity.ml-agents-test.yml
+++ b/.yamato/com.unity.ml-agents-test.yml
  dependencies:
  {% for editor in test_editors %}
    {% for platform in test_platforms %}
-  - .yamato/com.unity.ml-agents-test.yml#test_{{ platform.name }}_{{ editor.version }}
+  - .yamato/com.unity.ml-agents-test.yml#test_com.unity.ml-agents_{{ platform.name }}_{{ editor.version }}
-  - .yamato/com.unity.ml-agents-test.yml#test_{{ platform.name }}_{{ editor.version }}
+  - .yamato/com.unity.ml-agents-test.yml#test_com.unity.ml-agents_{{ platform.name }}_{{ editor.version }}
    {% endfor %}
  {% endfor %}
  triggers:
--- a/README.md
+++ b/README.md

 # Unity ML-Agents Toolkit

-[![docs badge](https://img.shields.io/badge/docs-reference-blue.svg)](https://github.com/Unity-Technologies/ml-agents/tree/release_3_docs/docs/)
+[![docs badge](https://img.shields.io/badge/docs-reference-blue.svg)](https://github.com/Unity-Technologies/ml-agents/tree/release_4_docs/docs/)

 [![license badge](https://img.shields.io/badge/license-Apache--2.0-green.svg)](LICENSE)


 ## Releases & Documentation

-**Our latest, stable release is `Release 3`. Click
-[here](https://github.com/Unity-Technologies/ml-agents/tree/release_3_docs/docs/Readme.md)
+**Our latest, stable release is `Release 4`. Click
+[here](https://github.com/Unity-Technologies/ml-agents/tree/release_4_docs/docs/Readme.md)
 to get started with the latest release of ML-Agents.**

 The table below lists all our releases, including our `master` branch which is
 | **Version** | **Release Date** | **Source** | **Documentation** | **Download** |
 |:-------:|:------:|:-------------:|:-------:|:------------:|
 | **master (unstable)** | -- | [source](https://github.com/Unity-Technologies/ml-agents/tree/master) | [docs](https://github.com/Unity-Technologies/ml-agents/tree/master/docs/Readme.md) | [download](https://github.com/Unity-Technologies/ml-agents/archive/master.zip) |
-| **Release 3** | **June 10, 2020** | **[source](https://github.com/Unity-Technologies/ml-agents/tree/release_3)** | **[docs](https://github.com/Unity-Technologies/ml-agents/tree/release_3_docs/docs/Readme.md)** | **[download](https://github.com/Unity-Technologies/ml-agents/archive/release_3.zip)** |
+| **Release 4** | **July 15, 2020** | **[source](https://github.com/Unity-Technologies/ml-agents/tree/release_4)** | **[docs](https://github.com/Unity-Technologies/ml-agents/tree/release_4_docs/docs/Readme.md)** | **[download](https://github.com/Unity-Technologies/ml-agents/archive/release_4.zip)** |
+| **Release 3** | June 10, 2020 | [source](https://github.com/Unity-Technologies/ml-agents/tree/release_3) | [docs](https://github.com/Unity-Technologies/ml-agents/tree/release_3_docs/docs/Readme.md) | [download](https://github.com/Unity-Technologies/ml-agents/archive/release_3.zip) |
-| **0.14.0** | February 13, 2020 | [source](https://github.com/Unity-Technologies/ml-agents/tree/0.14.0) | [docs](https://github.com/Unity-Technologies/ml-agents/tree/0.14.0/docs/Readme.md) | [download](https://github.com/Unity-Technologies/ml-agents/archive/0.14.0.zip) |
-| **0.13.1** | January 21, 2020 | [source](https://github.com/Unity-Technologies/ml-agents/tree/0.13.1) | [docs](https://github.com/Unity-Technologies/ml-agents/tree/0.13.1/docs/Readme.md) | [download](https://github.com/Unity-Technologies/ml-agents/archive/0.13.1.zip) |
-| **0.13.0** | January 8, 2020 | [source](https://github.com/Unity-Technologies/ml-agents/tree/0.13.0) | [docs](https://github.com/Unity-Technologies/ml-agents/tree/0.13.0/docs/Readme.md) | [download](https://github.com/Unity-Technologies/ml-agents/archive/0.13.0.zip) |
+
 ## Citation

 If you are a researcher interested in a discussion of Unity as an AI platform,
--- a/com.unity.ml-agents.extensions/package.json
+++ b/com.unity.ml-agents.extensions/package.json
 {
-	"name": "com.unity.ml-agents.extensions",
-	"displayName":"ML Agents Extensions",
-	"version": "0.0.1-preview",
-	"unity": "2018.4",
-	"description": "A source-only package for new features based on ML-Agents",
-	"dependencies": {
-        "com.unity.ml-agents": "1.1.0-preview"
-	}
+  "name": "com.unity.ml-agents.extensions",
+  "displayName": "ML Agents Extensions",
+  "version": "0.0.1-preview",
+  "unity": "2018.4",
+  "description": "A source-only package for new features based on ML-Agents",
+  "dependencies": {
+    "com.unity.ml-agents": "1.2.0-preview"
+  }
 }
--- a/com.unity.ml-agents/CHANGELOG.md
+++ b/com.unity.ml-agents/CHANGELOG.md
 - Fixed issue with FoodCollector, Soccer, and WallJump when playing with keyboard. (#4147, #4174)
 - Fixed a crash in StatsReporter when using threaded trainers with very frequent summary writes
 (#4201)
+- `mlagents-learn` will now raise an error immediately if `--num-envs` is greater than 1 without setting the `--env`
+argument. (#4203)

 ## [1.1.0-preview] - 2020-06-10
 ### Major Changes
 - `max_step` in the `TerminalStep` and `TerminalSteps` objects was renamed `interrupted`.
 - `beta` and `epsilon` in `PPO` are no longer decayed by default but follow the same schedule as learning rate. (#3940)
 - `get_behavior_names()` and `get_behavior_spec()` on UnityEnvironment were replaced by the `behavior_specs` property. (#3946)
- The first version of the Unity Environment Registry (Experimental) has been released. More information [here](https://github.com/Unity-Technologies/ml-agents/blob/release_3_docs/docs/Unity-Environment-Registry.md)(#3967)
+- The first version of the Unity Environment Registry (Experimental) has been released. More information [here](https://github.com/Unity-Technologies/ml-agents/blob/release_4_docs/docs/Unity-Environment-Registry.md)(#3967)
 - `use_visual` and `allow_multiple_visual_obs` in the `UnityToGymWrapper` constructor
 were replaced by `allow_multiple_obs` which allows one or more visual observations and
 vector observations to be used simultaneously. (#3981) Thank you @shakenes !
 - The format for trainer configuration has changed, and the "default" behavior has been deprecated.
-  See the [Migration Guide](https://github.com/Unity-Technologies/ml-agents/blob/release_3_docs/docs/Migrating.md) for more details. (#3936)
+  See the [Migration Guide](https://github.com/Unity-Technologies/ml-agents/blob/release_4_docs/docs/Migrating.md) for more details. (#3936)
 - Training artifacts (trained models, summaries) are now found in the `results/`
  directory. (#3829)
 - When using Curriculum, the current lesson will resume if training is quit and resumed. As such,
--- a/com.unity.ml-agents/Documentation~/com.unity.ml-agents.md
+++ b/com.unity.ml-agents/Documentation~/com.unity.ml-agents.md
 [unity ML-Agents Toolkit]: https://github.com/Unity-Technologies/ml-agents
 [unity inference engine]: https://docs.unity3d.com/Packages/com.unity.barracuda@latest/index.html
 [package manager documentation]: https://docs.unity3d.com/Manual/upm-ui-install.html
-[installation instructions]: https://github.com/Unity-Technologies/ml-agents/blob/release_3_docs/docs/Installation.md
+[installation instructions]: https://github.com/Unity-Technologies/ml-agents/blob/release_4_docs/docs/Installation.md
 [github repository]: https://github.com/Unity-Technologies/ml-agents
 [python package]: https://github.com/Unity-Technologies/ml-agents
 [execution order of event functions]: https://docs.unity3d.com/Manual/ExecutionOrder.html
--- a/com.unity.ml-agents/Runtime/Academy.cs
+++ b/com.unity.ml-agents/Runtime/Academy.cs
 * API. For more information on each of these entities, in addition to how to
 * set-up a learning environment and train the behavior of characters in a
 * Unity scene, please browse our documentation pages on GitHub:
- * https://github.com/Unity-Technologies/ml-agents/tree/release_3_docs/docs/
+ * https://github.com/Unity-Technologies/ml-agents/tree/release_4_docs/docs/
 */

 namespace Unity.MLAgents
    /// fall back to inference or heuristic decisions. (You can also set agents to always use
    /// inference or heuristics.)
    /// </remarks>
-    [HelpURL("https://github.com/Unity-Technologies/ml-agents/tree/release_3_docs/" +
+    [HelpURL("https://github.com/Unity-Technologies/ml-agents/tree/release_4_docs/" +
        "docs/Learning-Environment-Design.md")]
    public class Academy : IDisposable
    {
--- a/com.unity.ml-agents/Runtime/Agent.cs
+++ b/com.unity.ml-agents/Runtime/Agent.cs
    /// [OnDisable()]: https://docs.unity3d.com/ScriptReference/MonoBehaviour.OnDisable.html]
    /// [OnBeforeSerialize()]: https://docs.unity3d.com/ScriptReference/MonoBehaviour.OnBeforeSerialize.html
    /// [OnAfterSerialize()]: https://docs.unity3d.com/ScriptReference/MonoBehaviour.OnAfterSerialize.html
-    /// [Agents]: https://github.com/Unity-Technologies/ml-agents/blob/release_3_docs/docs/Learning-Environment-Design-Agents.md
-    /// [Reinforcement Learning in Unity]: https://github.com/Unity-Technologies/ml-agents/blob/release_3_docs/docs/Learning-Environment-Design.md
+    /// [Agents]: https://github.com/Unity-Technologies/ml-agents/blob/release_4_docs/docs/Learning-Environment-Design-Agents.md
+    /// [Reinforcement Learning in Unity]: https://github.com/Unity-Technologies/ml-agents/blob/release_4_docs/docs/Learning-Environment-Design.md
-    /// [Unity ML-Agents Toolkit manual]: https://github.com/Unity-Technologies/ml-agents/blob/release_3_docs/docs/Readme.md
+    /// [Unity ML-Agents Toolkit manual]: https://github.com/Unity-Technologies/ml-agents/blob/release_4_docs/docs/Readme.md
-    [HelpURL("https://github.com/Unity-Technologies/ml-agents/blob/release_3_docs/" +
+    [HelpURL("https://github.com/Unity-Technologies/ml-agents/blob/release_4_docs/" +
        "docs/Learning-Environment-Design-Agents.md")]
    [Serializable]
    [RequireComponent(typeof(BehaviorParameters))]
        /// for information about mixing reward signals from curiosity and Generative Adversarial
        /// Imitation Learning (GAIL) with rewards supplied through this method.
        ///
-        /// [Agents - Rewards]: https://github.com/Unity-Technologies/ml-agents/blob/release_3_docs/docs/Learning-Environment-Design-Agents.md#rewards
-        /// [Reward Signals]: https://github.com/Unity-Technologies/ml-agents/blob/release_3_docs/docs/ML-Agents-Overview.md#a-quick-note-on-reward-signals
+        /// [Agents - Rewards]: https://github.com/Unity-Technologies/ml-agents/blob/release_4_docs/docs/Learning-Environment-Design-Agents.md#rewards
+        /// [Reward Signals]: https://github.com/Unity-Technologies/ml-agents/blob/release_4_docs/docs/ML-Agents-Overview.md#a-quick-note-on-reward-signals
        /// </remarks>
        /// <param name="reward">The new value of the reward.</param>
        public void SetReward(float reward)
        /// for information about mixing reward signals from curiosity and Generative Adversarial
        /// Imitation Learning (GAIL) with rewards supplied through this method.
        ///
-        /// [Agents - Rewards]: https://github.com/Unity-Technologies/ml-agents/blob/release_3_docs/docs/Learning-Environment-Design-Agents.md#rewards
-        /// [Reward Signals]: https://github.com/Unity-Technologies/ml-agents/blob/release_3_docs/docs/ML-Agents-Overview.md#a-quick-note-on-reward-signals
+        /// [Agents - Rewards]: https://github.com/Unity-Technologies/ml-agents/blob/release_4_docs/docs/Learning-Environment-Design-Agents.md#rewards
+        /// [Reward Signals]: https://github.com/Unity-Technologies/ml-agents/blob/release_4_docs/docs/ML-Agents-Overview.md#a-quick-note-on-reward-signals
        ///</remarks>
        /// <param name="increment">Incremental reward value.</param>
        public void AddReward(float increment)
        /// implementing a simple heuristic function can aid in debugging agent actions and interactions
        /// with its environment.
        ///
-        /// [Demonstration Recorder]: https://github.com/Unity-Technologies/ml-agents/blob/release_3_docs/docs/Learning-Environment-Design-Agents.md#recording-demonstrations
-        /// [Actions]: https://github.com/Unity-Technologies/ml-agents/blob/release_3_docs/docs/Learning-Environment-Design-Agents.md#actions
+        /// [Demonstration Recorder]: https://github.com/Unity-Technologies/ml-agents/blob/release_4_docs/docs/Learning-Environment-Design-Agents.md#recording-demonstrations
+        /// [Actions]: https://github.com/Unity-Technologies/ml-agents/blob/release_4_docs/docs/Learning-Environment-Design-Agents.md#actions
        /// [GameObject]: https://docs.unity3d.com/Manual/GameObjects.html
        /// </remarks>
        /// <example>
        /// For more information about observations, see [Observations and Sensors].
        ///
        /// [GameObject]: https://docs.unity3d.com/Manual/GameObjects.html
-        /// [Observations and Sensors]: https://github.com/Unity-Technologies/ml-agents/blob/release_3_docs/docs/Learning-Environment-Design-Agents.md#observations-and-sensors
+        /// [Observations and Sensors]: https://github.com/Unity-Technologies/ml-agents/blob/release_4_docs/docs/Learning-Environment-Design-Agents.md#observations-and-sensors
        /// </remarks>
        public virtual void CollectObservations(VectorSensor sensor)
        {
        ///
        /// See [Agents - Actions] for more information on masking actions.
        ///
-        /// [Agents - Actions]: https://github.com/Unity-Technologies/ml-agents/blob/release_3_docs/docs/Learning-Environment-Design-Agents.md#actions
+        /// [Agents - Actions]: https://github.com/Unity-Technologies/ml-agents/blob/release_4_docs/docs/Learning-Environment-Design-Agents.md#actions
        /// </remarks>
        /// <seealso cref="OnActionReceived(float[])"/>
        public virtual void CollectDiscreteActionMasks(DiscreteActionMasker actionMasker)
        ///
        /// For more information about implementing agent actions see [Agents - Actions].
        ///
-        /// [Agents - Actions]: https://github.com/Unity-Technologies/ml-agents/blob/release_3_docs/docs/Learning-Environment-Design-Agents.md#actions
+        /// [Agents - Actions]: https://github.com/Unity-Technologies/ml-agents/blob/release_4_docs/docs/Learning-Environment-Design-Agents.md#actions
        /// </remarks>
        /// <param name="vectorAction">
        /// An array containing the action vector. The length of the array is specified
--- a/com.unity.ml-agents/Runtime/Demonstrations/DemonstrationRecorder.cs
+++ b/com.unity.ml-agents/Runtime/Demonstrations/DemonstrationRecorder.cs
    /// See [Imitation Learning - Recording Demonstrations] for more information.
    ///
    /// [GameObject]: https://docs.unity3d.com/Manual/GameObjects.html
-    /// [Imitation Learning - Recording Demonstrations]: https://github.com/Unity-Technologies/ml-agents/blob/release_3_docs/docs//Learning-Environment-Design-Agents.md#recording-demonstrations
+    /// [Imitation Learning - Recording Demonstrations]: https://github.com/Unity-Technologies/ml-agents/blob/release_4_docs/docs//Learning-Environment-Design-Agents.md#recording-demonstrations
    /// </remarks>
    [RequireComponent(typeof(Agent))]
    [AddComponentMenu("ML Agents/Demonstration Recorder", (int)MenuGroup.Default)]
--- a/com.unity.ml-agents/Runtime/DiscreteActionMasker.cs
+++ b/com.unity.ml-agents/Runtime/DiscreteActionMasker.cs
        ///
        /// See [Agents - Actions] for more information on masking actions.
        ///
-        /// [Agents - Actions]: https://github.com/Unity-Technologies/ml-agents/blob/release_3_docs/docs/Learning-Environment-Design-Agents.md#actions
+        /// [Agents - Actions]: https://github.com/Unity-Technologies/ml-agents/blob/release_4_docs/docs/Learning-Environment-Design-Agents.md#actions
        /// </remarks>
        /// <param name="branch">The branch for which the actions will be masked.</param>
        /// <param name="actionIndices">The indices of the masked actions.</param>
--- a/docs/FAQ.md
+++ b/docs/FAQ.md
 Scene Inspector to a value greater than 0. Alternatively, it is possible to
 manually set `done` conditions for episodes from within scripts for custom
 episode-terminating events.
+
+## "File name" cannot be opened because the developer cannot be verified.
+
+If you have downloaded the repository using the github website on macOS 10.15 (Catalina)
+or later, you may see this error when attempting to play scenes in the Unity project.
+Workarounds include installing the package using the Unity Package Manager (this is
+the officially supported approach - see [here](Installation.md)), or following the
+instructions [here](https://support.apple.com/en-us/HT202491) to verify the relevant
+files on your machine on a file-by-file basis.
--- a/docs/Installation-Anaconda-Windows.md
+++ b/docs/Installation-Anaconda-Windows.md
 the ml-agents Conda environment by typing `activate ml-agents`)_:

 ```sh
-git clone --branch release_3 https://github.com/Unity-Technologies/ml-agents.git
+git clone --branch release_4 https://github.com/Unity-Technologies/ml-agents.git
-The `--branch release_3` option will switch to the tag of the latest stable
+The `--branch release_4` option will switch to the tag of the latest stable
 release. Omitting that will get the `master` branch which is potentially
 unstable.

--- a/docs/Installation.md
+++ b/docs/Installation.md
 of our tutorials / guides assume you have access to our example environments).

 ```sh
-git clone --branch release_3 https://github.com/Unity-Technologies/ml-agents.git
+git clone --branch release_4 https://github.com/Unity-Technologies/ml-agents.git
-The `--branch release_3` option will switch to the tag of the latest stable
+The `--branch release_4` option will switch to the tag of the latest stable
 release. Omitting that will get the `master` branch which is potentially
 unstable.

 ML-Agents Toolkit for your purposes. If you plan to contribute those changes
-back, make sure to clone the `master` branch (by omitting `--branch release_3`
+back, make sure to clone the `master` branch (by omitting `--branch release_4`
 from the command above). See our
 [Contributions Guidelines](../com.unity.ml-agents/CONTRIBUTING.md) for more
 information on contributing to the ML-Agents Toolkit.
--- a/docs/Training-on-Amazon-Web-Service.md
+++ b/docs/Training-on-Amazon-Web-Service.md
 2. Clone the ML-Agents repo and install the required Python packages

   ```sh
-   git clone --branch release_3 https://github.com/Unity-Technologies/ml-agents.git
+   git clone --branch release_4 https://github.com/Unity-Technologies/ml-agents.git
   cd ml-agents/ml-agents/
   pip3 install -e .
   ```
--- a/utils/make_readme_table.py
+++ b/utils/make_readme_table.py
    ReleaseInfo("release_1", "1.0.0", "0.16.0", "April 30, 2020"),
    ReleaseInfo("release_2", "1.0.2", "0.16.1", "May 20, 2020"),
    ReleaseInfo("release_3", "1.1.0", "0.17.0", "June 10, 2020"),
+    ReleaseInfo("release_4", "1.2.0", "0.18.0", "July 15, 2020"),
 ]

 MAX_DAYS = 150  # do not print releases older than this many days
--- a/utils/validate_release_links.py
+++ b/utils/validate_release_links.py
    "docs/Versioning.md": None,
    "com.unity.ml-agents/CHANGELOG.md": None,
    "utils/make_readme_table.py": None,
-    "utils/validate_doc_versions.py": None,
+    "utils/validate_release_links.py": None,
 }


-        "https://github.com/Unity-Technologies/ml-agents/blob/release_3_docs/Food.md"
+        "https://github.com/Unity-Technologies/ml-agents/blob/release_4_docs/Food.md"
-        "https://github.com/Unity-Technologies/ml-agents/blob/release_3/Foo.md"
+        "https://github.com/Unity-Technologies/ml-agents/blob/release_4/Foo.md"
-        "git clone --branch release_3 https://github.com/Unity-Technologies/ml-agents.git"
+        "git clone --branch release_4 https://github.com/Unity-Technologies/ml-agents.git"
    )
    assert RELEASE_PATTERN.search(
        "https://github.com/Unity-Technologies/ml-agents/blob/release_123_docs/Foo.md"
--- a/utils/validate_versions.py
+++ b/utils/validate_versions.py
    "gym-unity/gym_unity",
 ]

-UNITY_PACKAGE_JSON_PATH = "com.unity.ml-agents/package.json"
+MLAGENTS_PACKAGE_JSON_PATH = "com.unity.ml-agents/package.json"
+MLAGENTS_EXTENSIONS_PACKAGE_JSON_PATH = "com.unity.ml-agents.extensions/package.json"
+
 ACADEMY_PATH = "com.unity.ml-agents/Runtime/Academy.cs"

 PYTHON_VERSION_FILE_TEMPLATE = """# Version of the library that will be used to upload to pypi
    if csharp_version is not None:
        package_version = csharp_version + "-preview"
        print(
-            f"Setting package version to {package_version} in {UNITY_PACKAGE_JSON_PATH}"
+            f"Setting package version to {package_version} in {MLAGENTS_PACKAGE_JSON_PATH}"
+            f" and {MLAGENTS_EXTENSIONS_PACKAGE_JSON_PATH}"
+        set_extension_package_version(package_version)
-    with open(UNITY_PACKAGE_JSON_PATH, "r") as f:
+    with open(MLAGENTS_PACKAGE_JSON_PATH, "r") as f:
-    with open(UNITY_PACKAGE_JSON_PATH, "w") as f:
+    with open(MLAGENTS_PACKAGE_JSON_PATH, "w") as f:
+        json.dump(package_json, f, indent=2)
+        f.write("\n")
+
+
+def set_extension_package_version(new_version: str) -> None:
+    with open(MLAGENTS_EXTENSIONS_PACKAGE_JSON_PATH, "r") as f:
+        package_json = json.load(f)
+    package_json["dependencies"]["com.unity.ml-agents"] = new_version
+    with open(MLAGENTS_EXTENSIONS_PACKAGE_JSON_PATH, "w") as f:
        json.dump(package_json, f, indent=2)
        f.write("\n")