Added local precommit hook to autogenerate markdown documentation using pydoc-markdown

.pre-commit-config.yaml

         name: validate release links
         language: script
         entry: utils/validate_release_links.py
+    -   id: generate-markdown-docs
+        name: generate markdown docs
+        language: script
+        entry: utils/generate_markdown_docs.py --package_dirs ml-agents-envs
+        pass_filenames: false

docs/Python-API-Documentation.md

-# Python Low level API Documentation
-
-## Table of Contents
+# Table of Contents
 
 * [mlagents\_envs.env\_utils](#mlagents_envs.env_utils)
   * [get\_platform](#mlagents_envs.env_utils.get_platform)
 not necessarily correspond to a fixed simulation time increment.
 
 <a name="mlagents_envs.base_env.DecisionStep"></a>
-## DecisionStep
+## DecisionStep Objects
 
 ```python
 class DecisionStep(NamedTuple)
  the agent during this simulation step.
 
 <a name="mlagents_envs.base_env.DecisionSteps"></a>
-## DecisionSteps
+## DecisionSteps Objects
 
 ```python
 class DecisionSteps(Mapping)
 - `spec`: The BehaviorSpec for the DecisionSteps
 
 <a name="mlagents_envs.base_env.TerminalStep"></a>
-## TerminalStep
+## TerminalStep Objects
 
 ```python
 class TerminalStep(NamedTuple)
  - agent_id is an int and an unique identifier for the corresponding Agent.
 
 <a name="mlagents_envs.base_env.TerminalSteps"></a>
-## TerminalSteps
+## TerminalSteps Objects
 
 ```python
 class TerminalSteps(Mapping)
 - `spec`: The BehaviorSpec for the TerminalSteps
 
 <a name="mlagents_envs.base_env.ActionTuple"></a>
-## ActionTuple
+## ActionTuple Objects
 
 ```python
 class ActionTuple(_ActionTupleBase)
 The dtype of a discrete action.
 
 <a name="mlagents_envs.base_env.ActionSpec"></a>
-## ActionSpec
+## ActionSpec Objects
 
 ```python
 class ActionSpec(NamedTuple)
 Creates an ActionSpec that is homogenously discrete
 
 <a name="mlagents_envs.base_env.DimensionProperty"></a>
-## DimensionProperty
+## DimensionProperty Objects
 
 ```python
 class DimensionProperty(IntFlag)
 The observations are unordered.
 
 <a name="mlagents_envs.base_env.ObservationType"></a>
-## ObservationType
+## ObservationType Objects
 
 ```python
 class ObservationType(Enum)
 of the agent.
 
 <a name="mlagents_envs.base_env.ObservationSpec"></a>
-## ObservationSpec
+## ObservationSpec Objects
 
 ```python
 class ObservationSpec(NamedTuple)
 - observation_type is an enum of ObservationType.
 
 <a name="mlagents_envs.base_env.BehaviorSpec"></a>
-## BehaviorSpec
+## BehaviorSpec Objects
 
 ```python
 class BehaviorSpec(NamedTuple)
 - action_spec is an ActionSpec NamedTuple.
 
 <a name="mlagents_envs.base_env.BaseEnv"></a>
-## BaseEnv
+## BaseEnv Objects
 
 ```python
 class BaseEnv(ABC)
 # mlagents\_envs.communicator
 
 <a name="mlagents_envs.communicator.Communicator"></a>
-## Communicator
+## Communicator Objects
 
 ```python
 class Communicator()
 # mlagents\_envs.environment
 
 <a name="mlagents_envs.environment.UnityEnvironment"></a>
-## UnityEnvironment
+## UnityEnvironment Objects
 
 ```python
 class UnityEnvironment(BaseEnv)
 # mlagents\_envs.registry.unity\_env\_registry
 
 <a name="mlagents_envs.registry.unity_env_registry.UnityEnvRegistry"></a>
-## UnityEnvRegistry
+## UnityEnvRegistry Objects
 
 ```python
 class UnityEnvRegistry(Mapping)
 Converts a local yaml file into a Python dictionary
 
 <a name="mlagents_envs.registry.binary_utils.ZipFileWithProgress"></a>
-## ZipFileWithProgress
+## ZipFileWithProgress Objects
 
 ```python
 class ZipFileWithProgress(ZipFile)
 # mlagents\_envs.registry.remote\_registry\_entry
 
 <a name="mlagents_envs.registry.remote_registry_entry.RemoteRegistryEntry"></a>
-## RemoteRegistryEntry
+## RemoteRegistryEntry Objects
 
 ```python
 class RemoteRegistryEntry(BaseRegistryEntry)
 # mlagents\_envs.registry.base\_registry\_entry
 
 <a name="mlagents_envs.registry.base_registry_entry.BaseRegistryEntry"></a>
-## BaseRegistryEntry
+## BaseRegistryEntry Objects
 
 ```python
 class BaseRegistryEntry()
 # mlagents\_envs.side\_channel.raw\_bytes\_channel
 
 <a name="mlagents_envs.side_channel.raw_bytes_channel.RawBytesChannel"></a>
-## RawBytesChannel
+## RawBytesChannel Objects
 
 ```python
 class RawBytesChannel(SideChannel)
 # mlagents\_envs.side\_channel.outgoing\_message
 
 <a name="mlagents_envs.side_channel.outgoing_message.OutgoingMessage"></a>
-## OutgoingMessage
+## OutgoingMessage Objects
 
 ```python
 class OutgoingMessage()
 # mlagents\_envs.side\_channel.engine\_configuration\_channel
 
 <a name="mlagents_envs.side_channel.engine_configuration_channel.EngineConfigurationChannel"></a>
-## EngineConfigurationChannel
+## EngineConfigurationChannel Objects
 
 ```python
 class EngineConfigurationChannel(SideChannel)
 # mlagents\_envs.side\_channel.side\_channel\_manager
 
 <a name="mlagents_envs.side_channel.side_channel_manager.SideChannelManager"></a>
-## SideChannelManager
+## SideChannelManager Objects
 
 ```python
 class SideChannelManager()
 # mlagents\_envs.side\_channel.stats\_side\_channel
 
 <a name="mlagents_envs.side_channel.stats_side_channel.StatsSideChannel"></a>
-## StatsSideChannel
+## StatsSideChannel Objects
 
 ```python
 class StatsSideChannel(SideChannel)
 # mlagents\_envs.side\_channel.incoming\_message
 
 <a name="mlagents_envs.side_channel.incoming_message.IncomingMessage"></a>
-## IncomingMessage
+## IncomingMessage Objects
 
 ```python
 class IncomingMessage()
 # mlagents\_envs.side\_channel.float\_properties\_channel
 
 <a name="mlagents_envs.side_channel.float_properties_channel.FloatPropertiesChannel"></a>
-## FloatPropertiesChannel
+## FloatPropertiesChannel Objects
 
 ```python
 class FloatPropertiesChannel(SideChannel)
 # mlagents\_envs.side\_channel.environment\_parameters\_channel
 
 <a name="mlagents_envs.side_channel.environment_parameters_channel.EnvironmentParametersChannel"></a>
-## EnvironmentParametersChannel
+## EnvironmentParametersChannel Objects
 
 ```python
 class EnvironmentParametersChannel(SideChannel)
 # mlagents\_envs.side\_channel.side\_channel
 
 <a name="mlagents_envs.side_channel.side_channel.SideChannel"></a>
-## SideChannel
+## SideChannel Objects
 
 ```python
 class SideChannel(ABC)
 
 The type of side channel used. Will influence how the data is
 processed in the environment.
-

ml-agents-envs/pydoc-config.yaml

+# config to specify which modules will be used to render api docs
+folder: docs
+modules:
+    - name: mlagents_envs
+      file_name: Python-API-Documentation.md
+      submodules:
+        - env_utils
+        - base_env
+        - communicator
+        - environment
+        - registry
+        - registry.unity_env_registry
+        - registry.binary_utils
+        - registry.remote_registry_entry
+        - registry.base_registry_entry
+        - side_channel
+        - side_channel.raw_bytes_channel
+        - side_channel.outgoing_message
+        - side_channel.engine_configuration_channel
+        - side_channel.side_channel_manager
+        - side_channel.stats_side_channel
+        - side_channel.incoming_message
+        - side_channel.float_properties_channel
+        - side_channel.environment_parameters_channel
+        - side_channel.side_channel

utils/generate_markdown_docs.py

+#!/usr/bin/env python3
+import os
+import subprocess
+import sys
+import yaml
+import argparse
+import hashlib
+
+
+# pydoc-markdown -I . -m module_name --render_toc > doc.md
+
+
+def hash_file(filename):
+    if os.path.exists(filename):
+        hasher = hashlib.md5()
+        with open(filename, "rb") as file_to_hash:
+            buffer = file_to_hash.read()
+            hasher.update(buffer)
+        return hasher.hexdigest()
+    else:
+        return 0
+
+
+def remove_trailing_whitespace(filename):
+    nchanged = 0
+    with open(filename, "rb") as f:
+        code1 = f.read().decode()
+    lines = [line.rstrip() for line in code1.splitlines()]
+    while lines and not lines[-1]:
+        lines.pop(-1)
+    lines.append("")  # always end with a newline
+    code2 = "\n".join(lines)
+    if code1 != code2:
+        nchanged += 1
+        with open(filename, "wb") as f:
+            f.write(code2.encode())
+
+
+if __name__ == "__main__":
+    parser = argparse.ArgumentParser()
+    parser.add_argument("--package_dirs", nargs="+")
+    args = parser.parse_args()
+
+    ok = False
+    return_code = 0
+    for package_dir in args.package_dirs:
+        config_path = os.path.join(os.getcwd(), package_dir, "pydoc-config.yaml")
+        print(config_path)
+        with open(config_path) as config_file:
+            config = yaml.full_load(config_file)
+            for module in config["modules"]:
+                module_name = module["name"]
+                submodules = module["submodules"]
+                output_file_name = f"./{config['folder']}/{module['file_name']}"
+                old_hash = hash_file(output_file_name)
+                module_args = []
+                for submodule in submodules:
+                    module_args.append("-m")
+                    module_args.append(f"{module_name}.{submodule}")
+                with open(output_file_name, "w") as output_file:
+                    subprocess_args = [
+                        "pydoc-markdown",
+                        "-I",
+                        f"./{package_dir}",
+                        *module_args,
+                        "--render-toc",
+                    ]
+                    subprocess.check_call(subprocess_args, stdout=output_file)
+                remove_trailing_whitespace(output_file_name)
+                new_hash = hash_file(output_file_name)
+                ok = old_hash == new_hash
+
+        return_code = 0 if ok else 1
+
+    sys.exit(return_code)