unity-tech-cn
/
ml-agents
46
14
派生 39
代码 合并请求 版本发布
浏览代码

[WIP] Unity Environment Registry (#3967) 

* [WIP] Unity Environment Registry

[JIRA ticket](https://jira.unity3d.com/browse/MLA-997)
[Design Document](https://docs.google.com/document/d/1bFQ3_oXsA80FMou8kwqYxC53kqG5L3i0mbTQUH4shY4/edit#)

In This PR : Prototype of the Unity Environment Registry
Uploaded the 3DBall and Basic Environments for mac only

How to use on Python :
```python
from mlagents_envs.registry import UnityEnvRegistry

registry = UnityEnvRegistry()

print(registry["3DBall"].description)
env = registry["3DBall"].make()
env.reset()

for i in range(10):
   print(i)
   env.step()

env.close()
```

* Other approach:
 - UnityEnvRegistry is no longer static and needs to be instantiated
 - Providing a default_registry that will contains our environments
 - Added a functionality to register RemoteRegistryEntry with a yaml file

* Some extra verification of the url : The binary will have a hash of the url in its name to make sure the right environ...
/docs-update
GitHub 5 年前
当前提交
e6a84ded
共有 10 个文件被更改，包括 586 次插入 和 0 次删除
  1. 1
      com.unity.ml-agents/CHANGELOG.md
  2. 1
      ml-agents-envs/setup.py
  3. 60
      docs/Unity-Environment-Registry.md
  4. 44
      ml-agents-envs/mlagents_envs/tests/test_registry.py
  5. 4
      ml-agents-envs/mlagents_envs/registry/__init__.py
  6. 56
      ml-agents-envs/mlagents_envs/registry/base_registry_entry.py
  7. 224
      ml-agents-envs/mlagents_envs/registry/binary_utils.py
  8. 71
      ml-agents-envs/mlagents_envs/registry/remote_registry_entry.py
  9. 125
      ml-agents-envs/mlagents_envs/registry/unity_env_registry.py

1
com.unity.ml-agents/CHANGELOG.md
查看文件


- `max_step` in the `TerminalStep` and `TerminalSteps` objects was renamed `interrupted`.
- `beta` and `epsilon` in `PPO` are no longer decayed by default but follow the same schedule as learning rate. (#3940)
- `get_behavior_names()` and `get_behavior_spec()` on UnityEnvironment were replaced by the `behavior_specs` property. (#3946)
- The first version of the Unity Environment Registry (Experimental) has been released. More information [here](https://github.com/Unity-Technologies/ml-agents/blob/master/docs/Unity-Environment-Registry.md)(#3967)
### Minor Changes
#### com.unity.ml-agents (C#)
- `ObservableAttribute` was added. Adding the attribute to fields or properties on an Agent will allow it to generate

1
ml-agents-envs/setup.py
查看文件


"numpy>=1.14.1,<2.0",
"Pillow>=4.2.1",
"protobuf>=3.6",
"pyyaml>=5.1.1",
],
python_requires=">=3.5",
cmdclass={"verify": VerifyVersionCommand},

60
docs/Unity-Environment-Registry.md
查看文件


# Unity Environment Registry [Experimental]
The Unity Environment Registry is a database of pre-built Unity environments that can be easily used without having to install the Unity Editor. It is a great way to get started with our [UnityEnvironment API](Python-API.md).
## Loading an Environment from the Registry
To get started, you can access the default registry we provide with our [Example Environments](Learning-Environment-Examples.md). The Unity Environment Registry implements a _Mapping_, therefore, you can access an entry with its identifier with the square brackets `[ ]`. Use the following code to list all of the environment identifiers present in the default registry:
```python
from mlagents_envs.registry import default_registry
environment_names = list(default_registry.keys())
for name in environment_names:
print(name)
```
The `make()` method on a registry value will return a `UnityEnvironment` ready to be used. All arguments passed to the make method will be passed to the constructor of the `UnityEnvironment` as well. Refer to the documentation on the [Python-API](Python-API.md) for more information about the arguments of the `UnityEnvironment` constructor. For example, the following code will create the environment under the identifier `"my-env"`, reset it, perform a few steps and finally close it:
```python
from mlagents_envs.registry import default_registry
env = default_registry["my-env"].make()
env.reset()
for _ in range(10):
env.step()
env.close()
```
## Create and share your own registry
In order to share the `UnityEnvironemnt` you created, you must :
- [Create a Unity executable](Learning-Environment-Executable.md) of your environment for each platform (Linux, OSX and/or Windows)
- Place each executable in a `zip` compressed folder
- Upload each zip file online to your preferred hosting platform
- Create a `yaml` file that will contain the description and path to your environment
- Upload the `yaml` file online
The `yaml` file must have the following format :
```yaml
environments:
- <environment-identifier>:
expected_reward: <expected-reward-float>
description: <description-of-the-environment>
linux_url: <url-to-the-linux-zip-folder>
darwin_url: <url-to-the-osx-zip-folder>
win_url: <url-to-the-windows-zip-folder>
additional_args:
- <an-optional-list-of-command-line-arguments-for-the-executable>
- ...
```
Your users can now use your environment with the following code :
```python
from mlagents_envs.registry import UnityEnvRegistry
registry = UnityEnvRegistry()
registry.register_from_yaml("url-or-path-to-your-yaml-file")
```
__Note__: The `"url-or-path-to-your-yaml-file"` can be either a url or a local path.

44
ml-agents-envs/mlagents_envs/tests/test_registry.py
查看文件


import shutil
import os
from mlagents_envs.registry import default_registry, UnityEnvRegistry
from mlagents_envs.registry.remote_registry_entry import RemoteRegistryEntry
from mlagents_envs.registry.binary_utils import get_tmp_dir
BASIC_ID = "Basic"
def delete_binaries():
tmp_dir, bin_dir = get_tmp_dir()
shutil.rmtree(tmp_dir)
shutil.rmtree(bin_dir)
def create_registry():
reg = UnityEnvRegistry()
entry = RemoteRegistryEntry(
BASIC_ID,
0.0,
"Description",
"https://storage.googleapis.com/mlagents-test-environments/1.0.0/linux/Basic.zip",
"https://storage.googleapis.com/mlagents-test-environments/1.0.0/darwin/Basic.zip",
"https://storage.googleapis.com/mlagents-test-environments/1.0.0/windows/Basic.zip",
)
reg.register(entry)
return reg
def test_basic_in_registry():
assert BASIC_ID in default_registry
os.environ["TERM"] = "xterm"
delete_binaries()
registry = create_registry()
for worker_id in range(2):
assert BASIC_ID in registry
env = registry[BASIC_ID].make(
base_port=6005, worker_id=worker_id, no_graphics=True
)
env.reset()
env.step()
assert len(env.behavior_specs) == 1
env.close()

4
ml-agents-envs/mlagents_envs/registry/__init__.py
查看文件


from mlagents_envs.registry.unity_env_registry import ( # noqa F401
default_registry,
UnityEnvRegistry,
)

56
ml-agents-envs/mlagents_envs/registry/base_registry_entry.py
查看文件


from abc import abstractmethod
from typing import Any, Optional
from mlagents_envs.base_env import BaseEnv
class BaseRegistryEntry:
def __init__(
self,
identifier: str,
expected_reward: Optional[float],
description: Optional[str],
):
"""
BaseRegistryEntry allows launching a Unity Environment with its make method.
:param identifier: The name of the Unity Environment.
:param expected_reward: The cumulative reward that an Agent must receive
for the task to be considered solved.
:param description: A description of the Unity Environment. Contains human
readable information about potential special arguments that the make method can
take as well as information regarding the observation, reward, actions,
behaviors and number of agents in the Environment.
"""
self._identifier = identifier
self._expected_reward = expected_reward
self._description = description
@property
def identifier(self) -> str:
"""
The unique identifier of the entry
"""
return self._identifier
@property
def expected_reward(self) -> Optional[float]:
"""
The cumulative reward that an Agent must receive for the task to be considered
solved.
"""
return self._expected_reward
@property
def description(self) -> Optional[str]:
"""
A description of the Unity Environment the entry can make.
"""
return self._description
@abstractmethod
def make(self, **kwargs: Any) -> BaseEnv:
"""
This method creates a Unity BaseEnv (usually a UnityEnvironment).
"""
raise NotImplementedError(
f"The make() method not implemented for entry {self.identifier}"
)

224
ml-agents-envs/mlagents_envs/registry/binary_utils.py
查看文件


import urllib.request
import tempfile
import os
import uuid
import shutil
import glob
import yaml
import hashlib
from zipfile import ZipFile
from sys import platform
from typing import Tuple, Optional, Dict, Any
from mlagents_envs.logging_util import get_logger
logger = get_logger(__name__)
# The default logical block size is 8192 bytes (8 KB) for UFS file systems.
BLOCK_SIZE = 8192
def get_local_binary_path(name: str, url: str) -> str:
"""
Returns the path to the executable previously downloaded with the name argument. If
None is found, the executable at the url argument will be downloaded and stored
under name for future uses.
:param name: The name that will be given to the folder containing the extracted data
:param url: The URL of the zip file
"""
NUMBER_ATTEMPTS = 5
path = get_local_binary_path_if_exists(name, url)
if path is None:
logger.debug(
f"Local environment {name} not found, downloading environment from {url}"
)
for attempt in range(NUMBER_ATTEMPTS): # Perform 5 attempts at downloading the file
if path is not None:
break
try:
download_and_extract_zip(url, name)
except IOError:
logger.debug(
f"Attempt {attempt + 1} / {NUMBER_ATTEMPTS} : Failed to download"
)
path = get_local_binary_path_if_exists(name, url)
if path is None:
raise FileNotFoundError(
f"Binary not found, make sure {url} is a valid url to "
"a zip folder containing a valid Unity executable"
)
return path
def get_local_binary_path_if_exists(name: str, url: str) -> Optional[str]:
"""
Recursively searches for a Unity executable in the extracted files folders. This is
platform dependent : It will only return a Unity executable compatible with the
computer's OS. If no executable is found, None will be returned.
:param name: The name/identifier of the executable
:param url: The url the executable was downloaded from (for verification)
"""
_, bin_dir = get_tmp_dir()
extension = None
if platform == "linux" or platform == "linux2":
extension = "*.x86_64"
if platform == "darwin":
extension = "*.app"
if platform == "win32":
extension = "*.exe"
if extension is None:
raise NotImplementedError("No extensions found for this platform.")
url_hash = "-" + hashlib.md5(url.encode()).hexdigest()
path = os.path.join(bin_dir, name + url_hash, "**", extension)
candidates = glob.glob(path, recursive=True)
if len(candidates) == 0:
return None
else:
for c in candidates:
# Unity sometimes produces another .exe file that we must filter out
if "UnityCrashHandler64" not in c:
return c
return None
def get_tmp_dir() -> Tuple[str, str]:
"""
Returns the path to the folder containing the downloaded zip files and the extracted
binaries. If these folders do not exist, they will be created.
:retrun: Tuple containing path to : (zip folder, extracted files folder)
"""
MLAGENTS = "ml-agents-binaries"
TMP_FOLDER_NAME = "tmp"
BINARY_FOLDER_NAME = "binaries"
mla_directory = os.path.join(tempfile.gettempdir(), MLAGENTS)
if not os.path.exists(mla_directory):
os.makedirs(mla_directory)
os.chmod(mla_directory, 16877)
zip_directory = os.path.join(tempfile.gettempdir(), MLAGENTS, TMP_FOLDER_NAME)
if not os.path.exists(zip_directory):
os.makedirs(zip_directory)
os.chmod(zip_directory, 16877)
bin_directory = os.path.join(tempfile.gettempdir(), MLAGENTS, BINARY_FOLDER_NAME)
if not os.path.exists(bin_directory):
os.makedirs(bin_directory)
os.chmod(bin_directory, 16877)
return (zip_directory, bin_directory)
def download_and_extract_zip(url: str, name: str) -> None:
"""
Downloads a zip file under a URL, extracts its contents into a folder with the name
argument and gives chmod 755 to all the files it contains. Files are downloaded and
extracted into special folders in the temp folder of the machine.
:param url: The URL of the zip file
:param name: The name that will be given to the folder containing the extracted data
"""
zip_dir, bin_dir = get_tmp_dir()
url_hash = "-" + hashlib.md5(url.encode()).hexdigest()
binary_path = os.path.join(bin_dir, name + url_hash)
if os.path.exists(binary_path):
shutil.rmtree(binary_path)
# Download zip
try:
request = urllib.request.urlopen(url, timeout=30)
except urllib.error.HTTPError as e: # type: ignore
e.msg += " " + url
raise
zip_size = int(request.headers["content-length"])
zip_file_path = os.path.join(zip_dir, str(uuid.uuid4()) + ".zip")
with open(zip_file_path, "wb") as zip_file:
downloaded = 0
while True:
buffer = request.read(BLOCK_SIZE)
if not buffer:
# There is nothing more to read
break
downloaded += len(buffer)
zip_file.write(buffer)
downloaded_percent = downloaded / zip_size * 100
print_progress(f" Downloading {name}", downloaded_percent)
print("")
# Extraction
with ZipFileWithProgress(zip_file_path, "r") as zip_ref:
zip_ref.extract_zip(f" Extracting {name}", binary_path) # type: ignore
print("")
# Clean up zip
print_progress(f" Cleaning up {name}", 0)
os.remove(zip_file_path)
# Give permission
for f in glob.glob(binary_path + "/**/*", recursive=True):
# 16877 is octal 40755, which denotes a directory with permissions 755
os.chmod(f, 16877)
print_progress(f" Cleaning up {name}", 100)
print("")
def print_progress(prefix: str, percent: float) -> None:
"""
Displays a single progress bar in the terminal with value percent.
:param prefix: The string that will precede the progress bar.
:param percent: The percent progression of the bar (min is 0, max is 100)
"""
BAR_LEN = 20
percent = min(100, max(0, percent))
bar_progress = min(int(percent / 100 * BAR_LEN), BAR_LEN)
bar = "|" + "\u2588" * bar_progress + " " * (BAR_LEN - bar_progress) + "|"
str_percent = "%3.0f%%" % percent
print(f"{prefix} : {bar} {str_percent} \r", end="", flush=True)
def load_remote_manifest(url: str) -> Dict[str, Any]:
"""
Converts a remote yaml file into a Python dictionary
"""
tmp_dir, _ = get_tmp_dir()
try:
request = urllib.request.urlopen(url, timeout=30)
except urllib.error.HTTPError as e: # type: ignore
e.msg += " " + url
raise
manifest_path = os.path.join(tmp_dir, str(uuid.uuid4()) + ".yaml")
with open(manifest_path, "wb") as manifest:
while True:
buffer = request.read(BLOCK_SIZE)
if not buffer:
# There is nothing more to read
break
manifest.write(buffer)
try:
result = load_local_manifest(manifest_path)
finally:
os.remove(manifest_path)
return result
def load_local_manifest(path: str) -> Dict[str, Any]:
"""
Converts a local yaml file into a Python dictionary
"""
with open(path) as data_file:
return yaml.safe_load(data_file)
class ZipFileWithProgress(ZipFile):
"""
This is a helper class inheriting from ZipFile that allows to display a progress
bar while the files are being extracted.
"""
def extract_zip(self, prefix: str, path: str) -> None:
members = self.namelist()
path = os.fspath(path)
total = len(members)
n = 0
for zipinfo in members:
self.extract(zipinfo, path, None) # type: ignore
n += 1
print_progress(prefix, n / total * 100)

71
ml-agents-envs/mlagents_envs/registry/remote_registry_entry.py
查看文件


from sys import platform
from typing import Optional, Any, List
from mlagents_envs.environment import UnityEnvironment
from mlagents_envs.base_env import BaseEnv
from mlagents_envs.registry.binary_utils import get_local_binary_path
from mlagents_envs.registry.base_registry_entry import BaseRegistryEntry
class RemoteRegistryEntry(BaseRegistryEntry):
def __init__(
self,
identifier: str,
expected_reward: Optional[float],
description: Optional[str],
linux_url: Optional[str],
darwin_url: Optional[str],
win_url: Optional[str],
additional_args: Optional[List[str]] = None,
):
"""
A RemoteRegistryEntry is an implementation of BaseRegistryEntry that uses a
Unity executable downloaded from the internet to launch a UnityEnvironment.
__Note__: The url provided must be a link to a `.zip` file containing a single
compressed folder with the executable inside. There can only be one executable
in the folder and it must be at the root of the folder.
:param identifier: The name of the Unity Environment.
:param expected_reward: The cumulative reward that an Agent must receive
for the task to be considered solved.
:param description: A description of the Unity Environment. Contains human
readable information about potential special arguments that the make method can
take as well as information regarding the observation, reward, actions,
behaviors and number of agents in the Environment.
:param linux_url: The url of the Unity executable for the Linux platform
:param darwin_url: The url of the Unity executable for the OSX platform
:param win_url: The url of the Unity executable for the Windows platform
"""
super().__init__(identifier, expected_reward, description)
self._linux_url = linux_url
self._darwin_url = darwin_url
self._win_url = win_url
self._add_args = additional_args
def make(self, **kwargs: Any) -> BaseEnv:
"""
Returns the UnityEnvironment that corresponds to the Unity executable found at
the provided url. The arguments passed to this method will be passed to the
constructor of the UnityEnvironment (except for the file_name argument)
"""
url = None
if platform == "linux" or platform == "linux2":
url = self._linux_url
if platform == "darwin":
url = self._darwin_url
if platform == "win32":
url = self._win_url
if url is None:
raise FileNotFoundError(
f"The entry {self.identifier} does not contain a valid url for this "
"platform"
)
path = get_local_binary_path(self.identifier, url)
if "file_name" in kwargs:
kwargs.pop("file_name")
args: List[str] = []
if "additional_args" in kwargs:
if kwargs["additional_args"] is not None:
args += kwargs["additional_args"]
if self._add_args is not None:
args += self._add_args
kwargs["additional_args"] = args
return UnityEnvironment(file_name=path, **kwargs)

125
ml-agents-envs/mlagents_envs/registry/unity_env_registry.py
查看文件


from typing import Dict, Iterator, Any, List
from collections.abc import Mapping
from mlagents_envs.registry.base_registry_entry import BaseRegistryEntry
from mlagents_envs.registry.binary_utils import (
load_local_manifest,
load_remote_manifest,
)
from mlagents_envs.registry.remote_registry_entry import RemoteRegistryEntry
class UnityEnvRegistry(Mapping):
"""
### UnityEnvRegistry
Provides a library of Unity environments that can be launched without the need
of downloading the Unity Editor.
The UnityEnvRegistry implements a Map, to access an entry of the Registry, use:
```python
registry = UnityEnvRegistry()
entry = registry[<environment_identifyier>]
```
An entry has the following properties :
* `identifier` : Uniquely identifies this environment
* `expected_reward` : Corresponds to the reward an agent must obtained for the task
to be considered completed.
* `description` : A human readable description of the environment.
To launch a Unity environment from a registry entry, use the `make` method:
```python
registry = UnityEnvRegistry()
env = registry[<environment_identifyier>].make()
```
"""
def __init__(self):
self._REGISTERED_ENVS: Dict[str, BaseRegistryEntry] = {}
self._manifests: List[str] = []
self._sync = True
def register(self, new_entry: BaseRegistryEntry) -> None:
"""
Registers a new BaseRegistryEntry to the registry. The
BaseRegistryEntry.identifier value will be used as indexing key.
If two are more environments are registered under the same key, the most
recentry added will replace the others.
"""
self._REGISTERED_ENVS[new_entry.identifier] = new_entry
def register_from_yaml(self, path_to_yaml: str) -> None:
"""
Registers the environments listed in a yaml file (either local or remote). Note
that the entries are registered lazily: the registration will only happen when
an environment is accessed.
The yaml file must have the following format :
```yaml
environments:
- <identifier of the first environment>:
expected_reward: <expected reward of the environment>
description: | <a multi line description of the environment>
<continued multi line description>
linux_url: <The url for the Linux executable zip file>
darwin_url: <The url for the OSX executable zip file>
win_url: <The url for the Windows executable zip file>
- <identifier of the second environment>:
expected_reward: <expected reward of the environment>
description: | <a multi line description of the environment>
<continued multi line description>
linux_url: <The url for the Linux executable zip file>
darwin_url: <The url for the OSX executable zip file>
win_url: <The url for the Windows executable zip file>
- ...
```
:param path_to_yaml: A local path or url to the yaml file
"""
self._manifests.append(path_to_yaml)
self._sync = False
def _load_all_manifests(self) -> None:
if not self._sync:
for path_to_yaml in self._manifests:
if path_to_yaml[:4] == "http":
manifest = load_remote_manifest(path_to_yaml)
else:
manifest = load_local_manifest(path_to_yaml)
for env in manifest["environments"]:
remote_entry_args = list(env.values())[0]
remote_entry_args["identifier"] = list(env.keys())[0]
self.register(RemoteRegistryEntry(**remote_entry_args))
self._manifests = []
self._sync = True
def clear(self) -> None:
"""
Deletes all entries in the registry.
"""
self._REGISTERED_ENVS.clear()
self._manifests = []
self._sync = True
def __getitem__(self, identifier: str) -> BaseRegistryEntry:
"""
Returns the BaseRegistryEntry with the provided identifier. BaseRegistryEntry
can then be used to make a Unity Environment.
:param identifier: The identifier of the BaseRegistryEntry
:returns: The associated BaseRegistryEntry
"""
self._load_all_manifests()
if identifier not in self._REGISTERED_ENVS:
raise KeyError(f"The entry {identifier} is not present in the registry.")
return self._REGISTERED_ENVS[identifier]
def __len__(self) -> int:
self._load_all_manifests()
return len(self._REGISTERED_ENVS)
def __iter__(self) -> Iterator[Any]:
self._load_all_manifests()
yield from self._REGISTERED_ENVS
default_registry = UnityEnvRegistry()
default_registry.register_from_yaml(
"https://storage.googleapis.com/mlagents-test-environments/1.0.0/manifest.yaml"
) # noqa E501
撰写 预览
正在加载...
取消
保存