* [Semantics] Modified the semantics for the documentation
* [Semantics] Updated the images
* [Semantics] Made further changes to the docs based of the comments received
This PR makes the following changes:
* Moves clipping of continuous control model into model itself. Output is now always [-1, 1].
* Internal model values are now clipped between [-3, 3] before being rescaled to [-1, 1] for output. * This improves training performance by providing a wider range of values within which the pdf of the gaussian can fall. Output of [-1, 1] is used to be more environment-creator friendly.
* Fixes issue where epsilon was erroneously being used to reconstruct old probabilities during PPO update, leading to reduced learning performance.
* Introduce ScaleAction() function within python to easily rescale values from [-1, 1] to arbitrary range.
* Re-train all CC models using improved algorithm. All performance levels are equal or improved. In the case of Crawler, improvement is drastic.
* Update documentation appropriately.
* Made miscellaneous minor code style and optimization improvements within environments.
The calculation of observation vectors is faulty. The old calculation does not reflect distances to the edges and it does not only yield results between -1 and 1. Since distance calculation would have been difficult in one line, I just replaced it by the relative position of the ball (only using two vectors instead of four). I've conducted 500K-step reinforcing trainings before and after the change and got enormously improved results. Contact me for screenshots of the tensorboard or just use the debugger and do the math.
* Initial Commit
Ported most functionalities, still need to :
- Documentation
- Add Comments
- Custom drawer for BrainParameters
- Fix the UnitTests
- Review Functionalities
* Added Custom Drawer for the Brain Parameters
* Improvements to the HubDrawer
* Modified the Brain Editors
* Minor bug fixes and UI changes
* Modified the Help Boxes of the Drawers
* Modified Brain class, renamed Initialize and made DecideAction virtual
* Fix the UnityTests
* Simpler Brain creation menu
* Renamed Internal Brain to Learning Brain
* modified the parameters to remove reference to External or Internal in the Protobuf objects
* Updated the protobuf generated files
* Fix the Pytests
* Removed the graph scope from the Learning Brain
* cleaner logic than try catch
* Removed the isExternal field of the brain and put the isTraining logic into LearningBrain and Training Hub
* Modified how the Brain finds the A...
The check for wether an agent has fallen off the platform was using a wrong value of 1 instead of 0.
This meant that the agent immediately started in a falling state and entered a thrashing cycle of resetting itself.
* Documentation Update
* addressed comments
* new images for the recorder
* Improvements to the docs
* Address the comments
* Core_ML typo
* Updated the links to inference repo
* Put back Inference-Engine.md
* fix typos : brain
* Readd deleted file
* fix typos
* Addressed comments
* Simplified rewards and observations; Determined better settings for training within a reasonable amount of time.
* Simplified Agent rewards; Added training section that discusses hyperparameters.
* Added note about DecisionFrequency.
* Updated screenshots and a small clarification in the text.
* Tested and updated using v0.6.
* Update a couple of images, minor text edit.
* Replace with more recent training stats.
* resolve a couple of minor review commnts.
* Increased the recommended batch and buffer size hyperparameter values.
* Fix 2 typos.
* Wording and filepath changes to tutorials
* Retake editor images to match v0.6
Retake editor images so that the filepaths and Brain names match what they actually are.
* Add blurb about using the --load flag in the intro guide, and typo fix.
* Add section in tutorial to create multiple area learning environment.
* Add mention of Done() method in agent design
* Update Learning-Environment-Create-New.md
Section : Final Editor Setup - Step 3. It says:
Drag the Brain RollerBallPlayer from the Project window to the RollerAgent Brain field.
Should say:
Drag the Brain RollerBallBrain from the Project window to the RollerAgent Brain field.
* Develop black format fix (#1998)
* fixed the format
* changed the circleci config
* [Gym] Added no_graphics argument (#1997)
> Added the no_graphics argument to the gym interface. #1413
* [Documentation] SetReward method (#1996)
Added a paragraph in the docs/Learning-Environment-Design-Agents.md document regarding the use of SetReward and how it is different from AddReward
* [Documentation] Added information for the environments the trainer cannot train with the default configurations (#1995)
* Format gym_unity using black
* Update Learning-Environment-Create-New.md
- Clarify that training is done in the original ml-agents project folder
- Remove mistype
- In the future it could help to show the user that they can copy the config folder and run training in a new project folder so they don't have to mix project settings in the original config folder
* Update Learning-Environment-Create-New.md
Add file paths
* Feature Deprecation : Online Behavioral Cloning
In this PR :
- Delete the online_bc_trainer
- Delete the tests for online bc
- delete the configuration file for online bc training
* Deleting the BCTeacherHelper.cs Script
TODO :
- Remove usages in the scene
- Documentation Edits
*DO NOT MERGE*
* IMPORTANT : REMOVED ALL IL SCENES
- Removed all the IL scenes from the Examples folder
* Removed all mentions of online BC training in the Documentation
* Made a note in the Migrating.md doc about the removal of the Online BC feature.
* Modified the Academy UI to remove the control checkbox and replaced it with a train in the editor checkbox
* Removed the Broadcast functionality from the non-Learning brains
* Bug fix
* Note that the scenes are broken since the BroadcastHub has changed
* Modified the LL-API for Python to remove the broadcasting functiuonality.
* All unit tests are running
* Modifie...
* Feature Deprecation : Online Behavioral Cloning
In this PR :
- Delete the online_bc_trainer
- Delete the tests for online bc
- delete the configuration file for online bc training
* Deleting the BCTeacherHelper.cs Script
TODO :
- Remove usages in the scene
- Documentation Edits
*DO NOT MERGE*
* IMPORTANT : REMOVED ALL IL SCENES
- Removed all the IL scenes from the Examples folder
* Removed all mentions of online BC training in the Documentation
* Made a note in the Migrating.md doc about the removal of the Online BC feature.
* Modified the Academy UI to remove the control checkbox and replaced it with a train in the editor checkbox
* Removed the Broadcast functionality from the non-Learning brains
* Bug fix
* Note that the scenes are broken since the BroadcastHub has changed
* Modified the LL-API for Python to remove the broadcasting functiuonality.
* All unit tests are running
* Modified the scen...
* 1 to 1 Brain to Agent
This is a work in progess
In this PR :
- Deleted all Brain Objects
- Moved the BrainParameters into the Agent
- Gave the Agent a Heuristic method (see Balance Ball for example)
- Modified the Communicator and ModelRunner : Put can only take one agent at a time
- Made the IBrain Interface with RequestDecision and DecideAction method
No changes made to Python
[Design Doc](https://docs.google.com/document/d/1hBhBxZ9lepGF4H6fc6Hu6AW7UwOmnyX3trmgI3HpOmo/edit#)
* Removing editorconfig
* Updating BallanceBall scene
* grammar mistake
* Clearing the Agents of the Model runner
* Added Documentation on IBrain
* Modified comments on GiveModel
* Introduced a factory
* Split Learning Brain in two
* Changes to walljump
* Fixing the Unit tests
* Renaming the Brain to Policy
* Heuristic now has priority over training
* Edited code comments
* Fixing bugs
* Develop one to one scene edits...
Convert the UnitySDK to a Packman Package.
- Separate Examples into a sample project.
- Move core UnitySDK Code into com.unity.ml-agents.
- Create asmdefs for the ml-agents package.
- Add package validation tests for win/linux/max.
- Update protobuf generation scripts.
- Add Barracuda as a package dependency for ML-Agents. (users no longer have to install it themselves).
* Add the VectorSensor to the CollectObservation call
* Example of API change for BalanceBall
* Modified the Examples
* Changes to the migrating doc
* Editing the docs
* Update docs/Learning-Environment-Design-Agents.md
Co-Authored-By: Chris Elion <chris.elion@unity3d.com>
* Update docs/Migrating.md
Co-Authored-By: Chris Elion <chris.elion@unity3d.com>
* Update docs/Migrating.md
Co-Authored-By: Chris Elion <chris.elion@unity3d.com>
* Update docs/Getting-Started-with-Balance-Ball.md
Co-Authored-By: Chris Elion <chris.elion@unity3d.com>
* addressing comments
* Removed the MLAgents.Sensor namespace
* Removing the MLAgents.Sensor namespace from the tests
* Editing the migrating docs
Co-authored-by: Chris Elion <celion@gmail.com>
* Update Learning-Environment-Create-New.md (#3356)
* Update Learning-Environment-Create-New.md
In the "Final Editor Setup" , I think their should be a Step to add Decision Parameters Script and it says Decision Period from 1 to 20.
Without this their was no action taken by the RolerAgent. After adding this step it worked for me.
* Update docs/Learning-Environment-Create-New.md
Co-Authored-By: Chris Elion <celion@gmail.com>
* Update docs/Learning-Environment-Create-New.md
Co-Authored-By: Chris Elion <celion@gmail.com>
Co-authored-by: Chris Elion <celion@gmail.com>
* migration fixes
Co-authored-by: Medhavi Monish <39962268+MedhaviMonish@users.noreply.github.com>
* Update Learning-Environment-Create-New.md
In the "Final Editor Setup" , I think their should be a Step to add Decision Parameters Script and it says Decision Period from 1 to 20.
Without this their was no action taken by the RolerAgent. After adding this step it worked for me.
* Update docs/Learning-Environment-Create-New.md
Co-Authored-By: Chris Elion <celion@gmail.com>
* Update docs/Learning-Environment-Create-New.md
Co-Authored-By: Chris Elion <celion@gmail.com>
Co-authored-by: Chris Elion <celion@gmail.com>
* [skip ci] Renamed methods in the Agent class
WARNING, the user when implementing obsolete methods will see the message :Member `old method` overrides obsolete member `old method`. Add the Obsolete attribute to `old method`. It will not suggest the new method to override.
* [skip ci] Updated the example environment
* [skip ci] Updated migrating and changelog
* [skip ci] Editing the docs
* [skip ci] Missing docs
* :+1
* Update docs/Getting-Started-with-Balance-Ball.md
Co-Authored-By: Chris Elion <chris.elion@unity3d.com>
* Update docs/Learning-Environment-Create-New.md
Co-Authored-By: Chris Elion <chris.elion@unity3d.com>
* Update docs/Learning-Environment-Create-New.md
Co-Authored-By: Chris Elion <chris.elion@unity3d.com>
* [skip ci] documentation changes
* [skip ci] Update docs/Getting-Started-with-Balance-Ball.md
* [skip ci] Update docs/Getting-Started-with-Balance-Ball.md
* [skip ci] Update docs/Gett...
* Merge agent & best practices doc. Plus other fixes
* Fix overly long lines
* Merge Getting Started and Basic Guides
* Rename guide and update links appropriately
* Fix broken link
* Improvements to Learning-Environment-Create-New.md
- Changed the ordered list to use "1."
- Trimmed down text
- Removed reference to materials as those are in the Example Envs project
* Incorporated PR feedback + new images.
* factor in feedback
removed unnecessary configs
updated the agent image
* Formatting fix
* Improvements to Key Components section of ML-Agents Overview
- Moved some documentation from Learning-Environment-Design.
- Added the trainers vs LL-API separation.
- Made a note about gym-unity.
- Some update to the Agent/Behavior sections
- Updated diagrams to reflect new side channels. Made Behavior type a consistent color.
* Reorganizing the overview file and creating new (empty) sections
This change defines the new structure for the overview doc. Subsequent commits will fill in the sections and rewrite existing sections.
* Reorganizing the main Training ML-Agents page
Re-organizes into feature-specific sections that somewhat mirror the previous commit of reorganizing the overview doc.
Subsequent commits will populate these empty sections.
* Adding Deep RL
- Update ML-Agents-Overview with description of DeepRL training algorithms
- Decribe the common and trainer-specific hyperparams in Training-ML-Agents.
- Removed ...
mlagents.trainers.exception.UnityTrainerException: The hyper-parameter memory_size could not be found for the <class 'mlagents.trainers.ppo.trainer.PPOTrainer'> trainer of brain RollerBall.
* Update Dockerfile
* Separate send environment data from reset (#4128)
* Fixed a typo on ML-Agents-Overview.md (#4130)
Fixed redundant "to" word from the sentence since it is probably a typo in document.
* Updated the badge’s link to point to the newest doc version
* Replaced all of the doc to release_3_doc
* Fix 3DBall and 3DBallHard SAC regressions (#4132)
* Move memory validation to settings
* Update docs
* Add settings test
* Update to release_3 in installation.md (#4144)
* rename to SideChannelManager +backcompat (#4137)
* Remove comment about logo with --help (#4148)
* [bugfix] Make FoodCollector heuristic playable (#4147)
* Make FoodCollector heuristic playable
* Update changelog
* script to check for old release links and references (#4153)
* Remove package validation suite from Project (#4146)
* RayPerceptionSensor: handle empty and invalid tags (#4155...
* doc updates
getting started page now uses consistent run-id
re-order create-new docs to have less back/forth between unity and text editor
* add link explaining decisions where we tell the reader to modify its parameter
* Updated Learning-Environment-Create-New.md with a section on parallel unity instances.
* Added trailing whitespace to Learning Environment Create New md file.
* Added trailing whitespace to Learning Environment Create New md file after fixes.
* Minor updates.
* Minor updates.
* Whitespace fixes.
Arthur Juliani
GitHub
Joe Ward
Marwan Mattar
unityjeffrey
Deric Pang
Vincent(Yuan) Gao
Vincent-Pierre BERGES
Jeffrey Shih
Ervin T
Chris Goy
Chris Elion
Andrew Cohen