fe4472cb
Add decoders, distributions, encoders, layers, networks, and utils
4 年前
89890bf2
Update with newest changes
4 年前
1955af9e
[feature] Add experimental PyTorch support ( #4335 )
* Begin porting work
* Add ResNet and distributions
* Dynamically construct actor and critic
* Initial optimizer port
* Refactoring policy and optimizer
* Resolving a few bugs
* Share more code between tf and torch policies
* Slightly closer to running model
* Training runs, but doesn’t actually work
* Fix a couple additional bugs
* Add conditional sigma for distribution
* Fix normalization
* Support discrete actions as well
* Continuous and discrete now train
* Mulkti-discrete now working
* Visual observations now train as well
* GRU in-progress and dynamic cnns
* Fix for memories
* Remove unused arg
* Combine actor and critic classes. Initial export.
* Support tf and pytorch alongside one another
* Prepare model for onnx export
* Use LSTM and fix a few merge errors
* Fix bug in probs calculation
* Optimize np -> tensor operations
* Time action sample funct...
4 年前
f2a8c421
add torch saver test
4 年前
70197342
Add torch saver test
Add torch saver test
4 年前
ba7eb360
Merge branch 'master' into develop-torch-save-rp
4 年前
a74c904a
Merge branch 'master' into develop-saver-name
4 年前
25454a48
adding tests
4 年前
108fac9a
Replace torch.detach().cpu().numpy() with a utils method
4 年前
07e82899
update torch saver test
4 年前
44fa3a65
Moved the tests around
4 年前
328353bc
Torch : Saving/Loading of the reward providers ( #4405 )
* Saving the reward providers
* adding tests
* Moved the tests around
* Update ml-agents/mlagents/trainers/tests/torch/saver/test_saver_reward_providers.py
* Update ml-agents/mlagents/trainers/tests/torch/saver/test_saver_reward_providers.py
* Update ml-agents/mlagents/trainers/tests/torch/saver/test_saver_reward_providers.py
Co-authored-by: Ruo-Ping (Rachel) Dong <ruoping.dong@unity3d.com>
* Update ml-agents/mlagents/trainers/tests/torch/saver/test_saver_reward_providers.py
Co-authored-by: Ruo-Ping (Rachel) Dong <ruoping.dong@unity3d.com>
Co-authored-by: Ruo-Ping (Rachel) Dong <ruoping.dong@unity3d.com>
4 年前
80b7a6d3
Update ml-agents/mlagents/trainers/tests/torch/saver/test_saver_reward_providers.py
4 年前
fdd343b2
more use of item() and additional tests
4 年前
88eff042
Merge branch 'master' into develop-saver-name
4 年前
82bd7fd0
Update ml-agents/mlagents/trainers/tests/torch/saver/test_saver_reward_providers.py
4 年前
56feb8af
update test_saver_reward_providers.py
4 年前
4dda2983
Update ml-agents/mlagents/trainers/tests/torch/saver/test_saver_reward_providers.py
Co-authored-by: Ruo-Ping (Rachel) Dong <ruoping.dong@unity3d.com>
4 年前
48f217b9
Rename Saver to ModelSaver ( #4402 )
Rename Saver to ModelSaver to avoid confusion with tf.Saver
4 年前
83e21972
Update ml-agents/mlagents/trainers/tests/torch/saver/test_saver_reward_providers.py
Co-authored-by: Ruo-Ping (Rachel) Dong <ruoping.dong@unity3d.com>
4 年前
f4f1a8d9
merge master into trainer-plugin branch
4 年前
12e15e29
Fix on GAIL Torch when using actions ( #4407 )
4 年前
498934f9
Replace torch.detach().cpu().numpy() with a utils method ( #4406 )
* Replace torch.detach().cpu().numpy() with a utils method
* Using item() in place of to_numpy()
* more use of item() and additional tests
4 年前
fd1dc3a6
Merge branch 'master' into develop-torch-omp
4 年前
7b4d0865
[Bug fix] Fix bug in GAIL gradient penalty ( #4425 )
4 年前
4e93cb6e
[torch] Restructure PyTorch encoders ( #4421 )
* Move linear encoding to NetworkBody
* moved encoders to processors (#4420 )
* fix bad merge
* Get it running
* Replace mentions of visual_encoders
* Remove output_size property
* Fix tests
* Fix some references
* Revert test_simple_rl
* Fix networks test
* Make curiosity test more accomodating
* Rename total_input_size
* [Bug fix] Fix bug in GAIL gradient penalty (#4425 ) (#4426 )
Co-authored-by: Vincent-Pierre BERGES <vincentpierre@unity3d.com>
* Up number of steps
* Rename to visual_processors and vector_processors
Co-authored-by: andrewcoh <54679309+andrewcoh@users.noreply.github.com>
Co-authored-by: Andrew Cohen <andrew.cohen@unity3d.com>
Co-authored-by: Vincent-Pierre BERGES <vincentpierre@unity3d.com>
4 年前
4e6d46cc
[tests] Add tests for Torch PPO ( #4429 )
4 年前
beb5eb30
[bug-fix] Fixes for Torch SAC and tests ( #4408 )
* Fixes for Torch SAC and tests
* FIx recurrent sac test
* Properly update normalization for SAC-continuous
* Fix issue with log ent coef reporting in SAC Torch
4 年前
6f534366
Add torch_utils class, auto-detect CUDA availability ( #4403 )
* Add torch_utils
* Use torch from torch_utils
* Add torch to banned modules in CI
* Better import error handling
* Fix flake8 errors
* Address comments
* Move networks to GPU if enabled
* Switch to torch_utils
* More flake8 problems
* Move reward providers to GPU/CPU
* Remove anothere set default tensor
* Fix banned import in test
4 年前
676f5f7c
[refactor] Refactor GAIL to use new encoder structure ( #4433 )
Co-authored-by: Ervin Teng <ervin@unity3d.com>
Co-authored-by: Vincent-Pierre BERGES <vincentpierre@unity3d.com>
4 年前
60eacc0d
Merge branch 'master' into develop-adjust-cpu-settings
4 年前
bf6506fc
[feature] Add small CNN for grids 5x5 and up ( #4434 )
4 年前
3997b14b
Merge branch 'master' into develop-hybrid-actions
4 年前
7754ad7b
Don't run value during inference
4 年前
85602279
add action_out to dist
4 年前
181bdec0
-
4 年前
4e4ad7b0
Don't run value during policy evaluate, optimized soft update function ( #4501 )
* Don't run value during inference
* Execute critic with LSTM
* Address comments
* Unformat
* Optimized soft update
* Move soft update to model utils
* Add test for soft update
4 年前
f9ff3efe
Merge branch 'develop-policyonly' into develop-sac-targetq
4 年前
7c0aa77b
Merge branch 'develop-actions-out' into develop-hybrid-actions
4 年前
60b76790
Random Network Distillation for Torch ( #4473 )
* initial commit
* works with Pyramids
* added unit tests and a separate config file
* Adding first batch of documentation
* adding in the docs that rnd is only for PyTorch
* adding newline at the end of the config files
* adding some docs
* Code comments
* no normalization of the reward
* Fixing the tests
* [skip ci]
* [skip ci] Make sure RND will only work for Torch by editing the config file
* [skip ci] Additional information in the Documentation
* Remove the _has_updated_once flag
4 年前
e471bd8b
Refactoring of the tests folder for the trainers ( #4510 )
* Refactoring of the tests folder for the trainers
* Fixing issues
* Fixing issues
* Fixing issues
4 年前
400e14cb
[Bug-fix] RND would not be saved correctly. Added tests ( #4514 )
4 年前
db37db34
fixing errors
4 年前
44c9879e
action models
4 年前
a3bf96fd
Merge branch 'master' into hh/develop/gridsensor-tests
4 年前
c494bfcc
trains successfully
4 年前
8dec4771
Add hybrid actions to SAC
4 年前
e686a785
removed abstract class
4 年前
81342148
Revert "Add hybrid actions to SAC"
This reverts commit a759b36a51df4f8f1fd296f9f148269f0f026e42.
4 年前
63757004
experiment with 1/1 test
4 年前
35b88994
simple rl tests pass
4 年前
b3bc7896
Cherrypick bug fixes to release_9_branch ( #4617 )
* [bug-fix] Don't load non-wrapped policy (#4593 )
* pin cattrs version
* cap PyTorch version
* use v2 action and pin python version (#4568 )
Co-authored-by: Ervin T <ervin@unity3d.com>
Co-authored-by: Chris Elion <chris.elion@unity3d.com>
4 年前
e5f14400
Merge branch 'master' into develop-hybrid-actions-singleton
4 年前
601f02a8
update simple rl tests
4 年前
e4db5dc5
ActionSpec and ActionBuffer ( #4578 )
4 年前
7827ca06
add ActionSpec; test_simple_rl torch passes
4 年前
be723c66
Change BrainParametersProto to support ActionSpec ( #4579 )
4 年前
da978fc6
add separate hybrid test file
4 年前
6e23bafd
ActionFlattener Refactor
4 年前
8013e544
ignoring Instance of 'AbstractContextManager' has no 'enter_context' member (no-member)
4 年前
b5dd43f2
[bug-fix] Don't load non-wrapped policy ( #4593 )
* Always initialize non-wrapped policy
* Load ghosted policy
* Update changelog
* Resume test
* Add test
* Add torch test and fix torch.
4 年前
e0ef30a5
[bug-fix] Change entropy computation and loss reporting in Torch to match TF ( #4538 )
* Proper dimensions for entropy, sum before bonus in PPO
* Make entropy reporting same as TF
* Always use separate critic
* Revert to shared
* Remove unneeded extra line
* Change entropy shape in test
* Change another entropy shape
* Add entropy summing to evaluate_actions
* Add notes about torch.abs(policy_loss)
4 年前
cb8e4d25
Add ActionSpec ( #4586 )
Co-authored-by: Ervin T <ervin@unity3d.com>
4 年前
b40e7793
fix mlagents-envs tests
4 年前
9689cf2c
remove *_action_* from function names
4 年前
64e998a2
[bug-fix] Use float64 when converting np.ndarray to torch.tensor, cap Torch version to 1.7.x ( #4610 )
* Use float64 in GAIL tests
* Use float32 when converting np arrays by default
* Enforce torch 1.7.x or below
* Add comment about Windows install
* Adjust tests
4 年前
590adc01
make_fake_trajectory/step take ActionSpec arg
4 年前
96452986
Initial commit for multi head attention
4 年前
a3a9a56b
Merge branch 'exp-multi-head-attention' into exp-bullet-hell
4 年前
9e08be87
Merge branch 'master' into release_9_branch_merge
4 年前
97dfa142
fix action_spec refs
4 年前
b853e5ba
Action buffer ( #4612 )
Co-authored-by: Ervin T <ervin@unity3d.com>
Co-authored-by: Vincent-Pierre BERGES <vincentpierre@unity3d.com>
4 年前
3c96a3a2
Action Model ( #4580 )
Co-authored-by: Ervin T <ervin@unity3d.com>
Co-authored-by: Vincent-Pierre BERGES <vincentpierre@unity3d.com>
4 年前
0e28dd8f
add static method to create continuous/discrete
4 年前
88d3ec3e
Merge master into hybrid actions staging branch ( #4704 )
4 年前
ccd7cc4c
fix recurrent sac test
4 年前
ae920478
resolve conflicts
4 年前
87a7ccf8
use int64 steps, check for NaN actions ( #4607 )
* use int64 steps
* check for NaN actions
Co-authored-by: Ruo-Ping Dong <ruoping.dong@unity3d.com>
4 年前
23800f33
Merge branch 'master' into develop-action-spec
4 年前
85a7c0f7
[bug-fix] Add clipping to PyTorch policy, fix initialization ( #4649 )
4 年前
733bffbf
use int64 steps, check for NaN actions ( #4607 ) ( #4654 )
* use int64 steps
* check for NaN actions
Co-authored-by: Ruo-Ping Dong <ruoping.dong@unity3d.com>
Co-authored-by: Chris Elion <chris.elion@unity3d.com>
4 年前
b6d10456
removed action_spec.size
4 年前
8175d558
[bug-fix] Fix BC module + action clipping ( #4667 )
4 年前
2a8c6800
[bug-fix] Add clipping to PyTorch policy, fix initialization ( #4649 ) ( #4662 )
4 年前
e14e1c4d
Improvements and new tests
4 年前
953cb6bb
Merge branch 'master' into develop-windows-delay
4 年前
ee5313e4
Merge branch 'master' into develop-windows-delay
4 年前
f0ed3a38
Cherry-pick BC fixes to Release 10 ( #4668 )
4 年前
afd16cc9
rename make_x to creat_x/remove redundant properties
4 年前
5b9aab58
fix advanced vis encoder simple rl
4 年前
95892058
reduce recurrent step/increase batch size
4 年前
e5cc57f9
rename create random to random action
4 年前
b863af57
Removing TensorFlow Trainers
4 年前
278911a5
Fix staging tests ( #4708 )
4 年前
94c59e31
C# changes for hybrid action spaces ( #4587 )
* Add hybrid action capability flag (#4576 )
* Change BrainParametersProto to support ActionSpec (#4579 )
* Assign new BrainParametersProto fields based on capabilities (#4581 )
* ActionBuffer with hybrid actions for RemotePolicy (#4592 )
* Barracuda inference for hybrid actions (#4611 )
* Refactor BarracudaModel loader checks (#4629 )
* Export separate nodes for continuous/discrete actions (#4655 )
* Separate continuous/discrete actions in AgentActionProto (#4698 )
* Force different nodes for new and deprecated action output (#4705 )
4 年前
f6355ba9
Merge branch 'develop-action-spec' into develop-action-buffer
4 年前
bd917c9c
action buffer passes continuous
4 年前
b36fcf16
discrete runs/cont passes
4 年前
ad951493
debugging discrete
4 年前
fcf6471e
2d discrete passes
4 年前
056630d7
sac continuous and discrete train
4 年前
990f801a
Develop hybrid action staging ( #4702 )
Co-authored-by: Ervin T <ervin@unity3d.com>
Co-authored-by: Vincent-Pierre BERGES <vincentpierre@unity3d.com>
Co-authored-by: Ruo-Ping Dong <ruoping.dong@unity3d.com>
Co-authored-by: Chris Elion <chris.elion@unity3d.com>
4 年前
735fcd52
[WIP] Refactor trainers to use list of obs rather than vec and vis obs
4 年前
85e4db33
bc tests pass
4 年前
b8f22fd7
Update second half of tests
4 年前
93ca1409
fixing the tests
4 年前
7a5cc9ec
Merge master into develop-rm-tf
4 年前
24fd9b3c
torch reward providers all pass
4 年前
dee6b805
fixed bug in discrete
4 年前
c1587bce
Solving merge conflicts
4 年前
4ebc6c44
ml-agents-envs pass
4 年前
ded1f79b
Merge pull request #4732 from Unity-Technologies/goal-sensors
Adds SensorTypes and GoalSensors
4 年前
b5d1c071
Merge branch 'master' into develop-action-buffer
4 年前
0d2f8887
Merge remote-tracking branch 'origin/master' into goal-conditioning
# Conflicts:
# ml-agents-envs/mlagents_envs/base_env.py
# ml-agents-envs/mlagents_envs/rpc_utils.py
# ml-agents/mlagents/trainers/tests/mock_brain.py
# ml-agents/mlagents/trainers/tests/simple_test_envs.py
4 年前
25dfd883
Merge branch 'master' into develop-centralizedcritic
4 年前
ad5f878c
[refactor] Remove critic pass during inference ( #4743 )
4 年前
11687f8d
[cherry-pick] Cherry-pick #4743 into Release 11 ( #4756 )
4 年前
903d3afe
Merge pull request #4707 from Unity-Technologies/develop-rm-tf
Removing TensorFlow Trainers
4 年前
14378aa5
Merging master
4 年前
293bd20b
fix torch test_ppo
4 年前
1a1070b1
forgot a file
4 年前
230497f5
fix torch utils test
4 年前
a545859e
fix torch test policy
4 年前
8cb050ef
WIP Made initial changes to enale dimension properties and added attention module
4 年前
498b1ee6
Merge branch 'develop-action-buffer' into develop-hybrid-actions-singleton
4 年前
a73f7d73
Turn down gain on GAIL discriminator output ( #4762 )
4 年前
06f1f254
1:1 and continuous/discrete train
4 年前
b6bb01b9
Turn down gain on GAIL discriminator output ( #4762 ) ( #4772 )
4 年前
c3699de8
merging master and addressing comments
4 年前
29d94c7c
Merge pull request #4734 from Unity-Technologies/develop-obs-as-list
Refactor trainers to use list of obs rather than vec and vis obs
4 年前
1d234d1d
bc works
4 年前
719c969c
addressing comments. ObservationSpec is no longer a list
4 年前
4bba4e8e
Renaming ObservationSpec to SensorSpec
4 年前
c0d01baf
Merge branch 'master' into merge-release11-master
4 年前
8dee7970
Fixing the tests
4 年前
5f0f7e3a
fix reward provider tests
4 年前
88b8f4b4
replace use_discrete with action_sizes in simple_rl
4 年前
c5a057d2
renaming obs_spec variables
4 年前
44ed3258
Merging master
4 年前
449712b0
renaming sensor_spec to sensor_specS
4 年前
35769b53
Merge branch 'develop-action-buffer' into develop-hybrid-actions-singleton
4 年前
272affe0
preliminary aciton model tests
4 年前
17496265
move AgentAction, ActionLogProbs, and ActionFlattener to separate files
4 年前
76ebc20c
Merge remote-tracking branch 'origin/master' into r12-to-master
4 年前
d984af1f
action model and network tests
4 年前
458fee17
Merge pull request #4763 from Unity-Technologies/develop-att
WIP Made initial changes to enable dimension properties and added attention module
4 年前
330fc1d0
Merge branch 'master' into develop-centralizedcritic-mm
4 年前
60309d8f
fix torch policy tests
4 年前
519c5f47
merging master
4 年前
8ed14762
Merge branch 'develop-hybrid-actions-singleton' into develop-hybrid-actions-csharp
4 年前
11e2f5e4
remove unused imports test_hybrid
4 年前
6ffbf209
fix imports in test utils
4 年前
7387a77f
remove pylint ( #4836 )
* remove pylint
* remove other pylint disables
4 年前
9689449f
Refactor of attention ( #4840 )
Co-authored-by: Vincent-Pierre BERGES <vincentpierre@unity3d.com>
4 年前
0b4b0992
Rename more files
4 年前
7c37c759
Fix some mis-renamings
4 年前
701c1a3f
fix test torch distributions
4 年前
b7e6efa3
Allow setting maximum number of elements in self-attention to None ( #4841 )
* separate entity encoder and RSA
* clean up args in mha
* more cleanups
* fixed tests
* entity embeddings have no max option
* Add exceptions for variable export
* Fix test
* Add docstrings
Co-authored-by: Andrew Cohen <andrew.cohen@unity3d.com>
4 年前
56972f56
WIP integrate attention to networkbody
4 年前
aba633b2
Merge branch 'develop-attention-refactor' into develop-centralizedcritic-mm
4 年前
e88558c3
fix torch test policy
4 年前
631ac7f4
fixed tests
4 年前
30a09c6f
Merge branch 'develop-attention-refactor' into develop-centralizedcritic-mm
4 年前
22f42f5b
fix torch test ppo
4 年前
eb78a477
Add default init/gain to LinearEncoder ( #4846 )
4 年前
7f8e6a0d
fix tests
4 年前
0ac990e0
add LayerNorm ( #4847 )
4 年前
4bf182aa
fix tensorflow test simple rl
4 年前
a7d04be6
Merge branch 'develop-hybrid-actions-singleton' into develop-hybrid-actions-csharp
4 年前
8c42dcc7
fix tensorflow test ppo
4 年前
5039b65a
Merge branch 'master' into develop-att-network-integration
4 年前
0a876b9c
Fix typos
4 年前
2085e17c
Merge branch 'master' into develop-centralizedcritic-mm
4 年前
e3de0406
Plurals
4 年前
180d3e20
Merge branch 'develop-centralizedcritic-mm' into develop-cc-teammanager
4 年前
0024a286
merge ervin's new stuff
4 年前
12e1fc28
[feature] Hybrid SAC ( #4574 )
4 年前
7af25330
fixed torch test sac
4 年前
9bcd3c39
fix 2d sac
4 年前
7b230bdf
Change seed for two offending tests
4 年前
b0c02ee0
Merge branch 'develop-hybrid-actions-csharp' into develop-actionmodel-csharp
4 年前
fc756e5a
Formatting
4 年前
a0876939
Extend test time
4 年前
880d390b
Change seed
4 年前
b4d8cf54
Change learning rate
4 年前
67ad9651
Merge pull request #4825 from Unity-Technologies/sensor-types
[WIP] Observation Types
4 年前
8660b1c2
merging master
4 年前
a02cf933
Add predict minimum attention test ( #4853 )
4 年前
24d2f335
fixing test
4 年前
38fc2536
addresing some comments
4 年前
01e0ee00
refactor entityembedding/network body ( #4857 )
4 年前
89b6c949
use singular entity embedding ( #4873 )
4 年前
6dafe05c
fix tests
4 年前
457fb612
Merge branch 'master' of https://github.com/Unity-Technologies/ml-agents
4 年前
52b011d6
_
4 年前
03c905b2
Fix equation for entropy
4 年前
396bc43c
Merging master
4 年前
d4455936
Merge pull request #4869 from Unity-Technologies/fix-normal-entropy
Fix equation for entropy
4 年前
b7c7d773
Adding some tests
4 年前
6f3ea7b8
_
4 年前
aaec009a
Formatting
4 年前
2f48cb82
Fixing a test
4 年前
bd4bc66b
Merge branch 'master' into fix-numti-env-delayed-spawn
4 年前
77eecc6b
Merge branch 'master' into develop-att-network-integration
4 年前
7e47f94b
addressing comments
4 年前
d7f549f9
Run pytest on GPU ( #4865 )
* make tests device-friendly
* mark all tests in test_simple_rl
4 年前
c27a95f0
Make a self encoder before EntityEmbedding
4 年前
ff70c5c4
Merge branch 'master' into goal-conditioning-new
4 年前
fd007f53
Attempting to use EntityEmbedding directly as processor
4 年前
f5ec393b
added a test to make sure that a mask of all zeros or all ones would not break backpropagation
4 年前
1cff7848
no need for large number of steps in test
4 年前
2fb87e4f
Merge branch 'master' into reward-dist
4 年前
212ebfb9
Merge pull request #4844 from Unity-Technologies/develop-att-network-integration
Integrate attention to networkbody
4 年前
9d70220e
Merge remote-tracking branch 'origin/master' into MLA-1734-demo-provider
4 年前
64fc7f43
Buffer key enums ( #4907 )
4 年前
b6f88d6d
Merge branch 'develop-base-teammanager' into develop-agentprocessor-teammanager
4 年前
543f22bc
fix test_networks
4 年前
1831044a
Update SAC to use separate policy
4 年前
5022d710
Add additional logic to avoid load being called on every advance ( #4934 )
4 年前
c7054d76
Use attention tests from master
4 年前
6828713c
fix saver test
4 年前
da6a55a0
Revert "Use attention tests from master"
This reverts commit 78e052be8f36381bb6857817ff0f505716be83b9.
4 年前
24ee4bd5
Merge remote-tracking branch 'origin/develop-critic-optimizer' into develop-critic-optimizer
4 年前
bac2fb68
Use attention from master
4 年前
66742dc8
test for SharedActorCritic
4 年前
c87bce9e
Merge branch 'master' into develop-base-teammanager
4 年前
3f7d68b8
fix test policy
4 年前
531695fb
adjust step size gail visual ppo
4 年前
219e773b
Merge branch 'develop-fix-lstms' into develop-critic-op-lstm
4 年前
e1b94b8b
Merge branch 'master' into develop-var-len-obs-feature
4 年前
dc8e8494
Merge branch 'master' into develop-critic-optimizer
4 年前
e4f51ca7
Merge remote-tracking branch 'origin/master' into MLA-1734-demo-provider
4 年前
d4438878
Merge branch 'develop-base-teammanager' into develop-agentprocessor-teammanager
4 年前
4a33be31
Tweak SAC tests
4 年前
40f51774
Fix PPO tests
4 年前
180f7d03
Fix SAC test
4 年前
c3bc8991
cleanup, don't store mask
4 年前
e46a86ad
Merge branch 'master' into develop-superpush-int
4 年前
15d512f9
Merge branch 'master' into hh/develop/dodgeball
4 年前
08db7c2f
Merge branch 'develop-agentprocessor-teammanager' into develop-coma2-trainer-mm
4 年前
1fc3640e
Change AgentAction back to 0 pad and add tests
4 年前
338af2ec
Move the Critic into the Optimizer ( #4939 )
Co-authored-by: Ervin Teng <ervin@unity3d.com>
4 年前
4c947151
Merge branch 'main' into hh/develop/dodgeball
4 年前
61781a1a
Merge branch 'main' into develop-agentprocessor-teammanager
4 年前
9060da06
Merge branch 'develop-agentprocessor-teammanager' into develop-coma2-trainer
4 年前
56d4c1f9
Fix to-flat and add tests
4 年前
e2d46ca0
Merge branch 'develop-agentprocessor-teammanager' into develop-coma2-trainer
4 年前
8562471e
add inital coma optimizer tests
4 年前
43955c5b
get value estimate test
4 年前
06c147f8
Merge remote-tracking branch 'origin/main' into goal-conditioning-new
# Conflicts:
# Project/Assets/ML-Agents/Examples/Crawler/Prefabs/CrawlerBase.prefab
# Project/Assets/ML-Agents/Examples/GridWorld/Prefabs/Area.prefab
# Project/Assets/ML-Agents/Examples/GridWorld/Scenes/GridWorld.unity
# Project/ProjectSettings/TagManager.asset
# com.unity.ml-agents/Runtime/Sensors/CameraSensor.cs
# com.unity.ml-agents/Runtime/Sensors/VectorSensor.cs
# ml-agents/mlagents/trainers/torch/networks.py
# ml-agents/mlagents/trainers/torch/utils.py
4 年前
d36a5242
Python Dataflow for Group Manager ( #4926 )
* Make buffer type-agnostic
* Edit types of Apped method
* Change comment
* Collaborative walljump
* Make collab env harder
* Add group ID
* Add collab obs to trajectory
* Fix bug; add critic_obs to buffer
* Set group ids for some envs
* Pretty broken
* Less broken PPO
* Update SAC, fix PPO batching
* Fix SAC interrupted condition and typing
* Fix SAC interrupted again
* Remove erroneous file
* Fix multiple obs
* Update curiosity reward provider
* Update GAIL and BC
* Multi-input network
* Some minor tweaks but still broken
* Get next critic observations into value estimate
* Temporarily disable exporting
* Use Vince's ONNX export code
* Cleanup
* Add walljump collab YAML
* Lower max height
* Update prefab
* Update prefab
* Collaborative Hallway
* Set num teammates to 2
* Add config and group ids to HallwayCollab
* Fix bug with hallway collab
* E...
4 年前
c8137dcd
Merge branch 'main' into develop-superpush-int
4 年前
f16ce486
Update v2-staging from main (March 15) ( #5123 )
4 年前
921ba4f0
Update v2-staging from main (March 15) ( #5123 )
4 年前
ba2af269
[coma2] Make group extrinsic reward part of extrinsic ( #5033 )
* Make group extrinsic part of extrinsic
* Fix test and init
* Fix tests and bug
* Add baseline loss to TensorBoard
4 年前
d24b0966
[bug-fix] Fix memory leak when using LSTMs ( #5048 )
* Detach memory before storing
* Add test
* Evaluate with no_grad
4 年前
ebe45056
Merge branch 'main' into release_14_branch-to-main
4 年前
d2635e58
Action slice ( #5047 )
* add slice function to agent action
* add type/docstring to slice
* add test
4 年前
95f62362
add test
4 年前
853b44d5
torch coma tests: lstm, cur, gail
4 年前
46461986
pass sensor name through to ObservationSpec ( #5036 )
4 年前
fc5d0a3f
[bug-fix] Fix save/restore critic, add test ( #5062 )
* Fix save/restore critic, add test
* Rename module for PPO
* Use correct policy in test
4 年前
970f1d40
Merge remote-tracking branch 'origin/v2-staging' into MLA-1634-ObservationSpec
4 年前
cd349985
add negative constant extrinsic to gail
4 年前
ffca08c4
Upgrade PyTorch version for python 3.9 ( #5028 )
4 年前
1f026c70
Merge branch 'main' into develop-superpush-branch-cleanup
4 年前
e547f26c
adjust step size
4 年前
ce872033
Revert "Merge branch 'main' into develop-superpush-branch-cleanup"
This reverts commit 5bea802525381f931a5e0f8b8778fe27a12f03af, reversing
changes made to cee3524e85161e13689d95f66bc6bff994d2cdfd.
4 年前
8f35bdd3
POCA trainer ( #5005 )
Co-authored-by: Ervin Teng <ervin@unity3d.com>
Co-authored-by: Ruo-Ping Dong <ruoping.dong@unity3d.com>
Co-authored-by: Chris Elion <chris.elion@unity3d.com>
Co-authored-by: Vincent-Pierre BERGES <vincentpierre@unity3d.com>
4 年前
9e77d7e1
Merge branch 'main' into develop-soccer-groupman
4 年前
e81e038b
Fix end episode for POCA, add warning for group reward if not POCA ( #5113 )
* Fix end episode for POCA, add warning for group reward if not POCA
* Add missing imports
4 年前
63169e2c
[cherry-pick] Fix group rewards for POCA, add warning for non-POCA trainers ( #5120 )
* Fix end episode for POCA, add warning for group reward if not POCA (#5113 )
* Fix end episode for POCA, add warning for group reward if not POCA
* Add missing imports
* Use np.any, which is faster
4 年前
ef3d6e0d
Adding Hypernetwork modules and unit tests ( #5141 )
4 年前
41dd16e8
Merge branch 'main' into release_15_mm
4 年前
c108da4a
[bug-fix] Fix POCA LSTM, pad sequences in the back ( #5206 )
* Pad buffer at the end
* Fix padding in optimizer value estimate
* Fix additional bugs and POCA
* Fix groupmate obs, add tests
* Update changelog
* Improve tests
* Address comments
* Fix poca test
* Fix buffer test
* Increase entropy for Hallway
* Add EOF newline
* Fix Behavior Name
* Address comments
(cherry picked from commit 2ce6810846ba9268e4fb5fb082fa54e90414c980)
3 年前
bed4bf36
Load individual elements if state dict load fails ( #5213 )
Co-authored-by: Vincent-Pierre BERGES <vincentpierre@unity3d.com>
Co-authored-by: Ervin T. <ervin@unity3d.com>
(cherry picked from commit ac4f43cf18b98d0fc7063b9b831e07429f7ea39e)
3 年前
18be47e8
Merge branch 'main' into develop-soccer-groupman-mod
4 年前
81705d6d
Goal conditioning integration ( #5142 )
* Adding Hypernetwork modules and unit tests
* Edits
* Integration of the hypernetowrk to the trainer
* Update ml-agents/mlagents/trainers/torch/networks.py
Co-authored-by: Arthur Juliani <awjuliani@gmail.com>
* Making the default hyper and added the conditioning type None
* Reducing the number of hypernetwork layers
* addressing comments
Co-authored-by: Arthur Juliani <awjuliani@gmail.com>
4 年前
d4716caa
Merge branch 'main' into goal-conditioning-sensors-3
4 年前
c05ec9af
Fix groupmate obs, add tests
4 年前
b3499848
Improve tests
4 年前
6e04aaf3
Fix poca test
4 年前
42105f23
add load different reward tests
4 年前
98dcb548
test convolutions can be loaded properly
4 年前
2e5b1352
add check that layers still have different dimensions
4 年前
cb1f5462
Update ml-agents/mlagents/trainers/tests/torch/saver/test_saver.py
Co-authored-by: Vincent-Pierre BERGES <vincentpierre@unity3d.com>
4 年前
f3d586bc
Update ml-agents/mlagents/trainers/tests/torch/saver/test_saver.py
Co-authored-by: Vincent-Pierre BERGES <vincentpierre@unity3d.com>
4 年前
c5589b59
[bug-fix] Fix POCA LSTM, pad sequences in the back ( #5206 )
* Pad buffer at the end
* Fix padding in optimizer value estimate
* Fix additional bugs and POCA
* Fix groupmate obs, add tests
* Update changelog
* Improve tests
* Address comments
* Fix poca test
* Fix buffer test
* Increase entropy for Hallway
* Add EOF newline
* Fix Behavior Name
* Address comments
4 年前
9dfe6c7f
Load individual elements if state dict load fails ( #5213 )
Co-authored-by: Vincent-Pierre BERGES <vincentpierre@unity3d.com>
Co-authored-by: Ervin T. <ervin@unity3d.com>
4 年前
51adab1c
Fix the attention module embedding size
3 年前
353b1566
Fix the attention module embedding size ( #5272 )
* Fix the attention module embedding size
* editing the changelog
3 年前
4846cf0f
Merge branch 'main' into develop-api-documentation-update
Updating with main.
3 年前
806f04bd
Readding the validation of the minimal cnn input size ( #5345 ) ( #5346 )
3 年前
15440c24
Readding the validation of the minimal cnn input size ( #5345 )
3 年前
bb07eb45
Adding a fully connected visual encoder for super small visual input + tests ( #5351 )
* initial commit for a fully connected visual encoder
* adding a test
* addressing comments
* Fixing error with minimal size of fully connected network
* adding documentation and changelog
3 年前
97b7d5c6
Merge branch 'main' into develop-api-documentation-update
Syncing with main.
3 年前
b767b66b
Exclude test_visual_encoder_trains from GPU test ( #5367 )
3 年前
fc6e8c35
[ 🐛 🔨 ] Fix sac target for continuous actions ( #5372 )
* Fix of the target entropy for continuous SAC
* Lowering required steps of test and remove unecessary unsqueeze
* Changing the target from -dim(a)^2 to -dim(a) by removing implicit broadcasting
3 年前
Ervin Teng
GitHub
Ruo-Ping Dong
vincentpierre
Anupam Bhatnagar
Andrew Cohen
HH
Arthur Juliani
Chris Elion
brccabral
Christopher Goy
Miguel Alonso Jr