浏览代码

linear value, no target

/develop/bisim-sac-transfer
yanchaosun 4 年前
当前提交
aa0e896f
共有 5 个文件被更改,包括 71 次插入16 次删除
  1. 14
      config/sac_transfer/3DBall.yaml
  2. 14
      config/sac_transfer/3DBallHard.yaml
  3. 7
      config/sac_transfer/3DBallHardTransfer.yaml
  4. 2
      ml-agents/mlagents/trainers/sac_transfer/network.py
  5. 50
      config/sac_transfer/3DBallHardTransfer1.yaml

14
config/sac_transfer/3DBall.yaml


init_entcoef: 0.5
reward_signal_steps_per_update: 10.0
encoder_layers: 2
policy_layers: 0
policy_layers: 1
value_layers: 2
feature_size: 32
separate_value_net: true
# separate_policy_train: true
reuse_encoder: false
value_layers: 0
feature_size: 64
# separate_value_net: true
separate_policy_train: true
# separate_value_train: true
separate_model_train: true
reuse_encoder: true
in_epoch_alter: false
in_batch_alter: true
use_op_buffer: false

14
config/sac_transfer/3DBallHard.yaml


init_entcoef: 1.0
reward_signal_steps_per_update: 10.0
encoder_layers: 2
policy_layers: 0
policy_layers: 1
value_layers: 2
feature_size: 32
separate_value_net: true
# separate_policy_train: true
reuse_encoder: false
value_layers: 0
feature_size: 64
# separate_value_net: true
separate_policy_train: true
# separate_value_train: true
separate_model_train: true
reuse_encoder: true
in_epoch_alter: false
in_batch_alter: true
use_op_buffer: false

7
config/sac_transfer/3DBallHardTransfer.yaml


save_replay_buffer: false
init_entcoef: 1.0
reward_signal_steps_per_update: 10.0
encoder_layers: 2
encoder_layers: 1
forward_layers: 0
forward_layers: 1
# separate_value_train: true
reuse_encoder: false
in_epoch_alter: false
in_batch_alter: false

use_transfer: true
load_model: true
train_model: false
transfer_path: "results/sac-ball-lintest/3DBall"
transfer_path: "results/sac_model_ball_ml1/3DBall"
network_settings:
normalize: true
hidden_units: 64

2
ml-agents/mlagents/trainers/sac_transfer/network.py


self.processed_vector_in,
vis_encode_type,
encoder_layers=encoder_layers,
scope="target_enc",
scope="encoding",
reuse=True
)
if separate_train:

50
config/sac_transfer/3DBallHardTransfer1.yaml


behaviors:
3DBallHard:
trainer_type: sac_transfer
hyperparameters:
learning_rate: 0.0003
learning_rate_schedule: linear
batch_size: 256
buffer_size: 50000
buffer_init_steps: 0
tau: 0.005
steps_per_update: 10.0
save_replay_buffer: false
init_entcoef: 1.0
reward_signal_steps_per_update: 10.0
encoder_layers: 2
policy_layers: 1
forward_layers: 0
value_layers: 0
feature_size: 64
# separate_value_net: true
# separate_model_train: true
separate_policy_train: true
reuse_encoder: true
in_epoch_alter: false
in_batch_alter: false
use_op_buffer: false
use_var_predict: true
with_prior: false
predict_return: true
use_bisim: false
use_transfer: true
load_model: true
load_policy: true
train_policy: false
train_model: false
transfer_path: "results/sacmodel-ball-v0/3DBall"
network_settings:
normalize: true
hidden_units: 64
num_layers: 2
vis_encode_type: simple
reward_signals:
extrinsic:
gamma: 0.99
strength: 1.0
keep_checkpoints: 5
max_steps: 500000
time_horizon: 1000
summary_freq: 12000
threaded: true
正在加载...
取消
保存