浏览代码

Match3 example (#4515)

/MLA-1734-demo-provider
GitHub 4 年前
当前提交
90a9d214
共有 65 个文件被更改,包括 6579 次插入1 次删除
  1. 5
      com.unity.ml-agents.extensions/Documentation~/com.unity.ml-agents.extensions.md
  2. 7
      com.unity.ml-agents.extensions/Tests/Editor/Sensors/RigidBodySensorTests.cs
  3. 2
      com.unity.ml-agents/Runtime/SensorHelper.cs
  4. 21
      docs/Learning-Environment-Examples.md
  5. 8
      Project/Assets/ML-Agents/Examples/Match3.meta
  6. 67
      com.unity.ml-agents.extensions/Documentation~/Match3.md
  7. 3
      com.unity.ml-agents.extensions/Runtime/Match3.meta
  8. 3
      com.unity.ml-agents.extensions/Tests/Editor/Match3.meta
  9. 75
      config/ppo/Match3.yaml
  10. 77
      docs/images/match3.png
  11. 8
      Project/Assets/ML-Agents/Examples/Match3/Prefabs.meta
  12. 174
      Project/Assets/ML-Agents/Examples/Match3/Prefabs/Match3Heuristic.prefab
  13. 7
      Project/Assets/ML-Agents/Examples/Match3/Prefabs/Match3Heuristic.prefab.meta
  14. 170
      Project/Assets/ML-Agents/Examples/Match3/Prefabs/Match3VectorObs.prefab
  15. 7
      Project/Assets/ML-Agents/Examples/Match3/Prefabs/Match3VectorObs.prefab.meta
  16. 170
      Project/Assets/ML-Agents/Examples/Match3/Prefabs/Match3VisualObs.prefab
  17. 7
      Project/Assets/ML-Agents/Examples/Match3/Prefabs/Match3VisualObs.prefab.meta
  18. 8
      Project/Assets/ML-Agents/Examples/Match3/Scenes.meta
  19. 1001
      Project/Assets/ML-Agents/Examples/Match3/Scenes/Match3.unity
  20. 7
      Project/Assets/ML-Agents/Examples/Match3/Scenes/Match3.unity.meta
  21. 8
      Project/Assets/ML-Agents/Examples/Match3/Scripts.meta
  22. 373
      Project/Assets/ML-Agents/Examples/Match3/Scripts/Match3Agent.cs
  23. 3
      Project/Assets/ML-Agents/Examples/Match3/Scripts/Match3Agent.cs.meta
  24. 272
      Project/Assets/ML-Agents/Examples/Match3/Scripts/Match3Board.cs
  25. 11
      Project/Assets/ML-Agents/Examples/Match3/Scripts/Match3Board.cs.meta
  26. 102
      Project/Assets/ML-Agents/Examples/Match3/Scripts/Match3Drawer.cs
  27. 3
      Project/Assets/ML-Agents/Examples/Match3/Scripts/Match3Drawer.cs.meta
  28. 8
      Project/Assets/ML-Agents/Examples/Match3/TFModels.meta
  29. 1001
      Project/Assets/ML-Agents/Examples/Match3/TFModels/Match3VectorObs.onnx
  30. 14
      Project/Assets/ML-Agents/Examples/Match3/TFModels/Match3VectorObs.onnx.meta
  31. 1001
      Project/Assets/ML-Agents/Examples/Match3/TFModels/Match3VisualObs.nn
  32. 11
      Project/Assets/ML-Agents/Examples/Match3/TFModels/Match3VisualObs.nn.meta
  33. 233
      com.unity.ml-agents.extensions/Runtime/Match3/AbstractBoard.cs
  34. 3
      com.unity.ml-agents.extensions/Runtime/Match3/AbstractBoard.cs.meta
  35. 120
      com.unity.ml-agents.extensions/Runtime/Match3/Match3Actuator.cs
  36. 3
      com.unity.ml-agents.extensions/Runtime/Match3/Match3Actuator.cs.meta
  37. 49
      com.unity.ml-agents.extensions/Runtime/Match3/Match3ActuatorComponent.cs
  38. 3
      com.unity.ml-agents.extensions/Runtime/Match3/Match3ActuatorComponent.cs.meta
  39. 297
      com.unity.ml-agents.extensions/Runtime/Match3/Match3Sensor.cs
  40. 3
      com.unity.ml-agents.extensions/Runtime/Match3/Match3Sensor.cs.meta
  41. 43
      com.unity.ml-agents.extensions/Runtime/Match3/Match3SensorComponent.cs
  42. 3
      com.unity.ml-agents.extensions/Runtime/Match3/Match3SensorComponent.cs.meta
  43. 260
      com.unity.ml-agents.extensions/Runtime/Match3/Move.cs
  44. 3
      com.unity.ml-agents.extensions/Runtime/Match3/Move.cs.meta
  45. 152
      com.unity.ml-agents.extensions/Tests/Editor/Match3/AbstractBoardTests.cs
  46. 3
      com.unity.ml-agents.extensions/Tests/Editor/Match3/AbstractBoardTests.cs.meta
  47. 115
      com.unity.ml-agents.extensions/Tests/Editor/Match3/Match3ActuatorTests.cs
  48. 3
      com.unity.ml-agents.extensions/Tests/Editor/Match3/Match3ActuatorTests.cs.meta
  49. 314
      com.unity.ml-agents.extensions/Tests/Editor/Match3/Match3SensorTests.cs
  50. 3
      com.unity.ml-agents.extensions/Tests/Editor/Match3/Match3SensorTests.cs.meta
  51. 60
      com.unity.ml-agents.extensions/Tests/Editor/Match3/MoveTests.cs
  52. 3
      com.unity.ml-agents.extensions/Tests/Editor/Match3/MoveTests.cs.meta
  53. 3
      com.unity.ml-agents.extensions/Tests/Editor/Match3/match3obs0.png
  54. 88
      com.unity.ml-agents.extensions/Tests/Editor/Match3/match3obs0.png.meta
  55. 3
      com.unity.ml-agents.extensions/Tests/Editor/Match3/match3obs_special0.png
  56. 88
      com.unity.ml-agents.extensions/Tests/Editor/Match3/match3obs_special0.png.meta
  57. 3
      com.unity.ml-agents.extensions/Tests/Editor/Match3/match3obs_special1.png
  58. 88
      com.unity.ml-agents.extensions/Tests/Editor/Match3/match3obs_special1.png.meta

5
com.unity.ml-agents.extensions/Documentation~/com.unity.ml-agents.extensions.md


| _Runtime_ | Contains core C# APIs for integrating ML-Agents into your Unity scene. |
| _Tests_ | Contains the unit tests for the package. |
The Runtime directory currently contains three features:
* [Match-3 sensor and actuator](Match3.md)
* [Grid-based sensor](Grid-Sensor.md)
* Physics-based sensors
## Installation
The ML-Agents Extensions package is not currently available in the Package Manager. There are two
recommended ways to install the package:

7
com.unity.ml-agents.extensions/Tests/Editor/Sensors/RigidBodySensorTests.cs


bool isOK = SensorHelper.CompareObservation(sensor, expected, out errorMessage);
Assert.IsTrue(isOK, errorMessage);
}
public static void CompareObservation(ISensor sensor, float[,,] expected)
{
string errorMessage;
bool isOK = SensorHelper.CompareObservation(sensor, expected, out errorMessage);
Assert.IsTrue(isOK, errorMessage);
}
}
public class RigidBodySensorTests

2
com.unity.ml-agents/Runtime/SensorHelper.cs


if (expected[h, w, c] != output[tensorShape.Index(0, h, w, c)])
{
errorMessage = $"Expected and actual differed in position [{h}, {w}, {c}]. " +
"Expected: {expected[h, w, c]} Actual: {output[tensorShape.Index(0, h, w, c)]} ";
$"Expected: {expected[h, w, c]} Actual: {output[tensorShape.Index(0, h, w, c)]} ";
return false;
}
}

21
docs/Learning-Environment-Examples.md


does not train with the provided default training parameters.**
- Float Properties: None
- Benchmark Mean Reward: 1.75
## Match 3
![Match 3](images/match3.png)
- Set-up: Simple match-3 game. Matched pieces are removed, and remaining pieces
drop down. New pieces are spawned randomly at the top, with a chance of being
"special".
- Goal: Maximize score from matching pieces.
- Agents: The environment contains several independent Agents.
- Agent Reward Function (independent):
- .01 for each normal piece cleared. Special pieces are worth 2x or 3x.
- Behavior Parameters:
- None
- Observations and actions are defined with a sensor and actuator respectively.
- Float Properties: None
- Benchmark Mean Reward:
- 37.2 for visual observations
- 37.6 for vector observations
- 34.2 for simple heuristic (pick a random valid move)
- 37.0 for greedy heuristic (pick the highest-scoring valid move)

8
Project/Assets/ML-Agents/Examples/Match3.meta


fileFormatVersion: 2
guid: 85094c6352d9e43c497a54fef35e4d76
folderAsset: yes
DefaultImporter:
externalObjects: {}
userData:
assetBundleName:
assetBundleVariant:

67
com.unity.ml-agents.extensions/Documentation~/Match3.md


# Match-3 Game Support
We provide some utilities to integrate ML-Agents with Match-3 games.
## AbstractBoard class
The `AbstractBoard` is the bridge between ML-Agents and your game. It allows ML-Agents to
* ask your game what the "color" of a cell is
* ask whether the cell is a "special" piece type or not
* ask your game whether a move is allowed
* request that your game make a move
These are handled by implementing the `GetCellType()`, `IsMoveValid()`, and `MakeMove()` abstract methods.
The AbstractBoard also tracks the number of rows, columns, and potential piece types that the board can have.
#### `public abstract int GetCellType(int row, int col)`
Returns the "color" of piece at the given row and column.
This should be between 0 and NumCellTypes-1 (inclusive).
The actual order of the values doesn't matter.
#### `public abstract int GetSpecialType(int row, int col)`
Returns the special type of the piece at the given row and column.
This should be between 0 and NumSpecialTypes (inclusive).
The actual order of the values doesn't matter.
#### `public abstract bool IsMoveValid(Move m)`
Check whether the particular `Move` is valid for the game.
The actual results will depend on the rules of the game, but we provide the `SimpleIsMoveValid()` method
that handles basic match3 rules with no special or immovable pieces.
#### `public abstract bool MakeMove(Move m)`
Instruct the game to make the given move. Returns true if the move was made.
Note that during training, a move that was marked as invalid may occasionally still be
requested. If this happens, it is safe to do nothing and request another move.
## Move struct
The Move struct encapsulates a swap of two adjacent cells. You can get the number of potential moves
for a board of a given size with. `Move.NumPotentialMoves(NumRows, NumColumns)`. There are two helper
functions to create a new `Move`:
* `public static Move FromMoveIndex(int moveIndex, int maxRows, int maxCols)` can be used to
iterate over all potential moves for the board by looping from 0 to `Move.NumPotentialMoves()`
* `public static Move FromPositionAndDirection(int row, int col, Direction dir, int maxRows, int maxCols)` creates
a `Move` from a row, column, and direction (and board size).
## `Match3Sensor` and `Match3SensorComponent` classes
The `Match3Sensor` generates observations about the state using the `AbstractBoard` interface. You can
choose whether to use vector or "visual" observations; in theory, visual observations should perform
better because they are 2-dimensional like the board, but we need to experiment more on this.
A `Match3SensorComponent` generates a `Match3Sensor` at runtime, and should be added to the same GameObject
as your `Agent` implementation. You do not need to write any additional code to use them.
## `Match3Actuator` and `Match3ActuatorComponent` classes
The `Match3Actuator` converts actions from training or inference into a `Move` that is sent to` AbstractBoard.MakeMove()`
It also checks `AbstractBoard.IsMoveValid` for each potential move and uses this to set the action mask for Agent.
A `Match3ActuatorComponent` generates a `Match3Actuator` at runtime, and should be added to the same GameObject
as your `Agent` implementation. You do not need to write any additional code to use them.
# Setting up match-3 simulation
* Implement the `AbstractBoard` methods to integrate with your game.
* Give the `Agent` rewards when it does what you want it to (match multiple pieces in a row, clears pieces of a certain
type, etc).
* Add the `Agent`, `AbstractBoard` implementation, `Match3SensorComponent`, and `Match3ActuatorComponent` to the same
`GameObject`.
* Call `Agent.RequestDecision()` when you're ready for the `Agent` to make a move on the next `Academy` step. During
the next `Academy` step, the `MakeMove()` method on the board will be called.

3
com.unity.ml-agents.extensions/Runtime/Match3.meta


fileFormatVersion: 2
guid: 569f8fa2b7dd477c9b71f09e9d633832
timeCreated: 1600465975

3
com.unity.ml-agents.extensions/Tests/Editor/Match3.meta


fileFormatVersion: 2
guid: 77b0212dde404f7c8ce9aac13bd550b8
timeCreated: 1601332716

75
config/ppo/Match3.yaml


behaviors:
Match3VectorObs:
trainer_type: ppo
hyperparameters:
batch_size: 64
buffer_size: 12000
learning_rate: 0.0003
beta: 0.001
epsilon: 0.2
lambd: 0.99
num_epoch: 3
learning_rate_schedule: constant
network_settings:
normalize: true
hidden_units: 128
num_layers: 2
vis_encode_type: match3
reward_signals:
extrinsic:
gamma: 0.99
strength: 1.0
keep_checkpoints: 5
max_steps: 5000000
time_horizon: 1000
summary_freq: 10000
threaded: true
Match3VisualObs:
trainer_type: ppo
hyperparameters:
batch_size: 64
buffer_size: 12000
learning_rate: 0.0003
beta: 0.001
epsilon: 0.2
lambd: 0.99
num_epoch: 3
learning_rate_schedule: constant
network_settings:
normalize: true
hidden_units: 128
num_layers: 2
vis_encode_type: match3
reward_signals:
extrinsic:
gamma: 0.99
strength: 1.0
keep_checkpoints: 5
max_steps: 5000000
time_horizon: 1000
summary_freq: 10000
threaded: true
Match3SimpleHeuristic:
# Settings can be very simple since we don't care about actually training the model
trainer_type: ppo
hyperparameters:
batch_size: 64
buffer_size: 128
network_settings:
hidden_units: 4
num_layers: 1
max_steps: 5000000
summary_freq: 10000
threaded: true
Match3GreedyHeuristic:
# Settings can be very simple since we don't care about actually training the model
trainer_type: ppo
hyperparameters:
batch_size: 64
buffer_size: 128
network_settings:
hidden_units: 4
num_layers: 1
max_steps: 5000000
summary_freq: 10000
threaded: true

77
docs/images/match3.png

之前 之后
宽度: 297  |  高度: 320  |  大小: 22 KiB

8
Project/Assets/ML-Agents/Examples/Match3/Prefabs.meta


fileFormatVersion: 2
guid: 8519802844d8d4233b4c6f6758ab8322
folderAsset: yes
DefaultImporter:
externalObjects: {}
userData:
assetBundleName:
assetBundleVariant:

174
Project/Assets/ML-Agents/Examples/Match3/Prefabs/Match3Heuristic.prefab


%YAML 1.1
%TAG !u! tag:unity3d.com,2011:
--- !u!1 &3508723250470608007
GameObject:
m_ObjectHideFlags: 0
m_CorrespondingSourceObject: {fileID: 0}
m_PrefabInstance: {fileID: 0}
m_PrefabAsset: {fileID: 0}
serializedVersion: 6
m_Component:
- component: {fileID: 3508723250470608008}
- component: {fileID: 3508723250470608010}
- component: {fileID: 3508723250470608012}
- component: {fileID: 3508723250470608011}
- component: {fileID: 3508723250470608009}
- component: {fileID: 3508723250470608013}
- component: {fileID: 3508723250470608014}
m_Layer: 0
m_Name: Match3 Agent
m_TagString: Untagged
m_Icon: {fileID: 0}
m_NavMeshLayer: 0
m_StaticEditorFlags: 0
m_IsActive: 1
--- !u!4 &3508723250470608008
Transform:
m_ObjectHideFlags: 0
m_CorrespondingSourceObject: {fileID: 0}
m_PrefabInstance: {fileID: 0}
m_PrefabAsset: {fileID: 0}
m_GameObject: {fileID: 3508723250470608007}
m_LocalRotation: {x: 0, y: 0, z: 0, w: 1}
m_LocalPosition: {x: 0, y: 0, z: 0}
m_LocalScale: {x: 1, y: 1, z: 1}
m_Children: []
m_Father: {fileID: 3508723250774301920}
m_RootOrder: 0
m_LocalEulerAnglesHint: {x: 0, y: 0, z: 0}
--- !u!114 &3508723250470608010
MonoBehaviour:
m_ObjectHideFlags: 0
m_CorrespondingSourceObject: {fileID: 0}
m_PrefabInstance: {fileID: 0}
m_PrefabAsset: {fileID: 0}
m_GameObject: {fileID: 3508723250470608007}
m_Enabled: 1
m_EditorHideFlags: 0
m_Script: {fileID: 11500000, guid: 5d1c4e0b1822b495aa52bc52839ecb30, type: 3}
m_Name:
m_EditorClassIdentifier:
m_BrainParameters:
VectorObservationSize: 0
NumStackedVectorObservations: 1
VectorActionSize:
VectorActionDescriptions: []
VectorActionSpaceType: 0
m_Model: {fileID: 11400000, guid: c34da50737a3c4a50918002b20b2b927, type: 3}
m_InferenceDevice: 0
m_BehaviorType: 0
m_BehaviorName: Match3SmartHeuristic
TeamId: 0
m_UseChildSensors: 1
m_UseChildActuators: 1
m_ObservableAttributeHandling: 0
--- !u!114 &3508723250470608012
MonoBehaviour:
m_ObjectHideFlags: 0
m_CorrespondingSourceObject: {fileID: 0}
m_PrefabInstance: {fileID: 0}
m_PrefabAsset: {fileID: 0}
m_GameObject: {fileID: 3508723250470608007}
m_Enabled: 1
m_EditorHideFlags: 0
m_Script: {fileID: 11500000, guid: d982f0cd92214bd2b689be838fa40c44, type: 3}
m_Name:
m_EditorClassIdentifier:
agentParameters:
maxStep: 0
hasUpgradedFromAgentParameters: 1
MaxStep: 0
Board: {fileID: 0}
MoveTime: 0.25
MaxMoves: 500
UseSmartHeuristic: 1
--- !u!114 &3508723250470608011
MonoBehaviour:
m_ObjectHideFlags: 0
m_CorrespondingSourceObject: {fileID: 0}
m_PrefabInstance: {fileID: 0}
m_PrefabAsset: {fileID: 0}
m_GameObject: {fileID: 3508723250470608007}
m_Enabled: 1
m_EditorHideFlags: 0
m_Script: {fileID: 11500000, guid: abebb7ad4a5547d7a3b04373784ff195, type: 3}
m_Name:
m_EditorClassIdentifier:
DebugEdgeIndex: -1
--- !u!114 &3508723250470608009
MonoBehaviour:
m_ObjectHideFlags: 0
m_CorrespondingSourceObject: {fileID: 0}
m_PrefabInstance: {fileID: 0}
m_PrefabAsset: {fileID: 0}
m_GameObject: {fileID: 3508723250470608007}
m_Enabled: 1
m_EditorHideFlags: 0
m_Script: {fileID: 11500000, guid: 6d852a063770348b68caa91b8e7642a5, type: 3}
m_Name:
m_EditorClassIdentifier:
Rows: 9
Columns: 8
NumCellTypes: 6
NumSpecialTypes: 2
RandomSeed: -1
BasicCellPoints: 1
SpecialCell1Points: 2
SpecialCell2Points: 3
--- !u!114 &3508723250470608013
MonoBehaviour:
m_ObjectHideFlags: 0
m_CorrespondingSourceObject: {fileID: 0}
m_PrefabInstance: {fileID: 0}
m_PrefabAsset: {fileID: 0}
m_GameObject: {fileID: 3508723250470608007}
m_Enabled: 1
m_EditorHideFlags: 0
m_Script: {fileID: 11500000, guid: 08e4b0da54cb4d56bfcbae22dd49ab8d, type: 3}
m_Name:
m_EditorClassIdentifier:
ForceHeuristic: 1
--- !u!114 &3508723250470608014
MonoBehaviour:
m_ObjectHideFlags: 0
m_CorrespondingSourceObject: {fileID: 0}
m_PrefabInstance: {fileID: 0}
m_PrefabAsset: {fileID: 0}
m_GameObject: {fileID: 3508723250470608007}
m_Enabled: 1
m_EditorHideFlags: 0
m_Script: {fileID: 11500000, guid: 530d2f105aa145bd8a00e021bdd925fd, type: 3}
m_Name:
m_EditorClassIdentifier:
ObservationType: 0
--- !u!1 &3508723250774301855
GameObject:
m_ObjectHideFlags: 0
m_CorrespondingSourceObject: {fileID: 0}
m_PrefabInstance: {fileID: 0}
m_PrefabAsset: {fileID: 0}
serializedVersion: 6
m_Component:
- component: {fileID: 3508723250774301920}
m_Layer: 0
m_Name: Match3Heuristic
m_TagString: Untagged
m_Icon: {fileID: 0}
m_NavMeshLayer: 0
m_StaticEditorFlags: 0
m_IsActive: 1
--- !u!4 &3508723250774301920
Transform:
m_ObjectHideFlags: 0
m_CorrespondingSourceObject: {fileID: 0}
m_PrefabInstance: {fileID: 0}
m_PrefabAsset: {fileID: 0}
m_GameObject: {fileID: 3508723250774301855}
m_LocalRotation: {x: 0, y: 0, z: 0, w: 1}
m_LocalPosition: {x: 0, y: 0, z: 0}
m_LocalScale: {x: 1, y: 1, z: 1}
m_Children:
- {fileID: 3508723250470608008}
m_Father: {fileID: 0}
m_RootOrder: 0
m_LocalEulerAnglesHint: {x: 0, y: 0, z: 0}

7
Project/Assets/ML-Agents/Examples/Match3/Prefabs/Match3Heuristic.prefab.meta


fileFormatVersion: 2
guid: 2fafdcd0587684641b03b11f04454f1b
PrefabImporter:
externalObjects: {}
userData:
assetBundleName:
assetBundleVariant:

170
Project/Assets/ML-Agents/Examples/Match3/Prefabs/Match3VectorObs.prefab


%YAML 1.1
%TAG !u! tag:unity3d.com,2011:
--- !u!1 &2118285883905619929
GameObject:
m_ObjectHideFlags: 0
m_CorrespondingSourceObject: {fileID: 0}
m_PrefabInstance: {fileID: 0}
m_PrefabAsset: {fileID: 0}
serializedVersion: 6
m_Component:
- component: {fileID: 2118285883905619878}
m_Layer: 0
m_Name: Match3VectorObs
m_TagString: Untagged
m_Icon: {fileID: 0}
m_NavMeshLayer: 0
m_StaticEditorFlags: 0
m_IsActive: 1
--- !u!4 &2118285883905619878
Transform:
m_ObjectHideFlags: 0
m_CorrespondingSourceObject: {fileID: 0}
m_PrefabInstance: {fileID: 0}
m_PrefabAsset: {fileID: 0}
m_GameObject: {fileID: 2118285883905619929}
m_LocalRotation: {x: 0, y: 0, z: 0, w: 1}
m_LocalPosition: {x: 0, y: 0, z: 0}
m_LocalScale: {x: 1, y: 1, z: 1}
m_Children:
- {fileID: 2118285884327540686}
m_Father: {fileID: 0}
m_RootOrder: 0
m_LocalEulerAnglesHint: {x: 0, y: 0, z: 0}
--- !u!1 &2118285884327540673
GameObject:
m_ObjectHideFlags: 0
m_CorrespondingSourceObject: {fileID: 0}
m_PrefabInstance: {fileID: 0}
m_PrefabAsset: {fileID: 0}
serializedVersion: 6
m_Component:
- component: {fileID: 2118285884327540686}
- component: {fileID: 2118285884327540684}
- component: {fileID: 2118285884327540682}
- component: {fileID: 2118285884327540685}
- component: {fileID: 2118285884327540687}
- component: {fileID: 2118285884327540683}
- component: {fileID: 2118285884327540680}
m_Layer: 0
m_Name: Match3 Agent
m_TagString: Untagged
m_Icon: {fileID: 0}
m_NavMeshLayer: 0
m_StaticEditorFlags: 0
m_IsActive: 1
--- !u!4 &2118285884327540686
Transform:
m_ObjectHideFlags: 0
m_CorrespondingSourceObject: {fileID: 0}
m_PrefabInstance: {fileID: 0}
m_PrefabAsset: {fileID: 0}
m_GameObject: {fileID: 2118285884327540673}
m_LocalRotation: {x: 0, y: 0, z: 0, w: 1}
m_LocalPosition: {x: 0, y: 0, z: 0}
m_LocalScale: {x: 1, y: 1, z: 1}
m_Children: []
m_Father: {fileID: 2118285883905619878}
m_RootOrder: 0
m_LocalEulerAnglesHint: {x: 0, y: 0, z: 0}
--- !u!114 &2118285884327540684
MonoBehaviour:
m_ObjectHideFlags: 0
m_CorrespondingSourceObject: {fileID: 0}
m_PrefabInstance: {fileID: 0}
m_PrefabAsset: {fileID: 0}
m_GameObject: {fileID: 2118285884327540673}
m_Enabled: 1
m_EditorHideFlags: 0
m_Script: {fileID: 11500000, guid: 5d1c4e0b1822b495aa52bc52839ecb30, type: 3}
m_Name:
m_EditorClassIdentifier:
m_BrainParameters:
VectorObservationSize: 0
NumStackedVectorObservations: 1
VectorActionSize:
VectorActionDescriptions: []
VectorActionSpaceType: 0
m_Model: {fileID: 11400000, guid: 9e89b8e81974148d3b7213530d00589d, type: 3}
m_InferenceDevice: 0
m_BehaviorType: 0
m_BehaviorName: Match3VectorObs
TeamId: 0
m_UseChildSensors: 1
m_UseChildActuators: 1
m_ObservableAttributeHandling: 0
--- !u!114 &2118285884327540682
MonoBehaviour:
m_ObjectHideFlags: 0
m_CorrespondingSourceObject: {fileID: 0}
m_PrefabInstance: {fileID: 0}
m_PrefabAsset: {fileID: 0}
m_GameObject: {fileID: 2118285884327540673}
m_Enabled: 1
m_EditorHideFlags: 0
m_Script: {fileID: 11500000, guid: d982f0cd92214bd2b689be838fa40c44, type: 3}
m_Name:
m_EditorClassIdentifier:
agentParameters:
maxStep: 0
hasUpgradedFromAgentParameters: 1
MaxStep: 0
Board: {fileID: 0}
MoveTime: 0.25
MaxMoves: 500
--- !u!114 &2118285884327540685
MonoBehaviour:
m_ObjectHideFlags: 0
m_CorrespondingSourceObject: {fileID: 0}
m_PrefabInstance: {fileID: 0}
m_PrefabAsset: {fileID: 0}
m_GameObject: {fileID: 2118285884327540673}
m_Enabled: 1
m_EditorHideFlags: 0
m_Script: {fileID: 11500000, guid: abebb7ad4a5547d7a3b04373784ff195, type: 3}
m_Name:
m_EditorClassIdentifier:
DebugEdgeIndex: -1
--- !u!114 &2118285884327540687
MonoBehaviour:
m_ObjectHideFlags: 0
m_CorrespondingSourceObject: {fileID: 0}
m_PrefabInstance: {fileID: 0}
m_PrefabAsset: {fileID: 0}
m_GameObject: {fileID: 2118285884327540673}
m_Enabled: 1
m_EditorHideFlags: 0
m_Script: {fileID: 11500000, guid: 6d852a063770348b68caa91b8e7642a5, type: 3}
m_Name:
m_EditorClassIdentifier:
Rows: 9
Columns: 8
NumCellTypes: 6
NumSpecialTypes: 2
RandomSeed: -1
--- !u!114 &2118285884327540683
MonoBehaviour:
m_ObjectHideFlags: 0
m_CorrespondingSourceObject: {fileID: 0}
m_PrefabInstance: {fileID: 0}
m_PrefabAsset: {fileID: 0}
m_GameObject: {fileID: 2118285884327540673}
m_Enabled: 1
m_EditorHideFlags: 0
m_Script: {fileID: 11500000, guid: 08e4b0da54cb4d56bfcbae22dd49ab8d, type: 3}
m_Name:
m_EditorClassIdentifier:
ForceRandom: 0
--- !u!114 &2118285884327540680
MonoBehaviour:
m_ObjectHideFlags: 0
m_CorrespondingSourceObject: {fileID: 0}
m_PrefabInstance: {fileID: 0}
m_PrefabAsset: {fileID: 0}
m_GameObject: {fileID: 2118285884327540673}
m_Enabled: 1
m_EditorHideFlags: 0
m_Script: {fileID: 11500000, guid: 530d2f105aa145bd8a00e021bdd925fd, type: 3}
m_Name:
m_EditorClassIdentifier:
ObservationType: 0

7
Project/Assets/ML-Agents/Examples/Match3/Prefabs/Match3VectorObs.prefab.meta


fileFormatVersion: 2
guid: 6944ca02359f5427aa13c8551236a824
PrefabImporter:
externalObjects: {}
userData:
assetBundleName:
assetBundleVariant:

170
Project/Assets/ML-Agents/Examples/Match3/Prefabs/Match3VisualObs.prefab


%YAML 1.1
%TAG !u! tag:unity3d.com,2011:
--- !u!1 &3019509691567202678
GameObject:
m_ObjectHideFlags: 0
m_CorrespondingSourceObject: {fileID: 0}
m_PrefabInstance: {fileID: 0}
m_PrefabAsset: {fileID: 0}
serializedVersion: 6
m_Component:
- component: {fileID: 3019509691567202569}
m_Layer: 0
m_Name: Match3VisualObs
m_TagString: Untagged
m_Icon: {fileID: 0}
m_NavMeshLayer: 0
m_StaticEditorFlags: 0
m_IsActive: 1
--- !u!4 &3019509691567202569
Transform:
m_ObjectHideFlags: 0
m_CorrespondingSourceObject: {fileID: 0}
m_PrefabInstance: {fileID: 0}
m_PrefabAsset: {fileID: 0}
m_GameObject: {fileID: 3019509691567202678}
m_LocalRotation: {x: 0, y: 0, z: 0, w: 1}
m_LocalPosition: {x: 0, y: 0, z: 0}
m_LocalScale: {x: 1, y: 1, z: 1}
m_Children:
- {fileID: 3019509692332007777}
m_Father: {fileID: 0}
m_RootOrder: 0
m_LocalEulerAnglesHint: {x: 0, y: 0, z: 0}
--- !u!1 &3019509692332007790
GameObject:
m_ObjectHideFlags: 0
m_CorrespondingSourceObject: {fileID: 0}
m_PrefabInstance: {fileID: 0}
m_PrefabAsset: {fileID: 0}
serializedVersion: 6
m_Component:
- component: {fileID: 3019509692332007777}
- component: {fileID: 3019509692332007779}
- component: {fileID: 3019509692332007781}
- component: {fileID: 3019509692332007778}
- component: {fileID: 3019509692332007776}
- component: {fileID: 3019509692332007780}
- component: {fileID: 3019509692332007783}
m_Layer: 0
m_Name: Match3 Agent
m_TagString: Untagged
m_Icon: {fileID: 0}
m_NavMeshLayer: 0
m_StaticEditorFlags: 0
m_IsActive: 1
--- !u!4 &3019509692332007777
Transform:
m_ObjectHideFlags: 0
m_CorrespondingSourceObject: {fileID: 0}
m_PrefabInstance: {fileID: 0}
m_PrefabAsset: {fileID: 0}
m_GameObject: {fileID: 3019509692332007790}
m_LocalRotation: {x: 0, y: 0, z: 0, w: 1}
m_LocalPosition: {x: 0, y: 0, z: 0}
m_LocalScale: {x: 1, y: 1, z: 1}
m_Children: []
m_Father: {fileID: 3019509691567202569}
m_RootOrder: 0
m_LocalEulerAnglesHint: {x: 0, y: 0, z: 0}
--- !u!114 &3019509692332007779
MonoBehaviour:
m_ObjectHideFlags: 0
m_CorrespondingSourceObject: {fileID: 0}
m_PrefabInstance: {fileID: 0}
m_PrefabAsset: {fileID: 0}
m_GameObject: {fileID: 3019509692332007790}
m_Enabled: 1
m_EditorHideFlags: 0
m_Script: {fileID: 11500000, guid: 5d1c4e0b1822b495aa52bc52839ecb30, type: 3}
m_Name:
m_EditorClassIdentifier:
m_BrainParameters:
VectorObservationSize: 0
NumStackedVectorObservations: 1
VectorActionSize:
VectorActionDescriptions: []
VectorActionSpaceType: 0
m_Model: {fileID: 11400000, guid: 48d14da88fea74d0693c691c6e3f2e34, type: 3}
m_InferenceDevice: 0
m_BehaviorType: 0
m_BehaviorName: Match3VisualObs
TeamId: 0
m_UseChildSensors: 1
m_UseChildActuators: 1
m_ObservableAttributeHandling: 0
--- !u!114 &3019509692332007781
MonoBehaviour:
m_ObjectHideFlags: 0
m_CorrespondingSourceObject: {fileID: 0}
m_PrefabInstance: {fileID: 0}
m_PrefabAsset: {fileID: 0}
m_GameObject: {fileID: 3019509692332007790}
m_Enabled: 1
m_EditorHideFlags: 0
m_Script: {fileID: 11500000, guid: d982f0cd92214bd2b689be838fa40c44, type: 3}
m_Name:
m_EditorClassIdentifier:
agentParameters:
maxStep: 0
hasUpgradedFromAgentParameters: 1
MaxStep: 0
Board: {fileID: 0}
MoveTime: 0.25
MaxMoves: 500
--- !u!114 &3019509692332007778
MonoBehaviour:
m_ObjectHideFlags: 0
m_CorrespondingSourceObject: {fileID: 0}
m_PrefabInstance: {fileID: 0}
m_PrefabAsset: {fileID: 0}
m_GameObject: {fileID: 3019509692332007790}
m_Enabled: 1
m_EditorHideFlags: 0
m_Script: {fileID: 11500000, guid: abebb7ad4a5547d7a3b04373784ff195, type: 3}
m_Name:
m_EditorClassIdentifier:
DebugEdgeIndex: -1
--- !u!114 &3019509692332007776
MonoBehaviour:
m_ObjectHideFlags: 0
m_CorrespondingSourceObject: {fileID: 0}
m_PrefabInstance: {fileID: 0}
m_PrefabAsset: {fileID: 0}
m_GameObject: {fileID: 3019509692332007790}
m_Enabled: 1
m_EditorHideFlags: 0
m_Script: {fileID: 11500000, guid: 6d852a063770348b68caa91b8e7642a5, type: 3}
m_Name:
m_EditorClassIdentifier:
Rows: 9
Columns: 8
NumCellTypes: 6
NumSpecialTypes: 2
RandomSeed: -1
--- !u!114 &3019509692332007780
MonoBehaviour:
m_ObjectHideFlags: 0
m_CorrespondingSourceObject: {fileID: 0}
m_PrefabInstance: {fileID: 0}
m_PrefabAsset: {fileID: 0}
m_GameObject: {fileID: 3019509692332007790}
m_Enabled: 1
m_EditorHideFlags: 0
m_Script: {fileID: 11500000, guid: 08e4b0da54cb4d56bfcbae22dd49ab8d, type: 3}
m_Name:
m_EditorClassIdentifier:
ForceRandom: 0
--- !u!114 &3019509692332007783
MonoBehaviour:
m_ObjectHideFlags: 0
m_CorrespondingSourceObject: {fileID: 0}
m_PrefabInstance: {fileID: 0}
m_PrefabAsset: {fileID: 0}
m_GameObject: {fileID: 3019509692332007790}
m_Enabled: 1
m_EditorHideFlags: 0
m_Script: {fileID: 11500000, guid: 530d2f105aa145bd8a00e021bdd925fd, type: 3}
m_Name:
m_EditorClassIdentifier:
ObservationType: 2

7
Project/Assets/ML-Agents/Examples/Match3/Prefabs/Match3VisualObs.prefab.meta


fileFormatVersion: 2
guid: aaa471bd5e2014848a66917476671aed
PrefabImporter:
externalObjects: {}
userData:
assetBundleName:
assetBundleVariant:

8
Project/Assets/ML-Agents/Examples/Match3/Scenes.meta


fileFormatVersion: 2
guid: e033fb0df67684ebf961ed115870ff10
folderAsset: yes
DefaultImporter:
externalObjects: {}
userData:
assetBundleName:
assetBundleVariant:

1001
Project/Assets/ML-Agents/Examples/Match3/Scenes/Match3.unity
文件差异内容过多而无法显示
查看文件

7
Project/Assets/ML-Agents/Examples/Match3/Scenes/Match3.unity.meta


fileFormatVersion: 2
guid: 2e09c5458f1494f9dad9cd6d09dff964
DefaultImporter:
externalObjects: {}
userData:
assetBundleName:
assetBundleVariant:

8
Project/Assets/ML-Agents/Examples/Match3/Scripts.meta


fileFormatVersion: 2
guid: be7a27f4291944d3dba4f696e1af4209
folderAsset: yes
DefaultImporter:
externalObjects: {}
userData:
assetBundleName:
assetBundleVariant:

373
Project/Assets/ML-Agents/Examples/Match3/Scripts/Match3Agent.cs


using System;
using UnityEngine;
using Unity.MLAgents;
using Unity.MLAgents.Actuators;
using Unity.MLAgents.Extensions.Match3;
namespace Unity.MLAgentsExamples
{
/// <summary>
/// State of the "game" when showing all steps of the simulation. This is only used outside of training.
/// The state diagram is
///
/// | <--------------------------------------- ^
/// | |
/// v |
/// +--------+ +-------+ +-----+ +------+
/// |Find | ---> |Clear | ---> |Drop | ---> |Fill |
/// |Matches | |Matched| | | |Empty |
/// +--------+ +-------+ +-----+ +------+
///
/// | ^
/// | |
/// v |
///
/// +--------+
/// |Wait for|
/// |Move |
/// +--------+
///
/// The stats advances each "MoveTime" seconds.
/// </summary>
enum State
{
/// <summary>
/// Guard value, should never happen.
/// </summary>
Invalid = -1,
/// <summary>
/// Look for matches. If there are matches, the next state is ClearMatched, otherwise WaitForMove.
/// </summary>
FindMatches = 0,
/// <summary>
/// Remove matched cells and replace them with a placeholder value.
/// </summary>
ClearMatched = 1,
/// <summary>
/// Move cells "down" to fill empty space.
/// </summary>
Drop = 2,
/// <summary>
/// Replace empty cells with new random values.
/// </summary>
FillEmpty = 3,
/// <summary>
/// Request a move from the Agent.
/// </summary>
WaitForMove = 4,
}
public enum HeuristicQuality
{
/// <summary>
/// The heuristic will pick any valid move at random.
/// </summary>
RandomValidMove,
/// <summary>
/// The heuristic will pick the move that scores the most points.
/// This only looks at the immediate move, and doesn't consider where cells will fall.
/// </summary>
Greedy
}
public class Match3Agent : Agent
{
[HideInInspector]
public Match3Board Board;
public float MoveTime = 1.0f;
public int MaxMoves = 500;
public HeuristicQuality HeuristicQuality = HeuristicQuality.RandomValidMove;
State m_CurrentState = State.WaitForMove;
float m_TimeUntilMove;
private int m_MovesMade;
private System.Random m_Random;
private const float k_RewardMultiplier = 0.01f;
void Awake()
{
Board = GetComponent<Match3Board>();
var seed = Board.RandomSeed == -1 ? gameObject.GetInstanceID() : Board.RandomSeed + 1;
m_Random = new System.Random(seed);
}
public override void OnEpisodeBegin()
{
base.OnEpisodeBegin();
Board.InitSettled();
m_CurrentState = State.FindMatches;
m_TimeUntilMove = MoveTime;
m_MovesMade = 0;
}
private void FixedUpdate()
{
if (Academy.Instance.IsCommunicatorOn)
{
FastUpdate();
}
else
{
AnimatedUpdate();
}
// We can't use the normal MaxSteps system to decide when to end an episode,
// since different agents will make moves at different frequencies (depending on the number of
// chained moves). So track a number of moves per Agent and manually interrupt the episode.
if (m_MovesMade >= MaxMoves)
{
EpisodeInterrupted();
}
}
void FastUpdate()
{
while (true)
{
var hasMatched = Board.MarkMatchedCells();
if (!hasMatched)
{
break;
}
var pointsEarned = Board.ClearMatchedCells();
AddReward(k_RewardMultiplier * pointsEarned);
Board.DropCells();
Board.FillFromAbove();
}
while (!HasValidMoves())
{
// Shuffle the board until we have a valid move.
Board.InitSettled();
}
RequestDecision();
m_MovesMade++;
}
void AnimatedUpdate()
{
m_TimeUntilMove -= Time.deltaTime;
if (m_TimeUntilMove > 0.0f)
{
return;
}
m_TimeUntilMove = MoveTime;
var nextState = State.Invalid;
switch (m_CurrentState)
{
case State.FindMatches:
var hasMatched = Board.MarkMatchedCells();
nextState = hasMatched ? State.ClearMatched : State.WaitForMove;
if (nextState == State.WaitForMove)
{
m_MovesMade++;
}
break;
case State.ClearMatched:
var pointsEarned = Board.ClearMatchedCells();
AddReward(k_RewardMultiplier * pointsEarned);
nextState = State.Drop;
break;
case State.Drop:
Board.DropCells();
nextState = State.FillEmpty;
break;
case State.FillEmpty:
Board.FillFromAbove();
nextState = State.FindMatches;
break;
case State.WaitForMove:
while (true)
{
// Shuffle the board until we have a valid move.
bool hasMoves = HasValidMoves();
if (hasMoves)
{
break;
}
Board.InitSettled();
}
RequestDecision();
nextState = State.FindMatches;
break;
default:
throw new ArgumentOutOfRangeException();
}
m_CurrentState = nextState;
}
bool HasValidMoves()
{
foreach (var move in Board.ValidMoves())
{
return true;
}
return false;
}
public override void Heuristic(in ActionBuffers actionsOut)
{
var discreteActions = actionsOut.DiscreteActions;
discreteActions[0] = GreedyMove();
}
int GreedyMove()
{
var pointsByType = new[] { Board.BasicCellPoints, Board.SpecialCell1Points, Board.SpecialCell2Points };
var bestMoveIndex = 0;
var bestMovePoints = -1;
var numMovesAtCurrentScore = 0;
foreach (var move in Board.ValidMoves())
{
var movePoints = HeuristicQuality == HeuristicQuality.Greedy ? EvalMovePoints(move, pointsByType) : 1;
if (movePoints < bestMovePoints)
{
// Worse, skip
continue;
}
if (movePoints > bestMovePoints)
{
// Better, keep
bestMovePoints = movePoints;
bestMoveIndex = move.MoveIndex;
numMovesAtCurrentScore = 1;
}
else
{
// Tied for best - use reservoir sampling to make sure we select from equal moves uniformly.
// See https://en.wikipedia.org/wiki/Reservoir_sampling#Simple_algorithm
numMovesAtCurrentScore++;
var randVal = m_Random.Next(0, numMovesAtCurrentScore);
if (randVal == 0)
{
// Keep the new one
bestMoveIndex = move.MoveIndex;
}
}
}
return bestMoveIndex;
}
int EvalMovePoints(Move move, int[] pointsByType)
{
// Counts the expected points for making the move.
var moveVal = Board.GetCellType(move.Row, move.Column);
var moveSpecial = Board.GetSpecialType(move.Row, move.Column);
var (otherRow, otherCol) = move.OtherCell();
var oppositeVal = Board.GetCellType(otherRow, otherCol);
var oppositeSpecial = Board.GetSpecialType(otherRow, otherCol);
int movePoints = EvalHalfMove(
otherRow, otherCol, moveVal, moveSpecial, move.Direction, pointsByType
);
int otherPoints = EvalHalfMove(
move.Row, move.Column, oppositeVal, oppositeSpecial, move.OtherDirection(), pointsByType
);
return movePoints + otherPoints;
}
int EvalHalfMove(int newRow, int newCol, int newValue, int newSpecial, Direction incomingDirection, int[] pointsByType)
{
// This is a essentially a duplicate of AbstractBoard.CheckHalfMove but also counts the points for the move.
int matchedLeft = 0, matchedRight = 0, matchedUp = 0, matchedDown = 0;
int scoreLeft = 0, scoreRight = 0, scoreUp = 0, scoreDown = 0;
if (incomingDirection != Direction.Right)
{
for (var c = newCol - 1; c >= 0; c--)
{
if (Board.GetCellType(newRow, c) == newValue)
{
matchedLeft++;
scoreLeft += pointsByType[Board.GetSpecialType(newRow, c)];
}
else
break;
}
}
if (incomingDirection != Direction.Left)
{
for (var c = newCol + 1; c < Board.Columns; c++)
{
if (Board.GetCellType(newRow, c) == newValue)
{
matchedRight++;
scoreRight += pointsByType[Board.GetSpecialType(newRow, c)];
}
else
break;
}
}
if (incomingDirection != Direction.Down)
{
for (var r = newRow + 1; r < Board.Rows; r++)
{
if (Board.GetCellType(r, newCol) == newValue)
{
matchedUp++;
scoreUp += pointsByType[Board.GetSpecialType(r, newCol)];
}
else
break;
}
}
if (incomingDirection != Direction.Up)
{
for (var r = newRow - 1; r >= 0; r--)
{
if (Board.GetCellType(r, newCol) == newValue)
{
matchedDown++;
scoreDown += pointsByType[Board.GetSpecialType(r, newCol)];
}
else
break;
}
}
if ((matchedUp + matchedDown >= 2) || (matchedLeft + matchedRight >= 2))
{
// It's a match. Start from counting the piece being moved
var totalScore = pointsByType[newSpecial];
if (matchedUp + matchedDown >= 2)
{
totalScore += scoreUp + scoreDown;
}
if (matchedLeft + matchedRight >= 2)
{
totalScore += scoreLeft + scoreRight;
}
return totalScore;
}
return 0;
}
}
}

3
Project/Assets/ML-Agents/Examples/Match3/Scripts/Match3Agent.cs.meta


fileFormatVersion: 2
guid: d982f0cd92214bd2b689be838fa40c44
timeCreated: 1598221207

272
Project/Assets/ML-Agents/Examples/Match3/Scripts/Match3Board.cs


using Unity.MLAgents.Extensions.Match3;
using UnityEngine;
namespace Unity.MLAgentsExamples
{
public class Match3Board : AbstractBoard
{
public int RandomSeed = -1;
public const int k_EmptyCell = -1;
[Tooltip("Points earned for clearing a basic cell (cube)")]
public int BasicCellPoints = 1;
[Tooltip("Points earned for clearing a special cell (sphere)")]
public int SpecialCell1Points = 2;
[Tooltip("Points earned for clearing an extra special cell (plus)")]
public int SpecialCell2Points = 3;
(int, int)[,] m_Cells;
bool[,] m_Matched;
System.Random m_Random;
void Awake()
{
m_Cells = new (int, int)[Columns, Rows];
m_Matched = new bool[Columns, Rows];
m_Random = new System.Random(RandomSeed == -1 ? gameObject.GetInstanceID() : RandomSeed);
InitRandom();
}
public override bool MakeMove(Move move)
{
if (!IsMoveValid(move))
{
return false;
}
var originalValue = m_Cells[move.Column, move.Row];
var (otherRow, otherCol) = move.OtherCell();
var destinationValue = m_Cells[otherCol, otherRow];
m_Cells[move.Column, move.Row] = destinationValue;
m_Cells[otherCol, otherRow] = originalValue;
return true;
}
public override int GetCellType(int row, int col)
{
return m_Cells[col, row].Item1;
}
public override int GetSpecialType(int row, int col)
{
return m_Cells[col, row].Item2;
}
public override bool IsMoveValid(Move m)
{
if (m_Cells == null)
{
return false;
}
return SimpleIsMoveValid(m);
}
public bool MarkMatchedCells(int[,] cells = null)
{
ClearMarked();
bool madeMatch = false;
for (var i = 0; i < Rows; i++)
{
for (var j = 0; j < Columns; j++)
{
// Check vertically
var matchedRows = 0;
for (var iOffset = i; iOffset < Rows; iOffset++)
{
if (m_Cells[j, i].Item1 != m_Cells[j, iOffset].Item1)
{
break;
}
matchedRows++;
}
if (matchedRows >= 3)
{
madeMatch = true;
for (var k = 0; k < matchedRows; k++)
{
m_Matched[j, i + k] = true;
}
}
// Check vertically
var matchedCols = 0;
for (var jOffset = j; jOffset < Columns; jOffset++)
{
if (m_Cells[j, i].Item1 != m_Cells[jOffset, i].Item1)
{
break;
}
matchedCols++;
}
if (matchedCols >= 3)
{
madeMatch = true;
for (var k = 0; k < matchedCols; k++)
{
m_Matched[j + k, i] = true;
}
}
}
}
return madeMatch;
}
/// <summary>
/// Sets cells that are matched to the empty cell, and returns the score earned.
/// </summary>
/// <returns></returns>
public int ClearMatchedCells()
{
var pointsByType = new[] { BasicCellPoints, SpecialCell1Points, SpecialCell2Points };
int pointsEarned = 0;
for (var i = 0; i < Rows; i++)
{
for (var j = 0; j < Columns; j++)
{
if (m_Matched[j, i])
{
var speciaType = GetSpecialType(i, j);
pointsEarned += pointsByType[speciaType];
m_Cells[j, i] = (k_EmptyCell, 0);
}
}
}
ClearMarked(); // TODO clear here or at start of matching?
return pointsEarned;
}
public bool DropCells()
{
var madeChanges = false;
// Gravity is applied in the negative row direction
for (var j = 0; j < Columns; j++)
{
var writeIndex = 0;
for (var readIndex = 0; readIndex < Rows; readIndex++)
{
m_Cells[j, writeIndex] = m_Cells[j, readIndex];
if (m_Cells[j, readIndex].Item1 != k_EmptyCell)
{
writeIndex++;
}
}
// Fill in empties at the end
for (; writeIndex < Rows; writeIndex++)
{
madeChanges = true;
m_Cells[j, writeIndex] = (k_EmptyCell, 0);
}
}
return madeChanges;
}
public bool FillFromAbove()
{
bool madeChanges = false;
for (var i = 0; i < Rows; i++)
{
for (var j = 0; j < Columns; j++)
{
if (m_Cells[j, i].Item1 == k_EmptyCell)
{
madeChanges = true;
m_Cells[j, i] = (GetRandomCellType(), GetRandomSpecialType());
}
}
}
return madeChanges;
}
public (int, int)[,] Cells
{
get { return m_Cells; }
}
public bool[,] Matched
{
get { return m_Matched; }
}
// Initialize the board to random values.
public void InitRandom()
{
for (var i = 0; i < Rows; i++)
{
for (var j = 0; j < Columns; j++)
{
m_Cells[j, i] = (GetRandomCellType(), GetRandomSpecialType());
}
}
}
public void InitSettled()
{
InitRandom();
while (true)
{
var anyMatched = MarkMatchedCells();
if (!anyMatched)
{
return;
}
ClearMatchedCells();
DropCells();
FillFromAbove();
}
}
void ClearMarked()
{
for (var i = 0; i < Rows; i++)
{
for (var j = 0; j < Columns; j++)
{
m_Matched[j, i] = false;
}
}
}
int GetRandomCellType()
{
return m_Random.Next(0, NumCellTypes);
}
int GetRandomSpecialType()
{
// 1 in N chance to get a type-2 special
// 2 in N chance to get a type-1 special
// otherwise 0 (boring)
var N = 10;
var val = m_Random.Next(0, N);
if (val == 0)
{
return 2;
}
if (val <= 2)
{
return 1;
}
return 0;
}
}
}

11
Project/Assets/ML-Agents/Examples/Match3/Scripts/Match3Board.cs.meta


fileFormatVersion: 2
guid: 6d852a063770348b68caa91b8e7642a5
MonoImporter:
externalObjects: {}
serializedVersion: 2
defaultReferences: []
executionOrder: 0
icon: {instanceID: 0}
userData:
assetBundleName:
assetBundleVariant:

102
Project/Assets/ML-Agents/Examples/Match3/Scripts/Match3Drawer.cs


using UnityEngine;
using Unity.MLAgents.Extensions.Match3;
namespace Unity.MLAgentsExamples
{
public class Match3Drawer : MonoBehaviour
{
public int DebugMoveIndex = -1;
static Color[] s_Colors = new[]
{
Color.red,
Color.green,
Color.blue,
Color.cyan,
Color.magenta,
Color.yellow,
Color.gray,
Color.black,
};
private static Color s_EmptyColor = new Color(0.5f, 0.5f, 0.5f, .25f);
void OnDrawGizmos()
{
// TODO replace Gizmos for drawing the game state with proper GameObjects and animations.
var cubeSize = .5f;
var cubeSpacing = .75f;
var matchedWireframeSize = .5f * (cubeSize + cubeSpacing);
var board = GetComponent<Match3Board>();
if (board == null)
{
return;
}
for (var i = 0; i < board.Rows; i++)
{
for (var j = 0; j < board.Columns; j++)
{
var value = board.Cells != null ? board.GetCellType(i, j) : Match3Board.k_EmptyCell;
if (value >= 0 && value < s_Colors.Length)
{
Gizmos.color = s_Colors[value];
}
else
{
Gizmos.color = s_EmptyColor;
}
var pos = new Vector3(j, i, 0);
pos *= cubeSpacing;
var specialType = board.Cells != null ? board.GetSpecialType(i, j) : 0;
if (specialType == 2)
{
Gizmos.DrawCube(transform.TransformPoint(pos), cubeSize * new Vector3(1f, .5f, .5f));
Gizmos.DrawCube(transform.TransformPoint(pos), cubeSize * new Vector3(.5f, 1f, .5f));
Gizmos.DrawCube(transform.TransformPoint(pos), cubeSize * new Vector3(.5f, .5f, 1f));
}
else if (specialType == 1)
{
Gizmos.DrawSphere(transform.TransformPoint(pos), .5f * cubeSize);
}
else
{
Gizmos.DrawCube(transform.TransformPoint(pos), cubeSize * Vector3.one);
}
Gizmos.color = Color.yellow;
if (board.Matched != null && board.Matched[j, i])
{
Gizmos.DrawWireCube(transform.TransformPoint(pos), matchedWireframeSize * Vector3.one);
}
}
}
// Draw valid moves
foreach (var move in board.AllMoves())
{
if (DebugMoveIndex >= 0 && move.MoveIndex != DebugMoveIndex)
{
continue;
}
if (!board.IsMoveValid(move))
{
continue;
}
var (otherRow, otherCol) = move.OtherCell();
var pos = new Vector3(move.Column, move.Row, 0) * cubeSpacing;
var otherPos = new Vector3(otherCol, otherRow, 0) * cubeSpacing;
var oneQuarter = Vector3.Lerp(pos, otherPos, .25f);
var threeQuarters = Vector3.Lerp(pos, otherPos, .75f);
Gizmos.DrawLine(transform.TransformPoint(oneQuarter), transform.TransformPoint(threeQuarters));
}
}
}
}

3
Project/Assets/ML-Agents/Examples/Match3/Scripts/Match3Drawer.cs.meta


fileFormatVersion: 2
guid: abebb7ad4a5547d7a3b04373784ff195
timeCreated: 1598221188

8
Project/Assets/ML-Agents/Examples/Match3/TFModels.meta


fileFormatVersion: 2
guid: 504c8f923fdf448e795936f2900a5fd4
folderAsset: yes
DefaultImporter:
externalObjects: {}
userData:
assetBundleName:
assetBundleVariant:

1001
Project/Assets/ML-Agents/Examples/Match3/TFModels/Match3VectorObs.onnx
文件差异内容过多而无法显示
查看文件

14
Project/Assets/ML-Agents/Examples/Match3/TFModels/Match3VectorObs.onnx.meta


fileFormatVersion: 2
guid: 9e89b8e81974148d3b7213530d00589d
ScriptedImporter:
fileIDToRecycleName:
11400000: main obj
11400002: model data
externalObjects: {}
userData:
assetBundleName:
assetBundleVariant:
script: {fileID: 11500000, guid: 683b6cb6d0a474744822c888b46772c9, type: 3}
optimizeModel: 1
forceArbitraryBatchSize: 1
treatErrorsAsWarnings: 0

1001
Project/Assets/ML-Agents/Examples/Match3/TFModels/Match3VisualObs.nn
文件差异内容过多而无法显示
查看文件

11
Project/Assets/ML-Agents/Examples/Match3/TFModels/Match3VisualObs.nn.meta


fileFormatVersion: 2
guid: 48d14da88fea74d0693c691c6e3f2e34
ScriptedImporter:
fileIDToRecycleName:
11400000: main obj
11400002: model data
externalObjects: {}
userData:
assetBundleName:
assetBundleVariant:
script: {fileID: 11500000, guid: 19ed1486aa27d4903b34839f37b8f69f, type: 3}

233
com.unity.ml-agents.extensions/Runtime/Match3/AbstractBoard.cs


using System;
using System.Collections.Generic;
using UnityEngine;
namespace Unity.MLAgents.Extensions.Match3
{
public abstract class AbstractBoard : MonoBehaviour
{
/// <summary>
/// Number of rows on the board
/// </summary>
public int Rows;
/// <summary>
/// Number of columns on the board
/// </summary>
public int Columns;
/// <summary>
/// Maximum number of different types of cells (colors, pieces, etc).
/// </summary>
public int NumCellTypes;
/// <summary>
/// Maximum number of special types. This can be zero, in which case
/// all cells of the same type are assumed to be equivalent.
/// </summary>
public int NumSpecialTypes;
/// <summary>
/// Returns the "color" of the piece at the given row and column.
/// This should be between 0 and NumCellTypes-1 (inclusive).
/// The actual order of the values doesn't matter.
/// </summary>
/// <param name="row"></param>
/// <param name="col"></param>
/// <returns></returns>
public abstract int GetCellType(int row, int col);
/// <summary>
/// Returns the special type of the piece at the given row and column.
/// This should be between 0 and NumSpecialTypes (inclusive).
/// The actual order of the values doesn't matter.
/// </summary>
/// <param name="row"></param>
/// <param name="col"></param>
/// <returns></returns>
public abstract int GetSpecialType(int row, int col);
/// <summary>
/// Check whether the particular Move is valid for the game.
/// The actual results will depend on the rules of the game, but we provide SimpleIsMoveValid()
/// that handles basic match3 rules with no special or immovable pieces.
/// </summary>
/// <param name="m"></param>
/// <returns></returns>
public abstract bool IsMoveValid(Move m);
/// <summary>
/// Instruct the game to make the given move. Returns true if the move was made.
/// Note that during training, a move that was marked as invalid may occasionally still be
/// requested. If this happens, it is safe to do nothing and request another move.
/// </summary>
/// <param name="m"></param>
/// <returns></returns>
public abstract bool MakeMove<