Steven Leal
4 年前
当前提交
54ae757a
共有 6 个文件被更改,包括 147 次插入 和 230 次删除
-
74com.unity.perception/Documentation~/Randomization/Index.md
-
50com.unity.perception/Documentation~/Randomization/Scenarios.md
-
29com.unity.perception/Documentation~/Randomization/RandomizerTags.md
-
42com.unity.perception/Documentation~/Randomization/Randomizers.md
-
182com.unity.perception/Documentation~/Randomization/Tutorial.md
|
|||
# Scenarios |
|||
|
|||
Scenarios have three responsibilities: |
|||
1. Controlling the execution flow of your simulation |
|||
2. Customizing the application of random parameters in your project |
|||
3. Defining constants that can be configured externally from a built Unity player |
|||
1. Controlling the execution flow of your simulation |
|||
2. Defining a list of randomizers |
|||
3. Defining constants that can be configured externally from a built Unity player |
|||
|
|||
By default, the perception package includes one ready-made scenario, the `FixedLengthScenario` class. This scenario runs each iteration for a fixed number of frames and is compatible with the Run in USim window for cloud simulation execution. |
|||
|
|||
## Scenario Cloud Execution (USim) |
|||
|
|||
Users can utilize Unity's Unity Simulation (USim) service to execute a scenario in the cloud through the perception package's Run in USim window. To open this window from the Unity editor using the top menu bar, navigate to `Window -> Run in USim`. |
|||
|
|||
From the newly opened editor window, customize the following settings to configure a new USim run: |
|||
1. **Run Name** - the name of the USim run (example: TestRun0) |
|||
2. **Total Iterations** - The number of scenario iterations to complete during the run |
|||
3. **Instance Count** - The number of USim worker instances to distribute execution between |
|||
4. **Main Scene** - The Unity scene to execute |
|||
5. **Scenario** - The scenario to execute |
|||
6. **USim Worker Config** - the type of USim worker instance to execute the scenario with. Determines per instance specifications such as the number of CPU cores, amount of memory, and presence of a GPU for accelerated execution. |
|||
By default, the perception package includes one ready-made scenario, the `FixedFrameLengthScenario` class. This scenario is useful for when all created parameters have target GameObjects configured directly in the `ParameterConfiguration` and the scenario execution requires little modification. |
|||
NOTE: To execute a scenario using the Run in USim window, the scenario class must implement the USimScenario class. |
|||
More commonly, users will find the need to create their own Scenario class. Below is an overview of the more common scenario properties and methods a user can override: |
|||
|
|||
## Custom Scenarios |
|||
|
|||
For use cases where the scenario should run for an arbitrary number of frames, implementing a custom scenario may be necessary. Below are the two most common scenario properties a user might want to override to implement custom scenario iteration conditions: |
|||
3. **Initialize** - actions to complete before the scenario has begun iterating |
|||
4. **Setup** - actions to complete at the beginning of each iteration |
|||
5. **Teardown** - actions to complete at the end of each iteration |
|||
6. **OnComplete** - actions to complete after the scenario as completed |
|||
|
|||
Scenarios define constants from which to expose global simulation behaviors like a starting iteration value or a total iteration count. Users can serialize these scenario constants to JSON, modify them in an external program, and finally reimport the JSON constants at runtime to configure their simulation even after their project has been built. Below is an example of the constants used in the `FixedLengthScenario` class: |
|||
Scenarios define constants from which to expose global simulation behaviors like a starting iteration value or a total iteration count. Users can serialize these scenario constants to JSON, modify them in an external program, and finally reimport the JSON constants at runtime to configure their simulation even after their project has been built. Below is an example of the constants class used in the `FixedLengthScenario` class: |
|||
public class Constants |
|||
public class Constants : USimConstants |
|||
public int iterationFrameLength = 1; |
|||
public int startingIteration; |
|||
public int totalIterations = 1000; |
|||
public int framesPerIteration = 1; |
|||
A few key things to note here: |
|||
1. Make sure to include the [Serializable] attribute on a constant class. This will ensure that the constants can be manipulated from the Unity inspector. |
|||
2. By default, UnityEngine.Object class references cannot be serialized to JSON in a meaningful way. This includes Monobehaviors and SerializedObjects. For more information on what can and can't be serialized, take a look at the [Unity JsonUtility manual](https://docs.unity3d.com/ScriptReference/JsonUtility.html). |
|||
3. A scenario class's Serialize() and Deserialized() methods can be overriden to implement custom serialization strategies. |
|||
|
|||
There are a few key things to note here: |
|||
1. The constants class will need to inherit from USimConstants to be compatible with the Run in USim window. Deriving from USimConstants will add a few key properties to the constants class that are needed to coordinate a USim run. |
|||
2. Make sure to include the [Serializable] attribute on a constant class. This will ensure that the constants can be manipulated from the Unity inspector. |
|||
3. By default, UnityEngine.Object class references cannot be serialized to JSON in a meaningful way. This includes Monobehaviors and SerializedObjects. For more information on what can and can't be serialized, take a look at the [Unity JsonUtility manual](https://docs.unity3d.com/ScriptReference/JsonUtility.html). |
|||
4. A scenario class's Serialize() and Deserialized() methods can be overriden to implement custom serialization strategies. |
|
|||
# Randomizer Tags |
|||
|
|||
RandomizerTags are the primary mechanism by which randomizers query for a certain subset of GameObjects to randomize within a simulation. |
|||
|
|||
More specifically, RandomizerTags are components that can be added to GameObjects to register them with the active scenario's TagManager. This TagManager is aware of all objects with tags in the scene and can be queried to find all GameObjects that contain a specific tag. Below is a simple example of a ColorRandomizer querying for all GameObjects with a ColorRandomizerTag that it will apply a random material base color to: |
|||
|
|||
``` |
|||
[Serializable] |
|||
[AddRandomizerMenu("Perception/Color Randomizer")] |
|||
public class ColorRandomizer : Randomizer |
|||
{ |
|||
static readonly int k_BaseColor = Shader.PropertyToID("_BaseColor"); |
|||
|
|||
public ColorHsvaParameter colorParameter; |
|||
|
|||
protected override void OnIterationStart() |
|||
{ |
|||
var taggedObjects = tagManager.Query<ColorRandomizerTag>(); |
|||
foreach (var taggedObject in taggedObjects) |
|||
{ |
|||
var renderer = taggedObject.GetComponent<MeshRenderer>(); |
|||
renderer.material.SetColor(k_BaseColor, colorParameter.Sample()); |
|||
} |
|||
} |
|||
} |
|||
``` |
|||
|
|||
RandomizerTags can also be used to customize how randomizers apply their randomizations to a particular GameObject. Visit [part two of the randomization tutorial]() to explore an in depth example of implementing a LightRandomizer that does exactly this. |
|||
|
|
|||
# Randomizers |
|||
|
|||
Randomizers encapsulate specific randomization activities to perform during the execution of a randomized simulation. For example, randomizers exist for spawning objects, repositioning lights, varying the color of objects, etc. Randomizers expose random parameters to their inspector interface to further customize these variations. Users can add a set of randomizers to a scenario in order to define an ordered list randomization activities to perform during the lifecycle of a simulation. |
|||
|
|||
To define an entirely new randomizer, derive the Randomizer class and implement one or more of the methods listed in the section below to randomize GameObjects during the runtime of a simulation. |
|||
|
|||
|
|||
## Randomizer Hooks |
|||
|
|||
1. OnCreate() - called when the Randomizer is added or loaded to a scenario |
|||
2. OnIterationStart() - called at the start of a new scenario iteration |
|||
3. OnIterationEnd() - called the after a scenario iteration has completed |
|||
4. OnScenarioComplete() - called the after the entire scenario has completed |
|||
5. OnStartRunning() - called on the first frame a Randomizer is enabled |
|||
6. OnStopRunning() - called on the first frame a disabled Randomizer is updated |
|||
7. OnUpdate() - executed every frame for enabled Randomizers |
|||
|
|||
|
|||
## Randomizer Coding Example |
|||
|
|||
Below is the code for the sample rotation randomizer included with the perception package: |
|||
|
|||
``` |
|||
[Serializable] |
|||
[AddRandomizerMenu("Perception/Rotation Randomizer")] |
|||
public class RotationRandomizer : Randomizer |
|||
{ |
|||
public Vector3Parameter rotation = new Vector3Parameter(); |
|||
|
|||
protected override void OnIterationStart() |
|||
{ |
|||
var taggedObjects = tagManager.Query<RotationRandomizerTag>(); |
|||
foreach (var taggedObject in taggedObjects) |
|||
taggedObject.transform.rotation = Quaternion.Euler(rotation.Sample()); |
|||
} |
|||
} |
|||
``` |
|||
|
|||
There are a few key things to note from this example: |
|||
1. Make sure to add the [Serializable] tag to all randomizer implementations to ensure that the randomizer can be customized and saved within the Unity Editor. |
|||
2. The [AddRandomizerMenu] attribute customizes the "Add Randomizer" sub menu path in the scenario inspector for a particular randomizer. In this example, the RotationRandomizer can be added to a scenario by opening the add randomizer menu and clicking `Perception -> Rotation Randomizer`. |
|||
3. The line `var taggedObjects = tagManager.Query<RotationRandomizerTag>();` uses RandomizerTags in combination with the current Scenario's tagManager to query for all objects with RotationRandomizerTags to obtain the subset of GameObjects within the simulation that need to have their rotations randomzied. To learn more about how RandomizerTags work, visit the [RandomizerTags doc](RandomizerTags.md). |
|
|||
# Randomization Tutorial |
|||
|
|||
This goal of this tutorial is to walk users through an example randomized perception project that explores the following activities: |
|||
1. Creating a parameter configuration |
|||
2. Customizing parameters and samplers |
|||
4. Configuring a scenario to run the simulation |
|||
5. Configure the perception camera |
|||
6. Building a simulation runtime |
|||
7. Modifying scenario constants |
|||
|
|||
By the end of this guide, the user should have a new project that generates the perception data necessary to train a model to identify a cube from a solid colored background. |
|||
|
|||
Note: Before beginning the tutorial, follow [this guide](../SetupSteps.md) to install the perception package into a new Unity project. |
|||
|
|||
|
|||
## Step 1: Create Scene and GameObjects |
|||
1. Create a new scene using `File -> New Scene` |
|||
2. Use the key combo `Ctrl+S` to save and name the new scene |
|||
3. Create a new cube GameObject |
|||
1. Create a new cube GameObject by navigating to `GameObject -> 3D Object -> Cube` in the menubar |
|||
2. Rename the new cube GameObject "Cube" by double clicking on the new GameObject that appeared in the hierarchy |
|||
3. Reset the cube's transform component by right clicking on transform component in the cube's inpector and clicking `Reset` |
|||
4. Create a new background GameObject |
|||
1. Create a new quad GameObject and rename it "Background" |
|||
2. Set the background quad's position to (0, 0, 2) to set it behind the cube and make the quad large enough to fill the camera by change the quad's scale to (30, 30, 1) |
|||
5. In the *MeshRenderer* component of the *Cube* and *Background* GameObjects, set `Lighting -> Cast shadows -> Off` to prevent the two objects from casting shadows on each other |
|||
|
|||
|
|||
## Step 2: Create Parameter Configuration |
|||
1. Create a new empty GameObject by using `GameObject -> Create Empty` from the menubar |
|||
2. Rename the new empty GameObject "Config" by double clicking on the new GameObject that appeared in the hierarchy |
|||
3. To add a new Parameter Configuration component to the Config GameObject, click on the GameObject in the hierarchy and then click the `Add Component` button in the inspector window. Select `Randomization -> Parameter Configuration` to add the new component. |
|||
|
|||
|
|||
## Step 3: Create and Customize Parameters |
|||
In this step, we will configure 6 parameters to randomize the scene: *CubePosition*, *CubeRotation*, *CubeScale*, *CubeColor*, *BackgroundColor*, and *CameraRotation* |
|||
|
|||
#### Parameter 1: Cube Position |
|||
1. Create a new Vector3 parameter by clicking *Add New Parameter -> Vector3* on the parameter configuration inspector |
|||
2. Rename the parameter "CubePosition" by typing the text box next to the blue text indicating the parameter's type |
|||
3. Click the *Target GameObject* checkbox and select the *Cube* GameObject in the target object selector. Select the property *position* from the property dropdown. |
|||
4. Consider using the following sampler values: |
|||
* X : Uniform [-5, 5] |
|||
* Y : Uniform [-5, 5] |
|||
* Z : Constant [Value = 0] |
|||
|
|||
#### Parameter 2: Cube Rotation |
|||
1. Create a new Vector3 parameter named "CubeRotation" |
|||
2. Select the *Cube* GameObject as the target GameObject and select the property *Transform.eulerAngles* from the property dropdown |
|||
3. Consider using the following component values: |
|||
* X : Uniform [0, 360] |
|||
* Y : Uniform [0, 360] |
|||
* Z : Uniform [0, 360] |
|||
|
|||
#### Parameter 3: Cube Scale |
|||
1. Create a new Vector3 parameter named "CubeScale" |
|||
2. Select the *Cube* GameObject as the target GameObject and select the property *Transform.localScale* from the property dropdown |
|||
3. Consider using the following component values: |
|||
* X : Uniform [0.5, 2] |
|||
* Y : Uniform [0.5, 2] |
|||
* Z : Uniform [0.5, 2] |
|||
4. To ensure that the X, Y, and Z samplers all sample equal scale values, copy the X sampler's random seed to all three samplers |
|||
|
|||
#### Parameter 4: Cube Color |
|||
1. Create a new ColorHSVA parameter named "CubeColor" |
|||
2. Skip setting the target GameObject. We will be using this parameter from within the scenario instead. |
|||
3. Consider using the following component values: |
|||
* Hue : Uniform [0, 1] |
|||
* Saturation : Uniform [0, 1] |
|||
* Value : Uniform [0.25, 1] |
|||
* Alpha : Constant [Value = 0] |
|||
|
|||
#### Parameter 5: Background Color |
|||
1. Create a new ColorHSVA parameter named "BackgroundColor" |
|||
2. Skip setting the target GameObject. We will be using this parameter from within the scenario instead. |
|||
3. Consider using the following component values: |
|||
* Hue : Uniform [0, 1] |
|||
* Saturation : Uniform [0, 1] |
|||
* Value : Uniform [0.25, 1] |
|||
* Alpha : Constant [Value = 0] |
|||
|
|||
#### Parameter 6: Camera Rotation |
|||
1. Create a new Vector3 parameter named "CameraRotation" |
|||
2. Select the *Main Camera* GameObject as the target GameObject and select the property *Transform.eulerAngles* from the property dropdown |
|||
3. Consider using the following component values: |
|||
* X : Constant [Value = 0] |
|||
* Y : Constant [Value = 0] |
|||
* Z : Uniform [0, 360] |
|||
|
|||
|
|||
## Step 4: Configure Scenario |
|||
1. Right click on the *Scripts* folder in the project hierarchy and select `Create -> C# Script`. Name the script "CubeScenario" and press enter. |
|||
2. Double click on the new "CubeScenario" script to open it for edit |
|||
3. In your code editor, paste the following C# code into the CubeScenario script: |
|||
``` |
|||
using System; |
|||
using UnityEngine; |
|||
using UnityEngine.Perception.Randomization.Configuration; |
|||
using UnityEngine.Perception.Randomization.Parameters; |
|||
using UnityEngine.Perception.Randomization.Scenarios; |
|||
|
|||
public class CubeScenario : Scenario<CubeScenario.Constants> |
|||
{ |
|||
[Serializable] |
|||
public class Constants |
|||
{ |
|||
public int totalIterations = 1000; |
|||
} |
|||
|
|||
public override bool isIterationComplete => currentIterationFrame >= 1; |
|||
public override bool isScenarioComplete => currentIteration >= constants.totalIterations; |
|||
|
|||
public ParameterConfiguration config; |
|||
public GameObject background; |
|||
public GameObject cube; |
|||
|
|||
ColorHsvaParameter m_BackgroundColorParameter; |
|||
ColorHsvaParameter m_CubeColorParameter; |
|||
Material m_BackgroundMaterial; |
|||
Material m_CubeMaterial; |
|||
static readonly int k_BaseColor = Shader.PropertyToID("_BaseColor"); |
|||
|
|||
public override void OnInitialize() |
|||
{ |
|||
m_BackgroundColorParameter = config.GetParameter<ColorHsvaParameter>("BackgroundColor"); |
|||
m_CubeColorParameter = config.GetParameter<ColorHsvaParameter>("CubeColor"); |
|||
m_BackgroundMaterial = background.GetComponent<MeshRenderer>().material; |
|||
m_CubeMaterial = cube.GetComponent<MeshRenderer>().material; |
|||
} |
|||
|
|||
public override void OnIterationSetup() |
|||
{ |
|||
m_BackgroundMaterial.SetColor(k_BaseColor, m_BackgroundColorParameter.Sample()); |
|||
m_CubeMaterial.SetColor(k_BaseColor, m_CubeColorParameter.Sample()); |
|||
} |
|||
} |
|||
``` |
|||
So what is this CubeScenario script accomplishing? |
|||
1. The *Constants* nested class in this scenario script determines what scenario parameters can be JSON serialized. Only these parameters can be changed externally from a built player. In this example, we expose the number of total iterations the scenario will complete. |
|||
2. The overrided properties *isIterationComplete* and *isScenarioComplete* are checked before every frame to control the scenario's execution flow. In this case, the scenario will execute for only one frame for each iteration and continue executing until reaching the total iteration limit set by the *totalIterations* field in the constants class. |
|||
3. In Unity, manipulating the color of a material is a shader specific task that cannot be accomplished directly from a color parameter's target GameObject setting. Instead we: |
|||
1. Expose a reference to the parameter configuration this scenario's inspector as the public script variable |
|||
2. Cache the ID of the *_BaseColor* shader property |
|||
3. Override the OnInitialize() method to cache a few references. First, we lookup the parameters *BackgroundColor* and *CubeColor* by name from the the parameter configuration. Second, we grab the references to the materials attached to the cube and background GameObjects when the simulation starts. |
|||
4. Override the OnIterationSetup() method to apply randomly sampled color values to the shaders of the cached materials at the beginning of each scenario iteration |
|||
4. Back in the Unity editor, navigate to the inspector of the *Config* GameObject and use `Add Component -> CubeScenario` to add the new CubeScenario component to your parameter configuration. |
|||
5. Open the constants dropdown and confirm how many iterations the scenario should run (the default is 1000) |
|||
6. Use the *backgroundColorParameter* and *cubeColorParameter* dropdowns to inform the script of which parameters in the configuration to use for the BackgroundColor and CubeColor respectively. |
|||
7. Select the *Background* and *Cube* GameObjects from their respective GameObject field selectors |
|||
8. Confirm that the scenario and parameter configuration are composed properly by clicking the play button in the editor. In the Game window, a cube of different sizes, rotations, scales, and colors should be appearing against a color changing background. |
|||
9. To serialize the constants used in the scenario to JSON for external modification after this project has been built into a runtime, click the *Serialize Config* button on the parameter configuration |
|||
|
|||
|
|||
## Step 5: Configure Perception Camera |
|||
Read through the [general perception getting started guide](../GettingStarted.md) before completing the following steps: |
|||
1. (For URP projects) Add GroundTruthRendererFeature |
|||
2. Add a *PerceptionCamera* component to the MainCamera |
|||
3. Label the cube |
|||
1. Add a *Labeling* component to the cube GameObject |
|||
2. Create a *LabelingConfiguration* asset |
|||
3. Add the cube label to the new configuration |
|||
4. Select the new configuration asset from the perception camera |
|||
4. Enter play mode to confirm that labeled data is being generated |
|||
|
|||
|
|||
## Step 6: Build Simulation Runtime |
|||
1. Create a new sub-folder under "Assets" in the Project Hierarchy named "BuildConfigurations" |
|||
2. Right click on the BuildConfigurations folder and use `Create -> Build -> Empty Build Configuration` to create a new build configuration asset |
|||
3. Rename the build configuration asset "TutorialBuild" |
|||
4. Copy the settings from the example build configuration screenshot below: |
|||
![TutorialBuild](Images/TutorialBuild.png) |
|||
5. Click the "Build" button in the upper righthand corner of the build configuration inspector to create an executable of the tutorial scene. |
|||
|
|||
|
|||
## Step 7: Modify Scenario Constants |
|||
**Note**: Make sure that the "Deserialize On Start" field is checked in the scenario's inspector. A player built without this field checked will cause the player to not load serialized constants. |
|||
|
|||
1. Navigate to the folder created during the build from the previous step of this tutorial (example: C:\projects\RandomizationTutorial\Builds\Tutorial) |
|||
2. Open the "_Data" folder (example: Tutorial_Data) and then open the "StreamingAssets" folder |
|||
3. Inside the folder should be the JSON scenario constants serialized from the parameter configuration in Step 4 |
|||
4. Edit this JSON file to update the scenario constants used in the player |
|||
5. Confirm the new constants are deserialized at runtime by executing the simulation |
撰写
预览
正在加载...
取消
保存
Reference in new issue