Merge pull request #218 from Unity-Technologies/scenario-lifecycle-hooks-refactor

Scenario lifecycle hooks refactor

com.unity.perception/CHANGELOG.md

 
 The color of keypoints and connections are now reported in the annotation definition json file for keypoint templates.
 
+The PerceptionScenario abstract class has been added to abstract perception data capture specific functionality from the vanilla scenario lifecycle. 
+
 ### Changed
 
 Renamed all appearances of the term `KeyPoint` within types and names to `Keypoint`.
 The scenario inspector buttons serialize and deserialize have been refactored to open a file explorer generate and import JSON configurations.
 
 Randomizer tags now use OnEnable and OnDisable to manage lifecycle. This allows the user to toggle them on and off in the editor.
+
+The randomizer methods OnCreate(), OnStartRunning(), and OnStopRunning() are now deprecated and have been replaced with OnAwake(), OnEnable() and OnDisable() respectively to better reflect the existing MonoBehaviour lifecycle methods.
 
 ### Deprecated
 

com.unity.perception/Documentation~/Randomization/Scenarios.md

 
 Scenarios have three responsibilities:
 1. Controlling the execution flow of your simulation
-2. Defining a list of Randomizers
-3. Defining constants that can be configured externally from a built Unity player 
+2. Organizing a list of Randomizers
+3. Defining settings that can be configured externally from a built Unity player 
+
+## Scenario Lifecycle Hooks
+
+Scenarios have a number of lifecycle hooks that are called during execution. Below is a diagram visualizing the sequence of operations run by a typical scenario:
+
+<p align="center">
+<img src="Images/scenario-lifecycle-diagram.png" width="600"/>
+</p>
+
+
+## JSON Configuration
+
+Scenarios can be serialized to JSON, modified, and imported at runtime to configure simulation behavior even after a Unity player has been built. The following scenario settings can be serialized by default using the "Generate JSON Config" button on the scenario inspector:
+ 1. Scenario constants
+ 2. String, numeric, and boolean fields on Randomizers and Parameters 
+ 3. Constant, Uniform, and Normal sampler configurations
+ 
+ Here is an example of how to load a Scenario JSON configuration into a Windows Unity player using the `--scenario-config-file` flag:
+ `.\PerceptionTutorial.exe --scenario-config-file scenarioConfiguration.json`
+
+NOTE: To execute a Scenario using the Run in Unity Simulation window, the Scenario class must implement the UnitySimulationScenario class.
+
-4. **Main Scene** - The Unity scene to execute
-5. **Scenario** - The Scenario to execute
-6. **Sys-Param** - The system parameters or the hardware configuration of Unity Simulation worker instances to execute the Scenario with. Determines per instance specifications such as the number of CPU cores, amount of memory, and presence of a GPU for accelerated execution.
+4. **Sys-Param** - The system parameters or the hardware configuration of Unity Simulation worker instances to execute the Scenario with. Determines per instance specifications such as the number of CPU cores, amount of memory, and presence of a GPU for accelerated execution.
+5. **Json Configuration** - This field is optional. You can select a json scenario configuration within your project to configure your Unity Simulation run instead of using the Scenario settings presently configured in the Editor.
-NOTE: To execute a Scenario using the Run in Unity Simulation window, the Scenario class must implement the UnitySimulationScenario class.
-
-## Custom Scenarios
+## Implementing Custom Scenarios
-1. **isIterationComplete** - determines the conditions that cause the end of a Scenario Iteration
-2. **isScenarioComplete** - determines the conditions that cause the end of a Scenario
-
+1. **isScenarioReadyToStart** - defines the conditions that determine when a Scenario can begin iterating
+1. **isIterationComplete** - defines the conditions that determine the end of a Scenario iteration
+2. **isScenarioComplete** - defines the conditions that determine the a Scenario to stop iterating
-## JSON Configuration
-
-Scenarios can be serialized to JSON, modified, and reimported at runtime to configure simulation behavior even after a Unity player has been built. Constants and Randomizer Sampler settings are the two primary sections generated when serializing a Scenario. Note that currently, only numerical samplers are serialized. Below is the contents of a JSON configuration file created when serializing the Scenario used in Phase 1 of the [Perception Tutorial](../Tutorial/TUTORIAL.md):
-```
-{
-  "constants": {
-    "framesPerIteration": 1,
-    "totalIterations": 100,
-    "instanceCount": 1,
-    "instanceIndex": 0,
-    "randomSeed": 123456789
-  },
-  "randomizers": {
-    "HueOffsetRandomizer": {
-      "hueOffset": {
-        "value": {
-          "range": {
-            "minimum": -180.0,
-            "maximum": 180.0
-          }
-        }
-      }
-    },
-    "RotationRandomizer": {
-      "rotation": {
-        "x": {
-          "range": {
-            "minimum": 0.0,
-            "maximum": 360.0
-          }
-        },
-        "y": {
-          "range": {
-            "minimum": 0.0,
-            "maximum": 360.0
-          }
-        },
-        "z": {
-          "range": {
-            "minimum": 0.0,
-            "maximum": 360.0
-          }
-        }
-      }
-    }
-  }
-}
-``` 
+### Abstract Scenario Classes
+There are a few abstract scenario classes to choose from depending on your requirements when deriving a custom scenario. Below is a list of these options and when it is appropriate to derive them for your use case:
+1. **Scenario<T>** - This is the most basic scenario class to derive if only the basic scenario lifecycle coordination tooling is necessary
+2. **PerceptionScenario<T>** - The perception scenario abstract class introduces some useful functionality for Scenarios that intend to utilize the Perception package's data capture tooling to generate datasets.
+3. **UnitySimulationScenario<T>** - This abstract Scenario class is useful for implementing custom perception Scenarios that are compatible with the "Run in Unity Simulation Window." The FixedLengthScenario class is an example of a Scenario deriving from the UnitySimulationScenario class.
-### Constants
-Constants can include properties such as starting Iteration value or total Iteration count, and you can always add your own custom constants. Below is an example of the Constants class used in the `FixedLengthScenario` class:
+### Scenario Constants
+Constants include properties such as starting iteration value or total iteration count that configure the lifecycle settings of the scenario. By deriving your own constants class you can  add your own custom Scenario settings for configuring different scenario properties. Below is an example of the Constants class used in the `FixedLengthScenario` class:
 ```
 [Serializable]
 public class Constants : UnitySimulationScenarioConstants
 ```
 
 There are a few key things to note here:
-1. The Constants class will need to inherit from `UnitySimulationScenarioConstants` to be compatible with the Run in Unity Simulation window. Deriving from `UnitySimulationScenarioConstants` will add a few key properties to the Constants class that are needed to coordinate a Unity Simulation run.
-2. Make sure to include the `[Serializable]` attribute on a constant class. This will ensure that the Constants can be manipulated from the Unity inspector.
+1. The Constants class will need to inherit from `UnitySimulationScenarioConstants` to be compatible with the Run in Unity Simulation window. Deriving from `UnitySimulationScenarioConstants` will add a few key properties to the Constants class that are needed to coordinate a Unity Simulation run. If running in Unity Simulation is not a requirement, a new scenario constants class can derive from the base ScenarioConstants class instead.
+2. Make sure to include the `[Serializable]` attribute on a constant class. This will ensure that the Constants can be properly configured from the Unity inspector.
-
-
-Follow the instructions below to generate a Scenario configuration file to modify your Scenario Constants and Randomizers in a built player:
-1. Click the _**Serialize Constants**_ button in the Scenario's inspector window. This will generate a `scenario_configuration.json` file and place it in the project's Assets/StreamingAssets folder.
-2. Build your player. The new player will have a [ProjectName]_Data/StreamingAssets folder. A copy of the `scenario_configuration.json` file previously constructed in the editor will be found in this folder.
-3. Change the contents of the `scenario_configuration.json` file. Any running player thereafter will utilize the newly authored values.

com.unity.perception/Editor/Randomization/Editors/ScenarioBaseEditor.cs

                     return;
                 Undo.RecordObject(m_Scenario, "Deserialized scenario configuration");
                 m_Scenario.DeserializeFromFile(filePath);
+                Debug.Log($"Deserialized scenario configuration from {Path.GetFullPath(filePath)}. " +
+                    "Using undo in the editor will revert these changes to your scenario.");
                 PlayerPrefs.SetString(k_ConfigFilePlayerPrefKey, filePath);
             };
 

com.unity.perception/Editor/Randomization/Utilities/StaticData.cs

             object obj = prop.serializedObject.targetObject;
             var elements = path.Split('.');
             if (parent)
-                elements = elements.Take(elements.Count() - 1).ToArray();
+                elements = elements.Take(elements.Length - 1).ToArray();
 
             foreach (var element in elements)
                 if (element.Contains("["))
             if (source == null)
                 return null;
             var type = source.GetType();
-            var f = type.GetField(name, BindingFlags.NonPublic | BindingFlags.Public | BindingFlags.Instance);
-            if (f == null)
+            var field = GetField(type, name);
+            if (field == null)
-                var p = type.GetProperty(name, BindingFlags.NonPublic | BindingFlags.Public | BindingFlags.Instance | BindingFlags.IgnoreCase);
-                return p == null ? null : p.GetValue(source, null);
+                var property = type.GetProperty(name, BindingFlags.NonPublic | BindingFlags.Public | BindingFlags.Instance | BindingFlags.IgnoreCase);
+                return property == null ? null : property.GetValue(source, null);
-
-            return f.GetValue(source);
+            return field.GetValue(source);
         }
 
         static object GetArrayValue(object source, string name, int index)
             while (index-- >= 0)
                 enumerator.MoveNext();
             return enumerator.Current;
+        }
+
+        public static FieldInfo GetField(Type type, string fieldName)
+        {
+            if (type == null)
+                return null;
+            const BindingFlags flags =
+                BindingFlags.Public | BindingFlags.NonPublic | BindingFlags.Instance |BindingFlags.DeclaredOnly;
+            var fields = type.GetFields(flags);
+            foreach (var field in fields)
+                if (field.Name == fieldName)
+                    return field;
+            return GetField(type.BaseType, fieldName);
         }
 
         public static bool IsSubclassOfRawGeneric(Type generic, Type toCheck)

com.unity.perception/Editor/Randomization/Utilities/UIElementsEditorUtilities.cs

 using System.Linq;
 using UnityEditor.UIElements;
 using UnityEngine;
+using UnityEngine.Perception.Randomization.Randomizers.SampleRandomizers;
 using UnityEngine.UIElements;
 
 namespace UnityEditor.Perception.Randomization
         /// <param name="containerElement">The element to place the created PropertyFields in</param>
         public static void CreatePropertyFields(SerializedProperty property, VisualElement containerElement)
         {
-            var fieldType = StaticData.GetManagedReferenceValue(property).GetType();
+            var obj = StaticData.GetManagedReferenceValue(property);
+            if (obj == null)
+                return;
+            var fieldType = obj.GetType();
             var iterator = property.Copy();
             var nextSiblingProperty = property.Copy();
             nextSiblingProperty.NextVisible(false);

com.unity.perception/Editor/Randomization/VisualElements/Randomizer/RandomizerElement.cs

             collapseToggle.RegisterCallback<MouseUpEvent>(evt => collapsed = !collapsed);
 
             var enabledToggle = this.Q<Toggle>("enabled");
-            enabledToggle.BindProperty(property.FindPropertyRelative("<enabled>k__BackingField"));
+            enabledToggle.BindProperty(property.FindPropertyRelative("m_Enabled"));
 
             var removeButton = this.Q<Button>("remove");
             removeButton.clicked += () => randomizerList.RemoveRandomizer(this);

com.unity.perception/Runtime/GroundTruth/PerceptionCamera.cs

 
         void CaptureRgbData(Camera cam)
         {
-            Profiler.BeginSample("CaptureDataFromLastFrame");
+
+            Profiler.BeginSample("CaptureDataFromLastFrame");
 
             // Record the camera's projection matrix
             SetPersistentSensorData("camera_intrinsic", ToProjectionMatrix3x3(cam.projectionMatrix));

com.unity.perception/Runtime/Randomization/Randomizers/Randomizer.cs

 using System;
 using System.Collections.Generic;
+using Unity.Entities;
 using UnityEngine.Perception.Randomization.Parameters;
 using UnityEngine.Perception.Randomization.Scenarios;
 
     [Serializable]
     public abstract class Randomizer
     {
-        bool m_PreviouslyEnabled;
-
-        [HideInInspector, SerializeField] internal bool collapsed;
+        [SerializeField, HideInInspector] bool m_Enabled = true;
+        [SerializeField, HideInInspector] internal bool collapsed;
-        [field: SerializeField, HideInInspector] public bool enabled { get; set; } = true;
+        public bool enabled
+        {
+            get => m_Enabled;
+            set
+            {
+                m_Enabled = value;
+                if (value)
+                    OnEnable();
+                else
+                    OnDisable();
+            }
+        }
 
         /// <summary>
         /// Returns the scenario containing this Randomizer
         /// <summary>
         /// OnCreate is called when the Randomizer is added or loaded to a scenario
         /// </summary>
-        protected virtual void OnCreate() { }
+        [Obsolete("Method OnCreate has been deprecated. Use OnAwake instead (UnityUpgradable)", true)]
+        protected virtual void OnCreate() =>
+            throw new NotSupportedException("OnCreate method has been deprecated");
-        /// OnIterationStart is called at the start of a new scenario iteration
+        /// OnAwake is called when the Randomizer is added or loaded to a scenario
-        protected virtual void OnIterationStart() { }
+        protected virtual void OnAwake() { }
-        /// OnIterationEnd is called the after a scenario iteration has completed
+        /// OnEnabled is called when the Randomizer becomes enabled and active
+        /// </summary>
+        protected virtual void OnEnable() { }
+
+        /// <summary>
+        /// OnDisable is called when the Randomizer becomes disabled
-        protected virtual void OnIterationEnd() { }
+        protected virtual void OnDisable() { }
-        /// OnScenarioComplete is called the after the entire scenario has completed
+        /// OnScenarioStart is called on the frame the scenario begins iterating
+        /// </summary>
+        protected virtual void OnScenarioStart() { }
+
+        /// <summary>
+        /// OnScenarioComplete is called the after the entire Scenario has completed
+        /// OnIterationStart is called at the start of a new Scenario iteration
+        /// </summary>
+        protected virtual void OnIterationStart() { }
+
+        /// <summary>
+        /// OnIterationEnd is called the after a Scenario iteration has completed
+        /// </summary>
+        protected virtual void OnIterationEnd() { }
+
+        /// <summary>
-        protected virtual void OnStartRunning() { }
+        [Obsolete("Method OnStartRunning has been deprecated. Use OnEnabled instead (UnityUpgradable)", true)]
+        protected virtual void OnStartRunning() =>
+            throw new NotSupportedException("OnStartRunning method has been deprecated");
-        protected virtual void OnStopRunning() { }
+        [Obsolete("Method OnStopRunning has been deprecated. Use OnDisable instead (UnityUpgradable)", true)]
+        protected virtual void OnStopRunning() =>
+            throw new NotSupportedException("OnStopRunning method has been deprecated");
 
         /// <summary>
         /// OnUpdate is executed every frame for enabled Randomizers
-        internal virtual void Create()
-        {
-            OnCreate();
-        }
+        #region InternalScenarioMethods
+        internal void Awake() => OnAwake();
+
+        internal void ScenarioStart() => OnScenarioStart();
-        internal virtual void IterationStart()
-        {
-            OnIterationStart();
-        }
+        internal void ScenarioComplete() => OnScenarioComplete();
-        internal virtual void IterationEnd()
-        {
-            OnIterationEnd();
-        }
+        internal void IterationStart() => OnIterationStart();
-        internal virtual void ScenarioComplete()
-        {
-            OnScenarioComplete();
-        }
+        internal void IterationEnd() => OnIterationEnd();
-        internal void Update()
-        {
-            if (enabled)
-            {
-                if (!m_PreviouslyEnabled)
-                {
-                    m_PreviouslyEnabled = true;
-                    OnStartRunning();
-                }
-                OnUpdate();
-            }
-            else if (m_PreviouslyEnabled)
-            {
-                m_PreviouslyEnabled = false;
-                OnStopRunning();
-            }
-        }
+        internal void Update() => OnUpdate();
+        #endregion
     }
 }

com.unity.perception/Runtime/Randomization/Randomizers/RandomizerExamples/Randomizers/BackgroundObjectPlacementRandomizer.cs

         GameObjectOneWayCache m_GameObjectOneWayCache;
 
         /// <inheritdoc/>
-        protected override void OnCreate()
+        protected override void OnAwake()
         {
             m_Container = new GameObject("BackgroundContainer");
             m_Container.transform.parent = scenario.transform;

com.unity.perception/Runtime/Randomization/Randomizers/RandomizerExamples/Randomizers/ForegroundObjectPlacementRandomizer.cs

         GameObjectOneWayCache m_GameObjectOneWayCache;
 
         /// <inheritdoc/>
-        protected override void OnCreate()
+        protected override void OnAwake()
         {
             m_Container = new GameObject("Foreground Objects");
             m_Container.transform.parent = scenario.transform;

com.unity.perception/Runtime/Randomization/Scenarios/FixedLengthScenario.cs

         /// <summary>
         /// Returns whether the current scenario iteration has completed
         /// </summary>
-        public override bool isIterationComplete => currentIterationFrame >= constants.framesPerIteration;
+        protected override bool isIterationComplete => currentIterationFrame >= constants.framesPerIteration;
     }
 }

com.unity.perception/Runtime/Randomization/Scenarios/ScenarioBase.cs

 using System;
 using System.Collections.Generic;
 using System.IO;
-using Unity.Simulation;
-using UnityEngine.Perception.GroundTruth;
 using UnityEngine.Perception.Randomization.Parameters;
 using UnityEngine.Perception.Randomization.Randomizers;
 using UnityEngine.Perception.Randomization.Samplers;
     [DefaultExecutionOrder(-1)]
     public abstract class ScenarioBase : MonoBehaviour
     {
-        const string k_ScenarioIterationMetricDefinitionId = "DB1B258E-D1D0-41B6-8751-16F601A2E230";
-        MetricDefinition m_IterationMetricDefinition;
-        /// The list of randomizers managed by this scenario
+        /// Returns the active parameter scenario in the scene
-        [SerializeReference] protected List<Randomizer> m_Randomizers = new List<Randomizer>();
+        public static ScenarioBase activeScenario
+        {
+            get => s_ActiveScenario;
+            private set
+            {
+                if (value != null && s_ActiveScenario != null && value != s_ActiveScenario)
+                    throw new ScenarioException("There cannot be more than one active Scenario");
+                s_ActiveScenario = value;
+            }
+        }
-        /// On some platforms, the simulation capture package cannot capture the first frame of output,
-        /// so this field is used to track whether the first frame has been skipped yet.
+        /// The current activity state of the scenario
-        protected bool m_SkipFrame = true;
+        public State state { get; private set; } = State.Initializing;
-        /// Setting this field to true will cause the scenario to enter an idle state. By default, scenarios will enter
-        /// the idle state after its isScenarioComplete property has returned true.
+        /// The list of randomizers managed by this scenario
-        protected bool m_Idle;
+        [SerializeReference] List<Randomizer> m_Randomizers = new List<Randomizer>();
 
         /// <summary>
         /// Enumerates over all enabled randomizers
         public IReadOnlyList<Randomizer> randomizers => m_Randomizers.AsReadOnly();
 
         /// <summary>
-        /// Returns the active parameter scenario in the scene
-        /// </summary>
-        public static ScenarioBase activeScenario
-        {
-            get
-            {
-#if UNITY_EDITOR
-
-                // This compiler define is required to allow samplers to
-                // iterate the scenario's random state in edit-mode
-                if (s_ActiveScenario == null)
-                    s_ActiveScenario = FindObjectOfType<ScenarioBase>();
-#endif
-                return s_ActiveScenario;
-            }
-            private set
-            {
-                if (value != null && s_ActiveScenario != null && value != s_ActiveScenario)
-                    throw new ScenarioException("There cannot be more than one active Scenario");
-                s_ActiveScenario = value;
-            }
-        }
-
-        /// <summary>
         /// Returns this scenario's non-typed serialized constants
         /// </summary>
         public abstract ScenarioConstants genericConstants { get; }
         public int currentIteration { get; protected set; }
 
         /// <summary>
+        /// The scenario will begin on the frame this property first returns true
+        /// </summary>
+        /// <returns>Whether the scenario should start this frame</returns>
+        protected abstract bool isScenarioReadyToStart { get; }
+
+        /// <summary>
-        public abstract bool isIterationComplete { get; }
+        protected abstract bool isIterationComplete { get; }
-        /// Returns whether the entire scenario has completed
+        /// Returns whether the scenario has completed
-        public abstract bool isScenarioComplete { get; }
+        protected abstract bool isScenarioComplete { get; }
 
         /// <summary>
         /// This method selects what the next iteration index will be. By default, the scenario will simply progress to
 
             var jsonText = File.ReadAllText(configFilePath);
             DeserializeFromJson(jsonText);
-
-            var absolutePath = Path.GetFullPath(configFilePath);
-#if UNITY_EDITOR
-            Debug.Log($"Deserialized scenario configuration from {absolutePath}. " +
-                "Using undo in the editor will revert these changes to your scenario.");
-#else
-            Debug.Log($"Deserialized scenario configuration from {absolutePath}");
+#if !UNITY_EDITOR
+            Debug.Log($"Deserialized scenario configuration from {Path.GetFullPath(configFilePath)}");
-        /// Resets SamplerState.randomState with a new seed value generated by hashing this Scenario's randomSeed
-        /// with its currentIteration
+        /// Deserialize scenario settings from a file passed through a command line argument
-        protected virtual void ResetRandomStateOnIteration()
+        /// <param name="commandLineArg">The command line argument to look for</param>
+        protected virtual void DeserializeFromCommandLine(string commandLineArg="--scenario-config-file")
-            SamplerState.randomState = SamplerUtility.IterateSeed((uint)currentIteration, genericConstants.randomSeed);
-        }
-
-        /// <summary>
-        /// OnAwake is called right after this scenario MonoBehaviour is created or instantiated
-        /// </summary>
-        protected virtual void OnAwake() { }
-
-        /// <summary>
-        /// OnStart is called after Awake but before the first Update method call
-        /// </summary>
-        protected virtual void OnStart()
-        {
-#if !UNITY_EDITOR
-                if (args[i] == "--scenario-config-file")
-                {
-                    filePath = args[i + 1];
-                    break;
-                }
+                if (args[i] != "--scenario-config-file")
+                    continue;
+                filePath = args[i + 1];
+                break;
             }
 
             if (string.IsNullOrEmpty(filePath))
                 return;
             }
 
-            try
-            {
-                DeserializeFromFile(filePath);
-            }
+            try { DeserializeFromFile(filePath); }
-                m_Idle = true;
+        }
+
+        /// <summary>
+        /// Resets SamplerState.randomState with a new seed value generated by hashing this Scenario's randomSeed
+        /// with its currentIteration
+        /// </summary>
+        protected virtual void ResetRandomStateOnIteration()
+        {
+            SamplerState.randomState = SamplerUtility.IterateSeed((uint)currentIteration, genericConstants.randomSeed);
+        }
+
+        #region LifecycleHooks
+        /// <summary>
+        /// OnAwake is called when this scenario MonoBehaviour is created or instantiated
+        /// </summary>
+        protected virtual void OnAwake() { }
+
+        /// <summary>
+        /// OnConfigurationImport is called before OnStart in the same frame. This method by default loads a scenario
+        /// settings from a file before the scenario begins.
+        /// </summary>
+        protected virtual void OnConfigurationImport()
+        {
+#if !UNITY_EDITOR
+            DeserializeFromCommandLine();
+        /// OnStart is called when the scenario first begins playing
+        /// </summary>
+        protected virtual void OnStart() { }
+
+        /// <summary>
+        /// OnIterationStart is called before a new iteration begins
+        /// </summary>
+        protected virtual void OnIterationStart() { }
+
+        /// <summary>
+        /// OnIterationStart is called after each iteration has completed
+        /// </summary>
+        protected virtual void OnIterationEnd() { }
+
+        /// <summary>
+        /// OnUpdate is called every frame while the scenario is playing
+        /// </summary>
+        protected virtual void OnUpdate() { }
+
+        /// <summary>
-        protected virtual void OnComplete()
+        protected virtual void OnComplete() { }
+
+        /// <summary>
+        /// OnIdle is called each frame after the scenario has completed
+        /// </summary>
+        protected virtual void OnIdle() { }
+
+        /// <summary>
+        /// Restart the scenario
+        /// </summary>
+        public void Restart()
-            DatasetCapture.ResetSimulation();
-            m_Idle = true;
+            if (state != State.Idle)
+                throw new ScenarioException(
+                    "A Scenario cannot be restarted until it is finished and has entered the Idle state");
+            currentIteration = 0;
+            currentIterationFrame = 0;
+            framesSinceInitialization = 0;
+            state = State.Initializing;
-        /// OnIdle is called each frame after the scenario has completed
+        /// Exit to playmode if in the Editor or quit the application if in a built player
-        protected virtual void OnIdle()
+        protected void Quit()
-            Manager.Instance.Shutdown();
-            if (!Manager.FinalUploadsDone)
-                return;
-                Application.Quit();
+            Application.Quit();
+        #endregion
+        #region MonoBehaviourMethods
+            OnAwake();
-                randomizer.Create();
+                randomizer.Awake();
-            m_IterationMetricDefinition = DatasetCapture.RegisterMetricDefinition(
-                "scenario_iteration", "Iteration information for dataset sequences",
-                Guid.Parse(k_ScenarioIterationMetricDefinitionId));
-            OnAwake();
         }
 
         /// <summary>
             activeScenario = null;
         }
 
-        void Start()
-        {
-            var randomSeedMetricDefinition = DatasetCapture.RegisterMetricDefinition(
-                "random-seed",
-                "The random seed used to initialize the random state of the simulation. Only triggered once per simulation.",
-                Guid.Parse("14adb394-46c0-47e8-a3f0-99e754483b76"));
-            DatasetCapture.ReportMetric(randomSeedMetricDefinition, new[] { genericConstants.randomSeed });
-            OnStart();
-        }
-
-            // TODO: remove this check when the perception camera can capture the first frame of output
-            if (m_SkipFrame)
+            switch (state)
-                m_SkipFrame = false;
-                return;
-            }
+                case State.Initializing:
+                    if (isScenarioReadyToStart)
+                    {
+                        OnConfigurationImport();
+                        state = State.Playing;
+                        OnStart();
+                        foreach (var randomizer in m_Randomizers)
+                            randomizer.ScenarioStart();
+                        IterationLoop();
+                    }
+                    break;
-            // Wait for any final uploads before exiting quitting
-            if (m_Idle)
-            {
-                OnIdle();
-                return;
+                case State.Playing:
+                    IterationLoop();
+                    break;
+
+                case State.Idle:
+                    OnIdle();
+                    break;
+
+                default:
+                    throw new ArgumentOutOfRangeException(
+                        $"Invalid state {state} encountered while updating scenario");
+        }
+        #endregion
+        void IterationLoop()
+        {
             // Increment iteration and cleanup last iteration
             if (isIterationComplete)
             {
                     randomizer.IterationEnd();
+                OnIterationEnd();
             }
 
             // Quit if scenario is complete
                     randomizer.ScenarioComplete();
                 OnComplete();
+                state = State.Idle;
+                OnIdle();
+                return;
-                DatasetCapture.StartNewSequence();
-                DatasetCapture.ReportMetric(m_IterationMetricDefinition, new[]
-                {
-                    new IterationMetricData { iteration = currentIteration }
-                });
+                OnIterationStart();
+            OnUpdate();
             foreach (var randomizer in activeRandomizers)
                 randomizer.Update();
 
         /// <summary>
         /// Append a randomizer to the end of the randomizer list
         /// </summary>
-        /// <param name="newRandomizer"></param>
+        /// <param name="newRandomizer">The Randomizer to add to the Scenario</param>
         public void AddRandomizer(Randomizer newRandomizer)
         {
             InsertRandomizer(m_Randomizers.Count, newRandomizer);
         /// <exception cref="ScenarioException"></exception>
         public void InsertRandomizer(int index, Randomizer newRandomizer)
         {
+            if (state != State.Initializing)
+                throw new ScenarioException("Randomizers cannot be added to the scenario after it has started");
             foreach (var randomizer in m_Randomizers)
                 if (randomizer.GetType() == newRandomizer.GetType())
                     throw new ScenarioException(
 #if UNITY_EDITOR
             if (Application.isPlaying)
-                newRandomizer.Create();
+                newRandomizer.Awake();
-            newRandomizer.Create();
+            newRandomizer.Awake();
 #endif
         }
 
         /// <param name="index">The index of the randomizer to remove</param>
         public void RemoveRandomizerAt(int index)
         {
+            if (state != State.Initializing)
+                throw new ScenarioException("Randomizers cannot be added to the scenario after it has started");
             m_Randomizers.RemoveAt(index);
         }
 
         {
             foreach (var randomizer in m_Randomizers)
             foreach (var parameter in randomizer.parameters)
-                try
-                {
-                    parameter.Validate();
-                }
-                catch (ParameterValidationException exception)
-                {
-                    Debug.LogException(exception, this);
-                }
+            {
+                try { parameter.Validate(); }
+                catch (ParameterValidationException exception) { Debug.LogException(exception, this); }
+            }
-        struct IterationMetricData
+        /// <summary>
+        /// Enum used to track the lifecycle of a Scenario
+        /// </summary>
+        public enum State
-            // ReSharper disable once NotAccessedField.Local
-            public int iteration;
+            Initializing,
+            Playing,
+            Idle
         }
     }
 }

com.unity.perception/Runtime/Randomization/Scenarios/Serialization/ScenarioSerializer.cs

 
         public static void SerializeToFile(ScenarioBase scenario, string filePath)
         {
-            Directory.CreateDirectory(Application.dataPath + "/StreamingAssets/");
+            var directory = Path.GetDirectoryName(filePath);
+            if (!string.IsNullOrEmpty(directory))
+                Directory.CreateDirectory(directory);
-            {
-            }
         }
 
         public static JObject SerializeToJsonObject(ScenarioBase scenario)

com.unity.perception/Runtime/Randomization/Scenarios/UnitySimulationScenario.cs

 namespace UnityEngine.Perception.Randomization.Scenarios
 {
     /// <summary>
-    /// Defines a scenario that is compatible with the Run in Unity Simulation window
+    /// A scenario must derive from this class to be compatible with the Run in
+    /// Unity Simulation window. The iterations of this scenario will be executed in parallel across a user specified
+    /// number of worker instances when run in Unity Simulation.
-    /// <typeparam name="T">The type of constants to serialize</typeparam>
-    public abstract class UnitySimulationScenario<T> : Scenario<T> where T : UnitySimulationScenarioConstants, new()
+    /// <typeparam name="T">The type of scenario constants to serialize</typeparam>
+    public abstract class UnitySimulationScenario<T> : PerceptionScenario<T>
+        where T : UnitySimulationScenarioConstants, new()
-        public sealed override bool isScenarioComplete => currentIteration >= constants.totalIterations;
-
-        /// <inheritdoc/>
-        protected sealed override void IncrementIteration()
+        protected override bool isScenarioReadyToStart
-            currentIteration += constants.instanceCount;
+            get
+            {
+                if (!Configuration.Instance.IsSimulationRunningInCloud() && !m_SkippedFirstFrame)
+                {
+                    m_SkippedFirstFrame = true;
+                    return false;
+                }
+                return true;
+            }
-        protected override void OnAwake()
-        {
-            // Don't skip the first frame if executing on Unity Simulation
-            if (Configuration.Instance.IsSimulationRunningInCloud())
-                m_SkipFrame = false;
-        }
+        protected sealed override bool isScenarioComplete => currentIteration >= constants.totalIterations;
-        protected override void OnStart()
+        protected override void OnConfigurationImport()
         {
             if (Configuration.Instance.IsSimulationRunningInCloud())
             {
             else
-                base.OnStart();
+                base.OnConfigurationImport();
+        }
+
+        /// <inheritdoc/>
+        protected sealed override void IncrementIteration()
+        {
+            currentIteration += constants.instanceCount;
         }
     }
 }

com.unity.perception/Tests/Editor/RandomizerEditorTests.cs

     {
         public GameObject testGameObject;
 
-        protected override void OnCreate()
+        protected override void OnAwake()
         {
             // This line should throw a NullReferenceException
             testGameObject.transform.position = Vector3.zero;

com.unity.perception/Tests/Runtime/Randomization/RandomizerTests/ExampleTransformRandomizer.cs

 
         public Transform transform;
 
-        protected override void OnCreate()
+        protected override void OnAwake()
         {
             transform = scenario.transform;
         }

com.unity.perception/Tests/Runtime/Randomization/ScenarioTests/ScenarioTests.cs

 using UnityEngine.TestTools;
 using Object = UnityEngine.Object;
 
-namespace RandomizationTests
+namespace RandomizationTests.ScenarioTests
-        FixedLengthScenario m_Scenario;
+        TestFixedLengthScenario m_Scenario;
 
         [SetUp]
         public void Setup()
         // TODO: update this function once the perception camera doesn't skip the first frame
         IEnumerator CreateNewScenario(int totalIterations, int framesPerIteration)
         {
-            m_Scenario = m_TestObject.AddComponent<FixedLengthScenario>();
+            m_Scenario = m_TestObject.AddComponent<TestFixedLengthScenario>();
-            yield return null; // Skip Start() frame
+            yield return null; // Skip first frame
             yield return null; // Skip first Update() frame
         }
 
             m_TestObject = new GameObject();
-            m_Scenario = m_TestObject.AddComponent<FixedLengthScenario>();
+            m_Scenario = m_TestObject.AddComponent<TestFixedLengthScenario>();
             m_Scenario.AddRandomizer(new RotationRandomizer());
 
             string RemoveWhitespace(string str) =>
         public void ScenarioConfigurationOverwrittenDuringDeserialization()
         {
             m_TestObject = new GameObject();
-            m_Scenario = m_TestObject.AddComponent<FixedLengthScenario>();
+            m_Scenario = m_TestObject.AddComponent<TestFixedLengthScenario>();
 
             var constants = new FixedLengthScenario.Constants
             {
             yield return CreateNewScenario(iterationCount, 1);
             for (var i = 0; i < iterationCount; i++)
             {
-                Assert.False(m_Scenario.isScenarioComplete);
+                Assert.True(m_Scenario.state == ScenarioBase.State.Playing);
-            Assert.True(m_Scenario.isScenarioComplete);
+            Assert.True(m_Scenario.state == ScenarioBase.State.Idle);
         }
 
         [UnityTest]

com.unity.perception/Runtime/Randomization/Scenarios/PerceptionScenario.cs

+using System;
+using Unity.Simulation;
+using UnityEngine.Perception.GroundTruth;
+
+namespace UnityEngine.Perception.Randomization.Scenarios
+{
+    /// <summary>
+    /// Derive this class to configure perception data capture while coordinating a scenario
+    /// </summary>
+    /// <typeparam name="T">The type of scenario constants to serialize</typeparam>
+    public abstract class PerceptionScenario<T> : Scenario<T> where T : ScenarioConstants, new()
+    {
+        /// <summary>
+        /// The guid used to identify this scenario's Iteration Metric Definition
+        /// </summary>
+        const string k_ScenarioIterationMetricDefinitionId = "DB1B258E-D1D0-41B6-8751-16F601A2E230";
+
+        /// <summary>
+        /// The metric definition used to report the current scenario iteration
+        /// </summary>
+        MetricDefinition m_IterationMetricDefinition;
+
+        /// <summary>
+        /// The scriptable render pipeline hook used to capture perception data skips the first frame of the simulation
+        /// when running locally, so this flag is used to track whether the first frame has been skipped yet.
+        /// </summary>
+        protected bool m_SkippedFirstFrame;
+
+        /// <inheritdoc/>
+        protected override bool isScenarioReadyToStart
+        {
+            get
+            {
+                if (!m_SkippedFirstFrame)
+                {
+                    m_SkippedFirstFrame = true;
+                    return false;
+                }
+                return true;
+            }
+        }
+
+        /// <inheritdoc/>
+        protected override void OnAwake()
+        {
+            m_IterationMetricDefinition = DatasetCapture.RegisterMetricDefinition(
+                "scenario_iteration", "Iteration information for dataset sequences",
+                Guid.Parse(k_ScenarioIterationMetricDefinitionId));
+        }
+
+        /// <inheritdoc/>
+        protected override void OnStart()
+        {
+            var randomSeedMetricDefinition = DatasetCapture.RegisterMetricDefinition(
+                "random-seed",
+                "The random seed used to initialize the random state of the simulation. Only triggered once per simulation.",
+                Guid.Parse("14adb394-46c0-47e8-a3f0-99e754483b76"));
+            DatasetCapture.ReportMetric(randomSeedMetricDefinition, new[] { genericConstants.randomSeed });
+        }
+
+        /// <inheritdoc/>
+        protected override void OnIterationStart()
+        {
+            DatasetCapture.StartNewSequence();
+            DatasetCapture.ReportMetric(m_IterationMetricDefinition, new[]
+            {
+                new IterationMetricData { iteration = currentIteration }
+            });
+        }
+
+        /// <inheritdoc/>
+        protected override void OnComplete()
+        {
+            DatasetCapture.ResetSimulation();
+            Manager.Instance.Shutdown();
+            Quit();
+        }
+
+        /// <summary>
+        /// Used to report a scenario iteration as a perception metric
+        /// </summary>
+        struct IterationMetricData
+        {
+            // ReSharper disable once NotAccessedField.Local
+            public int iteration;
+        }
+    }
+}

com.unity.perception/Runtime/Randomization/Scenarios/PerceptionScenario.cs.meta

+fileFormatVersion: 2
+guid: 78c9b59045074d7aa47e4911ba0e86fb
+timeCreated: 1613367040

com.unity.perception/Tests/Runtime/Randomization/ScenarioTests/TestFixedLengthScenario.cs

+using System;
+using UnityEngine.Perception.GroundTruth;
+using UnityEngine.Perception.Randomization.Scenarios;
+
+namespace RandomizationTests.ScenarioTests
+{
+    class TestFixedLengthScenario : FixedLengthScenario
+    {
+        protected override void OnComplete()
+        {
+            DatasetCapture.ResetSimulation();
+        }
+    }
+}

com.unity.perception/Tests/Runtime/Randomization/ScenarioTests/TestFixedLengthScenario.cs.meta

+fileFormatVersion: 2
+guid: 7e4a918b4c1442898a54ab08a83af01a
+timeCreated: 1614016112