LSTM models from 1.x will be incompatible with MLA 2.x (#5254)

* LSTM models from 1.x will be incompatible with MLA 2.x

* Adding a test and a new v2 model

* Make the Model Runner raise an error if using 1.0 model with LSTM

* adding a new model for hallway trained with 2.0

* reword error messages

* Only raise if error, not if warning

* Addressing comments: The legacy Barrauda memory generator and applier were removed. All code that checked for (memories + v1.X) have been removed since these will no longer be supported

* Modifying the changelog and the migrating guide with this change

* Fixing the merge issues

Co-authored-by: Chris Elion <chris.elion@unity3d.com>

Project/Assets/ML-Agents/Examples/Hallway/Prefabs/HallwayArea.prefab

   m_MotionVectors: 1
   m_LightProbeUsage: 1
   m_ReflectionProbeUsage: 1
+  m_RayTracingMode: 2
   m_RenderingLayerMask: 1
   m_RendererPriority: 0
   m_Materials:
   m_ProbeAnchor: {fileID: 0}
   m_LightProbeVolumeOverride: {fileID: 0}
   m_ScaleInLightmap: 1
+  m_ReceiveGI: 1
   m_PreserveUVs: 0
   m_IgnoreNormalsForChartDetection: 0
   m_ImportantGI: 0
   m_MotionVectors: 1
   m_LightProbeUsage: 1
   m_ReflectionProbeUsage: 1
+  m_RayTracingMode: 2
   m_RenderingLayerMask: 1
   m_RendererPriority: 0
   m_Materials:
   m_ProbeAnchor: {fileID: 0}
   m_LightProbeVolumeOverride: {fileID: 0}
   m_ScaleInLightmap: 1
+  m_ReceiveGI: 1
   m_PreserveUVs: 0
   m_IgnoreNormalsForChartDetection: 0
   m_ImportantGI: 0
   m_MotionVectors: 1
   m_LightProbeUsage: 1
   m_ReflectionProbeUsage: 1
+  m_RayTracingMode: 2
   m_RenderingLayerMask: 1
   m_RendererPriority: 0
   m_Materials:
   m_ProbeAnchor: {fileID: 0}
   m_LightProbeVolumeOverride: {fileID: 0}
   m_ScaleInLightmap: 1
+  m_ReceiveGI: 1
   m_PreserveUVs: 0
   m_IgnoreNormalsForChartDetection: 0
   m_ImportantGI: 0
   m_MotionVectors: 1
   m_LightProbeUsage: 1
   m_ReflectionProbeUsage: 1
+  m_RayTracingMode: 2
   m_RenderingLayerMask: 1
   m_RendererPriority: 0
   m_Materials:
   m_ProbeAnchor: {fileID: 0}
   m_LightProbeVolumeOverride: {fileID: 0}
   m_ScaleInLightmap: 1
+  m_ReceiveGI: 1
   m_PreserveUVs: 0
   m_IgnoreNormalsForChartDetection: 0
   m_ImportantGI: 0
   m_MotionVectors: 1
   m_LightProbeUsage: 1
   m_ReflectionProbeUsage: 1
+  m_RayTracingMode: 2
   m_RenderingLayerMask: 1
   m_RendererPriority: 0
   m_Materials:
   m_ProbeAnchor: {fileID: 0}
   m_LightProbeVolumeOverride: {fileID: 0}
   m_ScaleInLightmap: 1
+  m_ReceiveGI: 1
   m_PreserveUVs: 1
   m_IgnoreNormalsForChartDetection: 0
   m_ImportantGI: 0
   m_MotionVectors: 1
   m_LightProbeUsage: 1
   m_ReflectionProbeUsage: 1
+  m_RayTracingMode: 2
   m_RenderingLayerMask: 1
   m_RendererPriority: 0
   m_Materials:
   m_ProbeAnchor: {fileID: 0}
   m_LightProbeVolumeOverride: {fileID: 0}
   m_ScaleInLightmap: 1
+  m_ReceiveGI: 1
   m_PreserveUVs: 1
   m_IgnoreNormalsForChartDetection: 0
   m_ImportantGI: 0
   m_MotionVectors: 1
   m_LightProbeUsage: 1
   m_ReflectionProbeUsage: 1
+  m_RayTracingMode: 2
   m_RenderingLayerMask: 1
   m_RendererPriority: 0
   m_Materials:
   m_ProbeAnchor: {fileID: 0}
   m_LightProbeVolumeOverride: {fileID: 0}
   m_ScaleInLightmap: 1
+  m_ReceiveGI: 1
   m_PreserveUVs: 0
   m_IgnoreNormalsForChartDetection: 0
   m_ImportantGI: 0
   m_MotionVectors: 1
   m_LightProbeUsage: 1
   m_ReflectionProbeUsage: 1
+  m_RayTracingMode: 2
   m_RenderingLayerMask: 1
   m_RendererPriority: 0
   m_Materials:
   m_ProbeAnchor: {fileID: 0}
   m_LightProbeVolumeOverride: {fileID: 0}
   m_ScaleInLightmap: 1
+  m_ReceiveGI: 1
   m_PreserveUVs: 1
   m_IgnoreNormalsForChartDetection: 0
   m_ImportantGI: 0
   m_MotionVectors: 1
   m_LightProbeUsage: 1
   m_ReflectionProbeUsage: 1
+  m_RayTracingMode: 2
   m_RenderingLayerMask: 1
   m_RendererPriority: 0
   m_Materials:
   m_ProbeAnchor: {fileID: 0}
   m_LightProbeVolumeOverride: {fileID: 0}
   m_ScaleInLightmap: 1
+  m_ReceiveGI: 1
   m_PreserveUVs: 1
   m_IgnoreNormalsForChartDetection: 0
   m_ImportantGI: 0
   m_MotionVectors: 1
   m_LightProbeUsage: 1
   m_ReflectionProbeUsage: 1
+  m_RayTracingMode: 2
   m_RenderingLayerMask: 1
   m_RendererPriority: 0
   m_Materials:
   m_ProbeAnchor: {fileID: 0}
   m_LightProbeVolumeOverride: {fileID: 0}
   m_ScaleInLightmap: 1
+  m_ReceiveGI: 1
   m_PreserveUVs: 0
   m_IgnoreNormalsForChartDetection: 0
   m_ImportantGI: 0
   m_MotionVectors: 1
   m_LightProbeUsage: 1
   m_ReflectionProbeUsage: 1
+  m_RayTracingMode: 2
   m_RenderingLayerMask: 1
   m_RendererPriority: 0
   m_Materials:
   m_ProbeAnchor: {fileID: 0}
   m_LightProbeVolumeOverride: {fileID: 0}
   m_ScaleInLightmap: 1
+  m_ReceiveGI: 1
   m_PreserveUVs: 0
   m_IgnoreNormalsForChartDetection: 0
   m_ImportantGI: 0
   m_MotionVectors: 1
   m_LightProbeUsage: 1
   m_ReflectionProbeUsage: 1
+  m_RayTracingMode: 2
   m_RenderingLayerMask: 1
   m_RendererPriority: 0
   m_Materials:
   m_ProbeAnchor: {fileID: 0}
   m_LightProbeVolumeOverride: {fileID: 0}
   m_ScaleInLightmap: 1
+  m_ReceiveGI: 1
   m_PreserveUVs: 1
   m_IgnoreNormalsForChartDetection: 0
   m_ImportantGI: 0
   m_ClearFlags: 2
   m_BackGroundColor: {r: 0.5043253, g: 0.5998091, b: 0.64705884, a: 0}
   m_projectionMatrixMode: 1
+  m_GateFitMode: 2
+  m_FOVAxisMode: 0
-  m_GateFitMode: 2
   m_FocalLength: 50
   m_NormalizedViewPortRect:
     serializedVersion: 2
   m_MotionVectors: 1
   m_LightProbeUsage: 1
   m_ReflectionProbeUsage: 1
+  m_RayTracingMode: 2
   m_RenderingLayerMask: 1
   m_RendererPriority: 0
   m_Materials:
   m_ProbeAnchor: {fileID: 0}
   m_LightProbeVolumeOverride: {fileID: 0}
   m_ScaleInLightmap: 1
+  m_ReceiveGI: 1
   m_PreserveUVs: 1
   m_IgnoreNormalsForChartDetection: 0
   m_ImportantGI: 0
   m_MotionVectors: 1
   m_LightProbeUsage: 1
   m_ReflectionProbeUsage: 1
+  m_RayTracingMode: 2
   m_RenderingLayerMask: 1
   m_RendererPriority: 0
   m_Materials:
   m_ProbeAnchor: {fileID: 0}
   m_LightProbeVolumeOverride: {fileID: 0}
   m_ScaleInLightmap: 1
+  m_ReceiveGI: 1
   m_PreserveUVs: 0
   m_IgnoreNormalsForChartDetection: 0
   m_ImportantGI: 0
   m_MotionVectors: 1
   m_LightProbeUsage: 1
   m_ReflectionProbeUsage: 1
+  m_RayTracingMode: 2
   m_RenderingLayerMask: 1
   m_RendererPriority: 0
   m_Materials:
   m_ProbeAnchor: {fileID: 0}
   m_LightProbeVolumeOverride: {fileID: 0}
   m_ScaleInLightmap: 1
+  m_ReceiveGI: 1
   m_PreserveUVs: 0
   m_IgnoreNormalsForChartDetection: 0
   m_ImportantGI: 0
   m_MotionVectors: 1
   m_LightProbeUsage: 1
   m_ReflectionProbeUsage: 1
+  m_RayTracingMode: 2
   m_RenderingLayerMask: 1
   m_RendererPriority: 0
   m_Materials:
   m_ProbeAnchor: {fileID: 0}
   m_LightProbeVolumeOverride: {fileID: 0}
   m_ScaleInLightmap: 1
+  m_ReceiveGI: 1
   m_PreserveUVs: 1
   m_IgnoreNormalsForChartDetection: 0
   m_ImportantGI: 0
     VectorActionDescriptions: []
     VectorActionSpaceType: 0
     hasUpgradedBrainParametersWithActionSpec: 1
-  m_Model: {fileID: 11400000, guid: 317f4f8da7e4846b3aae0969781824a2, type: 3}
+  m_Model: {fileID: 5022602860645237092, guid: 5e4f7c94351f346859ff7e00810f5d34, type: 3}
   m_InferenceDevice: 2
   m_BehaviorType: 0
   m_BehaviorName: Hallway
   m_MotionVectors: 1
   m_LightProbeUsage: 1
   m_ReflectionProbeUsage: 1
+  m_RayTracingMode: 2
   m_RenderingLayerMask: 1
   m_RendererPriority: 0
   m_Materials:
   m_ProbeAnchor: {fileID: 0}
   m_LightProbeVolumeOverride: {fileID: 0}
   m_ScaleInLightmap: 1
+  m_ReceiveGI: 1
   m_PreserveUVs: 0
   m_IgnoreNormalsForChartDetection: 0
   m_ImportantGI: 0
   m_ClearFlags: 2
   m_BackGroundColor: {r: 0.46666667, g: 0.5647059, b: 0.60784316, a: 1}
   m_projectionMatrixMode: 1
+  m_GateFitMode: 2
+  m_FOVAxisMode: 0
-  m_GateFitMode: 2
   m_FocalLength: 50
   m_NormalizedViewPortRect:
     serializedVersion: 2
   m_MotionVectors: 1
   m_LightProbeUsage: 1
   m_ReflectionProbeUsage: 1
+  m_RayTracingMode: 2
   m_RenderingLayerMask: 1
   m_RendererPriority: 0
   m_Materials:
   m_ProbeAnchor: {fileID: 0}
   m_LightProbeVolumeOverride: {fileID: 0}
   m_ScaleInLightmap: 1
+  m_ReceiveGI: 1
   m_PreserveUVs: 0
   m_IgnoreNormalsForChartDetection: 0
   m_ImportantGI: 0
   m_Component:
   - component: {fileID: 4726744827719472}
   m_Layer: 0
-  m_Name: SymbolFinderArea
+  m_Name: HallwayArea
   m_TagString: Untagged
   m_Icon: {fileID: 0}
   m_NavMeshLayer: 0
   m_MotionVectors: 1
   m_LightProbeUsage: 1
   m_ReflectionProbeUsage: 1
+  m_RayTracingMode: 2
   m_RenderingLayerMask: 1
   m_RendererPriority: 0
   m_Materials:
   m_ProbeAnchor: {fileID: 0}
   m_LightProbeVolumeOverride: {fileID: 0}
   m_ScaleInLightmap: 1
+  m_ReceiveGI: 1
   m_PreserveUVs: 1
   m_IgnoreNormalsForChartDetection: 0
   m_ImportantGI: 0
   m_MotionVectors: 1
   m_LightProbeUsage: 1
   m_ReflectionProbeUsage: 1
+  m_RayTracingMode: 2
   m_RenderingLayerMask: 1
   m_RendererPriority: 0
   m_Materials:
   m_ProbeAnchor: {fileID: 0}
   m_LightProbeVolumeOverride: {fileID: 0}
   m_ScaleInLightmap: 1
+  m_ReceiveGI: 1
   m_PreserveUVs: 1
   m_IgnoreNormalsForChartDetection: 0
   m_ImportantGI: 0

com.unity.ml-agents/CHANGELOG.md

 - `GridSensor` has been refactored and moved to main package, with changes to both sensor interfaces and behaviors.
 Exsisting GridSensor created by extension package will not work in newer version. Previously trained models will
 need to be retrained. Please see the Migration Guide for more details. (#5256)
+- Models trained with 1.x versions of ML-Agents will no longer work at inference if they were trained using recurrent neural networks (#5254)
 
 ### Minor Changes
 #### com.unity.ml-agents / com.unity.ml-agents.extensions (C#)

com.unity.ml-agents/Runtime/Inference/ApplierImpl.cs

             }
         }
     }
-
-    internal class BarracudaMemoryOutputApplier : TensorApplier.IApplier
-    {
-        readonly int m_MemoriesCount;
-        readonly int m_MemoryIndex;
-
-        Dictionary<int, List<float>> m_Memories;
-
-        public BarracudaMemoryOutputApplier(
-            int memoriesCount,
-            int memoryIndex,
-            Dictionary<int, List<float>> memories)
-        {
-            m_MemoriesCount = memoriesCount;
-            m_MemoryIndex = memoryIndex;
-            m_Memories = memories;
-        }
-
-        public void Apply(TensorProxy tensorProxy, IList<int> actionIds, Dictionary<int, ActionBuffers> lastActions)
-        {
-            var agentIndex = 0;
-            var memorySize = (int)tensorProxy.shape[tensorProxy.shape.Length - 1];
-
-            for (var i = 0; i < actionIds.Count; i++)
-            {
-                var agentId = actionIds[i];
-                List<float> memory;
-                if (!m_Memories.TryGetValue(agentId, out memory)
-                    || memory.Count < memorySize * m_MemoriesCount)
-                {
-                    memory = new List<float>();
-                    memory.AddRange(Enumerable.Repeat(0f, memorySize * m_MemoriesCount));
-                }
-
-                for (var j = 0; j < memorySize; j++)
-                {
-                    memory[memorySize * m_MemoryIndex + j] = tensorProxy.data[agentIndex, j];
-                }
-
-                m_Memories[agentId] = memory;
-                agentIndex++;
-            }
-        }
-    }
 }

com.unity.ml-agents/Runtime/Inference/BarracudaModelExtensions.cs

                 });
             }
 
-            var modelVersion = model.GetVersion();
-            if (modelVersion < (int)BarracudaModelParamLoader.ModelApiVersion.MLAgents2_0)
-            {
-                foreach (var mem in model.memories)
-                {
-                    tensors.Add(new TensorProxy
-                    {
-                        name = mem.input,
-                        valueType = TensorProxy.TensorType.FloatingPoint,
-                        data = null,
-                        shape = TensorUtils.TensorShapeFromBarracuda(mem.shape)
-                    });
-                }
-            }
-
             tensors.Sort((el1, el2) => string.Compare(el1.name, el2.name, StringComparison.InvariantCulture));
 
             return tensors;
             var memory = (int)model.GetTensorByName(TensorNames.MemorySize)[0];
             if (memory > 0)
             {
-                if (modelVersion < (int)BarracudaModelParamLoader.ModelApiVersion.MLAgents2_0)
-                {
-                    foreach (var mem in model.memories)
-                    {
-                        names.Add(mem.output);
-                    }
-                }
-                else
-                {
-                    names.Add(TensorNames.RecurrentOutput);
-                }
+                names.Add(TensorNames.RecurrentOutput);
             }
 
             names.Sort(StringComparer.InvariantCulture);

com.unity.ml-agents/Runtime/Inference/BarracudaModelParamLoader.cs

         }
 
         /// <summary>
+        /// Checks that a model has the appropriate version.
+        /// </summary>
+        /// <param name="model">
+        /// The Barracuda engine model for loading static parameters
+        /// </param>
+        /// <returns>A FailedCheck containing the error message if the version of the model does not mach, else null</returns>
+        public static FailedCheck CheckModelVersion(Model model)
+        {
+            var modelApiVersion = model.GetVersion();
+            if (modelApiVersion < (int)ModelApiVersion.MinSupportedVersion)
+            {
+                return FailedCheck.Error(
+                    "Model was trained with a older version of the trainer than is supported. " +
+                    "Either retrain with an newer trainer, or use an older version of com.unity.ml-agents.\n" +
+                    $"Model version: {modelApiVersion} Minimum supported version: {(int)ModelApiVersion.MinSupportedVersion}"
+                );
+            }
+
+            if (modelApiVersion > (int)ModelApiVersion.MaxSupportedVersion)
+            {
+                return FailedCheck.Error(
+                    "Model was trained with a newer version of the trainer than is supported. " +
+                    "Either retrain with an older trainer, or update to a newer version of com.unity.ml-agents.\n" +
+                    $"Model version: {modelApiVersion}  Maximum supported version: {(int)ModelApiVersion.MaxSupportedVersion}"
+                );
+            }
+
+            var memorySize = (int)model.GetTensorByName(TensorNames.MemorySize)[0];
+
+            if (modelApiVersion == (int)ModelApiVersion.MLAgents1_0 && memorySize > 0)
+            {
+                // This block is to make sure that models that are trained with MLAgents version 1.x and have
+                // an LSTM (i.e. use the barracuda _c and _h inputs and outputs) will not work with MLAgents version
+                // 2.x. This is because Barracuda version 2.x will eventually drop support for the _c and _h inputs
+                // and only ML-Agents 2.x models will be compatible.
+                return FailedCheck.Error(
+                    "Models from com.unity.ml-agents 1.x that use recurrent neural networks are not supported in newer versions. " +
+                    "Either retrain with an newer trainer, or use an older version of com.unity.ml-agents.\n"
+                );
+            }
+            return null;
+
+        }
+
+
+
+        /// <summary>
         /// Factory for the ModelParamLoader : Creates a ModelParamLoader and runs the checks
         /// on it.
         /// </summary>
             }
 
             var modelApiVersion = model.GetVersion();
-            if (modelApiVersion < (int)ModelApiVersion.MinSupportedVersion || modelApiVersion > (int)ModelApiVersion.MaxSupportedVersion)
+            var versionCheck = CheckModelVersion(model);
+            if (versionCheck != null)
-                failedModelChecks.Add(
-                    FailedCheck.Warning($"Version of the trainer the model was trained with ({modelApiVersion}) " +
-                    $"is not compatible with the current range of supported versions:  " +
-                    $"({(int)ModelApiVersion.MinSupportedVersion} to {(int)ModelApiVersion.MaxSupportedVersion}).")
-                    );
-                return failedModelChecks;
+                failedModelChecks.Add(versionCheck);
             }
 
             var memorySize = (int)model.GetTensorByName(TensorNames.MemorySize)[0];
             if (memory > 0)
             {
                 var modelVersion = model.GetVersion();
-                var netHasMemories = false;
-                if (modelVersion < (int)BarracudaModelParamLoader.ModelApiVersion.MLAgents2_0)
-                {
-                    netHasMemories = tensorsNames.Any(x => x.EndsWith("_h")) &&
-                        tensorsNames.Any(x => x.EndsWith("_c"));
-                }
-                else
-                {
-                    netHasMemories = tensorsNames.Any(x => x == TensorNames.RecurrentInPlaceholder);
-                }
-                if (!netHasMemories)
+                if (!tensorsNames.Any(x => x == TensorNames.RecurrentInPlaceholder))
                 {
                     failedModelChecks.Add(
                             FailedCheck.Warning("The model does not contain a Recurrent Input Node but has memory_size.")
             // If there is no Recurrent Output but the model is Recurrent.
             if (memory > 0)
             {
-
-                var netHasMemories = false;
-                var modelVersion = model.GetVersion();
-                if (modelVersion < (int)BarracudaModelParamLoader.ModelApiVersion.MLAgents2_0)
-                {
-                    var memOutputs = model.memories.Select(x => x.output).ToList();
-                    netHasMemories = memOutputs.Any(x => x.EndsWith("_h")) &&
-                        memOutputs.Any(x => x.EndsWith("_c"));
-                }
-                else
-                {
-                    var allOutputs = model.GetOutputNames().ToList();
-                    netHasMemories = allOutputs.Any(x => x == TensorNames.RecurrentOutput);
-                }
-                if (!netHasMemories)
+                var allOutputs = model.GetOutputNames().ToList();
+                if (!allOutputs.Any(x => x == TensorNames.RecurrentOutput))
                 {
                     failedModelChecks.Add(
                         FailedCheck.Warning("The model does not contain a Recurrent Output Node but has memory_size.")

com.unity.ml-agents/Runtime/Inference/GeneratorImpl.cs

         }
     }
 
-    internal class BarracudaRecurrentInputGenerator : TensorGenerator.IGenerator
-    {
-        int m_MemoriesCount;
-        readonly int m_MemoryIndex;
-        readonly ITensorAllocator m_Allocator;
-
-        Dictionary<int, List<float>> m_Memories;
-
-        public BarracudaRecurrentInputGenerator(
-            int memoryIndex,
-            ITensorAllocator allocator,
-            Dictionary<int, List<float>> memories)
-        {
-            m_MemoryIndex = memoryIndex;
-            m_Allocator = allocator;
-            m_Memories = memories;
-        }
-
-        public void Generate(TensorProxy tensorProxy, int batchSize, IList<AgentInfoSensorsPair> infos)
-        {
-            TensorUtils.ResizeTensor(tensorProxy, batchSize, m_Allocator);
-
-            var memorySize = (int)tensorProxy.shape[tensorProxy.shape.Length - 1];
-            var agentIndex = 0;
-            for (var infoIndex = 0; infoIndex < infos.Count; infoIndex++)
-            {
-                var infoSensorPair = infos[infoIndex];
-                var info = infoSensorPair.agentInfo;
-                var offset = memorySize * m_MemoryIndex;
-                List<float> memory;
-                if (info.done)
-                {
-                    m_Memories.Remove(info.episodeId);
-                }
-                if (!m_Memories.TryGetValue(info.episodeId, out memory))
-                {
-                    for (var j = 0; j < memorySize; j++)
-                    {
-                        tensorProxy.data[agentIndex, j] = 0;
-                    }
-                    agentIndex++;
-                    continue;
-                }
-                for (var j = 0; j < memorySize; j++)
-                {
-                    if (j >= memory.Count)
-                    {
-                        break;
-                    }
-
-                    tensorProxy.data[agentIndex, j] = memory[j + offset];
-                }
-                agentIndex++;
-            }
-        }
-    }
-
     /// <summary>
     /// Generates the Tensor corresponding to the Previous Action input : Will be a two
     /// dimensional integer array of dimension [batchSize x actionSize].

com.unity.ml-agents/Runtime/Inference/ModelRunner.cs

                 D.logEnabled = m_Verbose;
 
                 barracudaModel = ModelLoader.Load(model);
+
+                var failedCheck = BarracudaModelParamLoader.CheckModelVersion(
+                        barracudaModel
+                    );
+                if (failedCheck != null)
+                {
+                    if (failedCheck.CheckType == BarracudaModelParamLoader.FailedCheck.CheckTypeEnum.Error)
+                    {
+                        throw new UnityAgentsException(failedCheck.Message);
+                    }
+                }
+
                 WorkerFactory.Type executionDevice;
                 switch (inferenceDevice)
                 {

com.unity.ml-agents/Runtime/Inference/TensorApplier.cs

                 }
             }
             m_Dict[TensorNames.RecurrentOutput] = new MemoryOutputApplier(memories);
-
-            if (modelVersion < (int)BarracudaModelParamLoader.ModelApiVersion.MLAgents2_0)
-            {
-                for (var i = 0; i < model?.memories.Count; i++)
-                {
-                    m_Dict[model.memories[i].output] =
-                        new BarracudaMemoryOutputApplier(model.memories.Count, i, memories);
-                }
-            }
         }
 
         /// <summary>

com.unity.ml-agents/Runtime/Inference/TensorGenerator.cs

             m_Dict[TensorNames.RecurrentInPlaceholder] =
                 new RecurrentInputGenerator(allocator, memories);
 
-            if (m_ApiVersion < (int)BarracudaModelParamLoader.ModelApiVersion.MLAgents2_0)
-            {
-                for (var i = 0; i < model.memories.Count; i++)
-                {
-                    m_Dict[model.memories[i].input] =
-                        new BarracudaRecurrentInputGenerator(i, allocator, memories);
-                }
-            }
-
             m_Dict[TensorNames.PreviousActionPlaceholder] =
                 new PreviousActionInputGenerator(allocator);
             m_Dict[TensorNames.ActionMaskPlaceholder] =

com.unity.ml-agents/Tests/Editor/Inference/ModelRunnerTest.cs

     [TestFixture]
     public class ModelRunnerTest
     {
+        const string k_hybrid_ONNX_recurr_v2 = "Packages/com.unity.ml-agents/Tests/Editor/TestModels/hybrid0vis8vec_2c_2_3d_v2_0.onnx";
+
-        const string k_discreteONNXPath = "Packages/com.unity.ml-agents/Tests/Editor/TestModels/discrete1vis0vec_2_3action_recurr_v1_0.onnx";
+        const string k_discreteONNXPath = "Packages/com.unity.ml-agents/Tests/Editor/TestModels/discrete1vis0vec_2_3action_obsolete_recurr_v1_0.onnx";
+        NNModel hybridONNXModelV2;
         NNModel continuousONNXModel;
         NNModel discreteONNXModel;
         NNModel hybridONNXModel;
         [SetUp]
         public void SetUp()
         {
+            hybridONNXModelV2 = (NNModel)AssetDatabase.LoadAssetAtPath(k_hybrid_ONNX_recurr_v2, typeof(NNModel));
+
             continuousONNXModel = (NNModel)AssetDatabase.LoadAssetAtPath(k_continuousONNXPath, typeof(NNModel));
             discreteONNXModel = (NNModel)AssetDatabase.LoadAssetAtPath(k_discreteONNXPath, typeof(NNModel));
             hybridONNXModel = (NNModel)AssetDatabase.LoadAssetAtPath(k_hybridONNXPath, typeof(NNModel));
             Assert.IsNotNull(hybridONNXModel);
             Assert.IsNotNull(continuousNNModel);
             Assert.IsNotNull(discreteNNModel);
+            Assert.IsNotNull(hybridONNXModelV2);
         }
 
         [Test]
             var modelRunner = new ModelRunner(continuousONNXModel, GetContinuous2vis8vec2actionActionSpec(), inferenceDevice);
             modelRunner.Dispose();
-            modelRunner = new ModelRunner(discreteONNXModel, GetDiscrete1vis0vec_2_3action_recurrModelActionSpec(), inferenceDevice);
-            modelRunner.Dispose();
+            Assert.Throws<UnityAgentsException>(() =>
+            {
+                // Cannot load a model trained with 1.x that has an LSTM
+                modelRunner = new ModelRunner(discreteONNXModel, GetDiscrete1vis0vec_2_3action_recurrModelActionSpec(), inferenceDevice);
+                modelRunner.Dispose();
+            });
-            modelRunner = new ModelRunner(discreteNNModel, GetDiscrete1vis0vec_2_3action_recurrModelActionSpec(), inferenceDevice);
+
+            Assert.Throws<UnityAgentsException>(() =>
+            {
+                // Cannot load a model trained with 1.x that has an LSTM
+                modelRunner = new ModelRunner(discreteNNModel, GetDiscrete1vis0vec_2_3action_recurrModelActionSpec(), inferenceDevice);
+                modelRunner.Dispose();
+            });
+            // This one was trained with 2.0 so it should not raise an error:
+            modelRunner = new ModelRunner(hybridONNXModelV2, new ActionSpec(2, new[] { 2, 3 }), inferenceDevice);
             modelRunner.Dispose();
         }
 
         [Test]
         public void TestRunModel()
         {
-            var actionSpec = GetDiscrete1vis0vec_2_3action_recurrModelActionSpec();
-            var modelRunner = new ModelRunner(discreteONNXModel, actionSpec, InferenceDevice.Burst);
+            var actionSpec = GetContinuous2vis8vec2actionActionSpec();
+            var modelRunner = new ModelRunner(continuousONNXModel, actionSpec, InferenceDevice.Burst);
+            var sensor_8 = new Sensors.VectorSensor(8, "VectorSensor8");
-            modelRunner.PutObservations(info1, new[] { sensor_21_20_3.CreateSensors()[0] }.ToList());
+            modelRunner.PutObservations(info1, new[] {
+                sensor_8,
+                sensor_21_20_3.CreateSensors()[0],
+                sensor_20_22_3.CreateSensors()[0] }.ToList());
-            modelRunner.PutObservations(info2, new[] { sensor_21_20_3.CreateSensors()[0] }.ToList());
+            modelRunner.PutObservations(info2, new[] {
+                sensor_8,
+                sensor_21_20_3.CreateSensors()[0],
+                sensor_20_22_3.CreateSensors()[0] }.ToList());
 
             modelRunner.DecideBatch();
 

com.unity.ml-agents/Tests/Editor/Inference/ParameterLoaderTest.cs

     public class ParameterLoaderTest
     {
         const string k_discrete_ONNX_v2 = "Packages/com.unity.ml-agents/Tests/Editor/TestModels/discrete_rank2_vector_v2_0.onnx";
+        const string k_hybrid_ONNX_recurr_v2 = "Packages/com.unity.ml-agents/Tests/Editor/TestModels/hybrid0vis8vec_2c_2_3d_v2_0.onnx";
+
-        const string k_discreteONNXPath = "Packages/com.unity.ml-agents/Tests/Editor/TestModels/discrete1vis0vec_2_3action_recurr_v1_0.onnx";
+        const string k_discreteONNXPath = "Packages/com.unity.ml-agents/Tests/Editor/TestModels/discrete1vis0vec_2_3action_obsolete_recurr_v1_0.onnx";
         const string k_hybridONNXPath = "Packages/com.unity.ml-agents/Tests/Editor/TestModels/hybrid0vis53vec_3c_2daction_v1_0.onnx";
         // NN model with single action output (deprecated, does not support hybrid action).
         // Same BrainParameters settings as the corresponding ONNX model.
         NNModel rank2ONNXModel;
+        NNModel hybridRecurrV2Model;
         NNModel continuousONNXModel;
         NNModel discreteONNXModel;
         NNModel hybridONNXModel;
             return validBrainParameters;
         }
 
+        BrainParameters GetRecurrHybridBrainParameters()
+        {
+            var validBrainParameters = new BrainParameters();
+            validBrainParameters.VectorObservationSize = 8;
+            validBrainParameters.NumStackedVectorObservations = 1;
+            validBrainParameters.ActionSpec = new ActionSpec(2, new int[] { 2, 3 });
+            return validBrainParameters;
+        }
+
         [SetUp]
         public void SetUp()
         {
             continuousNNModel = (NNModel)AssetDatabase.LoadAssetAtPath(k_continuousNNPath, typeof(NNModel));
             discreteNNModel = (NNModel)AssetDatabase.LoadAssetAtPath(k_discreteNNPath, typeof(NNModel));
             rank2ONNXModel = (NNModel)AssetDatabase.LoadAssetAtPath(k_discrete_ONNX_v2, typeof(NNModel));
+            hybridRecurrV2Model = (NNModel)AssetDatabase.LoadAssetAtPath(k_hybrid_ONNX_recurr_v2, typeof(NNModel));
             var go = new GameObject("SensorA");
             sensor_21_20_3 = go.AddComponent<Test3DSensorComponent>();
             sensor_21_20_3.Sensor = new Test3DSensor("SensorA", 21, 20, 3);
             Assert.IsNotNull(continuousNNModel);
             Assert.IsNotNull(discreteNNModel);
             Assert.IsNotNull(rank2ONNXModel);
+            Assert.IsNotNull(hybridRecurrV2Model);
         }
 
         [TestCase(true)]
                 model, validBrainParameters,
                 new ISensor[] { sensor_21_20_3.CreateSensors()[0] }, new ActuatorComponent[0]
             );
-            Assert.AreEqual(0, errors.Count()); // There should not be any errors
+            foreach (var e in errors)
+            {
+                Debug.Log(e.Message);
+            }
+            Assert.Greater(errors.Count(), 0); // There should be an error since LSTM v1.x is not supported
+        }
+
+        [Test]
+        public void TestCheckModelValidRecurrent()
+        {
+            var model = ModelLoader.Load(hybridRecurrV2Model);
+            var num_errors = 0; // A model trained with v2 should not raise errors
+            var validBrainParameters = GetRecurrHybridBrainParameters();
+
+            var errors = BarracudaModelParamLoader.CheckModel(
+                model, validBrainParameters,
+                new ISensor[] { sensor_8 }, new ActuatorComponent[0]
+            );
+            Assert.AreEqual(num_errors, errors.Count()); // There should not be any errors
+
+            var invalidBrainParameters = GetRecurrHybridBrainParameters();
+            invalidBrainParameters.ActionSpec = new ActionSpec(1, new int[] { 2, 3 });
+            errors = BarracudaModelParamLoader.CheckModel(
+                model, invalidBrainParameters,
+                new ISensor[] { sensor_8 }, new ActuatorComponent[0]
+            );
+            Assert.AreEqual(1, errors.Count()); // 1 continuous action instead of 2
+
+            invalidBrainParameters.ActionSpec = new ActionSpec(2, new int[] { 3, 2 });
+            errors = BarracudaModelParamLoader.CheckModel(
+                model, invalidBrainParameters,
+                new ISensor[] { sensor_8 }, new ActuatorComponent[0]
+            );
+            Assert.AreEqual(1, errors.Count()); // Discrete action branches flipped
         }
 
         [Test]

com.unity.ml-agents/Tests/Editor/TestModels/discrete1vis0vec_2_3action_v1_0.onnx.meta

 fileFormatVersion: 2
-guid: 9383297608d1d4530807d7109ee19d85
+guid: 68991653e04394f95b15a222253c0729
 ScriptedImporter:
   fileIDToRecycleName:
     11400000: main obj

docs/Migrating.md

 * The sensor will not further encode the data recieved from `GetObjectData()` anymore. The values
 recieved from `GetObjectData()` will be the observation sent to the trainer.
 
+### LSTM models from previous releases no longer supported
+The way the Unity Inference Engine processes LSTM (recurrent neural networks) has changed. As a result, models
+trained with previous versions of ML-Agents will not be usable at inference if they were trained with a `memory`
+setting in the `.yaml` config file.
+If you want to use a model that has a recurrent neural network in this release of ML-Agents, you need to train
+the model using the python trainer from this release.
+
 
 ## Migrating to Release 13
 ### Implementing IHeuristic in your IActuator implementations

Project/Assets/ML-Agents/Examples/Hallway/TFModels/Hallway.onnx.meta

+fileFormatVersion: 2
+guid: 5e4f7c94351f346859ff7e00810f5d34
+ScriptedImporter:
+  internalIDToNameTable: []
+  externalObjects: {}
+  serializedVersion: 2
+  userData: 
+  assetBundleName: 
+  assetBundleVariant: 
+  script: {fileID: 11500000, guid: 683b6cb6d0a474744822c888b46772c9, type: 3}
+  optimizeModel: 1
+  forceArbitraryBatchSize: 1
+  treatErrorsAsWarnings: 0
+  importMode: 1

com.unity.ml-agents/Tests/Editor/TestModels/discrete1vis0vec_2_3action_obsolete_recurr_v1_0.onnx.meta

+fileFormatVersion: 2
+guid: 9ecb2a56b0c6b42f7ad2b40ab97c5515
+ScriptedImporter:
+  internalIDToNameTable: []
+  externalObjects: {}
+  serializedVersion: 2
+  userData: 
+  assetBundleName: 
+  assetBundleVariant: 
+  script: {fileID: 11500000, guid: 683b6cb6d0a474744822c888b46772c9, type: 3}
+  optimizeModel: 1
+  forceArbitraryBatchSize: 1
+  treatErrorsAsWarnings: 0
+  importMode: 1

com.unity.ml-agents/Tests/Editor/TestModels/hybrid0vis8vec_2c_2_3d_v2_0.onnx.meta

+fileFormatVersion: 2
+guid: 2f6b2ae61d96a4555b60892a0ad924bb
+ScriptedImporter:
+  internalIDToNameTable: []
+  externalObjects: {}
+  serializedVersion: 2
+  userData: 
+  assetBundleName: 
+  assetBundleVariant: 
+  script: {fileID: 11500000, guid: 683b6cb6d0a474744822c888b46772c9, type: 3}
+  optimizeModel: 1
+  forceArbitraryBatchSize: 1
+  treatErrorsAsWarnings: 0
+  importMode: 1

Project/Assets/ML-Agents/Examples/Hallway/TFModels/Hallway.nn.meta

-fileFormatVersion: 2
-guid: 317f4f8da7e4846b3aae0969781824a2
-ScriptedImporter:
-  fileIDToRecycleName:
-    11400000: main obj
-    11400002: model data
-  externalObjects: {}
-  userData: 
-  assetBundleName: 
-  assetBundleVariant: 
-  script: {fileID: 11500000, guid: 19ed1486aa27d4903b34839f37b8f69f, type: 3}