Merge master to 0.8.0 (#265)

* addressed feedback

- new test for keypoint labeler
- added ClearMaterialProperties method to IGroundTruthGenerator
- Other fixes

* docs clarification

* Update CHANGELOG.md

* added missing interface method implementation for HDRP

* Amending dataset schema to describe correct coordinate system (#255)

The summary for the 3d bounding box states the coordinates are from the sensor's reference frame, but the string for the translational component says it's global.  I *think* this is just a typo in the doc, so correcting the description to match the summary for this type of entry.

* Adding supported version badges (#256)

* Cloud rendering support (#227)

* Adding support for cloud rendering in the run simulation window

* minor fixes

* minor update

* Update BoundingBox3DLabeler.cs

* Update BoundingBox3DLabeler.cs

* inner mesh scaling and disabled renderes

* A fix that prevents normal sampler returning values outside of min max (#258)

* Update pull_request_template.md (#257)

* Update com.unity.perception/CHANGELOG.md

* Increasing color variety in instance segmentation. (#261)

* Increasing color variety in instance segmentation.

* Fixing duplicated color issues in InstanceIdToColorMapping introduced after expanding range.

* Using other constants in constant for cleanliness

Co-authored-by: Mohsen Kamalzadeh <>
Co-authored-by: Wesley Smith <[email protected]>
Co-authored-by: Mohsen K <[email protected]>
Co-authored-by: Devin Miller (Unity) <[email protected]>
Co-authored-by: PriyeshWani <[email protected]>
Co-authored-by: Steve Borkman <[email protected]>

Author: 6 people

.github/pull_request_template.md

@@ -1,8 +1,7 @@
 # Peer Review Information:
-Information on any code, feature, documentation changes here
 
 ## Editor / Package versioning:
-**Editor Version Target (i.e. 19.3, 20.1)**: 2019.3
+**Editor Version Target**: 2019.4
 
 ## Dev Testing:
 **Tests Added**: 
@@ -11,8 +10,7 @@ Information on any code, feature, documentation changes here
 
 **At Risk Areas**: 
 
-**Notes + Expectations**: 
-
 ## Checklist
 - [ ] - Updated docs
 - [ ] - Updated changelog
+- [ ] - Updated test rail

README.md

@@ -6,6 +6,9 @@
 
 [![license badge](https://img.shields.io/badge/license-Apache--2.0-green.svg)](LICENSE.md)
 
+<img src="https://img.shields.io/badge/unity-2019.4-green.svg?style=flat-square" alt="unity 2019.4">
+<img src="https://img.shields.io/badge/unity-2020.2-green.svg?style=flat-square" alt="unity 2020.2">
+
 > com.unity.perception is in active development. Its features and API are subject to significant change as development progresses.
 
 

TestProjects/PerceptionHDRP/Packages/packages-lock.json

@@ -81,7 +81,7 @@
         "com.unity.collections": "0.9.0-preview.6",
         "com.unity.nuget.newtonsoft-json": "1.1.2",
         "com.unity.render-pipelines.core": "7.1.6",
-        "com.unity.simulation.capture": "0.0.10-preview.19",
+        "com.unity.simulation.capture": "0.0.10-preview.20",
         "com.unity.simulation.client": "0.0.10-preview.10",
         "com.unity.simulation.core": "0.0.10-preview.22"
       }
@@ -126,7 +126,7 @@
       "url": "https://packages.unity.com"
     },
     "com.unity.simulation.capture": {
-      "version": "0.0.10-preview.19",
+      "version": "0.0.10-preview.20",
       "depth": 1,
       "source": "registry",
       "dependencies": {

com.unity.perception/CHANGELOG.md

@@ -16,6 +16,7 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.
 ### Changed
 Expanded documentation on the Keypoint Labeler
 Updated Keypoint Labeler logic to only report keypoints for visible objects by default
+Increased color variety in instance segmentation images
 
 ### Deprecated
 
@@ -25,6 +26,8 @@ Updated Keypoint Labeler logic to only report keypoints for visible objects by d
 
 Fixed compiler warnings in projects with HDRP on 2020.1 and later
 
+Fixed a bug in the Normal Sampler where it would return values less than the passed in minimum value, or greater than the passed in maximum value, for random values very close to 0 or 1 respectively.
+
 ## [0.8.0-preview.2] - 2021-03-15
 
 ### Upgrade Notes
@@ -33,6 +36,8 @@ All appearances of the term `KeyPoint` have been renamed to `Keypoint`. If you h
 
 `ScenarioBase`'s `Awake()`, `Start()`, and `Update()` functions are now private. If you previously used these, replace the usages with `OnAwake()`, `OnStart()`, and `OnUpdate()`.
 
+The interface `IGroundTruthGenerator` now contains a new method named `ClearMaterialProperties` for disabling ground truth generation on a `Labeling` component or its associated `MaterialPropertyBlock`. Update your implementing classes to including this method.
+
 ### Known Issues
 
 ### Added
@@ -55,7 +60,7 @@ The newly added `LabelManager` class now enables custom Labelers to access the l
 
 Improved UI for `KeypointTemplate` and added useful default colors for keypoint and skeleton definitions.
 
-Added the ability to switch ground-truth labeling on or off for an object at runtime by enabling or disabling its `Labeling` component.
+Added the ability to switch ground truth generation on or off for an object at runtime by enabling or disabling its `Labeling` component. A new method named `ClearMaterialProperties()` in `IGroundTruthGenerator` handles this functionality.
 
 ### Changed
 

com.unity.perception/Documentation~/Schema/Synthetic_Dataset_Schema.md

@@ -216,7 +216,7 @@ bounding_box_3d {
   label_id:      <int>   -- Integer identifier of the label
   label_name:    <str>   -- String identifier of the label
   instance_id:   <str>   -- UUID of the instance.
-  translation {          -- 3d bounding box's center location in meters with respect to global coordinate system.
+  translation {          -- 3d bounding box's center location in meters with respect to the sensor's coordinate system
     x:           <float> -- The x coordinate
     y:           <float> -- The y coordinate
     z:           <float> -- The z coordinate

com.unity.perception/Editor/Randomization/Editors/RunInUnitySimulationWindow.cs

@@ -35,6 +35,8 @@ class RunInUnitySimulationWindow : EditorWindow
         Label m_PrevExecutionIdLabel;
         RunParameters m_RunParameters;
 
+        const string  m_SupportedGPUString = "NVIDIA";
+
         [MenuItem("Window/Run in Unity Simulation")]
         static void ShowWindow()
         {
@@ -166,6 +168,15 @@ static string IncrementRunName(string runName)
 
         async void RunInUnitySimulation()
         {
+            #if PLATFORM_CLOUD_RENDERING
+            if (!m_SysParamDefinitions[m_SysParamIndex].description.Contains(m_SupportedGPUString))
+            {
+                EditorUtility.DisplayDialog("Unsupported Sysparam",
+                    "The current selection of the Sysparam " + m_SysParamDefinitions[m_SysParamIndex].description +
+                    " is not supported by this build target. Please select a sysparam with GPU", "Ok");
+                return;
+            }
+            #endif
             m_RunParameters = new RunParameters
             {
                 runName = m_RunNameField.value,
@@ -228,7 +239,11 @@ void CreateLinuxBuildAndZip()
             {
                 scenes = new[] { m_RunParameters.currentOpenScenePath },
                 locationPathName = Path.Combine(projectBuildDirectory, $"{m_RunParameters.runName}.x86_64"),
+#if PLATFORM_CLOUD_RENDERING
+                target = BuildTarget.CloudRendering,
+#else
                 target = BuildTarget.StandaloneLinux64
+#endif
             };
             var report = BuildPipeline.BuildPlayer(buildPlayerOptions);
             var summary = report.summary;

com.unity.perception/Runtime/GroundTruth/IGroundTruthGenerator.cs

@@ -8,12 +8,21 @@ namespace UnityEngine.Perception.GroundTruth
     public interface IGroundTruthGenerator
     {
         /// <summary>
-        /// Called by <see cref="LabelManager"/> when first registered or when a Labeling is created at runtime.
+        /// Enables ground truth generation for a <see cref="Labeling"/> component or its associated <see cref="MaterialPropertyBlock"/>. This function is called by <see cref="LabelManager"/> when a <see cref="Labeling"/> component is registered, created, or enabled.
         /// </summary>
-        /// <param name="mpb">The MaterialPropertyBlock for the given meshRenderer. Can be used to set properties for custom rendering.</param>
-        /// <param name="renderer">The Renderer under the given Labeling.</param>
-        /// <param name="labeling">The Labeling component created</param>
-        /// <param name="instanceId">The instanceId assigned to the given Labeling instance.</param>
+        /// <param name="mpb">The <see cref="MaterialPropertyBlock"/> for the given <see cref="MeshRenderer"/>. Can be used to set properties for custom rendering.</param>
+        /// <param name="renderer">The <see cref="Renderer"/> under the given <see cref="LabelManager"/>.</param>
+        /// <param name="labeling">The <see cref="LabelManager"/> component that was registered, created, or enabled</param>
+        /// <param name="instanceId">The instanceId assigned to the given <see cref="LabelManager"/> instance.</param>
         void SetupMaterialProperties(MaterialPropertyBlock mpb, Renderer renderer, Labeling labeling, uint instanceId);
+
+        /// <summary>
+        /// Disables ground truth generation for a <see cref="Labeling"/> component or its associated <see cref="MaterialPropertyBlock"/>. This function is called by <see cref="LabelManager"/> when a <see cref="Labeling"/> component is disabled.
+        /// </summary>
+        /// <param name="mpb">The <see cref="MaterialPropertyBlock"/> for the given <see cref="MeshRenderer"/>. Can be used to set properties for custom rendering.</param>
+        /// <param name="renderer">The <see cref="Renderer"/> under the given <see cref="LabelManager"/>.</param>
+        /// <param name="labeling">The <see cref="LabelManager"/> component for which ground-truth generation should stop.</param>
+        /// <param name="instanceId">The instanceId assigned to the given <see cref="LabelManager"/> instance.</param>
+        void ClearMaterialProperties(MaterialPropertyBlock mpb, Renderer renderer, Labeling labeling, uint instanceId);
     }
 }

com.unity.perception/Runtime/GroundTruth/InstanceIdToColorMapping.cs

@@ -1,5 +1,6 @@
 using System;
 using System.Collections.Generic;
+using Unity.Profiling;
 
 namespace UnityEngine.Perception.GroundTruth
 {
@@ -21,42 +22,53 @@ public static class InstanceIdToColorMapping
         /// </summary>
         public const uint maxId = uint.MaxValue - ((256 * 256 * 256) * 2) + k_HslCount;
 
-        static Dictionary<uint, uint> s_IdToColorCache;
+        static uint[] s_IdToColorCache;
         static Dictionary<uint, uint> s_ColorToIdCache;
-        const uint k_HslCount = 64;
+        const uint k_HslCount = 1024;
         const uint k_ColorsPerAlpha = 256 * 256 * 256;
         const uint k_InvalidPackedColor = 255; // packed uint for color (0, 0, 0, 255);
-        public static readonly Color32 invalidColor = new Color(0, 0, 0, 255);
         static readonly float k_GoldenRatio = (1 + Mathf.Sqrt(5)) / 2;
-        const int k_HuesInEachValue = 30;
-
-        static void InitializeMaps()
-        {
+        const int k_HuesInEachValue = 64;
+        const uint k_Values = k_HslCount / k_HuesInEachValue;
 
-            s_IdToColorCache = new Dictionary<uint, uint>();
-            s_ColorToIdCache = new Dictionary<uint, uint>();
-
-            s_IdToColorCache[0] = k_InvalidPackedColor;
-            s_IdToColorCache[k_InvalidPackedColor] = 0;
+        /// <summary>
+        /// The color returned when an instanceId is not mapped to any color, and for clearing ground truth material properties on a <see cref="MaterialPropertyBlock"/>.
+        /// </summary>
+        public static readonly Color32 invalidColor = new Color(0, 0, 0, 255);
 
-            for (uint i = 1; i <= k_HslCount; i++)
+        private static ProfilerMarker k_InitializeMapsMarker = new ProfilerMarker(nameof(InitializeMaps));
+        internal static void InitializeMaps()
+        {
+            using (k_InitializeMapsMarker.Auto())
             {
-                var color = GenerateHSLValueForId(i);
-                s_IdToColorCache[i] = color;
-                s_ColorToIdCache[color] = i;
+                s_IdToColorCache = new uint[k_HslCount + 1];
+                s_ColorToIdCache = new Dictionary<uint, uint>();
+
+                s_IdToColorCache[0] = k_InvalidPackedColor;
+                s_ColorToIdCache[k_InvalidPackedColor] = 0;
+
+                for (uint i = 1; i <= k_HslCount; i++)
+                {
+                    var color = GenerateHSLValueForId(i);
+                    s_IdToColorCache[i] = color;
+                    s_ColorToIdCache.Add(color, i);
+                }
             }
         }
 
         static uint GenerateHSLValueForId(uint count)
         {
             count -= 1;
 
-            var ratio = count * k_GoldenRatio;
+            // assign hue based on golden ratio
+            var hueId = count % k_HuesInEachValue;
+            var ratio = hueId * k_GoldenRatio;
             var hue = ratio - Mathf.Floor(ratio);
 
-            count /= k_HuesInEachValue;
-            ratio = count * k_GoldenRatio;
-            var value = 1 - (ratio - Mathf.Floor(ratio));
+            var valueId = count / k_HuesInEachValue;
+
+            // avoid value 0
+            var value = 1 - (float)valueId / (k_Values + 1);
 
             var color = (Color32)Color.HSVToRGB(hue, 1f, value);
             color.a = 255;

com.unity.perception/Runtime/GroundTruth/Labelers/BoundingBox3DLabeler.cs

@@ -248,6 +248,9 @@ void ProcessLabel(Labeling labeledEntity)
                     // Need to convert all bounds into labeling mesh space...
                     foreach (var mesh in meshFilters)
                     {
+                        if (!mesh.GetComponent<Renderer>().enabled)
+                            continue;
+
                         var currentTransform = mesh.gameObject.transform;
                         // Grab the bounds of the game object from the mesh, although these bounds are axis-aligned,
                         // they are axis-aligned with respect to the current component's coordinate space. This, in theory
@@ -260,6 +263,8 @@ void ProcessLabel(Labeling labeledEntity)
                         // Apply the transformations on this object until we reach the labeled transform
                         while (currentTransform != labelTransform)
                         {
+                            transformedBounds.center = Vector3.Scale(transformedBounds.center, currentTransform.localScale);
+                            transformedBounds.center = currentTransform.localRotation * transformedBounds.center;
                             transformedBounds.center += currentTransform.localPosition;
                             transformedBounds.extents = Vector3.Scale(transformedBounds.extents, currentTransform.localScale);
                             transformedRotation *= currentTransform.localRotation;
@@ -287,7 +292,7 @@ void ProcessLabel(Labeling labeledEntity)
 
                     // Convert the combined bounds into world space
                     combinedBounds.center = labelTransform.TransformPoint(combinedBounds.center);
-                    combinedBounds.extents = Vector3.Scale(combinedBounds.extents,  labelTransform.localScale);
+                    combinedBounds.extents = Vector3.Scale(combinedBounds.extents,  labelTransform.lossyScale);
 
                     // Now convert all points into camera's space
                     var cameraCenter = cameraTransform.InverseTransformPoint(combinedBounds.center);