com.unity.test-framework.performance@2.8.0-preview

## [2.8.0] - 2021-03-16

Add overloads to measurements for overriding sample unit
Fix cases where cleanup throws an exception

## [2.7.0] - 2021-02-19

Reduce metadata overhead when running locally by caching dependencies
Remove the need for link.xml
Restructured documentation
Fixed method measurement IterationsPerMeasurement

Author: Unity Technologies

CHANGELOG.md

@@ -1,5 +1,10 @@
 # Changelog
 
+## [2.8.0] - 2021-03-16
+
+Add overloads to measurements for overriding sample unit
+Fix cases where cleanup throws an exception
+
 ## [2.7.0] - 2021-02-19
 
 Reduce metadata overhead when running locally by caching dependencies

Documentation~/measure-frames.md

@@ -6,7 +6,7 @@ Records time per frame by default and provides additional properties/methods to
 * **DontRecordFrametime()** - disables frametime measurement.
 * **ProfilerMarkers(...)** - sample profile markers per frame. Does not work for deep profiling and `Profiler.BeginSample()`
 * **SampleGroup(string name)** - name of the measurement, defaults to "Time" if unspecified.
-* **Scope()** - measures frame times in a given coroutine scope.
+* **Scope()** - measures frame times in a given coroutine scope. By default it uses a SampleGroup named "Time" with Milliseconds as measurement unit. You can also create your own SampleGroup, specifying a custom name and the measurement unit you want your results in, see [example 5](#example-5-specify-custom-samplegroup-in-the-scope).
 
 
 #### Example 1: Simple frame time measurement using default values of at least 7 frames and default WarmupCount (see description above).
@@ -67,3 +67,34 @@ public IEnumerator Test()
         .Run();
 }
 ```
+#### Example 5: Specify Custom SampleGroup in the Scope
+
+``` csharp
+[UnityTest, Performance]
+public IEnumerator Test()
+{
+    var sg = new SampleGroup("MarkerName", SampleUnit.Second);
+    using (Measure.Frames().Scope(sg))
+    {
+        yield return ...;
+    }
+}
+```
+
+#### Example 6: Sample profile markers per frame with custom SampleGroups that change sample unit
+
+``` csharp
+[UnityTest, Performance]
+public IEnumerator Test()
+{
+    var sampleGroup = new SampleGroup("Name", SampleUnit.Milliseconds);
+    var profileMarkersSampleGroups = new []{
+        new SampleGroup("MarkerName", SampleUnit.Second), 
+        new SampleGroup("MarkerName1", SampleUnit.Nanosecond)
+    };
+
+    yield return Measure.Frames().SampleGroup(sg).ProfilerMarkers(profileMarkersSampleGroups).Run();
+
+     ...
+}
+```

Documentation~/measure-method.md

@@ -5,6 +5,7 @@ Executes the provided method, sampling performance using the following additiona
 * **MeasurementCount(int n)** - number of measurements to capture, defaults to 9 if not specified.
 * **IterationsPerMeasurement(int n)** - number of method executions per measurement to use. If this value is not specified, the method is executed as many times as possible until approximately 100 ms has elapsed.
 * **SampleGroup(string name)** - name of the measurement, defaults to "Time" if unspecified.
+* **SampleGroup(SampleGroup sampleGroup)** - a sample group with a custom name and measurement unit. This will override the otherwise default value of "Time".
 * **GC()** - if specified, measures the total number of Garbage Collection allocation calls.
 * **SetUp(Action action)** - is called every iteration before method execution. Setup time is not measured.
 * **CleanUp(Action action)** - is called every iteration after method execution. Cleanup time is not measured.
@@ -22,7 +23,7 @@ public void Test()
 
 #### Example 2: Customize Measure.Method properties
 
-```
+``` csharp
 [Test, Performance]
 public void Test()
 {
@@ -36,3 +37,15 @@ public void Test()
         .Run();
 }
 ```
+
+#### Example 3: Measure.Method, SampleGroup and ProfilerMarkers
+
+``` csharp
+[Test, Performance]
+public void Test()
+{
+    var sg = new SampleGroup("Name", SampleUnit.Microsecond);
+    var sgMarker = new SampleGroup("MarkerName", SampleUnit.Seconds);
+    Measure.Method(() => { ... }).SampleGroup(sg).ProfilerMarkers(sgMarker).Run();
+}
+```

Documentation~/measure-profile-markers.md

@@ -2,6 +2,8 @@
 
 Used to record profiler markers. Profiler marker timings are recorded automatically and sampled within the scope of the `using` statement. Names should match profiler marker labels. Profiler markers are sampled once per frame. Sampling the same profiler marker per frame will result in the sum of all invocations. Note that deep and editor profiling is not available. Profiler markers created using `Profiler.BeginSample()` are not supported, switch to `ProfilerMarker` if possible. 
 
+You can also create your own SampleGroups, specifying a custom name and the measurement units you want your results in, see [example 2](#example-2-measuring-profiler-markers-in-a-scope-with-custom-samplegroups). 
+
 #### Example: Measuring profiler markers in a scope
 
 ``` csharp
@@ -22,3 +24,23 @@ public void Test()
     }
 }
 ```
+
+#### Example 2: Measuring profiler markers in a scope with custom SampleGroups
+
+``` csharp
+
+[UnityTest, Performance]
+public IEnumerator Test()
+{
+    var sampleGroups = new []{
+        new SampleGroup("Instantiate", SampleUnit.Second), 
+        new SampleGroup("Instantiate.Copy", SampleUnit.Nanosecond),
+        new SampleGroup("Instantiate.Awake", SampleUnit.Microsecond)
+    };
+
+    using (Measure.ProfilerMarkers(sampleGroups))
+    {
+        ...
+    }
+}
+```

Documentation~/measure-scope.md

@@ -1,8 +1,9 @@
 # Measure.Scope(string name = "Time")
 
 Measures execution time for the scope as a single time, for both synchronous and coroutine methods. Passing the name argument overrides the name of the created SampleGroup.
+The defualt SampleGroup is named "Time" and with Milliseconds as measurement unit. You can also create your own SampleGroup, specifying a custom name and the measurement unit you want your results in, see [example 2](#example-2-specify-custom-samplegroup).
 
-#### Example: Measuring a scope; execution time is measured for everything in the using statement
+#### Example 1: Measuring a scope; execution time is measured for everything in the using statement
 
 ``` csharp
 [Test, Performance]
@@ -14,3 +15,17 @@ public void Test()
     }
 }
 ```
+
+#### Example 2: Specify Custom SampleGroup
+
+``` csharp
+[Test, Performance]
+public void Test()
+{
+    var sampleGroup = new SampleGroup("Scope", SampleUnit.Microsecond);
+    using (Measure.Scope(sg))
+    {
+        ...
+    }
+}
+```

Editor/TestRunBuilder.cs

@@ -76,16 +76,14 @@ static List<string> GetPackageDependencies()
 
         public void Cleanup()
         {
-            if (File.Exists(Utils.TestRunPath))
-            {
-                File.Delete(Utils.TestRunPath);
-                File.Delete(Utils.TestRunPath + ".meta");
-            }
+            if (File.Exists(Utils.TestRunPath)) { File.Delete(Utils.TestRunPath); }
+            const string metaPath = Utils.TestRunPath + ".meta";
+            if (File.Exists(metaPath)) {File.Delete(metaPath);}
 
             if (EditorPrefs.GetBool(cleanResources) && Directory.Exists("Assets/Resources"))
             {
                 Directory.Delete("Assets/Resources/", true);
-                File.Delete("Assets/Resources.meta");
+                if(File.Exists("Assets/Resources.meta")) {File.Delete("Assets/Resources.meta");}
             }
 
             AssetDatabase.Refresh();

Runtime/Measure.cs

@@ -2,6 +2,7 @@
 using System.Diagnostics;
 using Unity.PerformanceTesting.Exceptions;
 using Unity.PerformanceTesting.Measurements;
+using Unity.PerformanceTesting.Runtime;
 using UnityEngine;
 using Object = UnityEngine.Object;
 
@@ -48,11 +49,21 @@ public static ScopeMeasurement Scope(string name = "Time")
         {
             return new ScopeMeasurement(name);
         }
+        
+        public static ScopeMeasurement Scope(SampleGroup sampleGroup)
+        {
+            return new ScopeMeasurement(sampleGroup);
+        }
 
         public static ProfilerMeasurement ProfilerMarkers(params string[] profilerMarkerLabels)
         {
             return new ProfilerMeasurement(profilerMarkerLabels);
         }
+        
+        public static ProfilerMeasurement ProfilerMarkers(params SampleGroup[] sampleGroups)
+        {
+            return new ProfilerMeasurement(sampleGroups);
+        }
 
         public static MethodMeasurement Method(Action action)
         {
@@ -70,24 +81,30 @@ public struct ScopeMeasurement : IDisposable
         private readonly SampleGroup m_SampleGroup;
         private readonly long m_StartTicks;
 
-        public ScopeMeasurement(string name)
+        public ScopeMeasurement(SampleGroup sampleGroup)
         {
-            m_SampleGroup = PerformanceTest.GetSampleGroup(name);
+            m_SampleGroup = PerformanceTest.GetSampleGroup(sampleGroup.Name);
             if (m_SampleGroup == null)
             {
-                m_SampleGroup = new SampleGroup(name);
+                m_SampleGroup = sampleGroup;
                 PerformanceTest.Active.SampleGroups.Add(m_SampleGroup);
             }
 
             m_StartTicks = Stopwatch.GetTimestamp();
             PerformanceTest.Disposables.Add(this);
         }
 
+        public ScopeMeasurement(string name) : this(new SampleGroup(name))
+        {
+        }
+
         public void Dispose()
         {
             var elapsedTicks = Stopwatch.GetTimestamp() - m_StartTicks;
             PerformanceTest.Disposables.Remove(this);
             var delta = TimeSpan.FromTicks(elapsedTicks).TotalMilliseconds;
+            
+            delta = Utils.ConvertSample(SampleUnit.Millisecond, m_SampleGroup.Unit, delta);
 
             Measure.Custom(m_SampleGroup, delta);
         }
@@ -97,15 +114,15 @@ public struct ProfilerMeasurement : IDisposable
     {
         private readonly ProfilerMarkerMeasurement m_Test;
 
-        public ProfilerMeasurement(string[] profilerMarkers)
+        public ProfilerMeasurement(SampleGroup[] sampleGroups)
         {
-            if (profilerMarkers == null)
+            if (sampleGroups == null)
             {
                 m_Test = null;
                 return;
             }
 
-            if (profilerMarkers.Length == 0)
+            if (sampleGroups.Length == 0)
             {
                 m_Test = null;
                 return;
@@ -115,10 +132,14 @@ public ProfilerMeasurement(string[] profilerMarkers)
             if (Application.isPlaying) Object.DontDestroyOnLoad(go);
             go.hideFlags = HideFlags.HideAndDontSave;
             m_Test = go.AddComponent<ProfilerMarkerMeasurement>();
-            m_Test.AddProfilerSample(profilerMarkers);
+            m_Test.AddProfilerSampleGroup(sampleGroups);
             PerformanceTest.Disposables.Add(this);
         }
 
+        public ProfilerMeasurement(string[] profilerMarkers): this(Utils.CreateSampleGroupsFromMarkerNames(profilerMarkers))
+        {
+        }
+
         public void Dispose()
         {
             PerformanceTest.Disposables.Remove(this);
@@ -127,4 +148,4 @@ public void Dispose()
             Object.DestroyImmediate(m_Test.gameObject);
         }
     }
-}
+}

Runtime/Measurements/FramesMeasurement.cs

@@ -1,6 +1,7 @@
 using System;
 using System.Collections;
 using System.Diagnostics;
+using Unity.PerformanceTesting.Runtime;
 using UnityEngine;
 using Debug = UnityEngine.Debug;
 using Object = UnityEngine.Object;
@@ -14,22 +15,34 @@ public class FramesMeasurement
         private const int k_ProbingMultiplier = 4;
         private const int k_MinIterations = 7;
 
-        private string[] m_ProfilerDefinitions;
-        private string m_SampleGroup = "Time";
+        private SampleGroup[] m_ProfilerSampleGroups;
+        private SampleGroup m_SampleGroup = new SampleGroup("FrameTime");
         private int m_DesiredFrameCount;
         private int m_Executions;
         private int m_Warmup = -1;
         private bool m_RecordFrametime = true;
 
         public FramesMeasurement ProfilerMarkers(params string[] profilerMarkerNames)
         {
-            m_ProfilerDefinitions = profilerMarkerNames;
+            m_ProfilerSampleGroups = Utils.CreateSampleGroupsFromMarkerNames(profilerMarkerNames);
+            return this;
+        }
+
+        public FramesMeasurement ProfilerMarkers(params SampleGroup[] sampleGroups)
+        {
+            m_ProfilerSampleGroups = sampleGroups;
             return this;
         }
 
         public FramesMeasurement SampleGroup(string name)
         {
-            m_SampleGroup = name;
+            m_SampleGroup.Name = name; 
+            return this;
+        }
+
+        public FramesMeasurement SampleGroup(SampleGroup sampleGroup)
+        {
+            m_SampleGroup = sampleGroup;
             return this;
         }
 
@@ -51,11 +64,14 @@ public FramesMeasurement DontRecordFrametime()
             return this;
         }
 
-        public ScopedFrameTimeMeasurement Scope(string name = "FrameTime")
+        public ScopedFrameTimeMeasurement Scope(string name = "Time")
         {
             return new ScopedFrameTimeMeasurement(name);
         }
-
+        public ScopedFrameTimeMeasurement Scope(SampleGroup sampleGroup)
+        {
+            return new ScopedFrameTimeMeasurement(sampleGroup);
+        }
         public IEnumerator Run()
         {
             if (m_Executions == 0 && m_Warmup >= 0)
@@ -67,7 +83,7 @@ public IEnumerator Run()
             yield return m_Warmup > -1 ? WaitFor(m_Warmup) : GetDesiredIterationCount();
             m_DesiredFrameCount = m_Executions > 0 ? m_Executions : m_DesiredFrameCount;
 
-            using (Measure.ProfilerMarkers(m_ProfilerDefinitions))
+            using (Measure.ProfilerMarkers(m_ProfilerSampleGroups))
             {
                 for (var i = 0; i < m_DesiredFrameCount; i++)
                 {
@@ -132,14 +148,17 @@ public struct ScopedFrameTimeMeasurement : IDisposable
         {
             private readonly FrameTimeMeasurement m_Test;
 
-            public ScopedFrameTimeMeasurement(string sampleGroup)
+            public ScopedFrameTimeMeasurement(SampleGroup sampleGroup)
             {
                 var go = new GameObject("Recorder");
                 if (Application.isPlaying) Object.DontDestroyOnLoad(go);
                 m_Test = go.AddComponent<FrameTimeMeasurement>();
-                m_Test.SampleGroup = new SampleGroup(sampleGroup);
+                m_Test.SampleGroup = sampleGroup;
                 PerformanceTest.Disposables.Add(this);
             }
+            public ScopedFrameTimeMeasurement(string name): this(new SampleGroup(name))
+            {
+            }
 
             public void Dispose()
             {